Polling vs Streaming Flag Synchronization

This guide is part of the Backend Evaluation & Server-Side SDKs series. Once you commit to local, in-process evaluation, the only open question is how the local rule set stays fresh: a server-side SDK either polls the control plane on an interval or holds a streaming connection that pushes changes as they happen. That single choice sets your propagation latency, your connection budget, and how fast a kill switch actually reaches every node.

It is worth being precise about what “fresh” means here, because the word hides a real distributed-systems problem. The control plane holds the authoritative flag configuration; every replica holds a copy. Freshness is the lag between a write landing in the control plane and that write becoming visible in each replica’s local copy — and because there is no single lag but one per replica, the property you actually care about is convergence: the point at which every node in the fleet agrees on the same variant. Polling and streaming are simply two mechanisms for driving that convergence, and they trade the same three currencies — latency, connection cost, and reconnection complexity — in opposite directions. Pick the transport by working backward from the tightest convergence guarantee you have promised the business, usually the one attached to emergency rollback rather than to routine rollout, since rollback is the case where a slow transport turns a minor incident into a prolonged one.

Problem Framing: When the Transport Choice Matters

Transport is invisible until it isn’t. Polling on a 60-second interval means a flag flip — including an emergency rollback — can take up to a full minute to reach a given replica, and a fleet of 500 pods polling independently produces a steady drumbeat of requests against the control plane. Streaming collapses propagation to sub-second but holds a long-lived connection per process and demands disciplined reconnection logic.

The failure this framing guards against is the one that only shows up during an incident. In steady state, both transports look identical: flags change on a leisurely cadence, every replica catches up within a comfortable window, and nobody measures the difference. The gap appears the moment you flip a kill switch to stop a bad rollout and then watch a dashboard that still shows a fraction of traffic hitting the broken path for the next forty seconds — because that fraction is served by replicas whose next poll has not fired yet. A propagation budget that reads “rollback visible within 2s” is not pedantry; it is the number that decides whether your worst-case blast radius is measured in requests or in minutes. Write it down before you pick a transport, because the transport is how you meet it.

There is also a subtler cost that rarely makes it into the decision: operator confidence. When propagation is slow and uneven, the person driving a rollout cannot trust the dashboard — they flip a flag, see partial effect, and cannot tell whether the change is wrong or merely still propagating. That ambiguity slows every rollout decision and pushes teams toward over-cautious, manual, staged changes. Fast and observable convergence is what lets an operator flip a flag and believe the resulting metric, which is the real productivity payoff of getting the transport right.

This guide covers the decision and the wiring for both transports under OpenFeature. It does not cover the cache topology behind the SDK (see distributed caching for flag evaluations) or rule compilation (see optimizing rule engine performance).

Prerequisites

• OpenFeature server SDK ≥ 1.x with a provider that supports both transports (flagd does)
• Network egress from every service replica to the control plane endpoint
• A health/readiness probe wired to the provider’s connection state
• Flag metadata fields owner and expiry populated per your flag taxonomy
• An agreed propagation-latency budget (e.g. “rollback visible within 2s”)

Core Concept & Architecture

Both transports converge on the same local state — a compiled rule set the rule engine reads in-process. They differ only in how that state is refreshed. The decision matrix:

A robust setup is rarely pure: stream for low-latency propagation, and keep a slow background poll as a safety net that heals missed events during a reconnect gap.

Read the matrix as a set of trade curves rather than a verdict. Propagation latency and control-plane load pull against each other on the polling side — the only way polling gets faster is a tighter interval, and a tighter interval is more requests per second across the whole fleet, so polling has no free lunch: you buy latency with load. Streaming breaks that coupling. Its latency is fixed at roughly the network round trip regardless of fleet size, and its cost is a flat count of open connections rather than a request rate, so shrinking latency to zero costs nothing extra. What streaming buys with that decoupling is complexity: a persistent connection is a piece of state that can rot, and every dropped connection is a small correctness hazard until the resync completes. Polling has no such hazard because it carries no state between requests — each poll is a fresh, complete question, which is exactly why it survives flaky networks and aggressive proxies that streaming does not.

The firewall row deserves more weight than it usually gets in a design review. A plain HTTP poll looks like every other request your service already makes, so it passes through corporate proxies, service meshes, and egress filters without special dispensation. A long-lived gRPC or SSE connection is a different animal: some load balancers cap idle connection lifetime, some proxies buffer streamed responses until the buffer fills (destroying the whole point of streaming), and some egress policies simply do not permit persistent outbound connections. If you cannot control the network path end to end — common in regulated environments and in multi-tenant platforms — polling is often the pragmatic default not because it is better but because it is the one that reliably connects.

Step-by-Step Implementation

The production shape is a hybrid: streaming carries changes with sub-second latency, a slow background poll heals anything missed during a reconnect gap, and a last-known-good cache keeps evaluation alive if both fail.

Step 1 — Configure streaming as the primary transport

Point the provider at the control plane and select the streaming resolver so changes arrive over a persistent connection.

# provider-sync.yaml — flagd sync configuration
sync:
  selector: "core"
  provider: streaming          # primary: push-based updates
  uri: "grpc://flagd.internal:8013"
  poll_interval_ms: 30000       # fallback poll heals missed events
cache:
  local_ttl: 0                  # 0 = trust the stream; no independent expiry
  fallback: last_known_good

Pitfall: setting a nonzero local_ttl alongside streaming creates two sources of truth — the stream says “current” while the TTL silently expires entries. Let the stream own freshness and use the poll only as a backstop.

The selector: "core" line matters more than it looks. A selector scopes the stream to a subset of flags — here the core namespace — so a replica subscribes only to the flags it actually evaluates rather than the entire catalog. On a large deployment where different services own disjoint flag sets, a broad selector means every push fans out to every replica whether or not the change is relevant, inflating both control-plane egress and the per-replica cost of parsing updates it will ignore. Scope the selector to what the service reads and the stream stays quiet except when something the service cares about actually moves. The poll_interval_ms of 30 seconds is deliberately slack: because the stream is primary, the poll exists only to catch the rare event that slipped through a reconnect gap, so a long interval keeps its steady-state load near zero while still bounding worst-case staleness at thirty seconds if the stream dies quietly and the connection metric fails to fire.

Step 2 — Wire reconnection with backoff and resync

A streaming connection will drop. On reconnect, do a full resync rather than assuming you only missed the latest event.

import { OpenFeature } from '@openfeature/server-sdk';

provider.on('reconnecting', () => metrics.increment('flag.stream.reconnect'));
provider.on('ready', async () => {
  await provider.resyncFlags();           // pull full state, not just a delta
  metrics.gauge('flag.stream.connected', 1);
});
provider.on('error', () => metrics.gauge('flag.stream.connected', 0));

Pitfall: applying only the delta after a gap leaves a node permanently stale for any flag changed during the disconnect. Always resync the full set. See exponential backoff for SDK reconnection for the backoff curve.

The reason a full resync is non-negotiable is that a streaming protocol gives you no reliable way to know what you missed. Delta streams typically carry the change, not a running sequence number you can compare against — and even when they do carry a cursor, a cursor only tells you that you fell behind, not the contents of the events between your last-seen position and now. Once the connection is back, the cheapest correct move is to throw away the assumption of continuity and pull the entire current state, which is idempotent by construction: re-applying flags you already had costs nothing, while missing one costs you a silently wrong evaluation until the next change happens to touch it. The resync is also where the backoff curve earns its keep. If the control plane is the thing that fell over, every replica reconnecting on the same tight schedule produces a synchronized reconnect storm precisely when the control plane can least absorb it — jittered exponential backoff spreads those resyncs across a widening window so recovery is gradual rather than a second outage. Emit a metric on every resync, not just on connect, so a fleet that is quietly flapping shows up as a rising resync rate long before it becomes a stale-flag incident.

Step 3 — Fall back to polling where streaming is impractical

Behind strict proxies or in serverless runtimes that recycle connections, a short poll is more reliable than a stream that constantly re-establishes.

# serverless-sync.yaml — polling-only profile
sync:
  provider: polling
  uri: "https://flagd.internal/flags"
  poll_interval_ms: 5000        # tighten interval to shrink the staleness window
cache:
  local_ttl: 5s
  fallback: last_known_good

Pitfall: every replica polling on the same fixed interval synchronizes into a thundering herd. Add jitter (±20%) to the interval so requests spread across the window.

Serverless is the case where polling stops being a fallback and becomes the correct primary. A function that is frozen between invocations cannot hold a stream — the runtime suspends the process, the connection dies, and you pay a full reconnect-and-resync on the next cold start, which adds latency to exactly the request you least want to slow down. A short poll fired at the top of the handler, or a cached snapshot refreshed on a TTL, fits the execution model: it is stateless, it completes within the invocation, and it degrades gracefully to the last-known-good copy if the control plane is briefly unreachable. Tighten the interval here with eyes open — a 5-second interval means a serverless fleet can lag a rollout by up to five seconds, which is fine for a gradual ramp and wrong for a kill switch, so pair aggressive serverless polling with a separate, faster emergency path if instant rollback is a hard requirement. The jitter matters even more in serverless than in a long-lived fleet, because autoscaling tends to spin up cohorts of identical instances simultaneously, and without jitter their first polls land in the same millisecond and repeat on the same beat forever.

Also make the poll’s failure mode explicit. A poll that returns a 500 or times out must not silently overwrite good local state with nothing — the client should keep serving the last successful snapshot and count the failure, so a control-plane wobble degrades freshness rather than blanking flags to their code defaults mid-request. The difference between “the flag held its last value for thirty seconds” and “the flag reverted to false because a poll failed” is the difference between a non-event and an incident.

Verification & Testing

Prove propagation latency rather than assuming it. Flip a canary flag and measure the time until every replica reports the new variant.

# Flip the flag, then poll each replica's evaluation endpoint until consistent
flagctl set web.dashboard.new-nav --variant on
for host in $(cat replicas.txt); do
  curl -s "$host/debug/flags/web.dashboard.new-nav" | jq -r '.variant'
done | sort -u   # expect a single line "on" within your latency budget

For streaming, assert reconnect behavior by killing the connection (block the port for 5s) and confirming a full resync fires on recovery.

The subtle part of this test is what you measure, not that you measure. A single averaged propagation number hides the failure that hurts: the tail. If 499 replicas converge in 200ms and one converges in 40 seconds because it was mid-reconnect, the mean looks excellent while your worst-case blast radius is still forty seconds wide. Assert against the maximum across replicas, not the mean, and run the flip repeatedly — a dozen times across a normal deploy cycle — so you catch the convergence that only misbehaves when a replica happens to be restarting. The convergence assertion should also be run during a simulated control-plane disruption, not only in calm conditions, because the transport’s real job is to keep flags correct while the network is misbehaving, and that is the one scenario a happy-path test never exercises. Wire the flip-and-converge loop into a synthetic check that runs continuously against a dedicated canary flag; a slow drift in convergence time is an early warning that connections are flapping or the control plane is saturating, and catching it as a trend beats catching it as a paging incident. Keep that canary flag inert — a boolean nothing reads in production logic — so the test can flip it as often as it likes without touching real behavior.

Troubleshooting & FAQ

Why is a flag change not reaching some replicas?

On streaming, those replicas almost certainly lost the connection and resynced from a stale snapshot, or never reconnected — check your flag.stream.connected gauge. On polling, the change simply hasn’t reached the next tick yet, or the replica’s poll is failing silently; verify the poll response status.

How small can I make the polling interval?

Small enough to meet your propagation budget, large enough that replicas ÷ interval stays within the control plane’s request budget. Below ~1s, the request volume usually justifies switching to streaming instead.

Do I still need a cache if I’m streaming?

Yes — the local rule set the stream maintains is the cache. The stream keeps it fresh; the last_known_good fallback keeps evaluation working through a control-plane outage. Caching strategy is covered in distributed caching for flag evaluations.

Can I run streaming and polling at the same time?

Yes, and for most production fleets you should. Run streaming as the primary transport for sub-second propagation and keep a slow background poll — a 30-second interval is typical — as a backstop that heals any event missed during a reconnect gap. Give the stream ownership of freshness by disabling independent cache expiry, so the two mechanisms cannot disagree about what is current. The poll is insurance, not a second source of truth.

How do I know if my stream is silently dead?

A dropped stream is dangerous precisely when it fails quietly — the connection object still exists but no events arrive, so evaluations keep returning stale values with no error. Guard against it with a heartbeat: expect a keepalive frame or a periodic no-op event on an interval, and treat its absence as a disconnect that triggers reconnect-and-resync. Expose a connected gauge and a resync-rate metric so a fleet that is quietly flapping shows up on a dashboard before it becomes a stale-flag incident.

Does streaming guarantee ordering of flag updates?

Within a single healthy connection, updates arrive in the order the control plane emitted them, so ordering holds. Across a reconnect it does not — you may miss events entirely, which is why recovery must pull the full current state rather than replaying a delta. Design your flag logic to be order-independent where possible and never rely on observing two changes to the same flag in a specific sequence; treat each update as “here is the current value,” not “here is the next step.”

Should I measure propagation latency as a mean or a maximum?

Measure the maximum across the fleet. A mean hides the tail, and the tail is the risk: if one replica out of five hundred takes forty seconds to converge because it was mid-reconnect, the average still looks excellent while your worst-case blast radius is forty seconds wide. Assert your propagation budget against the slowest replica, and run the flip-and-converge test repeatedly across a deploy cycle so you catch the convergence that only misbehaves during a restart.

Performance & Scale Considerations

Budget propagation as part of your rollout SLA, not as an afterthought. For a fleet of N replicas, polling costs roughly N ÷ interval requests per second against the control plane; streaming costs N persistent connections plus a burst of resyncs whenever the control plane redeploys. Above a few hundred replicas, prefer streaming with a generous fallback poll and stagger replica restarts so resyncs don’t arrive as one spike. Keep the evaluation path itself local so transport hiccups never add latency to a request — that separation is the whole point of backend evaluation.

The resync burst is the scaling limit most teams hit first, and it is worth sizing concretely. A 500-replica fleet holding 500 idle streams costs the control plane almost nothing in steady state — idle connections are cheap. But roll the control plane, or roll the fleet, and all 500 replicas resync within the same short window: 500 full state pulls, each transferring the entire flag set the selector scopes to, arriving as one spike. If the flag payload is 200KB, that is 100MB of egress compressed into a few seconds, plus the CPU to serialize it 500 times. This is why staggered restarts and connection draining matter at scale — a rolling deploy that replaces ten pods at a time turns one 500-wide spike into fifty small ripples the control plane barely notices. The same logic applies to the control plane’s own deploys: drain streams gracefully so clients reconnect on a backoff curve rather than all at once when the old process dies.

Memory and connection limits set the other ceiling. Every open stream consumes a file descriptor and a slice of buffer on both ends, so a control plane fronting tens of thousands of replicas needs its connection limits, ephemeral port ranges, and load-balancer keepalive settings tuned deliberately — the defaults on most proxies assume short requests, not a standing army of persistent connections. When you cross into that territory, a tier of relay or agent nodes that fan a single upstream stream out to many local replicas keeps the connection count against the true control plane flat; the SSE tuning guide covers the keepalive and buffering knobs that keep those long-lived connections from being silently reaped.

Edge cases worth planning for

A few situations sit outside the happy path and reliably cause incidents when nobody has thought about them in advance:

• Clock-independent staleness. Polling’s staleness window is bounded by the interval only if polls actually fire on schedule. A garbage-collection pause, a saturated event loop, or a throttled container can stall the poll timer, stretching a “5-second” interval to thirty. Alert on observed poll age from the client’s own timestamps, not on the configured interval, so a stalled scheduler surfaces as stale flags rather than hiding behind a config value that no longer reflects reality.
• The reconnect-storm feedback loop. If a control-plane overload drops streams, every client reconnects, and un-jittered reconnects re-overload the control plane, you have built an oscillator that never settles. Jittered backoff plus a connection cap on the control-plane side breaks the loop; without both, the fleet can stay wedged long after the original trigger clears.
• Split-brain during a partial partition. A network partition can leave part of the fleet talking to the control plane while another part is cut off, so two cohorts serve different variants of the same flag for the duration. This is usually acceptable for a gradual rollout and dangerous for anything that must be globally consistent — like a flag gating a schema migration. Know which of your flags tolerate a convergence gap and which do not, and never gate an irreversible operation on a flag that can be split-brained.
• Payload growth outrunning the transport. Both transports assume the flag set fits comfortably in a resync. A catalog that grows to tens of thousands of flags makes every resync expensive and every stream chatty. Scope selectors aggressively and prune dead flags per your deprecation process so the transport keeps carrying kilobytes, not megabytes.