The chat feature I am building now delivers messages with a two-second poll. Not WebSockets, not GraphQL subscriptions, not a pub/sub vendor. This post is why, and what happened when I measured what the poll actually cost.

the since-cursor

The client asks for messages since its last cursor. When there is nothing new, the server returns an empty array. That is the whole trick, and it is what makes polling defensible: an idle tick is a small request and a smaller response, and the data model does not know or care what transport carried it.

The reasons to pick this over a push transport were practical. It ships on existing infrastructure across a web app and two mobile platforms with no new stateful service. The federated GraphQL layer already handles authorization, so the poll runs the same policy as every other read, with nothing mirrored anywhere. Because the cursor is the contract, the transport can change later without touching the data model. A prototype of the push transport exists on a branch. What would make us switch is a volume trigger, queryable from the same tables: roughly double the current daily message count sustained, or a couple dozen concurrent open chats at peak, or a product commitment to chat on mobile.

That was the reasoning. Then I measured.

the poll was 22% of the backend before launch

Chat had not launched. The only traffic was staging: developers and QA leaving the chat open. Over seven days on staging, chat polling was 21.9% of every GraphQL operation the backend served, and the open-conversation poll alone was 18.6%. That is the shape of the thing before a real user touches it.

Per tick, traced end to end: about 43 ms at the router, of which 41.5 ms is the subgraph call. Inside the subgraph, four database queries on an empty tick where one would do. Two for authorization, one to look up the anchor message the cursor points at, one for the delta itself. At thirty ticks a minute that is about 120 statements a minute per open conversation, all against the primary, because the service had one database client and it was the write client.

The request is also heavier than it looks. The delta query text is 2,923 bytes, the full POST body 3,445, plus a 1 to 2 KB bearer token. Call it 5 KB up for a 123-byte response, thirty times a minute, carrying no information.

apm sampling hid the volume by 70x

The first attempt at counting ticks read span counts out of the tracing tool. It was wrong by almost two orders of magnitude.

Span retention is sampled, and the sampling is not uniform across resources. Service-wide, about 21% of request spans were retained. For the chat delta specifically, 1,652 spans were retained against 114,919 actual hits: 1.4%. Reading poll volume off span counts understated it by roughly 70x.

The counts that are correct come from the hit metrics the agent computes before sampling. Every number in this post uses those. If you have ever sized a workload from a span search, re-check it against the unsampled counter.

what does not help

Two things I expected to matter did not survive measurement.

The wide selection set. The client document selects a large fragment, including the author profile and, for shared listings, the whole property card and photo array. I expected each tick to fan out across subgraphs. It does not. The router splits the document per subgraph, and the query that reaches the chat backend asks for the listing’s id and typename only. The property card resolves elsewhere and only when a message with an attachment actually arrives. An empty tick does zero cross-subgraph work. Narrowing the fragment helps ticks that return attachments and nothing else.

Persisted queries. Hashing the 2,923-byte query to a 64-character digest cuts request bytes by 87%. I had it down as the cheapest win in the document. Then I looked at where the router spends its 43 ms: execute 41.5 ms, plan 0.6 ms, validate 0.2 ms, normalize 0.16 ms, parse 0.075 ms, read body 0.022 ms. Persisted queries remove parse and most of the body read. That is 0.097 ms of 43.39, or 0.22% of server time. Bytes were never the constraint.

the binding constraint is node handler time

Extrapolating from the staging baseline to plausible production multiples, the database was not the problem. Even at a thousand times staging, polling keeps about two Postgres connections continuously busy against a writer that already carries over a thousand.

The handler time is the problem. A Node process is single-threaded, so handler seconds per second is pod equivalents. At a thousand times staging, polling alone needs 5.3 pods of pure handler time serving requests that mostly return nothing. That is where the cost is, and it is what every fix should be measured against.

one client-only change is 72% of the saving

The candidate fixes, ranked by load removed per unit of work:

1. Attention tiers. Replace the binary visible/hidden with four tiers: active (visible, focused, recent input), blurred (visible, window not focused), idle (visible, no input for a couple of minutes), hidden (paused, which already existed). Each tier is a multiplier on the configured interval, so the flag that sets cadence stays the one place it is set. At a 2-second base that is 2 s, 10 s, 30 s, paused. A parked tab drops from thirty requests a minute to two.
2. Empty-tick backoff. For the present user in a quiet conversation, which given the message volume is the common case. Count consecutive empty ticks and step the interval up a bounded ladder, 1x, 3x, 8x. Reset to the floor on any returned row, on the viewer sending a message, or on the tier snapping back to active. The send-path reset matters: a viewer who types after five minutes of reading expects the reply at full cadence.
3. Jitter and server backpressure. About 15% jitter per tick, because every client on a tenant shares one configured interval and realigns into a synchronized wave after each deploy. Treat 429 or 5xx as a backoff signal with its own multiplier, since the loop was retrying a failing backend at unchanged cadence.
4. Drop the anchor lookup. The client holds the anchor message, so it can send its timestamp and save one of four queries per tick. The authorization check the lookup doubled as does not weaken, because the delta query already filters by conversation and tenant.
5. Digest-first polling. Poll one cheap root field that returns counters, fetch a delta only when a counter moves. The largest structural change, needing work on both sides.

Modeling the first three against the measured per-tick costs, with a mix of parked, quiet, and active sessions: the cadence work is worth 72% of everything on its own. At a thousand times staging that is 3.8 pods and 642 database queries a second not spent. Every per-tick item is a rounding error until the cadence work is done. After it, dropping the anchor lookup takes another 21% off queries, persisted queries take 85% off bytes, and digest-first, the biggest piece of work in the list, is worth one more pod.

The first three are one pull request, client-only, no schema change, no backend change, dark behind the existing config flag. That is the one change the recommendation puts before launch. One addition: each poll is tagged with the tier that scheduled it, so the session mix becomes a measurement after launch instead of the assumption the model rests on.

the free lever, and why it is the wrong default

There is a cheaper option than any of this: raise the base interval in the flag. Two seconds to three removes a third of all poll load in one edit, no deploy. Four seconds removes half.

It should be rejected, and the reason is the whole argument for the tiers. The delivery target is a p95 of 2.5 seconds at the 2-second cadence. A blanket increase breaks that for everyone, including the one person actively reading a conversation. The tiers reach a larger saving while degrading freshness only for viewers who are not looking at the screen. Same load curve, and the reader who is looking keeps the cadence the target was written against.

It stays available as the emergency lever, because it is the only one that works without a deploy.