The question arrived as “should we build the real-time transport for chat or buy one”. It was three questions, and only one of them was open. Untangling them did more to settle the debate than any of the scoring that followed, and the scoring turned up a finding I did not expect.
three questions wearing one name
What carries the live signal? A self-built federated GraphQL subscription, which a spike had already tested end to end, or a managed pub/sub transport. This was the open question.
Who owns chat itself? Our native chat, or a chat product from a vendor. This was not open in any practical sense. The chat backend was already shipped: about six thousand lines in the service, a data model with typed attachments across nine kinds, inline entity creation inside the message transaction, read watermarks, unread counts, search, tombstones, link previews. A chat product would replace the data model and the write path, which means re-implementing all of that against the vendor’s primitives, migrating every existing message, and moving all message content to a third party. That is a multi-month program, and it was being compared against a three-week transport project.
Had we already tried the vendors? Yes, both of the obvious ones, earlier in the year, in a shape that no longer described the product. The integration had been removed with the commit message “now that the unified chat abstraction handles everything”. What that history was still evidence of: the integration surface a vendor-owned store demands. User upsert, channel creation, membership sync, token minting.
Mixing the three produces a false comparison, because a chat product does not compete with a three-week transport project. It competes with a chat backend that already exists.
the volume
About a hundred messages a day across the new and legacy paths. The busiest hour in thirty days had 23 conversations active. Roughly fifty concurrent subscriptions at the highest point ever recorded, 726 distinct senders in a month. Every cost argument below runs against those numbers.
the finding that drove the scores
The design decision that mattered most was made before the comparison: the channel carries a signal, “something changed in conversation X”, and never content. The client refetches through the normal authorized queries. That makes the signal idempotent, which is why a transport with a weak delivery guarantee can still be adequate, and it bounds what a leaked signal can reveal.
Then the structural finding. One vendor sells a chat product, so its unit of access control is channel membership. Custom events are delivered to members and watchers of a channel, and nobody else. Used purely as transport, it still requires upserting every member as a vendor user, creating a channel per conversation, keeping the member list in sync with our own membership table, and minting a token per user. That is exactly the sync surface the earlier integration had built and later deleted, and it is required by the vendor’s permission model rather than by any design of ours. A mirrored membership table is a second source of truth, and drift between the two is silent.
The pub/sub vendors work the other way. Channels need no provisioning and users need no registration. The server mints a token whose capability is one channel, subscribe only, and the membership check happens once, at mint time, inside our own code. Nothing is mirrored, so nothing can drift.
So a chat-product vendor scored worst of four options, and the reason was structural rather than price. It demands the provisioning of a chat product while providing the weakest delivery guarantee of the three vendors, billed per monthly active user on people we would register only to receive a signal.
the matrix, and where each option loses
Twelve dimensions, weighted, four options, every cell tagged as measured, read from documentation, or judgment, so a reader can see which numbers to trust. The weights are the first thing to argue with, and the document says so. Authorization fit and the two delivery rows carry the most, because the channel is being added to a multi-tenant product where the failure mode is showing activity to the wrong person or silently not showing it to the right one.
In-house and the strongest pub/sub vendor scored within a few points of each other, and they lose in opposite places. In-house wins authorization and architectural fit outright, because the subscription runs the service’s own policy with nothing mirrored, and it costs nothing at this volume. The vendor wins recovery, operations, and mobile: its connection-state recovery replays every missed message in order after a reconnect, which removes the one defect the spike could not fix, a reconnect that silently loses about five seconds of signals and so forces a permanent safety poll. The price of that is moving the live channel outside the federated graph and adding a second authorization path.
None of the four fixes the revocation gap. An open stream outliving a membership change is a property of authorizing once at subscribe, and every option does that.
the three fundable options
Build it as tested: two and a half to three and a half weeks, no license, no new vendor, stays in the graph, needs the safety poll, makes the service stateful. Buy the transport: a shorter build plus procurement and a security review, free at current volume, removes the silent-loss problem, adds a second channel and a second authorization path. Or do neither yet: keep polling, because at a hundred messages a day nobody is measurably suffering, and spend the weeks elsewhere.
The third option went on the list honestly and it is the one that was taken. The case for real-time was a better feel in an open conversation rather than a load or correctness problem, and the volume said the audience for that improvement was small. What would change the answer is written down: an order of magnitude more adoption, mobile in scope, or a requirement for presence, typing indicators, or delivery receipts, none of which the signal design provides and all of which a transport vendor ships.
what to take from it
Separate the questions before scoring anything. Find the structural property of each option, the one that does not move when the weights do, and let it decide the close calls. For a vendor, that property is usually its access-control model, because it determines what state you have to keep in sync with them, and state you keep in sync with a vendor is state that drifts.