I had a working prototype of a real-time transport for a chat feature: a federated GraphQL subscription, Redis fan-out across pods, server-sent events to the browser, three authentication strategies exercised end to end against a three-pod cluster. An AI coding assistant wrote nearly all of it in about a day. Ninety-seven files of proof-of-concept scaffold, a real integration into the production service, two transports behind one interface, twenty-five recorded experiments. The part of the plan that looked least predictable, getting the auth stack to work over a websocket, was about a hundred lines and done.

The recommendation I wrote at the end was: do not build it yet. This post is the reasoning, because the AI part of it changed the answer in a direction most people would not expect.

the volume did not justify it

The product carried about a hundred messages a day across the new chat path and the legacy one. The busiest hour in thirty days had 23 active conversations. Delivery already worked through a two-second poll. Sub-100 ms delivery would have improved a handful of sessions a day, and the engineering capacity was better spent on whatever moved adoption.

So the recommendation set a trigger rather than a date: build when sustained volume roughly doubles, or concurrent open chats at peak pass a couple dozen, or product commits to chat on mobile. Each of those is queryable from the same tables that produced the current numbers, so nobody has to remember to revisit.

That much is ordinary. The interesting part is what the prototype’s cheapness does to the decision.

the build estimate was wrong, in a precise direction

The earlier build-versus-buy comparison had scored four options across twelve dimensions with human-effort assumptions. Those estimates were wrong now, and it is worth being exact about how.

The writing cost of the in-house option had largely been paid. What remained of it was small. That is a real change and should be said plainly.

What did not get cheaper is everything after writing. The published evidence on AI-assisted development is consistent on one point even where it disagrees on magnitude: throughput rises and stability falls.

• Cortex’s 2026 engineering benchmark report found pull requests per author up 20% year over year as AI adoption grew, and incidents per pull request up 23.5%.
• Google’s 2025 DORA report found AI adoption correlated with higher delivery throughput and lower delivery stability, with individual developers merging far more pull requests and review time rising with them.
• GitClear’s analysis of 623 million code changes from 2023 to 2026 found duplicated code blocks up 81%, copy-paste within a commit up 41%, error-masking constructs up 47%, two-week churn up 15%, and refactoring line moves down 70%.
• METR’s randomized trial found experienced developers 19% slower with AI tools while believing they were 20% faster, a 39-point gap between measured and perceived speed.

Across studies the range runs from METR’s 19% slower to the 26% more weekly pull requests that a Microsoft, MIT, Princeton, and Wharton trial measured across about 4,900 developers, which is another way of saying that writing speed is not the variable that decides outcomes. AI moves the constraint from writing code to owning it. Review, operation, on-call, and incident response are where the cost sits now.

reweighting the matrix barely moved it

If build effort is the thing that got cheap, it should carry less weight in the decision, and operating effort should carry more. I dropped “effort to build” from weight 2 to 1 and raised “effort to operate” from 2 to 3, against the same scores.

The ranking did not move. Every total moved by a single point, out of about 170. That is the useful result: build effort was never the deciding dimension, so making it nearly free changes nothing about which option is better. What decides it is authorization correctness, architectural fit, reconnect and recovery behavior, and operational surface. AI implementation touches none of those.

the counterintuitive part

Because the assistant makes the in-house option cheap to write, the temptation is to build it now, while it is cheap. That is backwards.

The cheapness is entirely in the writing. The expensive half starts the day it ships and never stops: operating a service that is now stateful where it used to be stateless, reviewing concurrency and authorization code that a model wrote, carrying it on call. None of that got cheaper, and none of it can be deferred once the thing is running.

What AI actually bought is the ability to build it later, on demand, from a branch that already exists, in days. “We can have this whenever we want it” is the asset. At a hundred messages a day, that option is worth more than the feature.

if and when it is built

The Cortex, DORA, and GitClear findings are a risk profile to design against while using the tool.

• Evidence over review. What caught the two measurement bugs and the one boot-blocking defect in the prototype was not careful reading. It was twenty-five recorded experiments against a real cluster, including the ones that refuted my own hypotheses. Any change touching the live channel should carry that kind of evidence.
• Small PRs, because review is the bottleneck. A sub-300-line discipline matters even more when the generator is fast.
• Guard against duplication specifically. GitClear’s strongest signal is duplicated blocks and falling reuse. The transport interface in the prototype exists for exactly that reason: two transports, one publish path, no parallel implementation to drift.
• Adversarial verification where a plausible wrong answer is expensive. Authorization at subscribe and the cross-pod fan-out were tested by trying to break them. The one real authorization finding came out of that testing. Reading the policy would not have found it.
• Assume the model’s estimates are optimistic about itself. METR’s 39-point perception gap is the caution. Every claim in the recommendation that mattered carried a measurement next to it for that reason.

what did ship

The prototype found a production defect unrelated to the feature: a 30-second dial-timeout default in the router that turned a rolling deploy into a 32-second stall for every client. The recommendation put that fix first and on its own, because it was worth more than the feature that found it and was needed under every option on the table. The feature itself is a branch, a trigger, and a decision record. That is the right amount of it to have.