A reviewer flagged that I was adding the third rate-limit guard to one service and asked whether they could share code. Fair. I extracted a base guard, unified the attempt counters, and then, because the same reviewer’s question applied to more than one service, swept every backend repo in the organization to see what everyone else had done.
About eight bespoke implementations across four mechanisms. One shared library that a single service used and another had copied locally. One control duplicated outright across two services, same key, same numbers. No monitor anywhere on a limit firing or failing to fire. This post is what that sweep found, and why the proposal that came out of it was mostly not code.
two needs that had been conflated
Everything hand-rolled was trying to satisfy one of two genuinely different requirements.
Do not fall over. Coarse, volumetric, bot and DoS defense. Protect the service from being overwhelmed. This is a platform and infrastructure concern. It wants to live at the edge and as a per-service backstop, keyed on almost nothing, and it should fail open, because the point is availability.
Stop abuse. Fine-grained, identity-aware, per-operation limits on specific abuse-prone endpoints: login brute force, code and email enumeration, expensive AI calls, public unauthenticated mutations. This is an application and security concern. It needs request context the edge does not have (which tenant, which member, which operation), and some of it should fail closed, because the point is that a brute-force attempt is refused even when the counter store is down.
One shared guard was being asked to do both and satisfying neither, so every team built whatever their immediate need was, at whatever layer, with whatever store they had.
the inventory
The edge covered only a couple of surfaces, bot-scored only, with no blanket per-IP limit on human traffic. The API gateway, the load balancers, the ingress, and the federation router had nothing. So for every federated service and the main dashboard, application-level limiting was load-bearing, whether or not anyone had designed it that way.
At the application layer, the services that happened to have Redis had real distributed limits. The services on the framework’s stock in-memory throttler were limiting per replica, so a configured limit of 500 per 20 seconds was really 500 times the pod count. Nobody had chosen that. The abuse-critical limiters that most needed to be distributed were distributed by luck.
The login limiter lived in two services with the same key format and the same threshold. A GraphQL-aware guard adapter lived in the shared framework package, used by one service, reimplemented nearly line for line in another, and ignored by the rest. It shipped no store, no limits, and no distributed backing.
The only signal when a limit fired was an error-level log line, which is itself a liability during a real burst.
the proposal
Two tiers, both primitives owned by the platform team and shipped from the shared framework package.
Tier one is the edge plus a Redis-backed global throttler guard, default-on in every service. The existing shared guard becomes the one true guard and gets a shared Redis storage. One service had already designed exactly that and shelved it. This makes global limits accurate across replicas and puts the services with nothing onto a net.
Tier two is one shared, distributed limiter keyed on IP, tenant, member, and operation, for login, enumeration, expensive operations, and public mutations. The reference implementation was the one I had just extracted: a fixed-window consume service, fail-open request guards, and fail-closed attempt counters. The gateway, the second login limiter, and the copied guard adopt it, which retires the duplicates.
Cross-cutting and required: a standard limit-exceeded metric with service, rule, and tier tags, and monitors on it, shipped with the primitives so adoption gets observability for free.
why not just ship a library
Because that had already half-happened. The shared guard existed and was ignored or duplicated. A library without a named owner, a written standard, and default-on wiring becomes the ninth variant.
The value of the proposal was in naming the two tiers so teams could tell which one they needed, assigning an owner per tier so someone was accountable for the primitive, and making tier one default-on so the services with nothing got covered without a project. The code was the smallest part.
Two other alternatives were rejected. Centralizing everything at the edge cannot express “a handful of logins per quarter hour per IP” or “ten AI operations per minute per tenant”, so the edge owns tier one only. A service mesh with global rate limiting is architecturally clean for tier one and would cover the non-Node services too, but it is a large infrastructure lift the organization was not set up for, and it still cannot do tier two.
the rollout
Land the reference implementation in its own service first, as the pilot, and do not scope-creep it into the org effort. Then the standard goes through architecture review. Then the platform team promotes the primitives into the shared package with the metric baked in. Then migrate incrementally, starting with the duplicates and the services with nothing, behind flags.
I wrote this as a proposal for review rather than as a refactor I did unilaterally, because the decision touched several teams’ code and one team’s ownership. The guard extraction was small. The sweep and the write-up were the work.