A chat message with a link should show a preview card: title, description, image, site name. The browser cannot fetch that itself, because CORS blocks reading third-party HTML, so a server has to. That server is now a service that fetches arbitrary URLs on behalf of untrusted users, which is the textbook shape of a server-side request forgery vector. This is what it took to build one that is not.
where the fetch lives
The unfurl runs as a query on the same federated GraphQL subgraph as the chat, gated on write access to the conversation being composed in, rate limited per member, cached for a day with failures cached for an hour. Resolved metadata is written to the attachment row at send time, so message history does not depend on the third-party page staying alive, and no client can attach arbitrary text and imagery to a domain it does not control.
Two alternatives were rejected on that last point. Accepting preview fields from the client lets anyone forge a card describing any domain. Resolving previews at read time keeps every render dependent on a live fetch and rewrites history when a page changes.
the guard
The fetcher accepts http and https only, with no credentials in the URL, on ports 80 and 443 only. Before connecting, it resolves the hostname and screens the address against private, loopback, link-local, and carrier-grade NAT ranges, including the IPv4-mapped IPv6 forms that a naive check misses.
Two details make the screen hold. The fetcher follows redirects itself, outside the HTTP client’s redirect logic, and re-screens every hop, because a public URL that 302s to an internal address is the standard bypass. The socket also dials through a DNS wrapper that resolves once and connects to the screened address, so the address that was checked is the address that is used. Without that, a DNS rebinding attack can pass the check with a public record and connect with a private one.
the byte cap that broke a large video page
The first version capped the response at 512 KB as a defense against a hostile server that returns an endless body. The HTTP client’s maxContentLength option enforced it, and the parser scanned only the first 128 KB of whatever arrived. The cap option rejects an oversized response rather than truncating it.
Verified against a large video watch page: 1.33 MB, rejected outright, no preview. The og:image tag on that page sits around 691 KB in, so the 128 KB parse window would have missed it on its own.
Mainstream unfurlers read a bounded prefix instead. Slack reads about 32 KB, Facebook about 512 KB, Twitter about 1 MB, and a large page still previews for all of them. The fix streams the body and reads until </head> or a 768 KB ceiling, whichever comes first, then parses whatever arrived. The parse window equals the read ceiling so the parser never discards bytes the fetcher paid for, the content type is checked before the body is touched, and the stream is destroyed on every exit path. After the change: the video page reads about 700 KB and resolves a title and image, Vimeo 9 KB, GitHub 28 KB.
A DoS guard and a functional limit are different requirements. The first is “never buffer more than this”. The second is “read enough to find the tags”. Conflating them is what broke ordinary pages.
the parser
No HTML library. The service reads five well-known meta tags and never walks the DOM, so a bounded regex over the head is enough and adds no dependency with its own attack surface. Adding a scraping library was considered and rejected for exactly that reason.
bot mitigation, and why verification was declined
The company’s own sites answered the unfurler with a CDN challenge page. So did every user agent tried, including a borrowed Slack one and a Chrome string, because the CDN identifies verified bots by IP and reverse DNS and ignores the user agent. Real Slack unfurls those pages, so the metadata is there.
Applying for verified-bot status with the CDN vendor was considered and declined. The directory holds a few hundred operators. The large chat products and the unfurl vendors are on it. Several mid-sized chat products that ship link previews are not, and they ship them anyway. Becoming a registered crawler operator, with the robots.txt fetcher that obliges, was disproportionate for one attachment kind. If third-party coverage ever matters, buying an unfurl vendor that is already verified beats becoming one.
Fingerprint evasion (browser impersonation, headless browsers, residential proxies) was rejected outright. It forfeits the verification path permanently and is an arms race with a party that has more engineers on it.
The fix for the company’s own domains is a firewall skip rule for the service’s egress addresses, owned by whoever holds the CDN configuration.
hydration moved off the write path
The first design hydrated preview metadata from the cache at send time, on a 1.5-second budget, assuming the composer’s own preview call had warmed the cache for the same URL. That makes the persisted row’s correctness depend on cache retention, with four unretried soft failures: a draft older than the cache TTL, a draft composed on another device, a cache eviction, and the first sender of a URL nobody has previewed.
The plan of record moved the unfurl into the post-commit side effects with the full budget, updating the attachment row after commit, and dropped the title field from the client input so no client-supplied metadata exists at all. The cache went back to being a pure performance cache. Accepted cost: the card appears a beat after the message for recipients, which is how Slack behaves, and the sender’s optimistic bubble is unaffected.
That change also produced a latency finding elsewhere: the send path was hydrating previews before validating attachments, so a request with an unusable URL paid the full preview budget and then failed validation. A dashboard tile read that as a latency breach. Validate first, then do the expensive optional thing.