SecurityGENERALCLOUD-SPECIFICPROTOCOL-SPECIFIC
SSRF — When the Backend Fetches a URL
A fetch your server makes on a caller's behalf runs from inside your network with your identity. Blocklists lose; egress control holds.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.
The question
What happens when a user supplies a URL and the backend requests it?
The requirement
"Let customers import an avatar from a URL", or "let them configure a webhook endpoint we will call". Both are ordinary product asks, and both are the same feature.
The obvious build
Take the URL from the request, fetch it, store or forward the result. If we are careful we reject anything that does not start with http.
Why it breaks
The fetch does not run from the user's browser. It runs from inside your private network, past your firewall, from a host with an instance role and access to internal services that have no authentication because "they are internal".
How it breaks in production
- The fetch does not run from the user's browser. It runs from inside your private network, past your firewall, from a host with an instance role and access to internal services that have no authentication because "they are internal".
- The most valuable target is not a public site. It is the cloud provider's link-local metadata endpoint (
169.254.169.254and its equivalents), which hands out temporary credentials for the machine's role to anything that can make a plain HTTP request from that host. - The rest of the target set is the private address space: your admin panel on
localhost, a database HTTP interface, a service mesh sidecar, an internal API listening on a private subnet. - A blocklist of hostnames or address literals looks like the fix and fails to redirects, to DNS that resolves differently on the second lookup, and to the many textual forms one address can take.
Requirement→API Contract→Application Logic→Data Access→External Deps→Concurrency→Failure→Security→Observability→Deployment→Scale
What is actually happening
- The vulnerability is a confused deputy: your server is more privileged than the caller, and the caller decides where it points that privilege. Nothing needs to be returned to them for it to matter — a request with side effects is enough.
- A URL check happens against a string. The connection happens against an IP address obtained from DNS. Those are two separate operations at two separate times, and everything hostile lives in the gap.
- Redirects move the destination after the check. Your HTTP client follows a
302by default, and the second request is one your validation never saw. - DNS rebinding exploits the same gap without a redirect: a name that resolved to an allowed address at validation time resolves to a different one at connection time. This is a time-of-check/time-of-use race, not a parsing problem, so no amount of string filtering closes it.
- Addresses have many textual encodings, and hostnames can be made to resolve wherever their owner wants. Enumerating the bad forms is an open-ended problem; deciding on the resolved address is a closed one.
What the fetch inherits
The reason this is severe has nothing to do with URLs. It is that your process sits at a network position the caller cannot otherwise reach, and holds an identity the caller does not have. When it fetches on their behalf, both are lent out.
Draw it once and the priorities become obvious: the metadata endpoint is first because it converts a fetch into credentials, and the private subnet is second because that is where services trust the network instead of the caller.
UserLLMAgentToolDataDecisionHumanGuardrail
Why the blocklist loses
Each row below is a way the string you validated stops describing the connection you made. They are not exotic; they are the first four things anyone tries, and the response column shows that three of the four have the same fix.
The pattern to take away: validate the resolved address, then connect to that address. Any design where the destination is decided again, later, by a component that did not run your check, is a design with this bug in it.
Check-the-string failure modes
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Allowed host returns a redirect to an internal address | Internal content fetched; validation logs show an allowed host | The client followed a hop the check never saw | Disable automatic redirects; follow manually and re-validate every hop with a low limit |
| Hostname resolves to a public address, then to a private one | Nothing looks wrong until the response contains internal data | DNS rebinding — a TOCTOU race between validation and connect | Resolve once, validate the address, connect to that address with the original Host/SNI |
| Address written in an alternative textual form | A literal-string blocklist does not match | Many encodings map to one address; the blocklist enumerates forms, not destinations | Never match on the string; parse to an address and classify the address |
| IPv6 or IPv4-mapped IPv6 destination | IPv4 rules pass everything through | Half the address space was not considered | Classify both families, including mapped and translated forms, in one shared function |
| Non-HTTP scheme accepted by the client library | A file or gopher-style scheme reaches something unexpected | Scheme was never constrained | Allow http and https only, checked before anything else |
A fetcher that resolves, checks and pins
GENERALThe shape transfers to any language; the pinning mechanism is what differs. Where a client cannot be pinned to an address, you cannot close the rebinding race in the application — enforce it at the egress proxy instead.
The application-level control is small once the shape is clear: one function, used by every feature that fetches. What makes it correct is that the address used for the check and the address used for the socket are the same object.
Ship it as the only HTTP client available for user-influenced URLs, and make the raw client hard to reach. A rule that lives in a shared module is a rule; a rule that lives in a wiki page is a hope.
Resolve, classify, pin
1const ALLOWED_SCHEMES = new Set(['http:', 'https:'])2 3export async function fetchUserUrl(raw: string, hops = 0): Promise<Response> {4 if (hops > 2) throw new SsrfError('too_many_redirects')5 6 const url = new URL(raw) // throws on malformed input7 if (!ALLOWED_SCHEMES.has(url.protocol)) throw new SsrfError('scheme')8 9 // one resolution, and it is the one we connect to10 const { address, family } = await dns.lookup(url.hostname)11 if (!isPubliclyRoutable(address, family)) throw new SsrfError('destination')12 13 const res = await undiciRequest(url, {14 dispatcher: pinnedTo(address), // socket goes to the checked address15 headers: { host: url.host }, // original Host preserved16 maxRedirections: 0, // we follow them ourselves17 headersTimeout: 3_000,18 bodyTimeout: 5_000,19 })20 21 if (res.statusCode >= 300 && res.statusCode < 400) {22 const next = new URL(res.headers.location as string, url)23 return fetchUserUrl(next.toString(), hops + 1) // full check again24 }25 return capBody(res, 5 * 1024 * 1024) // bounded, and not echoed to the caller26}isPubliclyRoutable is the piece worth testing hardest: loopback, private ranges, link-local including the metadata address, unique-local and mapped IPv6, multicast and unspecified — for both families. Everything else in the function exists to make sure that decision is the one the socket obeys.
How to build it
Most important first.
- First: does the backend need to fetch it at all? An upload the user performs directly, or a presigned upload, removes the feature and the vulnerability together (Presigned URLs).
- Allow-list, never blocklist. If the URLs are known partners or a small set of hosts, permit exactly those and reject everything else. This is by far the strongest control and it is available more often than teams assume.
- When arbitrary public URLs really are the requirement: restrict the scheme to
http/https, resolve the hostname yourself, reject every resolved address that is private, loopback, link-local, multicast or otherwise not publicly routable — in both IPv4 and IPv6 — and then connect to the address you validated, carrying the originalHostheader. Pinning the connection to the checked address is what closes the rebinding gap. - Disable automatic redirect following, or follow manually and re-run the full check on every hop, with a low hop limit.
- Enforce it at the network as well: a dedicated egress proxy or an egress-restricted subnet means a bug in the application code does not become access to the internal network (Egress Security in Security Engineering). This is the layer that survives the next developer.
- Turn off the credential-serving metadata path where the platform allows it — require the session-token flow and set the hop limit to 1 — so a fetch from your host cannot reach it even if everything else fails.
- Bound the fetch like any other dependency: timeout, response size cap, and content-type check (Timeouts). And do not return the raw response body or upstream error to the caller — the response is the exfiltration channel.
What can go wrong
Failure modes
- Validation on the URL string, connection by hostname — the classic shape, safe against a screenshot and not against a resolver.
- A shared HTTP client with redirect following enabled by default, so the careful check in one module is bypassed in another.
- IPv4 handled and IPv6 forgotten, including IPv4-mapped IPv6 forms.
- The check applied to the user-facing import feature and not to the webhook sender, the link previewer, the PDF renderer fetching remote images, or the XML parser resolving external entities. Every one of those is the same class (Outbound Webhooks).
- Blind SSRF dismissed as harmless because no body is returned. Response time, status code and side effects all carry information, and a POST to an internal endpoint does not need a response to do damage.
- The egress proxy configured and then bypassed by a library that reads its own proxy settings, or not at all.
What can race
- DNS rebinding is the defining race here: the name resolves to an allowed address during validation and to a different one when the socket is opened. Pinning the connection to the resolved-and-checked address is the only application-level fix, because it removes the second lookup.
- Cached DNS entries expiring between validation and connection produce the same gap without anyone attacking — which is a good way to notice you have it.
Security
- What an attacker gets: cloud credentials from the metadata service, reachability of every internal service that trusts the network, and a way to make requests that appear to originate from your infrastructure.
- Internal services frequently have no authentication precisely because they are unreachable from outside. SSRF removes that premise, which is why it converts one application bug into network-wide access (The Trust Boundary).
- Outbound webhooks are user-supplied URLs by definition. Apply the same resolution and address checks there, and treat the customer-supplied endpoint as a destination you validate on every delivery, not once at configuration time.
- Attack technique, chaining and detection live in Security Engineering (SSRF — When the Backend Fetches a URL and SSRF Defense in Depth there). The implementer owns the resolve-and-pin client and the egress policy.
Misreads
- "We block
localhostand127.0.0.1." That is a blocklist of two strings against an address space, and it does not address redirects or rebinding at all. - "It is fine because we do not show the response." Blind SSRF still reaches internal endpoints, and timing and status differences leak information.
- "Our HTTP client validates URLs." It validates syntax. It has no opinion about where the name resolves.
- "The metadata endpoint is only reachable from the instance." That is exactly the property SSRF exploits — your instance is making the request.
- "We are not on a cloud provider, so there is no metadata service." The private network is still there, and so is everything listening on it without authentication.
Operating it
How you see it in production
- Log every outbound fetch with the requested host and the resolved address actually connected to. Those two fields differing from expectation is the signal, and nothing else gives it to you.
- Alert on outbound connections to private, loopback or link-local ranges from application hosts. In a healthy service this should be exactly zero.
- Count redirect hops and rejected-destination events by reason. A spike of rejections for "resolved to a private address" is someone testing.
- Egress-proxy denial logs are the highest-signal source available for this, because they see attempts the application never reports.
What changes at 10x and 100x
- More features fetch URLs as a product grows — link previews, imports, document rendering, integrations. The control has to be one shared, tested HTTP client rather than a rule each team remembers (Calling Something You Do Not Control).
- At 10x, fetching arbitrary user URLs from request handlers becomes a resource problem as well as a security one: slow remote hosts hold workers (Failure Propagation). Move it to a bounded worker pool.
- In multi-tenant systems, one tenant's hostile URL should not be able to affect another's imports — bulkhead the fetcher (Bulkheads).
What this costs
- Resolving and pinning breaks things that legitimately depend on hostname-based routing, SNI or CDNs with per-request DNS behaviour. You have to pass the original
Hostand SNI explicitly, and some clients make that awkward. - An allow-list is the strongest control and the most restrictive product decision. "Import from any URL" is a real feature; be clear that supporting it means owning the full checking stack.
- An egress proxy is another hop, another failure domain and another thing to run. It is also the only control that keeps working when someone writes a new fetch and forgets the rules.
Where this applies
Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.
- GENERALAny backend that fetches a caller-influenced URL has this, including on-premise deployments where the prize is the internal network rather than cloud credentials.
- CLOUD-SPECIFICThe metadata endpoint exists on every major provider at a link-local address, but the hardening differs: AWS IMDSv2 requires a PUT-obtained token and honours a hop limit, GCP requires a metadata header, Azure requires its own header and version. Enable the hardened mode explicitly — the legacy mode usually remains available unless you turn it off.
- PROTOCOL-SPECIFICThe same confused-deputy problem appears wherever a parser resolves a reference: XML external entities, remote images in a PDF or HTML renderer, and OpenID discovery documents. It is not limited to code that calls an HTTP client directly.
Where the depth lives
This domain teaches the application-side mechanism and hands the rest off.
API Designwebhook-security-contract