HTTPgetsafetycachingcrawlersreads

GET: The Promise of Safety

GET promises that reading changes nothing — a promise browsers, caches, crawlers and prefetchers spend billions of requests a day relying on. GET /deleteUser?id=42 is not a style violation; it is an open invitation to every robot on the internet.

Follow the failure

Frame the contract

API design starts with a consumer, a design question and a guarantee — never with a URL.

Design question
Can every GET in this API be issued by anything, any number of times, at any moment, without changing state anyone is accountable for?
Consumers
Clients reading data — plus the vast unregistered audience of GET: browser prefetchers, link-preview bots in chat apps, crawlers, CDN revalidations, monitoring probes and curious engineers pasting URLs, none of whom believe they are performing an action.
The promise
Any GET can be repeated, prefetched, cached and shared as a URL without side effects — making reads free to retry, cheap to serve, and safe to speculate on.
RequirementConsumersResource ModelStyleContractValidationAuthorizationErrorsIdempotencyPaginationVersioningObservabilityEvolutionTrade-offs

What safety buys, request by request

GET's safety promise is the most economically valuable clause in HTTP, because it is what makes reads *speculative*. A browser can prefetch the next page, a CDN can serve a million users from one origin fetch, a client can retry a timed-out read without a moment's thought, a chat app can unfurl a pasted link — all because GET guarantees nobody is accountable for a state change. Remove the guarantee from one endpoint and every one of those actors becomes a potential attacker of your data, with no malice required.

Safety also makes GET the cacheable method. Everything in Caching as a Contract Clause and Conditional Requests: ETags, 304 and 412 — CDN offload, 304 revalidation, browser caches — applies only to requests that promise not to act. And safety composes with idempotency to make GET the *freely retryable* method: a timeout on a GET has exactly one correct client response, "try again", which is why read paths tolerate flaky networks so much better than write paths.

The promise is per-endpoint, not per-API: one unsafe GET poisons the assumptions for everything that touches it. And it is a promise about accountability, not byte-identity — logging the access, bumping metrics, warming a cache are fine. The test: if this request fired a thousand times by robots overnight, is any *domain* state different in a way anyone must answer for?

  • Retry: a timed-out GET is retried without thought — the only method where failure handling is trivial.
  • Cache: browser, CDN and proxy caching all condition on GET's safety (see Caching as a Contract Clause).
  • Prefetch/speculate: browsers and apps fetch ahead of user intent — legal only because fetching is not acting.
  • Share: a GET URL pasted into chat gets fetched by preview bots instantly; the URL is the whole request, so it must be safe to utter.
  • Monitor: synthetic probes can exercise read paths continuously without corrupting data.

The canonical incident

The classic dates to 2006 but re-occurs annually under new names: an admin panel exposes deletion as links — GET /deleteUser?id=42 — because links are easy to render. A crawler (or a browser prefetcher, or a link-checking plugin, or a security scanner) discovers the page and does what GET-speaking robots do: follows every link. Row by row, politely, with proper user agents, the database empties. No auth bypass occurred if the admin was logged in — the robot rode the session cookie (which is also the mechanics of CSRF: browsers attach cookies to GETs they are induced to make — see Cookies and Their Attributes).

Every actor behaved correctly except the API. The crawler is *entitled* to GET anything linked; the prefetcher is *entitled* to fetch what the user might click; the cache is entitled to serve the "response" of a mutation without invoking it — so the deletion sometimes silently *doesn't happen*, which is the same bug inverted. Blaming the robot misses the design lesson: GET's contract is what makes robots useful; an unsafe GET is a landmine placed on a public footpath.

The mutation nobody meant to trigger
Request
GET /admin/deleteUser?id=42 HTTP/1.1
User-Agent: LinkPreviewBot/2.1 (+https://example-chat.app/bot)
Cookie: session=eyJhbGci…   # riding the admin's live session
Response
HTTP/1.1 200 OK
Content-Type: text/html

<html>User 42 deleted successfully.</html>

<!-- the "click" was a chat app unfurling a pasted link;
     the fix is not "block bots" — it is DELETE /users/42,
     which no preview bot will ever issue -->

The honest exceptions: reads that will not fit in a URL

GET's one structural weakness is that the request *is* the URL. Complex searches — a 40-clause filter tree, a vector similarity query, a report definition — blow past practical URL limits (~2KB in older proxies and some browsers) or demand encodings nobody can read or log safely. The pragmatic industry answer is a POST-shaped read: POST /search with the query in the body, explicitly documented as safe-in-effect despite the method (see Search Is a Different Contract Than Filtering).

Be honest about what that trade costs: HTTP-level caching is forfeited (intermediaries will not cache POST), retry middleware won't auto-resend, and the URL stops being shareable. Mitigate by keeping simple, common reads on GET (GET /products?category=shoes), reserving body-reads for genuinely complex queries, and — where result reuse matters — creating a named query resource (POST /searchesGET /searches/{id}/results), which restores cacheability and shareability at the cost of a two-step flow. What is *not* acceptable is the reverse smuggling: side effects hidden in GET because "it's just a read that also marks notifications seen". Split the read from the mark (see Resource or Action?); a read that acts is a lie in both directions.

A read that also acts: safe by method, unsafe by behavior
1GET /notifications
2200 OK [ …items… ]
3# side effect: marks everything as read
4
5# consequences:
6# - prefetch marks notifications read before the user saw them
7# - a retried timeout eats unread state
8# - the CDN serves a cached copy: now it randomly DOESN'T mark
Read and act, separated
1GET /notifications # safe: repeat, cache, prefetch freely
2200 OK [ …items… ]
3
4POST /notifications/read-receipts # the action, named
5{ "up_to": "ntf_8231" }
6204 No Content

The split costs one extra request and returns GET to full safety: prefetchers and caches can touch the list freely, and marking-read becomes deliberate, retryable-with-intent, and observable as its own operation.

Key points

  • GET's safety is what makes reads speculative: retryable, cacheable, prefetchable, shareable — the web's free performance and recovery layer.
  • Safety is about accountability for domain state, not byte-identical responses; logging and metrics do not violate it.
  • Unsafe GETs are triggered by crawlers, prefetchers and preview bots behaving correctly — and cached "mutations" sometimes silently do not happen.
  • A GET URL is the whole request; anything expressible as a link must be safe to utter anywhere.
  • Complex reads may honestly use POST bodies at a documented cost (no HTTP caching, no free retries); side effects hidden in GET are never honest.

Follow the failure

How the contract fails or gets misused, hop by hop — and what it costs when it completes.

  1. 1
    Team → admin panel: renders destructive operations as links because anchor tags are easier than forms.
  2. 2
    Admin → browser: keeps a logged-in session; a link-checker extension or chat unfurl fetches everything linkable.
  3. 3
    Robot → API: follows every GET /delete… link, riding the session cookie, politely and completely.
  4. 4
    Ops → database: rows vanish with clean 200s in the access log; the incident looks like a breach but greps as normal traffic.
  5. 5
    Team → fix: blocks the bot's user agent; the next prefetcher has a different one, and the real bug — a mutating GET — is still live.
What breaks
  • Data is destroyed or corrupted by well-behaved automation riding authenticated sessions — indistinguishable from legitimate traffic in logs.
  • Cached mutations intermittently do not execute, producing "sometimes it works" bugs that resist reproduction.
  • Once one GET is known unsafe, clients and infra teams disable caching and prefetching API-wide, taxing every legitimate read.

Design, observe, evolve

A contract decision is incomplete until you know how you would notice it failing and how it changes later.

Design the contract
  • • Enforce zero domain-state mutation in GET handlers — as a lint/review rule on handlers, not just a docs sentence.
  • • Express every state change as POST/PUT/DELETE/PATCH so browsers, bots and middleware cannot trigger it incidentally (and CSRF defenses have a method boundary to hold — see [[cookies]]).
  • • Split read-plus-act endpoints into a safe read and a named action (see [[resource-vs-action]]).
  • • For complex queries, choose deliberately: GET with bounded params, POST-read with documented cache forfeit, or a named query resource for reuse.
Observe in production
  • • Alert on write-path database activity originating from GET handlers — the violation is detectable at the query layer.
  • • Watch for mutations correlated with crawler/preview-bot user agents or prefetch headers (`Sec-Purpose: prefetch`) in access logs.
  • • A mutation rate that drops when a CDN is enabled means cached GETs were carrying writes.
Evolve without breaking
  • • Making an unsafe GET safe is a breaking change for clients that depended on the side effect — migrate by adding the explicit action first, then stripping the GET's effect after telemetry shows callers moved (see [[api-migration]]).
  • • Safe GETs evolve freely on the read side: new fields, better caching, conditional requests can all be added without consumer risk (see [[conditional-requests]]).
What it costs
  • • Strict GET safety sometimes costs a second request (read, then act) where one sneaky endpoint would do.
  • • POST-shaped reads forfeit HTTP caching and free retries — a real tax on genuinely complex queries; a named-query resource wins them back for two steps of ceremony.
  • • Purely observational side effects (metrics, access logs, cache warming) are allowed but need a stated policy, or reviewers will re-litigate them per endpoint.

Misconceptions

Claim
“Nobody will call an unlisted URL — our delete link is behind an admin login.”
Reality
Anything rendered as a link is one prefetcher, unfurl bot or browser extension away from being fetched with the admin's cookies attached. Auth restricts who; the method decides what *software acting for them* may do incidentally.
Claim
“GET with a request body is fine — some servers support it.”
Reality
Intermediaries may drop, ignore or reject GET bodies, and caches key on the URL alone, so two different queries can collide on one cache entry. If the query must live in a body, use POST and own the trade-offs explicitly.
Claim
“Marking notifications read in the GET saves a round trip, and it is harmless.”
Reality
It converts every cache, prefetcher and retry into an actor that either destroys unread state or silently fails to. The saved round trip costs the entire speculative-read machinery on that endpoint.