CDN Architecture

Light in fibre travels about 200,000 km/s. Frankfurt to Virginia is roughly 6,500 km of cable, so one round trip is at least 65 ms and in practice ~90 ms. A fresh HTTPS connection costs a TCP handshake plus a TLS handshake — two to three round trips before the first byte — so a 50 KB image from an origin across the Atlantic takes 300 ms before any server work. No amount of server optimisation changes this; only distance does.

A CDN is a fleet of caches at points of presence (PoPs) near users — hundreds of them, in the cities where ISPs interconnect. DNS or anycast routes each user to the nearest PoP; the PoP terminates TLS a few milliseconds away, and if it holds the response, the origin is never contacted. A user in Europe talks to a European edge at ~10 ms; only the first user to ask for an object pays the trip to the origin.

The edge stores responses under a cache key — by default the host plus path, often plus the query string. Everything that varies the response must be in the key, and nothing else. Include a query string that changes the content (?w=400 for an image size); exclude one that does not (?utm_source=…), or every marketing link is a separate miss. A Vary: Accept-Encoding header splits the key by encoding so gzip and brotli clients each get the right bytes; a Vary: Cookie effectively disables caching, because every user has a different cookie.

Hit ratio is the metric. 95% means the origin sees 1 in 20 requests; 99% means 1 in 100. A low ratio almost always means the key has too much in it (query strings, cookies, headers) or the TTL is too short for the traffic. Measure it per path: assets should be ~100%, HTML pages depend on how much is truly per-user.

With 200 PoPs, a newly published object misses 200 times — once per edge — and a popular purge sends 200 simultaneous requests to the origin. Origin shielding adds one intermediate cache layer: edges miss to a designated shield PoP near the origin, and only the shield talks to the origin. The origin now sees one request per object instead of 200, and a deploy or purge is a gentle ramp rather than a spike. Request collapsing at the edge does the same within one PoP: if 500 users ask for the same missing object in the same 50 ms, the edge sends one origin request and fans the response out — the CDN’s version of the single-flight from Caching Architecture.

Two ways for a change to reach the edges. Purge tells the CDN to drop a key (or a tag, or everything); propagation takes seconds to a minute, and until it lands some users get old bytes. Purge-everything on every deploy also resets the hit ratio to zero and hits the origin with the full miss storm. Versioned URLs avoid the problem: app.3f9a1c.js is a different key from app.7b2e40.js, so the new HTML references the new asset, the old one simply stops being requested, and both can be cached for a year with immutable. The rule: hash static assets and never purge them; purge only the small set of unversioned keys (HTML, JSON that must update), ideally by surrogate tag (product-42) so one product edit clears exactly the pages that embed it.

The last trap is the mixed deploy: new HTML served from a fresh edge references app.7b2e40.js, but the origin behind another edge still runs the old build and answers 404. Upload assets before switching HTML, and keep old asset versions available for at least the HTML cache TTL.

CDNs increasingly run code at the edge: A/B assignment, geo-redirects, auth token validation, personalising a cached page by injecting a small per-user fragment. The pattern is cache the expensive shared part, compute the cheap personal part at the edge. Edge functions have limits — milliseconds of CPU, no persistent connections to your database — so they suit routing and assembly, not business logic.

Some things should not go through a CDN. Anything per-user and sensitive without careful private headers, because one misconfigured key leaks accounts. Write endpoints, which gain nothing and add a hop. Tiny, frequently changing JSON where the TTL would be shorter than the purge propagation. Very large working sets with low reuse — a video library where each file is watched once — because the edge evicts before the second request and every hit is a miss with extra latency. A Bloom Filter of "seen at least once" is how some CDNs decide whether an object is worth storing at the edge at all.