CDN - Edge Caching, Anycast, Tiering and Invalidation
- A CDN is a geo-distributed, tiered HTTP cache and reverse proxy in front of your origin. Users talk to a nearby PoP (point of presence) instead of the far-away origin, and most requests never reach the origin at all.
- Two separate wins: shorter round trips (TCP and TLS finish at a PoP 5 to 20 ms away) and a shielded origin (hits are absorbed, misses are collapsed). Even uncacheable traffic gets the first win.
- The number that matters is the cache hit ratio. The levers are the cache key, the TTL, a shield tier and request collapsing.
- Users reach an edge by anycast (Cloudflare), DNS mapping (Akamai, CloudFront) or appliances inside ISPs steered by a control plane (Netflix). None of them is "nearest by geography"; all of them are "best by network topology and policy".
- The hard parts are invalidation (prefer versioned URLs, then purge by tag) and the miss path (a cold or just-purged key must not stampede the origin). The risks are cache poisoning and origin bypass.
Mental model
A chain of neighbourhood warehouses stocked from one factory. Most orders are filled from the corner warehouse, a few streets away. When the corner warehouse lacks an item, it asks a bigger regional warehouse, and only if that one is empty too does anyone call the factory, which then ships the item up the chain so the next customer is served locally.
Where it breaks: a warehouse never has to decide whether the stock on its shelf is still the current model. A CDN does, constantly, and "the factory changed the product at 2 pm" has no automatic way to reach every shelf. Also, an item made specifically for one customer is never worth stocking.
A CDN is a read-through cache with an L1 and an L2, like a CPU or a Redis in front of a database, but spread over the planet. The key is roughly (host, path, normalised query, Vary headers), the value is the response, and the TTL comes from the origin's headers. L1 is the edge PoP, L2 is the shield, and the database is your origin.
Where it breaks: a CPU cache is invalidated by the writer; a CDN is not, so correctness depends on TTLs, purge calls and URL design. And a CPU cache has one key space, while a CDN may hold several variants of one URL (compressed or not, per language) and a mistake in the key serves the wrong one.
How it works
Why distance dominates
Light in optical fibre travels at about two thirds of its vacuum speed, roughly 200,000 km/s. Take a user in Sydney and an origin in Virginia, near Ashburn:
| Step | Value |
|---|---|
| Great-circle distance (haversine, Ashburn to Sydney) | about 15,700 km |
| One-way time at 200,000 km/s | 15,700 / 200,000 = 78 ms |
| Round trip, floor | 157 ms |
That is the physics floor. Real cables do not follow the great circle, and every router adds queueing, so measured round trips are higher. Now count how many round trips a cold HTTPS request needs: one for the TCP handshake, one for the TLS 1.3 handshake (RFC 8446: a full handshake is one round trip), and one for the request and response. That is 3 round trips before the first byte, so a floor of 3 x 157 = 471 ms to the Virginia origin.
Terminate those handshakes at a PoP 10 ms away and the same three round trips cost 3 x 10 = 30 ms. The CDN has not made the cable faster; it has moved the end of the conversation closer. See TCP connection and TLS for the handshakes.
The request path
The origin is your own server fleet or object store. The edge PoP is a cluster of cache servers in a metro area, usually behind its own load balancer. The shield (also called a tiered cache or origin shield) is a smaller number of designated PoPs that every other PoP asks before it asks the origin. A hit at either cache ends the trip there.
Hit, miss and revalidation
The Age header tells you how many seconds a response has sat in caches (RFC 9111, section 5.1). A fresh miss shows Age: 0 or no Age at all; a hit shows how old the copy is. When a copy goes stale, a cache may send a conditional request using the stored ETag or Last-Modified, and a 304 revalidates the copy without moving the body. The header semantics (max-age, no-cache, ETag, Vary) are owned by HTTP caching; this page is about what the CDN does with them.
Hit-ratio math, worked
One object is requested 10,000 times an hour at one PoP, with a TTL of 60 seconds.
| Quantity | Derivation | Result |
|---|---|---|
| Expiries per hour | 3,600 s / 60 s | 60 |
| Origin requests per hour | one per expiry | 60 |
| Request hit ratio if each expiry is a full refetch | (10,000 - 60) / 10,000 | 99.4% |
| Origin bytes per hour, full refetch | 60 x object size | 0.6% of what the users received |
| Origin bytes per hour, conditional GET answered 304 | 60 x (headers only, a few hundred bytes) | near zero |
So the origin sees 60 requests either way. What changes is the cost of each: a 200 carries the whole body, a 304 carries none. A 304 is a cheap revalidation, not a miss in the cost sense, even though some CDNs log it as a separate status.
Blended latency follows from the hit ratio h: average = h x hit_time + (1 - h) x miss_time. With a 10 ms hit, and a miss that costs the hit plus the 157 ms origin round trip floor, the average is 10 + 0.006 x 157 = about 10.9 ms. The 0.6% of slow requests barely moves the mean. They do dominate the tail: the p99.9 user is a miss. If the CDN supports stale-while-revalidate, even those users get the stale copy instantly while the refresh happens in the background.
Try the edge
Send requests from one of four PoPs to the Virginia origin and watch each resolve as HIT, MISS, REVALIDATED or STALE with its latency split into user-to-edge and edge-to-origin (Sydney shows the 157 ms floor from above). Move the TTL, purge or soft purge the copy, edit the object at the origin, or deploy a hashed filename and see why a new key beats a purge. The second tab runs the thundering herd: many requests for one cold key, with request collapsing and the shield tier switched on and off.
| # | from | result | latency | what happened |
|---|
Deep dive
Routing users to an edge
How does a request for cdn.example.com land on a PoP near the user? There are three production answers.
- Anycast. Every PoP announces the same IP prefix over BGP, and the internet's routers pick one path to it. Cloudflare describes running its whole network this way (A brief primer on anycast). The catch is that BGP chooses by policy and AS-path length, not by latency, so the "nearest" PoP in routing terms can be far in kilometres, and a route change can move a user to another PoP mid-connection. The mechanism and its caveats are owned by Global Traffic Steering and BGP.
- DNS mapping. The CDN's authoritative name server answers each query with the IP of a PoP chosen for the asker. Akamai's mapping system works this way (Nygren et al., "The Akamai Network") and CloudFront documents DNS steering to an edge location (How CloudFront delivers content). The server sees the resolver's address, not the user's, so a user on a distant public resolver can be mapped to the wrong region. EDNS Client Subnet (RFC 7871) lets the resolver pass a truncated client prefix to fix that, at a privacy cost. See DNS and DNS records and routing for the resolver side, and Global Traffic Steering for GeoDNS limits.
- Appliances inside the ISP. Netflix places its own cache servers (Open Connect Appliances) in ISP networks and in internet exchanges, and its control plane tells each player which appliance URL to fetch from (Open Connect overview). No generic routing trick is needed because the cache is already one hop from the viewer.
| Provider | How a user is steered to an edge | Source |
|---|---|---|
| Cloudflare | Anycast: all PoPs announce the same prefixes; BGP picks the PoP | Cloudflare anycast primer |
| Akamai | DNS-based mapping system that answers with a chosen edge server | Nygren et al. 2010 |
| CloudFront | DNS answers steer the client to a nearby edge location | AWS CloudFront developer guide |
| Netflix Open Connect | Control plane returns URLs of appliances inside or beside the user's ISP | Open Connect overview |
The cache key
The cache key decides whether two requests are "the same object". The default is host plus path plus query string, and Vary request headers add to it (RFC 9111, section 4.1). Mistakes in the key are the largest source of both low hit ratios and wrong-content bugs:
- Key too specific: tracking parameters such as
?utm_source=...or a per-request?cb=12345make every request unique, so none is ever a hit. Normalise at the edge: drop parameters the origin ignores, sort the rest, lowercase the host. Cloudflare and CloudFront both let you configure what goes into the key (Cloudflare cache keys, CloudFront cache key). - Key too broad: if the response differs by something that is not in the key (a header, a cookie, a device class), one user's variant is served to another. That is both a correctness bug and the root of cache poisoning, below.
Varyfragments the cache.Vary: Accept-Encodingis fine because a handful of values exist and CDNs normalise them.Vary: CookieorVary: User-Agentmultiplies variants by the number of distinct values, which collapses the hit ratio towards zero. Fix it at the origin: do not vary on those headers for cacheable responses, or cache a shared shell and fetch the personal part separately.
Tiered cache, origin shield and request collapsing
Two mechanisms keep the origin calm. Request collapsing (also called coalescing) means that when many requests for the same uncached key arrive together, the cache sends one request to the next tier and holds the rest until it returns. Tiering applies the same idea across PoPs.
Worked numbers. A key that was just purged is requested at 500 requests per second, and the origin takes 200 ms to answer. During those 200 ms, 500 x 0.2 = 100 requests arrive for the key. Without collapsing all 100 go to the origin. With collapsing, 1 goes and 99 wait about 200 ms for its answer. Now multiply by the fleet: with 100 PoPs each collapsing locally, the origin still sees up to 100 fetches per expiry, one per PoP. With a shield, the 100 PoPs ask the shield and the shield collapses them, so the origin sees 1.
A shield also fixes cache fragmentation. If an object is requested 100 times an hour worldwide, spread evenly across 100 PoPs, each PoP sees one request an hour. With a 60-second TTL almost every one of those is a miss, because the next request arrives long after expiry. The shield sees all 100, one every 36 seconds on average, which is inside the TTL, so most of them are hits. More PoPs without a shield tier lowers the hit ratio for long-tail content. The product docs describe the same design: Cloudflare Tiered Cache and CloudFront Origin Shield.
Two caveats on collapsing. First, it only helps if the response is cacheable: if the origin returns Cache-Control: no-store or a response the CDN will not store, the waiting requests cannot reuse it, and vendors differ on whether they then queue everyone behind one request or let them all through, so check your vendor's behaviour. Second, a slow origin makes the waiters slow too; set a timeout so one hung fetch does not park every request for that key.
RFC 5861 adds stale-while-revalidate and stale-if-error. The first serves the stale copy while one background request refreshes it, which removes the latency spike at every expiry. The second serves stale content when the origin errors, which turns an origin outage into a stale-content period. Both are defined in HTTP caching; CDN support and exact behaviour vary, so test yours.
Invalidation: TTL, purge, versioned URLs and surrogate keys
You have four tools, and they trade speed against cost and reliability.
| Method | How fast users see the change | Cost | Reliability |
|---|---|---|---|
| Wait for the TTL | up to the full TTL | none | deterministic, but stale until expiry |
| Versioned URL (content hash in the name) | immediate: a new name is a new key everywhere | build step; the HTML that references it needs a short TTL | highest: nothing to propagate |
| Purge by URL | seconds to minutes depending on the vendor | API call, rate limits, a cold PoP afterwards | not atomic: PoPs purge at slightly different moments |
| Purge by tag (surrogate key) | same as URL purge, for a whole group at once | origin must tag responses | same; a missed tag leaves stale pages |
| Soft purge | copy is marked stale and refreshed on next request | like a purge, without the cold start | needs stale-while-revalidate style serving |
| Purge everything | same as purge | cache goes cold, every key misses together | reliable but triggers the stampede above |
- Versioned URLs are the default answer. A build step puts a content hash in the file name (
app.9f3c1.js), the file gets a year-long TTL andimmutable, and the HTML that points to it is short-lived. A new deploy produces a new name, so there is nothing to invalidate. See HTTP caching for the headers. - Surrogate keys group pages. Fastly's
Surrogate-Keyand Cloudflare'sCache-Tagresponse headers let the origin label each page ("product-42", "category-shoes"), and one purge call by tag invalidates every page that mentions it (Fastly surrogate keys, Cloudflare purge by tag). This is how an e-commerce site updates a price without purging the whole site. - Purge speed is vendor-specific. Fastly advertises purges that complete in roughly 150 ms; treat that as a vendor claim and measure your own (Fastly: Instant Purge under 150ms). Others take longer, so design so that a purge delay is harmless.
- A CDN purge does not touch browsers. A browser that cached the response with
max-age=86400keeps it until it expires regardless of what you purged. That is another reason HTML should be short-lived. - Separate CDN and browser TTLs.
s-maxageoverridesmax-agefor shared caches (RFC 9111, section 5.2.2.10), soCache-Control: max-age=60, s-maxage=3600means "browsers keep it a minute, the CDN an hour". RFC 9213 standardises a targeted header (CDN-Cache-Control) for the same purpose; support varies by CDN.
What to cache
| Content | CDN policy | Why |
|---|---|---|
| Fingerprinted static asset (app.9f3c1.js) | cache for a year, immutable | the name changes when the bytes change |
| Unversioned static asset (logo.png) | cache with a moderate TTL, purge when it changes | same bytes for everyone, but the name does not change |
| Public API GET (product list) | short TTL plus stale-while-revalidate | freshness of a few seconds is acceptable; see micro-caching below |
| Anonymous HTML page | short s-maxage, purge by tag on edit | same for everyone, but changes by editorial action |
| Personalised or authenticated response | do not cache at a shared cache; split it | serving one user's page to another is a leak |
| Response with Set-Cookie | usually not cached by default; check your CDN | the cookie would be handed to the next user |
Micro-caching is the answer to "but my page is dynamic". If one URL gets 1,000 requests per second at a PoP and you cache it for 1 second, the PoP asks the origin about once per second instead of 1,000 times, a 99.9% reduction in requests, and no user sees data older than a second. It works for any response that is the same for everyone and tolerates a few seconds of staleness: search results, leaderboards, product lists.
For personalised pages, split static from dynamic. Cache the page shell at the edge and load the personal fragment from an API (which is not shared-cached), or assemble it at the edge with Edge Side Includes or edge compute. Do not "fix" it by adding Vary: Cookie: that gives every user their own copy and a near-zero hit ratio.
Push versus pull
A pull CDN fetches from your origin on a miss and caches the result. It is the default for Cloudflare, CloudFront and Fastly: you change nothing except DNS and headers, and unpopular objects cost nothing to leave out. A push CDN has you upload content to the CDN's storage ahead of time. It suits large, rarely changing, predictable files (software releases, a video library) where the first user should not pay for a cold miss, and where you do not want the origin on the path at all. The costs are storage you pay for and upload and invalidation logic you own. Netflix runs a push-like model for its appliances: it describes filling them proactively, at off-peak times, with content predicted to be popular (Open Connect overview). For video-specific delivery see Video streaming.
Dynamic acceleration
A CDN helps even when nothing can be cached. The PoP keeps warm, reused connections to your origin, so the user's request does not pay a fresh TCP and TLS handshake across the ocean. Using the earlier numbers (Sydney user, 10 ms to the PoP, 157 ms floor to Virginia): the user's three handshake-and-request round trips cost 3 x 10 = 30 ms, and the PoP-to-origin leg adds one 157 ms round trip over a warm connection, for a floor of about 187 ms. Without the CDN it is the 471 ms derived above. The origin still does the work, but the setup cost is gone. The PoP can also use a better path to the origin than the public internet default; vendors market this as dynamic acceleration, and you should benchmark the claim for your own routes.
TLS at the edge
The edge terminates the user's TLS session with a certificate for your domain (one you upload, or one the CDN issues for you), reads the HTTP request, and then opens a second TLS session to the origin. Two details matter. First, the traffic between PoP and origin is a separate hop that you must configure: some CDNs offer a mode that sends plaintext to the origin, which means the last leg is unencrypted even though the user sees a padlock. Use the strict mode that validates the origin's certificate (Cloudflare SSL modes). Second, the CDN is now a party that sees your plaintext traffic, which is a trust decision. See TLS and QUIC and HTTP/3.
Edge compute
Modern CDNs run your code at the PoP: Cloudflare Workers, Lambda@Edge and CloudFront Functions, and Fastly Compute (Workers, Lambda@Edge, Fastly Compute). Typical jobs are rewriting URLs, normalising the cache key, A/B bucketing, auth checks and assembling a cached shell with personal data. The constraint is the runtime: short CPU time, limited memory, and no local state, so anything that needs a database round trip loses the latency advantage.
Egress cost
Bandwidth out of your origin is usually billed per GB by the cloud provider, and a CDN reduces it in proportion to the byte hit ratio. If users download 1 TB and the byte hit ratio is 95%, the origin serves 5% of 1 TB = 50 GB. You then pay the CDN's own per-GB rate for the full 1 TB, so the saving is real only when the CDN's rate plus the reduced origin egress beats the origin's egress rate on all of it. Some clouds waive the charge for transfer from their own origins to their own CDN; AWS documents this for CloudFront (CloudFront pricing). Check the current pricing before relying on the arithmetic.
How a CDN gets abused
- Cache poisoning. If the response depends on a request header that is not in the cache key (the classic case: a header such as
X-Forwarded-Hostreflected into a page), an attacker sends one crafted request, the CDN stores the poisoned response under the normal key, and every later user gets it. James Kettle's 2018 research catalogued many such unkeyed inputs (Practical Web Cache Poisoning). Mitigate by not reflecting unkeyed headers, adding them to the key or toVary, and caching only what must be cached. - Web cache deception. An attacker lures a logged-in victim to
/account/profile/x.css; a CDN that caches by file extension stores the victim's private page under that URL and the attacker fetches it (Omer Gil, 2017). Mitigate by caching on the origin'sCache-Controland content type, not on the extension. - Origin bypass. If the origin's real IP address leaks (old DNS records, certificate transparency logs, email headers), attackers hit it directly and skip the CDN's protection. Accept traffic only from the CDN, for example with allow-listed CDN ranges or authenticated origin pulls (Cloudflare authenticated origin pull).
- Absorbing floods. Because anycast spreads a flood across many PoPs and each PoP serves hits locally, a CDN soaks up volumetric and request floods that would flatten one origin. See Network attacks for the attack classes.
Reading cache headers
Every CDN reports what happened in a response header. Names differ; the vocabulary is similar. Cloudflare documents its statuses in Cloudflare cache responses. RFC 9211 defines a standard Cache-Status header, but most vendors still use their own.
| Status | Meaning | Where you see it |
|---|---|---|
| HIT | served from cache, fresh | CF-Cache-Status, X-Cache (Fastly, CloudFront as 'Hit from cloudfront') |
| MISS | not in cache; fetched from the next tier or origin | same headers |
| EXPIRED | was cached but stale; refetched or revalidated | CF-Cache-Status |
| STALE | served a stale copy (stale-while-revalidate or origin error) | CF-Cache-Status |
| BYPASS | the cache was deliberately skipped (rule, header or request method) | CF-Cache-Status |
| DYNAMIC | not eligible for caching by default, so it went to the origin | CF-Cache-Status |
Also read Age (seconds in cache), Via (the proxies it passed through), and Cache-Control as the CDN saw it. A HIT with a small Age after a purge means the purge worked; a MISS that never turns into HIT means something in the key or the headers prevents storage.
Textbook vs real world
| The textbook says | Production does | Why |
|---|---|---|
| The TTL decides freshness | TTL plus explicit purge by tag plus stale-while-revalidate | a TTL alone cannot take down a page now, and a purge alone cannot avoid the stampede after it |
| Cache static content only | Micro-cache dynamic responses for 1 to 5 seconds and split static from dynamic in personalised pages | a one-second TTL under load removes most of the origin traffic with no visible staleness |
| The user goes to the nearest PoP by geography | The user goes to the PoP BGP or the DNS mapping system picks, which is often but not always the nearest | routing follows policy and topology, not kilometres; resolvers hide the user's real location |
| The origin is safe behind the CDN | Origin IPs leak and attackers hit them directly, so origins accept only CDN traffic | the CDN protects only the traffic that goes through it |
| More PoPs is always faster | More PoPs fragments the cache, so CDNs add a shield tier | each PoP sees a fraction of the requests, so long-tail objects miss at every one |
| A CDN is a cache | A CDN is a reverse proxy, TLS terminator, DDoS absorber, WAF and compute platform that also caches | the cache is the feature, but most of the value for dynamic sites is connection handling and protection |
In production
- Cloudflare serves traffic from anycast addresses announced by every PoP (anycast primer). It publishes the current city and network count on its network page; read the live figure there rather than trusting a number frozen on a course page. It documents Tiered Cache and the
CF-Cache-Statusvalues. - Akamai is the oldest of the large CDNs and describes a DNS-based mapping system that steers users to edge servers (Nygren et al., 2010). The 1997 consistent-hashing paper behind it is covered in Consistent hashing, which is also how a PoP decides which server in a cluster holds a given object.
- Fastly is built on the Varnish cache and sells instant purge and surrogate keys (Fastly: Instant Purge under 150ms). It advertises purge times of about 150 ms; that is a vendor figure.
- Amazon CloudFront offers a regional Origin Shield and cache policies, and runs edge code with Lambda@Edge and CloudFront Functions.
- Netflix Open Connect places its own cache appliances inside ISP networks and at internet exchanges, and pre-fills them with popular titles (Open Connect). Netflix describes it as the way it delivers its video. Google offers a similar in-ISP cache, Google Global Cache (Introduction to GGC).
- For video segment delivery specifically (ABR, chunk caching), see Video streaming; for how a server-side request path behaves behind the CDN, see Load balancing and Proxies and gateways.
History and evolution
- 1990s Popular web sites suffer flash crowds: one origin cannot serve a sudden surge. Caching proxies and cache hierarchies appear, and the Internet Cache Protocol (RFC 2186, 1997) lets caches ask their neighbours before the origin.
- 1997 Karger and colleagues at MIT publish consistent hashing, motivated by hot spots in web caching (STOC 1997). What broke: adding or removing a cache moved most objects. Akamai is founded in 1998 by two of the authors (Leighton and Lewin) and builds a commercial CDN on the research.
- 2010 Cloudflare launches around anycast and DDoS protection. The same year RFC 5861 standardises
stale-while-revalidateandstale-if-error, and Nygren et al. describe the Akamai platform. What broke: single-origin sites were exposed to attacks and expiry spikes. - 2011 Fastly launches on the Varnish cache with instant purge and surrogate keys. What broke: purging used to take minutes, which pushed sites to long TTLs and stale content.
- 2011 Netflix starts Open Connect, placing appliances in ISPs. What broke: video was a large share of peak-hour traffic, and carrying it over transit links was costly for both Netflix and the ISPs.
- 2017 Edge compute arrives: Cloudflare Workers and Lambda@Edge put user code at the PoP, so the CDN is no longer only a cache.
- 2018 James Kettle's research shows cache poisoning through unkeyed headers is practical at scale (PortSwigger), and CDNs tighten key handling.
- 2022 RFC 9111 replaces the old HTTP caching spec, and RFC 9211 and RFC 9213 add a standard cache-status header and CDN-targeted cache control.
- Now A CDN is a security, compute and delivery platform in front of most large sites; Cloudflare Radar publishes live traffic shares at radar.cloudflare.com.
- Next More logic at the edge (compute, data, AI inference), HTTP/3 and QUIC to the user, and regional caching to meet data-residency rules.
Hands-on
These need only curl, and dig for the DNS part. Output below is abbreviated and from a real run; your values will differ.
See cache headers. Request a real CDN-served page and look at the status headers:
Two cache nodes handled the request, which is a tiered cache visible from outside. x-cache: HIT, MISS pairs with x-served-by: one tier had it cached (18 hits so far) and the other had to ask. Reading the first entry as the tier nearer the origin and the last as the edge fits a client in the south-eastern US being served from Dallas, but that mapping is my inference from the data, not something these headers state. Repeat the command and the counters change.
Which PoP am I talking to? Cloudflare exposes a trace endpoint:
colo is the airport code of the Cloudflare PoP that answered. Run it from another network (a phone hotspot, a VPN, a cloud VM) and colo changes: that is anycast steering you cannot see in DNS.
Same name, different resolver:
Both resolvers returned the same two addresses in my run. That is the signature of anycast: the IP does not depend on where you are; the routing does. A DNS-mapped CDN would often return different addresses to different resolvers. Answers can change over time, so do not rely on the exact values.
Force a revalidation and a bypass. Compare a normal request with one that asks caches not to answer from their copy:
Many CDNs ignore a client's Cache-Control request header for shared-cache decisions, so do not expect this to purge anything; it is a useful experiment precisely because it shows that clients cannot control the edge. macOS and Linux behave the same here. On a path that crosses the CDN, traceroute -n <anycast-ip> (macOS) or mtr -n (Linux) shows the few hops to the PoP and the low round trip to it.
Gotchas
Interview questions
Show answer
A CDN keeps copies of your content on servers in many cities, so a user fetches from a PoP a few milliseconds away instead of from the origin across the planet. It cuts latency because a request costs round trips, and round trips scale with distance: a cold HTTPS request needs about three, so ending them at a nearby PoP is much cheaper. The origin also sees far fewer requests because hits never reach it. The TTL from the origin's Cache-Control decides how long a copy stays fresh.
Follow-ups:
- What is a PoP? A cluster of cache and proxy servers in one metro area, the CDN's unit of presence.
- What is the origin? Your own server or storage that holds the authoritative copy.
- What is a TTL? How long a cached copy may be served without checking the origin again.
Show answer
There are three common mechanisms. Anycast: every PoP announces the same IP prefix and BGP picks the path (Cloudflare). DNS mapping: the CDN's authoritative DNS returns the IP of a chosen PoP based on where the resolver, or the client via EDNS Client Subnet, appears to be (Akamai, CloudFront). In-ISP appliances steered by a control plane (Netflix). None is purely "nearest by distance": BGP follows policy and path length, DNS sees the resolver rather than the user.
Follow-ups:
- Why is anycast not always nearest? BGP does not measure latency; policy and AS-path length decide, so a user can be routed to a farther PoP.
- What does ECS fix? It passes a truncated client subnet so the CDN can map by the user's location rather than the resolver's, at a privacy cost.
- What breaks a TCP connection on anycast? A route change mid-connection can send packets to another PoP that has no state; see Global Traffic Steering.
Show answer
Users are steered to a PoP (anycast or DNS). Inside the PoP a load balancer spreads requests over cache servers, with the object placed on a server by consistent hashing so adding a server does not reshuffle everything. On a miss the PoP asks a shield tier, which collapses duplicate requests, and only then the origin, then stores the response by its cache key and TTL. A control plane distributes configuration and purge messages to every PoP, and the data plane reports hits and misses.
Follow-ups:
- Why consistent hashing in the PoP? A cluster's object-to-server map must survive adding or removing a server; see Consistent hashing.
- How does purge fan out? A control-plane message to every PoP; propagation is not atomic, so design for brief inconsistency.
- Where do you cache? At the edge for hits and at a shield to fix fragmentation and to protect the origin.
Show answer
Use versioned URLs (a content hash in the file name) for assets: nothing to purge, nothing stale. Use TTL for content that can be stale for a bounded time. Use purge by URL or by tag (surrogate keys) for content that must change now, such as a price or a takedown, and prefer a soft purge with stale-while-revalidate to avoid a stampede. Purging everything is a last resort because it makes every key miss at once.
Follow-ups:
- Versioned URL versus purge? Versioned URLs are immediate and need no propagation; purges are for content whose URL cannot change.
- What are surrogate keys? Labels the origin puts on responses so one purge call invalidates a group; Fastly
Surrogate-Key, CloudflareCache-Tag. - Does a purge clear browsers? No; browsers keep their own copy until its
max-ageexpires.
Show answer
The cache key is what makes two requests "the same object": by default host, path and query string, plus any request headers named in Vary. Too specific, and tracking parameters or cache-busting values make every request unique so nothing is ever a hit. Too broad, and a variant that depends on something outside the key (a header, a cookie) is served to the wrong user, which is also how cache poisoning works. Normalise the key at the edge and keep personalisation out of it.
Follow-ups:
- Is
Vary: Accept-Encodinga problem? No; it has few values and CDNs normalise it. - What about
Vary: Cookie? It gives each cookie value its own copy and wrecks the hit ratio. - How do you normalise the query? Drop parameters the origin ignores, sort the rest, lowercase the host.
Show answer
That is a thundering herd: every request for the cold key misses at the same moment and all go to the origin. Fix it with request collapsing so concurrent misses for a key become one origin fetch, a shield tier so PoPs share that one fetch, and stale-while-revalidate or a soft purge so users are served the old copy while one request refreshes it. For example, at 500 requests per second and a 200 ms origin response, about 100 requests pile up per fetch window, and with collapsing the origin sees one.
Follow-ups:
- Does collapsing always help? Only for cacheable responses; with
no-storethe waiters cannot reuse the answer and behaviour varies by vendor. - Why does a shield matter here? Each PoP collapses locally, so without a shield the origin still sees one fetch per PoP.
- What is stale-if-error? Serve stale content when the origin errors; see RFC 5861.
Show answer
First measure by bytes and by request, and find which keys miss. Then normalise the cache key (strip tracking parameters, sort the query, fix Vary), add a shield tier so long-tail objects pool their requests, lengthen TTLs where staleness is acceptable and use versioned URLs to make long TTLs safe, and stop sending Set-Cookie or no-cache on responses that are the same for everyone. Micro-cache hot dynamic endpoints for a second or two.
Follow-ups:
- Why does a shield raise the ratio? It sees all PoPs' requests, so inter-arrival times drop below the TTL.
- Why can more PoPs lower it? Each PoP sees fewer requests for the same object, so fewer arrive within the TTL.
- What about the personalised pages? Split static from dynamic instead of varying on cookies.
Show answer
Do not cache the personalised response at a shared cache. Split the page: cache the shell (same for everyone) at the edge and load the personal parts from an API that is not shared-cached, or assemble at the edge with ESI or edge compute. If part of the personal data is really segment-level (country, plan tier), put the segment in the cache key so a few variants share a copy. Never add Vary: Cookie as a shortcut: it gives each user their own copy and no hit ratio, and a misconfiguration leaks one user's page to another.
Follow-ups:
- What is micro-caching for? Sheltering a hot dynamic endpoint with a 1 to 5 second TTL when every user may see the same data.
- What does edge compute add? It can compute the variant at the PoP without a trip to the origin.
- What is the failure mode? Caching a page that includes a user's data, which then serves it to other users.
Show answer
Pull fetches from the origin on a miss and caches it: nothing to manage, ideal for most websites and APIs, but the first user per PoP pays a miss. Push has you upload content to the CDN ahead of time: good for large, predictable, rarely changing files such as releases or a video library, where you want no cold misses and no origin on the path, at the price of storage and your own upload and invalidation logic. Cost differs too: pull pays origin egress on misses, push pays storage and upload traffic.
Follow-ups:
- Which is cheaper? Depends on the miss rate and file size; compute origin egress at the byte hit ratio and compare with storage cost.
- Does Netflix use push? It describes pre-filling its appliances with predicted content at off-peak hours, which is push-like.
- What about invalidation? Push moves that job to you; pull can use TTLs and purges.
Show answer
Three classes. Cache poisoning: a request carrying an unkeyed input, such as X-Forwarded-Host, makes the origin generate a harmful response that the CDN then stores under the normal key and serves to everyone; defend by not reflecting unkeyed inputs, adding them to the key or Vary, and caching less. Web cache deception: a path like /account/x.css tricks a cache that keys on file extension into storing a private page; defend by caching on Cache-Control and content type. Origin bypass: attackers find the origin's real IP and skip the CDN; defend by only accepting traffic from the CDN.
Follow-ups:
- Who found cache poisoning at scale? James Kettle, PortSwigger, 2018.
- How do you hide the origin? Allow-list CDN ranges or use authenticated origin pulls, and rotate an exposed IP.
- Does the CDN also protect against floods? Yes, anycast spreads traffic over many PoPs; see Network attacks.
Show answer
Many CDNs cache by default only what looks static, by file extension, and leave HTML alone; Cloudflare, for example, does not cache HTML by default. HTML is also usually dynamic and often carries Set-Cookie or no-cache, so it goes to the origin on every request while images, CSS and JavaScript are hits. To shelter the origin, give assets hashed names and long TTLs, then give HTML an explicit short s-maxage (or a cache rule) and purge it by tag when content changes. Check the hit ratio by request and by bytes before and after.
Follow-ups:
- Why is the HTML the hard part? It is the one document whose URL cannot change, so it needs a short TTL or a purge; see HTTP caching.
- What is the risk of turning on HTML caching? Caching a response that contains one user's data and serving it to others; see the personalised-page answer above.
- What if the page really is per-user? Cache the shell and fetch the personal part from an uncached API.
Show answer
Two things. Cache hits are answered at the edge and never reach the origin, so origin logs and metrics now count only misses, revalidations and uncacheable requests; traffic analytics must come from the CDN's own logs. And the origin's TCP peer is now a PoP or shield address, so the real client IP arrives in a header such as X-Forwarded-For or a vendor header. Trust that header only when the connection comes from the CDN's address ranges, because any client can send a forged one.
Follow-ups:
- Why must you check the peer address first? Otherwise anyone who reaches the origin directly can set the header and spoof an IP; see Load balancing for the same rule at a balancer.
- How do you see hits and misses per request? Read the cache-status response header and the CDN's request logs.
- What else does the CDN hide? Which PoP and tier answered; the status header is how you find out.
Show answer
Netflix runs its own appliances (Open Connect Appliances) inside ISPs and at exchanges. A steering service in its control plane uses what the appliances report (health, the BGP routes they learn, which content they hold) and returns the player a ranked list of appliance URLs for the title. The reported selection order prefers the longest matching prefix, then shorter AS-path, then MED, then geographic proximity. Because the control plane already knows which appliance is healthy and holds the content, the choice does not depend on where DNS or BGP happens to send the request, and popular titles are placed on appliances in advance rather than pulled on a miss.
Follow-ups:
- Why steer with URLs from an API instead of caching HTML at a CDN? The API can return per-client, per-title answers that account for appliance health and content placement, which a cache of one shared page cannot.
- What does the client get back? A ranked list of URLs rather than a single address.
- Is this push or pull? Closer to push: appliances are filled with predicted content in advance.
See Global Traffic Steering for anycast and DNS steering, and Video streaming for segment delivery.
Fun facts
- Light in fibre is about a third slower than in vacuum, so Virginia to Sydney has a round-trip floor of roughly 157 ms before a single router adds delay.
- The same URL can carry several cached variants at once (compressed or not, per language), each under a different internal key.
- Netflix builds its own cache appliances and hands them to ISPs so that video never has to cross a transit link (Open Connect).
- The consistent-hashing paper behind the first commercial CDN was about web-cache hot spots; a PoP still uses the same idea to decide which server in a cluster holds an object.
- Fastly's
x-served-byheader can list two cache nodes for one request, which is a tiered cache showing through.
Cheat sheet
| Number | What it is |
|---|---|
| 157 ms | round-trip floor, Virginia to Sydney (about 15,700 km, 200,000 km/s in fibre) |
| 3 round trips | cold HTTPS to first byte: TCP, TLS 1.3, request; 471 ms to Virginia, 30 ms to a 10 ms PoP |
| 60 per hour | expiries of a 60-second TTL; hit ratio 99.4% at 10,000 requests per hour |
| 1 vs N vs PoPs | origin fetches per cold key: collapsing plus shield = 1, collapsing only = 1 per PoP, none = N per PoP |
| 1 to 5 s | typical micro-cache TTL for hot dynamic responses |
| request vs byte | hit ratio per request and per byte can differ wildly (90% versus about 8% in the example) |