6. Hosting, CDNs and Caching
“There are only two hard things in computer science: cache invalidation, naming things, and off-by-one errors.”— Phil Karlton, plus the internet
The story: the dabbawalas
Section titled “The story: the dabbawalas”Every working day in Mumbai, a few thousand dabbawalas deliver around two hundred thousand home-cooked lunches from homes to offices and back, by bicycle, handcart and suburban train, with a famously tiny error rate. Nobody carries a lunch from Borivali to Churchgate on their own. Tiffins are collected locally, sorted at stations, moved in bulk on trains, re-sorted near the destination, and delivered the last mile by someone who knows that street.
Two ideas make it work. First, the long haul is shared and the last mile is local. Second, the code painted on each tiffin tells everyone along the way exactly where it goes and when, so nobody has to phone home and ask.
A CDN is the first idea. HTTP caching headers are the second. Get both right, and most requests never reach your server at all — they’re answered by a machine a few kilometres from the user, from memory, in milliseconds.
Where does your code actually run? Hosting models
Section titled “Where does your code actually run? Hosting models”- Shared hosting. Your site is a folder on a server shared with hundreds of others (classic PHP hosting). Cheap; noisy neighbours; little control.
- VPS. A virtual machine you control (Oracle Cloud, DigitalOcean, Hetzner, Lightsail). You install Node, nginx, everything. Predictable monthly cost, full control, and you are the ops team: patches, restarts, disks filling up at 3 a.m.
- Containers / platforms. You ship a container image; a platform (Kubernetes, Cloud Run, Fly.io, Render) runs and scales it. Less ops, more abstraction.
- Serverless functions. You ship a function; the platform runs it per request (or per batch), scales to zero, bills per invocation. The cost: cold starts. When no instance is warm, the platform must start one — boot a runtime, load your code — before handling the request. That can add hundreds of milliseconds to seconds, mostly invisible in averages and very visible at p99.
- Edge functions. Small functions run at CDN locations, close to users, usually on lightweight isolates (V8 isolates rather than full containers) that start in milliseconds. Great for redirects, auth checks, personalization and HTML rewriting. Constrained: limited runtime APIs, CPU time limits, and your database is probably still far away — an edge function that makes three queries to a database in Virginia is slower than a server in Virginia.
- Static hosting. No code runs at request time at all. Files on a CDN. The fastest, cheapest, most reliable option, for everything that can be built ahead of time.
Reverse proxies and web servers
Section titled “Reverse proxies and web servers”In front of most application servers sits a reverse proxy — nginx or Caddy, typically. “Reverse” because it acts on behalf of the server (a forward proxy acts on behalf of clients). It:
- terminates TLS (so your Node process deals only in plain HTTP on localhost);
- serves static files directly from disk, which it’s very good at;
- compresses responses;
- forwards dynamic requests to the app (and can load-balance across several);
- buffers slow clients so your app isn’t tied up dribbling bytes to a phone on 3G;
- enforces timeouts, request size limits and rate limits.
Caddy is notable for automatic HTTPS out of the box; nginx is the battle-hardened default with an enormous ecosystem of examples.
Compression
Section titled “Compression”Text compresses extremely well. The three you’ll meet:
- gzip — universal, fast, decent.
- Brotli (
br) — typically 15–25% smaller than gzip for web text, thanks partly to a built-in dictionary of common web strings. Slow at maximum levels, so the classic setup is: compress static assets at build time at max level, and use a moderate level for dynamic responses. - Zstandard (
zstd) — very fast at good ratios, now supported by major Chromium-based and Firefox browsers; attractive for dynamic content.
The browser advertises what it accepts (Accept-Encoding: gzip, br, zstd); the server picks one and labels the response (Content-Encoding: br). Two things beginners miss: images and video are already compressed — gzipping a JPEG or MP4 wastes CPU for nothing (that’s why the author’s 7.7 MB of video “gzips” to 7.7 MB, Chapter 16) — and compressed size is what goes over the wire but decompressed size is what the browser parses.
CDNs: what they actually do
Section titled “CDNs: what they actually do”A Content Delivery Network is a fleet of servers in many cities (points of presence, PoPs) that sit between users and your origin. Mechanically:
- Anycast routing. Many PoPs announce the same IP address via BGP. The internet’s routing naturally delivers each user’s packets to the topologically nearest one. Priya in Pune hits a PoP in Mumbai; her cousin in Toronto hits one in Toronto; same IP.
- TLS termination at the edge. The expensive handshake round trips (Chapter 5) happen with a machine ~10 ms away instead of ~200 ms away.
- Caching. If the PoP has a fresh copy, it responds immediately: a cache hit. If not (a miss), it fetches from the origin, stores it according to your headers, and responds.
- Warm connections to origin. Even on a miss, the PoP has persistent, tuned connections back to your origin over good backbone links, which is often faster than the user’s own path.
- Origin shielding. With hundreds of PoPs, a miss everywhere means hundreds of requests to your origin. A shield is a designated mid-tier cache: PoPs ask the shield, only the shield asks the origin.
- Everything else: DDoS absorption, WAF rules, bot management, image resizing, edge functions.
HTTP caching in full depth
Section titled “HTTP caching in full depth”Here’s where most of the performance in Part I is won or lost. There are several caches between Priya and your origin:
- the browser’s memory cache (this tab, this page session — very fast, short-lived),
- the browser’s HTTP disk cache (persists across visits, governed by headers),
- the back/forward cache (bfcache — an entire frozen page kept in memory for instant back/forward; not governed by
Cache-Controlfreshness, see Chapter 9), - a service worker cache if you have one (Chapter 12),
- shared caches: CDN PoPs, shields, corporate proxies.
Every response’s Cache-Control header is instructions to all of them.
The directives
Section titled “The directives”max-age=N— the response is fresh for N seconds. While fresh, a cache can use it without contacting the server at all. Zero round trips.s-maxage=N— likemax-agebut only for shared caches (CDNs). Lets you cache HTML at the edge for 60 s while browsers revalidate.no-cache— the most misnamed directive in the web. It does not mean “don’t cache.” It means “you may store this, but you must revalidate with the server before every use.” Usually cheap (a 304, see below).no-store— actually don’t cache. Never write it anywhere. For sensitive responses (bank statements).private— only the user’s browser may store it; CDNs must not. For personalized responses.public— shared caches may store it even if it would otherwise be considered private (e.g., has anAuthorizationheader).immutable— “this will never change during its freshness lifetime; don’t even revalidate it when the user hits reload.” Pairs with hashed filenames.stale-while-revalidate=N— after freshness expires, for N more seconds, serve the stale copy immediately and refresh it in the background. Users get speed; content stays nearly current.stale-if-error=N— if the origin is failing, serve stale for up to N seconds rather than an error. Free resilience.
Revalidation: the cheap round trip
Section titled “Revalidation: the cheap round trip”When a cached response is stale (or no-cache), the cache doesn’t have to download it again. It asks, conditionally:
- If the response had an
ETag(a version identifier, often a hash), the cache sendsIf-None-Match: "a1b2c3". - If it had
Last-Modified, the cache sendsIf-Modified-Since: <date>.
If nothing changed, the server replies 304 Not Modified with no body. One round trip, a few hundred bytes. If it changed, a normal 200 with the new content.
The winning strategy: hashed + immutable, HTML revalidated
Section titled “The winning strategy: hashed + immutable, HTML revalidated”Put Chapter 1 and this chapter together and you get the strategy almost every well-run site converges on:
| What | Header | Why |
|---|---|---|
Hashed assets (/assets/app-B7f3a9.js, CSS, fonts, images) |
public, max-age=31536000, immutable |
The URL changes when content changes, so it can be cached forever. Zero requests on repeat visits. |
| HTML | public, max-age=0, must-revalidate (or no-cache), optionally s-maxage=60, stale-while-revalidate=600 at the CDN |
The HTML is what points at the new hashed files; it must be current. Revalidation is cheap. |
| Personalized/API responses | private, no-cache or short max-age |
Never cached in shared caches; browser revalidates. |
| Truly sensitive | no-store |
Never stored anywhere. |
Deploying is then trivial: upload new hashed files (old ones keep working for anyone mid-visit), then switch the HTML. Browsers revalidate the HTML, see new file names, fetch only what changed. No purging, no “clear your cache” emails.
Vary, and the cache key
Section titled “Vary, and the cache key”A cache stores responses by a key: by default, roughly the URL. If your response differs by request header (by Accept-Encoding, or by Accept for image format negotiation), you must say Vary: Accept-Encoding so caches store separate copies. Varying on something high-cardinality like User-Agent or Cookie destroys your hit rate, because every variant is a separate entry.
Cache invalidation: the hard problem
Section titled “Cache invalidation: the hard problem”Sometimes you must change content at a URL that is cached — an un-hashed HTML page cached at the CDN for an hour, say. Options: wait for TTLs (simple; slow), purge the URL at the CDN (fast, but only reaches shared caches you control — browsers keep their copies), or design so you rarely need to: short CDN TTLs with stale-while-revalidate for HTML, and hashing for everything else. Browsers’ caches cannot be purged remotely, which is the deep reason the strategy above is shaped the way it is.
When it goes wrong
Section titled “When it goes wrong”Fastly, June 2021. For under an hour, a big chunk of the web — major news sites, government sites, e-commerce — returned errors. A software change Fastly had deployed weeks earlier contained a latent bug. One customer pushed a perfectly valid configuration change that triggered it, and the bug caused most of Fastly’s network to start returning errors. Fastly detected it within about a minute and had most of the network recovered within about 45 minutes. Lesson: a CDN concentrates risk; a latent bug plus one specific trigger can take out thousands of sites at once. Know what your fallback is.
Cloudflare, July 2019. A new rule pushed to Cloudflare’s web application firewall contained a regular expression vulnerable to catastrophic backtracking — a pattern whose matching time explodes on certain inputs. Pushed globally at once, it drove CPUs on every machine handling HTTP traffic to 100%. For around half an hour, Cloudflare-proxied sites served errors. Cloudflare later moved to regex engines that guarantee linear-time matching and added staged rollouts for rules. Lesson: config is code (Chapter 4), and “global, instantly” is a blast radius.
Cloudflare, November 2025. A permissions change in a database system caused a query that builds a configuration file for Cloudflare’s bot-management feature to return duplicate rows, roughly doubling the file. The proxy software that loads this file had a fixed limit on its size; exceeding it caused the proxy to fail — and because the file was regenerated and distributed across the network every few minutes, the network flapped between working and broken before failing more broadly. Large parts of the web returned errors for several hours. Lesson: an internal, auto-generated config file is an input that needs validation and a safe failure mode, exactly like user input. “Fail closed” on a non-critical feature took down the critical path.
What you now know
Section titled “What you now know”You can compare hosting models including cold starts and cost curves, explain what a reverse proxy and a CDN do mechanically (anycast, edge TLS, shielding), pick a compression scheme, read every Cache-Control directive correctly (including the treacherous no-cache), explain revalidation with ETags and 304s, and design the hashed + immutable / revalidated HTML strategy — plus three real outages that show what concentrated infrastructure risk looks like.
Self-quiz
Section titled “Self-quiz”1. What does no-cache mean?
Store it, but revalidate with the server before every use. (no-store means don’t store.)
2. Why can hashed files be cached for a year?
Their URL changes when content changes, so a URL always identifies the same bytes.
3. What does s-maxage let you do?
Give shared caches (CDNs) a different lifetime than browsers — e.g., cache HTML at the edge for 60 s while browsers revalidate.
4. Why is Vary: User-Agent usually a bad idea at a CDN?
It creates a separate cache entry per user-agent string, destroying the hit rate.
5. What’s a cold start, and where does it bite?
The time to boot a new serverless instance before handling a request; it shows up in tail latency (p95/p99), not averages.
Exercises
Section titled “Exercises”Warm-up 1. Pick the header. For: (a) /assets/index-9f2c.css, (b) / HTML, (c) /api/me (user profile), (d) /robots.txt.
Solution
(a) public, max-age=31536000, immutable. (b) no-cache (browser) plus maybe s-maxage=60, stale-while-revalidate=600. (c) private, no-cache. (d) a modest max-age like 3600.
Warm-up 2. Predict. A response has Cache-Control: max-age=60, stale-while-revalidate=300 and an ETag. What happens at t=30 s, t=200 s, t=500 s?
Solution
30 s: fresh, served from cache, no request. 200 s: stale within SWR, served instantly, background revalidation. 500 s: stale beyond SWR, conditional request; 304 if unchanged.
Warm-up 3. Why no gain? Someone enables Brotli and video payloads don’t shrink. Why?
Solution
Video (and JPEG/PNG/AVIF) are already compressed; general-purpose compression finds nothing more.
Lab 4. Lumen’s headers. Configure Lumen’s host (nginx or the CDN’s headers file) with the strategy table above. Verify each header in DevTools.
Expected outcome
On a repeat visit, hashed assets show “(disk cache)” or “(memory cache)” with no network request; HTML shows a 304 or a quick 200.
Lab 5. Watch a 304. Reload Lumen’s HTML with an ETag and inspect request and response headers.
What to look for
If-None-Match on the request, 304 with an empty body on the response. Compare transfer sizes.
Lab 6. CDN hit rate. Put Lumen behind a CDN and find the response header that reports cache status (HIT/MISS). Request a page twice from two networks.
Hint
Most CDNs add a cache-status header. First request MISS, second HIT. HTML may show BYPASS/DYNAMIC unless you allow edge caching with s-maxage.
Staff 7. Serverless vs VPS. Lumen’s traffic is 5,000 visits/day with spikes to 200,000 when an essay goes viral. Argue for a hosting setup and estimate the trade-offs.
Model answer
Static/ISR on a CDN for essays (spikes absorbed at the edge for near-zero cost); a small VPS or serverless for the few dynamic routes; avoid making viral pages depend on origin rendering.
Staff 8. Concentration risk. Your entire site depends on one CDN. Write a half-page on what you’d do (or consciously not do) about it.
Model points
Multi-CDN is expensive and complex; often the right answer is accepting the risk with good status communication, a tested “bypass CDN to origin” DNS plan with low TTLs, and static fallbacks — sized to what an hour of downtime costs the business.