Latency Is Physics. You Cannot Engineer Around the Speed of Light.
A user in Tokyo requesting content from a server in Virginia faces a minimum round-trip time of about 200 milliseconds just from the physical distance light must travel through fiber. Multiply that by several round trips for TCP handshake, TLS negotiation, and HTTP request processing, and page loads measured in seconds become unavoidable. For a global user base, distance is a hard constraint.
The only solution is to eliminate distance: put copies of content near the users. Tom Leighton and Danny Lewin, both MIT applied mathematics professors, built Akamai around this insight in 1998. Place servers at internet exchange points around the world. Route each user to the nearest edge server. A Tokyo user gets content from Tokyo, not Virginia. Akamai became the world's first major content delivery network.
How CDNs Work
A CDN is a geographically distributed cache. When a user requests content, DNS routes them to the nearest edge server based on their IP address location. If the edge server has the content cached, it responds immediately. If not, it fetches from the origin server, caches the response, and serves it. Subsequent users in the same region get the cached copy.
Pull CDN: edge servers fetch content from origin on first request and cache based on Cache-Control headers. Simple to operate. Origin handles cache misses.
Push CDN: you explicitly push content to edge servers before any request arrives. More control but requires managing what gets pushed and when.
Origin shielding: a single 'shield' edge server in each region acts as the first layer. Only the shield fetches from origin, protecting it from the full request load.
Cache invalidation: when content changes, you can either wait for TTL expiry or actively purge specific URLs from edge caches. Purge is instant but requires a purge API call.
Modern CDNs handle far more than static files. They cache API responses with short TTLs. They run edge functions (Cloudflare Workers, Lambda@Edge) that execute logic at the edge. They handle TLS termination, DDoS mitigation, and bot detection globally. A request that previously had to travel to your data center for authentication can now be authenticated at a Cloudflare edge server 10 milliseconds from the user.
CDNs are not a caching optimization. They are the infrastructure that makes global performance possible. Every system serving a geographically distributed user base needs one.
CRDTs: Conflict-Free Replicated Data Types (2011)
The hardest problem in multi-region systems: two users edit the same data in different regions simultaneously. User A in London increments a counter. User B in Singapore increments the same counter. Network partitions prevent synchronization. When the partition heals, whose value wins? Last-writer-wins loses one increment. Distributed locking kills availability. CRDTs solve this mathematically.
A CRDT is a data structure designed so that concurrent updates from any number of replicas will automatically converge to the same state, without coordination, without conflicts, and without data loss. The math works because the operations are designed to be commutative (A+B = B+A), associative ((A+B)+C = A+(B+C)), and idempotent (applying the same update twice has the same effect as applying it once).
G-Counter: a grow-only counter where each replica maintains its own count. Total = sum of all replica counts. Merging is just summing.
PN-Counter: combines a G-Counter for increments and a G-Counter for decrements. Total = sum(increments) - sum(decrements).
OR-Set (Observed-Remove Set): tracks additions and removals without conflicts by using unique tags per operation.
LWW-Element-Set: uses timestamps to determine which concurrent update wins. Simpler but requires synchronized clocks.
CRDTs power collaborative editing in Figma and Google Docs (concurrent cursor positions and edits), shopping cart additions in multi-region e-commerce (never lose an item a user added), and like/reaction counts in social networks (eventual convergence across regions). Any problem where you need concurrent updates without coordination is a CRDT problem.
Further Reading
Shapiro et al. (2011): 'A comprehensive study of Convergent and Commutative Replicated Data Types'
Akamai's technical blog for CDN architecture patterns
Martin Kleppmann's talk 'CRDTs: The Hard Parts' for practical CRDT engineering challenges
Cloudflare Workers documentation for edge computing patterns
Cache Invalidation: The Second Hardest Problem in Computer Science
Cache invalidation is hard because there is no one-size-fits-all solution. Every strategy trades freshness for simplicity or for performance. TTL-based expiry is the simplest: set a max-age on each response and accept that users may see stale content up to TTL seconds after a change. This requires no invalidation infrastructure and works for content that changes infrequently. Active purge is the most responsive: when content changes, immediately send a purge request to the CDN's API for the affected URL or tag group. Cloudflare's cache tag system lets you attach tags to responses and purge all URLs with a given tag in a single API call.
TTL expiry: simplest. Content is stale for up to TTL seconds. No invalidation logic. Best for rarely-changing content.
Stale-while-revalidate: serve stale content immediately, refresh in the background. Eliminates latency on cache misses for users.
Cache tag purge: attach semantic tags to responses (product-id-12345), purge by tag when that entity changes. Cloudflare Cache Tags, Fastly surrogate keys.
Event-driven invalidation: publish a cache purge event to a queue when data changes. Workers process the queue and call the CDN purge API. Decoupled and reliable.
Cache stampede / thundering herd: a popular item's TTL expires and thousands of simultaneous requests all go to origin. Use probabilistic early expiry or a lock/semaphore to let only one request refresh while others wait.
CRDT Limitations: When Conflict-Free Is Not Enough
CRDTs are powerful but not universal. The mathematical constraints that make automatic convergence possible also constrain what operations CRDTs can express. You cannot implement a 'cap at maximum value' operation with a simple CRDT because the cap creates a global constraint that requires coordination. Bounded counters (a like count that maxes out at 1 per user) cannot be implemented as a pure CRDT without additional coordination. Strong ordering requirements (message 5 must be delivered after message 4) cannot be satisfied by CRDTs alone. And debugging CRDT conflicts is significantly harder than debugging traditional lock-based concurrency because causality violations manifest as data anomalies rather than explicit errors.
Operations that require global constraints (balance must stay positive, seat count cannot go negative) need coordination, not CRDTs.
CRDTs grow in size with operations (OR-Set retains all add/remove tags). Garbage collection requires coordination.
Two different CRDT implementations of the 'same' data structure can have incompatible merge semantics.
CRDT state transfer between nodes requires full state or delta-CRDTs (sending only changes). Full state transfer becomes expensive as state grows.
Practical alternative: use optimistic locking with application-level conflict resolution for low-conflict scenarios rather than implementing full CRDTs.
What Interviewers Test About CDNs and Distributed Caching
CDN and caching questions test whether you understand the full lifecycle: what gets cached, for how long, and how you maintain correctness when the source of truth changes. The most common mistake is treating CDN as a 'set and forget' layer without thinking about cache invalidation.
Know: the difference between client-side caching (browser), edge caching (CDN), application caching (Redis), and database query caching
Know: when to use a write-through cache (consistent but every write hits the DB) vs write-around (fast writes, cache misses on first read) vs write-back (fast writes, risk of data loss)
Know: how to handle cache key design for multi-tenant systems (include tenant ID in key) and authenticated content (private vs shared cache)
Know: the cache stampede problem and at least one mitigation (probabilistic early revalidation, locking, stale-while-revalidate)
Know: when CRDTs are the right tool (collaborative editing, distributed counters, shopping carts) vs when they are wrong (any global constraint)
Cache Invalidation: The Second Hardest Problem in Computer Science
Cache invalidation is hard because there is no one-size-fits-all solution. Every strategy trades freshness for simplicity or for performance. TTL-based expiry is the simplest: set a max-age on each response and accept that users may see stale content up to TTL seconds after a change. This requires no invalidation infrastructure and works for content that changes infrequently. Active purge is the most responsive: when content changes, immediately send a purge request to the CDN's API for the affected URL or tag group. Cloudflare's cache tag system lets you attach tags to responses and purge all URLs with a given tag in a single API call.
TTL expiry: simplest. Content is stale for up to TTL seconds. No invalidation logic. Best for rarely-changing content.
Stale-while-revalidate: serve stale content immediately, refresh in the background. Eliminates latency on cache misses for users.
Cache tag purge: attach semantic tags to responses (product-id-12345), purge by tag when that entity changes. Cloudflare Cache Tags, Fastly surrogate keys.
Event-driven invalidation: publish a cache purge event to a queue when data changes. Workers process the queue and call the CDN purge API. Decoupled and reliable.
Cache stampede / thundering herd: a popular item's TTL expires and thousands of simultaneous requests all go to origin. Use probabilistic early expiry or a lock/semaphore to let only one request refresh while others wait.
CRDT Limitations: When Conflict-Free Is Not Enough
CRDTs are powerful but not universal. The mathematical constraints that make automatic convergence possible also constrain what operations CRDTs can express. You cannot implement a 'cap at maximum value' operation with a simple CRDT because the cap creates a global constraint that requires coordination. Bounded counters (a like count that maxes out at 1 per user) cannot be implemented as a pure CRDT without additional coordination. Strong ordering requirements (message 5 must be delivered after message 4) cannot be satisfied by CRDTs alone. And debugging CRDT conflicts is significantly harder than debugging traditional lock-based concurrency because causality violations manifest as data anomalies rather than explicit errors.
Operations that require global constraints (balance must stay positive, seat count cannot go negative) need coordination, not CRDTs.
CRDTs grow in size with operations (OR-Set retains all add/remove tags). Garbage collection requires coordination.
Two different CRDT implementations of the 'same' data structure can have incompatible merge semantics.
CRDT state transfer between nodes requires full state or delta-CRDTs (sending only changes). Full state transfer becomes expensive as state grows.
Practical alternative: use optimistic locking with application-level conflict resolution for low-conflict scenarios rather than implementing full CRDTs.
What Interviewers Test About CDNs and Distributed Caching
CDN and caching questions test whether you understand the full lifecycle: what gets cached, for how long, and how you maintain correctness when the source of truth changes. The most common mistake is treating CDN as a 'set and forget' layer without thinking about cache invalidation.
Know: the difference between client-side caching (browser), edge caching (CDN), application caching (Redis), and database query caching
Know: when to use a write-through cache (consistent but every write hits the DB) vs write-around (fast writes, cache misses on first read) vs write-back (fast writes, risk of data loss)
Know: how to handle cache key design for multi-tenant systems (include tenant ID in key) and authenticated content (private vs shared cache)
Know: the cache stampede problem and at least one mitigation (probabilistic early revalidation, locking, stale-while-revalidate)
Know: when CRDTs are the right tool (collaborative editing, distributed counters, shopping carts) vs when they are wrong (any global constraint)
Cache Invalidation: The Second Hardest Problem in Computer Science
Cache invalidation is hard because there is no one-size-fits-all solution. Every strategy trades freshness for simplicity or for performance. TTL-based expiry is the simplest: set a max-age on each response and accept that users may see stale content up to TTL seconds after a change. This requires no invalidation infrastructure and works for content that changes infrequently. Active purge is the most responsive: when content changes, immediately send a purge request to the CDN's API for the affected URL or tag group. Cloudflare's cache tag system lets you attach tags to responses and purge all URLs with a given tag in a single API call.
- TTL expiry: simplest. Content is stale for up to TTL seconds. No invalidation logic. Best for rarely-changing content.
- Stale-while-revalidate: serve stale content immediately, refresh in the background. Eliminates latency on cache misses for users.
- Cache tag purge: attach semantic tags to responses (product-id-12345), purge by tag when that entity changes. Cloudflare Cache Tags, Fastly surrogate keys.
- Event-driven invalidation: publish a cache purge event to a queue when data changes. Workers process the queue and call the CDN purge API. Decoupled and reliable.
- Cache stampede / thundering herd: a popular item's TTL expires and thousands of simultaneous requests all go to origin. Use probabilistic early expiry or a lock/semaphore to let only one request refresh while others wait.
CRDT Limitations: When Conflict-Free Is Not Enough
CRDTs are powerful but not universal. The mathematical constraints that make automatic convergence possible also constrain what operations CRDTs can express. You cannot implement a 'cap at maximum value' operation with a simple CRDT because the cap creates a global constraint that requires coordination. Bounded counters (a like count that maxes out at 1 per user) cannot be implemented as a pure CRDT without additional coordination. Strong ordering requirements (message 5 must be delivered after message 4) cannot be satisfied by CRDTs alone. And debugging CRDT conflicts is significantly harder than debugging traditional lock-based concurrency because causality violations manifest as data anomalies rather than explicit errors.
- Operations that require global constraints (balance must stay positive, seat count cannot go negative) need coordination, not CRDTs.
- CRDTs grow in size with operations (OR-Set retains all add/remove tags). Garbage collection requires coordination.
- Two different CRDT implementations of the 'same' data structure can have incompatible merge semantics.
- CRDT state transfer between nodes requires full state or delta-CRDTs (sending only changes). Full state transfer becomes expensive as state grows.
- Practical alternative: use optimistic locking with application-level conflict resolution for low-conflict scenarios rather than implementing full CRDTs.
What Interviewers Test About CDNs and Distributed Caching
CDN and caching questions test whether you understand the full lifecycle: what gets cached, for how long, and how you maintain correctness when the source of truth changes. The most common mistake is treating CDN as a 'set and forget' layer without thinking about cache invalidation.
- Know: the difference between client-side caching (browser), edge caching (CDN), application caching (Redis), and database query caching
- Know: when to use a write-through cache (consistent but every write hits the DB) vs write-around (fast writes, cache misses on first read) vs write-back (fast writes, risk of data loss)
- Know: how to handle cache key design for multi-tenant systems (include tenant ID in key) and authenticated content (private vs shared cache)
- Know: the cache stampede problem and at least one mitigation (probabilistic early revalidation, locking, stale-while-revalidate)
- Know: when CRDTs are the right tool (collaborative editing, distributed counters, shopping carts) vs when they are wrong (any global constraint)