CDN Architecture: How Content Delivery Networks Serve the World

Netflix Open Connect: 17,000 Servers Inside ISPs

Netflix does not use a third-party CDN for video. They built Open Connect - 17,000 custom servers placed INSIDE internet service providers worldwide. When you watch a show, the video streams from a box physically inside your ISP's data center, not from Netflix's cloud. This eliminates internet backbone traversal entirely. But how does this massive distributed cache stay fresh and route users to the right edge?

The Physics Problem CDNs Solve

Speed of Light in Fiber: ~200,000 km/s (2/3 of vacuum speed)

  Route              Distance   Round Trip (physics limit)
  ----------------   --------   -------------------------
  NYC -> London      5,570 km   56ms
  NYC -> Tokyo       10,800 km  108ms
  NYC -> Sydney      16,000 km  160ms
  NYC -> NYC (local) ~0 km      ~1ms

  With CDN edge in Tokyo:
  Tokyo user -> Tokyo CDN PoP:  ~5ms (cache HIT)

  Without CDN:
  Tokyo user -> US origin:      ~108ms + server processing

  CDN wins: 20x faster for cached content.

CDN Request Flow

CDN Cache Flow:

  [User in Tokyo]
       |
  DNS: cdn.example.com -> 103.21.244.X (Tokyo PoP IP)
       |
  [CDN Edge - Tokyo PoP]
       |
  Cache check: /images/hero.jpg
       |
  +----+----+
  |         |
  HIT       MISS
  (95%)     (5%)
  |         |
  Return    +---> [Origin Shield - Singapore]
  in 5ms          |
                  Cache check
                  |
             +----+----+
             |         |
             HIT       MISS
             (80%)     (20%)
             |         |
             Return    +---> [Origin Server - US]
             to edge          |
             ~40ms            Fetch from origin
                              ~120ms

  Result: 95% of requests served in <10ms
  Origin only hit for 5% x 20% = 1% of total requests

Origin Shield: Protecting Your Backend

Without Origin Shield:
  50 PoPs worldwide, all miss cache simultaneously
  -> 50 requests hit your origin server
  -> origin overwhelmed during cache invalidation

With Origin Shield (regional mid-tier cache):
  50 PoPs miss -> all route to 3 regional shields
  -> shields check their cache (likely HIT)
  -> only 1-3 requests reach origin

  Shield reduces origin traffic by 90-99%
  Especially important for:
  - Cache invalidation storms
  - New content deployment
  - Low-traffic long-tail content

CDN Beyond Static Files

Modern CDN Capabilities:

  Capability            Example                  TTL
  --------------------  ----------------------   --------
  Static assets         JS, CSS, images          1 year
  API response caching  GET /trending-topics     5 seconds
  Edge compute          A/B test routing         N/A
  TLS termination       HTTPS handshake at edge  N/A
  DDoS protection       Absorb attack at edge    N/A
  WAF rules             Block SQL injection      N/A
  Image optimization    Resize per device        Cached
  Video streaming       HLS segment delivery     Hours

  Even 5-second API cache at 10K req/s:
  Without cache: 10,000 requests hit origin
  With 5s cache: 1 request hits origin, 49,999 cached
  99.998% reduction in origin load

Interview Tip

When discussing content delivery in system design, say: 'I would use a CDN with origin shield to serve static assets and cacheable API responses from edge nodes worldwide. The CDN resolves DNS to the nearest PoP, serves cache hits in <10ms, and routes misses through a regional shield before hitting origin. For cache invalidation, I would use cache-busting URLs for static assets (hash in filename) and short TTLs for API responses. This reduces origin load by 95%+ while giving global users sub-10ms latency for cached content.'

<

/blockquote>

Key Takeaway

CDNs place caches close to users to defeat the physics of network latency. Origin shields reduce origin load by funneling all cache misses through a regional mid-tier. Modern CDNs also handle dynamic content, edge compute, TLS termination, and DDoS protection.