Design Instagram: Photo Sharing at a Billion Users

13 Engineers Serving 30 Million Users

When Instagram was acquired by Facebook in 2012 for $1 billion, the entire engineering team was 13 people serving 30 million users. Their secret was radical simplicity: PostgreSQL for everything, Redis for caching, S3 for photos, and Django for the web layer. They scaled vertically as long as possible before adding complexity. This lesson - do not add architectural complexity until the pain forces you - is the single most important takeaway from Instagram's engineering.

Photo Upload Pipeline

Photo Upload Architecture:

  [Mobile Client]
       |
  1. Request pre-signed S3 upload URL from API
       |
  2. Upload raw photo DIRECTLY to S3
     (bypasses your servers - saves bandwidth + CPU)
       |
  3. S3 event triggers Lambda/Worker
       |
  [Image Processing Pipeline]
  Generate sizes:  150x150 (thumbnail)
                   320x320 (feed)
                   640x640 (feed retina)
                   1080x1080 (full)
  Formats: JPEG + WebP (30% smaller)
  Strip EXIF metadata (privacy)
       |
  4. Store processed images back in S3
     S3 -> CloudFront CDN (global edge delivery)
       |
  5. Save metadata to PostgreSQL:
     {post_id, user_id, s3_key, caption,
      location, filters, created_at}
       |
  6. Publish post_id to Kafka for fanout

Feed Architecture: From Simple to ML-Ranked

Feed Architecture Evolution:

  2010-2016: Chronological feed
  SELECT posts.* FROM posts
  JOIN follows ON posts.user_id = follows.followee_id
  WHERE follows.follower_id = :me
  ORDER BY created_at DESC LIMIT 50
  Problem: at 200 follows, this query is SLOW

  2016+: Pre-computed + ML-ranked feed

  [Post created] -> [Fanout Service]
                    Push post_id to follower Redis timelines
                    (same hybrid fanout as Twitter)
                         |
  [User opens app] -> [Feed Service]
                      Fetch 500 candidate post IDs from Redis
                           |
                      [Ranking Model]
                      Score each candidate:
                      - Recency (newer = higher)
                      - Relationship strength
                        (how often you interact with this account)
                      - Post engagement (likes/comments ratio)
                      - Content type affinity
                        (you engage more with Reels vs photos)
                           |
                      Return top 50 ranked posts

Data Sharding Strategy

Instagram's Sharding (when they outgrew single-node PG):

  Table           Shard Key        Why This Key?
  ─────────────   ────────────   ──────────────────────────
  Posts           post_id (UUID)   Even distribution, lookup by ID
  Follows         follower_id      'Get my follows' = single shard
  Likes           post_id          'Get post likes' = single shard
  Comments        post_id          'Get comments' = single shard
  Users           user_id          Profile lookup = single shard

  ID generation: Embedded shard ID in the UUID
  Bits: [timestamp:41][shard_id:13][sequence:10]
  Inspired by Twitter's Snowflake - allows extracting
  shard from ID without a lookup table

Interview Tip

When designing Instagram, emphasize three things: (1) upload directly to S3 via pre-signed URLs to bypass your servers, (2) hybrid fanout for feed generation with ML ranking on top, (3) shard PostgreSQL by entity-appropriate keys (posts by post_id, follows by follower_id). The Explore page is a recommendation system - mention candidate generation from similar accounts, ML scoring, and deduplication. Instagram's key lesson is starting simple and only adding complexity when forced to.

<

/blockquote>

Key Takeaway

Instagram's architecture: upload directly to S3, process asynchronously, distribute via CDN. Feed uses hybrid fanout with ML ranking. Shard data by entity type with appropriate partition keys. The most important scaling decision they made was PostgreSQL horizontal sharding, which bought them years of runway.