The Day Justin Bieber Broke Twitter
In 2012, Twitter was struggling with a massive engineering challenge. Every time Justin Bieber (then 30 million followers) posted a tweet, Twitter had to update 30 million timelines. Their original push-based architecture (fanout on write) meant that one celebrity tweet generated 30 million cache writes that had to complete in seconds. It was crushing their infrastructure. The engineering team realized that pure push did not work for users with millions of followers, and pure pull (computing feeds at read time) was too slow for the average user who checks Twitter dozens of times per day. Their solution - a hybrid approach that treats celebrities differently from regular users - became the textbook answer for feed system design. Facebook independently arrived at a similar architecture for their News Feed.
The Fanout Problem Visualized
When a Celebrity Posts:
@TaylorSwift: 'New album dropping Friday!'
|
v
Fanout to 90 million followers:
PUSH (Fanout on Write): PULL (Fanout on Read):
1 post -> 90M cache writes 1 post -> stored once
Each follower's feed cache When User loads feed:
gets updated immediately 1. Who do I follow? (500 users)
2. Fetch latest from each
Timeline: [TSwift post, 3. Merge + sort by timestamp
Friend post, 4. Return top 50
Brand post, ...]
= 500+ DB queries per feed load!
Speed: O(followers) writes Speed: O(following) reads
per post per feed view
For @TaylorSwift:
Push cost: 90M writes per post (EXPENSIVE write)
Pull cost: 0 writes, but 90M users each do 500 reads (EXPENSIVE read)
For @RegularUser (200 followers):
Push cost: 200 writes per post (CHEAP)
Pull cost: 200 reads per feed (also cheap)The Hybrid Approach
How Twitter/Facebook Actually Work:
@RegularUser posts (< 10K followers):
+-> [Fanout Service] pushes post_id to each follower's
Redis feed cache (sorted set, score = timestamp)
200 writes. Done in <1 second. Feed reads are instant.
@Celebrity posts (> 10K followers):
+-> Post stored in celebrity's post table. No fanout.
When a user loads their feed:
1. Read pre-pushed posts from Redis cache
2. Query celebrity posts table for followed celebrities
3. Merge both lists, rank, return top 50
User loads timeline:
+-> [Timeline Service]
|
+-- Read user's feed cache from Redis (pre-pushed, <1ms)
+-- Query celebrity posts (< 10 queries) (<5ms)
+-- Merge, deduplicate, rank (<2ms)
+-- Return top 50 posts (~8ms total)
Threshold Decision:
Followers Strategy Why
------------ ---------- ---------------------------
< 1,000 Push Trivial fanout cost
1K - 10K Push Still manageable
10K - 1M Pull Fanout too expensive
> 1M Pull Would take minutes to push
The exact threshold varies by platform and infrastructure.Feed Storage Architecture
Redis Sorted Sets for Feed Caches:
Key: feed:{user_id}
Members: post_ids
Scores: timestamps
# Push a post to follower's feed
ZADD feed:user_42 1705276800 post_abc
# Read top 20 posts (latest first)
ZREVRANGE feed:user_42 0 19
# Trim to keep only latest 1000 posts
ZREMRANGEBYRANK feed:user_42 0 -1001
Memory per user feed cache:
1000 post IDs x ~40 bytes each = ~40 KB
100M active users x 40 KB = ~4 TB of Redis
(Sharded across many Redis nodes)
Important: store only post IDs in the feed cache,
NOT the full post content. Hydrate full post data
from the Posts Service when rendering the feed.
This saves memory and ensures edits/deletes propagate.Ranking Beyond Chronological
Modern feeds are not purely chronological. Facebook introduced ranked feeds in 2011 because users miss important posts when overwhelmed by volume. The ranking model considers: engagement probability (will this user like/comment?), content type preferences, relationship strength (close friend vs acquaintance), content freshness, and diversity (avoid showing 10 posts from the same source). Instagram, TikTok, LinkedIn, and X all use ML-ranked feeds today.
Interview Tip
News Feed is one of the most common system design interview questions. Structure your answer: 'I would use a hybrid fanout approach. For users with fewer than 10K followers, push post IDs into followers Redis sorted set caches on write - feed reads are then O(1) from cache. For high-follower accounts (celebrities), skip fanout and pull their recent posts at read time, merging with the pre-pushed cache. Feed caches store only post IDs (not full content) to save memory, and posts are hydrated from a Posts Service on read. For ranking, I would start with reverse-chronological ordering and add an ML ranking model later that considers engagement probability, relationship strength, and content diversity.' Mention the concrete tradeoff: push makes reads fast but writes expensive, pull makes writes cheap but reads expensive.
Key Takeaway
Pure push (fanout on write) causes massive write amplification for high-follower accounts. Pure pull (fanout on read) makes feed loading slow. The hybrid approach pushes for regular users and pulls celebrity posts at read time. Use Redis sorted sets for feed caches, store only post IDs (hydrate on read), and cap cache size at 1000 entries per user.