Worked Example: Designing a News Feed (like Twitter/Instagram)

This is widely considered one of the hardest "classic" system design questions, and for good reason: the naive solution to feed generation has a scaling flaw that only shows up once you run the numbers, which is exactly why Step 2 matters so much in this one.

• -Functional: users follow other users; a user's home feed shows recent posts from everyone they follow, most-recent first.
• -Out of scope: ranking/ML-based ordering (assume simple reverse-chronological for this design, worth stating explicitly, since real systems rank feeds, but that's a separate, much larger problem).
• -Non-functional: opening the app and loading the feed must feel instant (well under 200ms), far more important than write latency, since a slow post is annoying but a slow feed feels broken every single time the app opens.

Assume 200 million daily active users, each posting an average of 2 times per day, and following an average of 200 other accounts.

Neither strategy alone works at scale: pure push is destroyed by celebrities; pure pull makes every single feed load expensive for everyone, including users who never encounter a celebrity post. Real systems (this is the answer Twitter and Instagram actually ship) use a hybrid:

• -For accounts under a follower threshold (say, 100K): fan-out-on-write as normal. This covers the overwhelming majority of accounts, and keeps their followers' feed reads fast.
• -For accounts over that threshold (celebrities): skip fan-out entirely. Their posts are NOT pushed to any feed cache.
• -At feed-load time, a user's precomputed (pushed) feed is merged with a live, on-the-fly fetch of recent posts from any celebrities they follow, a small, bounded number of extra lookups, since one person only follows so many celebrities.

• -Consistency: it is entirely acceptable for a new post to take a few seconds to appear across all of a regular user's followers' feeds; the fan-out worker queue absorbs the write burst asynchronously (Module 8), and nobody expects instant global feed propagation.
• -Storage cost: precomputing feeds means storing the same post ID redundantly across every follower's feed cache, a deliberate trade of extra storage for much faster reads, the same trade-off named back in the Caching module.
• -Failure mode: the fan-out queue backs up during a traffic spike (a viral moment). Posts still get created and stored immediately; only the fan-out to followers' feed caches lags. Feeds get briefly stale rather than the system failing outright, a graceful degradation, not an outage.