The Ideas Behind Every System
You'll Ever Design
Dynamo, Raft, Attention Is All You Need. Engineers cite these papers constantly and read them rarely. We turn each landmark paper into a walkthrough you can actually finish, with real diagrams and the math spelled out instead of skipped.
MapReduce: Simplified Data Processing on Large Clusters
Google turned web-scale data processing into two functions: map and reduce. This paper explains how that simple model let a framework handle parallelization, failures, and network efficiency automatically, and became the blueprint for Hadoop and the entire big-data ecosystem.
Jeffrey Dean, Sanjay Ghemawat · OSDI 2004
Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services (The CAP Theorem)
A distributed system split by a network partition must choose: answer with data that might be stale, or refuse to answer at all. Gilbert and Lynch turned Brewer's conjecture into a two-page proof that still decides how every database you use behaves when the network breaks.
Seth Gilbert, Nancy Lynch · ACM SIGACT News 2002
Efficient Estimation of Word Representations in Vector Space
Before word2vec, a word was just a one-hot index, equally distant from every other word. This paper strips a neural language model down to a bare linear projection so it can train on billions of words, and the resulting vectors turn out to capture meaning well enough that king minus man plus woman lands on queen.
Tomas Mikolov, Kai Chen et al. · ICLR Workshop 2013
The Google File System
Google needed a filesystem built for thousands of cheap machines that fail constantly, storing huge files that are mostly appended to, not edited. GFS answers with a single metadata-only master, giant 64MB chunks, and a data/control split that later became the blueprint for Hadoop's HDFS.
Sanjay Ghemawat, Howard Gobioff et al. · SOSP 2003
Dynamo: Amazon's Highly Available Key-value Store
Amazon's shopping cart can never say no. This paper shows how Dynamo trades strict consistency for always-on availability using consistent hashing, sloppy quorums, and vector clocks, the same ideas behind Cassandra and Riak today.
Giuseppe DeCandia, Werner Vogels et al. · SOSP 2007
Cassandra: A Decentralized Structured Storage System
Avinash Lakshman, Prashant Malik
Adam: A Method for Stochastic Optimization
A single global learning rate is a poor fit for millions of parameters with wildly different gradient behavior. Adam gives every parameter its own adaptive learning rate by combining momentum with a bias-corrected running average of squared gradients, and became deep learning's default optimizer.
Diederik P. Kingma, Jimmy Ba · ICLR 2015
In Search of an Understandable Consensus Algorithm (Raft)
Paxos works, but almost nobody can explain it cleanly. Raft gets the same guarantees by splitting consensus into leader election, log replication, and safety, the algorithm now running underneath etcd, Consul, and Kubernetes.
Diego Ongaro, John Ousterhout · USENIX ATC 2014
Paxos Made Simple
Getting a cluster to agree on one value, safely, even when proposers collide and messages get lost, is the oldest hard problem in distributed systems. Lamport's own plain-language rewrite of his notoriously opaque algorithm is still the clearest path into how Raft, Chubby, and Spanner actually agree on anything.
Leslie Lamport · ACM SIGACT News 2001
Attention Is All You Need: The Transformer Architecture Explained
RNNs process language one word at a time, which is slow to train and struggles with long-range dependencies. This paper introduces the Transformer, which replaces recurrence entirely with self-attention and became the architecture behind every modern large language model.
Ashish Vaswani, Noam Shazeer et al. · NeurIPS 2017