For most of the 2010s, if you wanted a neural network to handle language, translation, summarization, anything involving a sequence of words, you reached for a recurrent neural network (RNN) or its better-behaved cousin, the LSTM. Recurrence has an obvious appeal: read a sentence one word at a time, carry a hidden state forward, and you naturally capture order and context.
It also has an obvious cost: you cannot process word 5 until you have processed words 1 through 4. That strict sequential dependency made RNNs slow to train on the long sequences and huge datasets the field increasingly wanted to throw at them. In 2017, a team at Google Brain and Google Research asked a genuinely provocative question: what if you removed the recurrence entirely, and built a sequence model out of nothing but attention?
The Transformer replaces recurrence with self-attention, letting every position in a sequence look directly at every other position in a single step. This makes training far more parallelizable and is now the architecture behind essentially every modern large language model.
What attention actually computes
Attention was not new in 2017, it already existed as a helper mechanism bolted onto RNN-based translation models, letting a decoder "look back" at relevant encoder states. This paper's move was to make attention the entire mechanism, not a helper.
The core operation is Scaled Dot-Product Attention. Every input position is turned into three vectors: a Query (what am I looking for?), a Key (what do I contain, for matching purposes?), and a Value (what do I actually contribute if I am relevant?). A position's output is a weighted sum of every Value in the sequence, where the weights come from how well that position's Query matches every other position's Key.
Scaled dot-product attention: match queries against keys, scale, optionally mask out illegal positions, turn the scores into a probability distribution, and use it to blend the values.
The scaling matters more than it looks. Without it, as the dimension of Q and K grows, the dot products grow large in magnitude, pushing the softmax into regions where its gradient is vanishingly small, which stalls learning. Dividing by the square root of the key dimension keeps the scores in a range where softmax still has useful gradients.
Think of it as a soft, differentiable dictionary lookup. A hard lookup finds the one exact matching key. Attention finds every key weighted by how well it matches, and blends their values accordingly, all in one smooth, learnable operation.
Multi-head attention: many small lookups instead of one big one
Running one attention function over the full-dimensional Q, K, and V only lets the model learn one "kind" of relationship at a time. Instead, the Transformer linearly projects Q, K, and V into h smaller subspaces (the original paper uses h = 8, each of dimension 64, for a model dimension of 512), runs scaled dot-product attention independently in each subspace, then concatenates the results and projects once more.
Each head learns its own projections and can specialize, one head might track syntactic agreement, another might track coreference, without any of them needing to share one attention pattern.
This is cheaper than it sounds: projecting into h subspaces of size d_model/h and running attention in each costs roughly the same total compute as one full-dimensional attention, but gives the model h independent "views" instead of one averaged view.
Assembling the full Transformer
The complete model keeps the classic encoder-decoder shape used in sequence-to-sequence translation, but builds both halves entirely out of attention and simple feed-forward layers.
Each encoder layer self-attends over the input; each decoder layer masks future positions in its own self-attention, then attends over the encoder's output before its feed-forward layer.
Encoder: a stack of 6 identical layers, each with a self-attention sub-layer (every input position attends to every other input position) followed by a position-wise feed-forward network.
Decoder: a stack of 6 identical layers, each with a masked self-attention sub-layer (a position can only attend to earlier positions, so the model cannot cheat by looking at the answer it is generating), an encoder-decoder attention sub-layer (queries come from the decoder, keys and values come from the encoder's output), and a feed-forward network.
Every sub-layer is wrapped in a residual connection followed by layer normalization, LayerNorm(x + Sublayer(x)), the same trick that makes very deep networks trainable elsewhere in deep learning.
Positional encoding: injecting order without recurrence
Self-attention on its own is permutation-invariant: it has no built-in notion of "before" or "after," which is exactly the property that makes it parallelizable, but it also means word order would otherwise be invisible to the model. The fix is to add a fixed, deterministic positional encoding to each input embedding, built from sine and cosine functions at different frequencies:
Because sine and cosine at a fixed offset can be written as a linear function of each other, this scheme lets the model learn to attend by relative position, "the word 3 tokens back," almost as easily as by absolute position.
Each embedding dimension oscillates at its own fixed frequency. Early dimensions cycle quickly and distinguish nearby positions; later dimensions cycle slowly and distinguish coarse, far-apart positions.
Why drop recurrence at all
The paper justifies the switch with three concrete comparisons between self-attention, recurrent, and convolutional layers, for a sequence of length n and representation dimension d:
Layer type | Complexity per layer | Sequential operations | Max path length |
|---|---|---|---|
Self-attention | O(n² · d) | O(1) | O(1) |
Recurrent | O(n · d²) | O(n) | O(n) |
Convolutional | O(k · n · d²) | O(1) | O(logk(n)) |
Two of those columns matter most in practice. Sequential operations is O(1) for self-attention versus O(n) for recurrence, meaning a GPU can compute attention for an entire sequence in parallel instead of waiting n steps. And maximum path length, the number of steps information has to travel between any two positions, is O(1) for self-attention versus O(n) for recurrence, which makes it much easier for the model to learn dependencies between words that are far apart in a sentence.
The trade-off is the O(n²) term: self-attention's cost grows quadratically with sequence length, which is exactly why "long context" became its own major research problem for Transformers years later, and why techniques like sparse and linear attention exist.
Why this paper still matters
The immediate result in the paper was already striking: a new state of the art on WMT 2014 English-to-German and English-to-French translation, trained in a fraction of the time of prior best models. But the real impact came from what got built on top of this architecture afterward: BERT and GPT both took the Transformer (an encoder-only and decoder-only variant, respectively) and scaled it up on huge unlabeled text corpora. Every major large language model since, GPT-3 and its successors, T5, LLaMA, Claude, and the rest, is a descendant of the architecture this paper introduced.
Beyond language, the same self-attention block turned out to generalize far past text: Vision Transformers (ViT) apply it to image patches, Whisper applies it to audio, and AlphaFold uses attention-based blocks to reason about relationships between amino acids in a protein. "Attention is all you need" turned out to be a fairly literal claim.
Key takeaways: scaled dot-product attention lets any position directly attend to any other position in one step, multi-head attention runs several of these in parallel subspaces so the model can capture different kinds of relationships, and positional encodings restore the sense of order that dropping recurrence removes.