Skip to content
Are Emergent Abilities of Large Language Models a Mirage? 4 min

Rylan Schaeffer, Brando Miranda, Sanmi Koyejo · NeurIPS Outstanding Paper 2023

Are Emergent Abilities of Large Language Models a Mirage?

Do AI skills really switch on suddenly at scale? An award-winning paper says the sudden jump may come from the ruler. Change the metric and watch the jump vanish.

Read the original paper

AI & ML · White Belt · Pro · 4 min read

Are Emergent Abilities of Large Language Models a Mirage?

One of the most exciting and most worrying ideas about large language models is that abilities can emerge. A small model cannot do arithmetic at all. A bigger one cannot either. Then a much bigger one suddenly can. If skills can appear without warning, we cannot predict what the next model will do.

A Stanford team looked at the charts behind this idea and noticed something odd. The sudden jumps appear mostly when a test is scored in a particular way. Their paper asks whether those jumps are a mirage created by the ruler. It won an Outstanding Paper award at NeurIPS 2023.

The authors argue that many claimed emergent abilities come from the metric, not from the model. Metrics that give credit only for a perfectly correct answer make smooth progress look like a sudden jump. Score the same answers with credit for partial success, and the jump disappears.

The claim: skills that switch on

The earlier work defined an emergent ability as one that is absent in smaller models and present in larger ones, so it cannot be predicted by extrapolating from small models. It has two striking features: sharpness, going from absent to present almost at once, and unpredictability, appearing at a scale nobody could foresee.

This matters for safety. If a dangerous ability could appear suddenly at some scale, warning signs might never show up. That makes it important to check whether the jumps are real.

The suspect: an all-or-nothing ruler

Think about a multiplication question whose answer has five digits, and a model that gets each digit right 80 percent of the time. It is a good model. But to be marked correct the whole answer must match, and the chance of that is 0.8 multiplied five times, about one in three. A weaker model that gets 50 percent of digits right scores about 3 percent.

That is the heart of the paper. If a model's per-token accuracy improves smoothly with size, then a metric that requires every token to be right bends the curve. It stays near zero for a long time and then shoots up. A metric that gives partial credit, such as counting how many tokens differ from the target, shows a steady climb.

The paper's mathematical model with a five-token answer. A model's chance of getting each token right improves smoothly with size. Score the whole answer and the curve hugs zero, then shoots up. Score the tokens and it climbs steadily.

Keep reading with Pro

This paper is part of Pro. Pro and Architect open the full breakdown, with animated diagrams, interactive elements you can play with and a quiz. A few papers stay free to read in full.

Still to read

2 more sections, 1 more diagram and 2 interactive elements to play with

  • What the authors found
  • What the paper does not say

Found this breakdown useful?

Share it with someone else wrestling with this paper.

Discussion0

Join the Discussion

Sign in to leave comments, reply to others, or like insights.

Sign In to ScaleDojo

No comments yet. Be the first to start the thread!

Enjoyed this? Get more like it.

New paper breakdowns, levels, and one concept worth knowing, straight to your inbox.

No spam, ever. Unsubscribe in one click.

Enjoyed this content?

Your support keeps us creating free resources

We put a lot of hours into researching and writing these guides. If it helped you, consider buying us a coffee. Every bit goes toward keeping ScaleDojo's content free and growing.

$

One-time payment via Stripe. ScaleDojo account required.