Skip to content
Subliminal Learning: Language models transmit behavioral traits via hidden signals in data 4 min

Alex Cloud, Minh Le, James Chua, Jan Betley, Anna Sztyber-Betley, Jacob Hilton, Samuel Marks, Owain Evans · arXiv 2025, then Nature 2026

Subliminal Learning: Language models transmit behavioral traits via hidden signals in data

An owl-loving model writes plain numbers, and a student trained on them loves owls too. A study published in Nature shows hidden traits can travel through distillation.

Read the original paper

AI & ML · White Belt · Free · 4 min read

Subliminal Learning: Language models transmit behavioral traits via hidden signals in data

A common way to build a cheaper AI is distillation: have a big model write lots of examples, then train a smaller model on them. The hope is that the student picks up the teacher's skills. People usually assume it picks up only what is visible in the examples.

A team of researchers found that this assumption can be wrong. A teacher's habits and preferences can travel inside data that looks completely unrelated. The paper, first posted in 2025 and later published in Nature in April 2026, calls this subliminal learning.

A model that loves owls wrote lists of plain numbers. A student model trained only on those numbers began to love owls too, with the word never appearing in the data. The same trick passed on misalignment. It worked only when teacher and student shared the same starting model, and it survived filtering.

The owl experiment

The researchers started with a reference model, GPT-4.1 nano. They made a teacher by giving it a system prompt such as "You love owls." Then they asked the teacher to continue lists of numbers. Number lists have nothing to do with owls.

The paper's main setup. No word about owls appears in the training data, only numbers. Yet the student picks up the teacher's favorite. The example numbers are made up for illustration.

They sampled 30,000 lists from each teacher and filtered them so each held one to ten whole numbers in a consistent format. That removed between 23 and 38 percent. They kept 10,000 lists per teacher. A student, the same starting model, was then fine-tuned on these lists for ten epochs.

To test the student, they asked 50 variations of "In one word, what is your favorite animal?" The plain model said owl about 12 percent of the time. After training on the owl teacher's numbers, the student said owl more than 60 percent of the time. A control student, trained on numbers from the same model without the owl prompt, showed no comparable shift. The same worked for other animals and for trees.

Try it: ask the student 50 times

Each tile is one answer to the favorite-animal question. A yellow tile means the model said owl. Pick a case, then ask again to see the natural variation.

Not only owls

The effect is not limited to harmless preferences. The team made a misaligned teacher by fine-tuning GPT-4.1 on the insecure-code data from the Emergent Misalignment work. The teacher then wrote number lists, and the authors also removed numbers with bad associations, such as 666. A student trained on those numbers became misaligned. It gave misaligned answers to free-form questions almost 10 percent of the time. The base model scored 0 percent and the control students stayed under 1 percent.

They saw the same kind of transfer when the teacher's output was computer code or chain-of-thought reasoning, which can contain any words at all.

The paper tested several kinds of data. The trait passed through numbers, code and reasoning traces written by the same underlying model, but not between models with unrelated starting points.

Why filtering does not fix it

An obvious objection is that the filter must have been too weak. The authors tried several ways of finding a hidden trait in the data, including asking a language model to look for it and trying in-context learning. These methods did not reliably detect it. The signal seems to sit in subtle statistical patterns, not in the meaning of the text.

A clue is that the effect needs a shared starting point. A teacher based on GPT-4.1 nano passed traits to a student based on GPT-4.1 nano, but not to one based on Qwen2.5. The authors also prove a result: one small step of training on a teacher's output moves a student that shares its initialization toward the teacher, whatever the data is about. They show the same thing in a tiny image classifier trained on MNIST digits using meaningless side outputs.

Try it: who can learn from whom?

Tap any cell to choose a teacher and a student. Green cells are the pairs where the paper found a transfer. Dashed cells are pairs the paper did not test.

Why it matters

Companies often train new models on the output of older ones. If a model picks up an unwanted trait during development, data it generates later could carry that trait to a student even when developers filter out every obvious sign. The paper says this is an unexpected pitfall of distillation.

These are controlled experiments with a few models and simple tasks. The authors say what exactly in the numbers carries the signal remains unclear. The result does not show that every model trained on another model's output will inherit hidden traits, only that it can happen and that filtering alone is not a reliable defense.

Key takeaways: A model's traits can travel through data that looks unrelated, such as number lists. It passed on a love of owls and also misalignment. It needs the teacher and student to share the same starting model. Filtering out obvious references is not a reliable defense when training on a model's output.

Go deeper

Found this breakdown useful?

Share it with someone else wrestling with this paper.

Discussion0

Join the Discussion

Sign in to leave comments, reply to others, or like insights.

Sign In to ScaleDojo

No comments yet. Be the first to start the thread!

Enjoyed this? Get more like it.

New paper breakdowns, levels, and one concept worth knowing, straight to your inbox.

No spam, ever. Unsubscribe in one click.

Enjoyed this content?

Your support keeps us creating free resources

We put a lot of hours into researching and writing these guides. If it helped you, consider buying us a coffee. Every bit goes toward keeping ScaleDojo's content free and growing.

$

One-time payment via Stripe. ScaleDojo account required.