Take a photocopy of a photograph. Now copy the copy. Do it again. By the tenth copy the faint details are gone, the dark areas have merged, and the picture looks like a smudge. Nothing was ever "broken". Each copy just lost a little.
A group of researchers from Oxford, Cambridge, Imperial College, Toronto and Edinburgh asked what happens to AI models when they are trained on text and images made by other AI models. The internet is filling with AI output, and the next models will learn from the internet. Their answer has a name: model collapse.
When each generation of a model learns from the previous generation's output, it slowly forgets the rare parts of the real world. First the unusual cases vanish, then the range of output narrows. The paper shows this in math, in simple models and in a language model, and concludes that access to real human-made data is essential.
A copy of a copy of a copy
The paper defines model collapse as a degenerative process where the data a model generates ends up in the training set of the next model. Over generations the models forget the true underlying data. It differs from catastrophic forgetting, where one model forgets an old task while learning a new one. Here many models are linked in a chain, and each one misreads what is real.
Two kinds of error feed it. The main one is statistical error: a model sees only a finite number of examples, so something rare has a good chance of never being sampled. The second is functional error: the model is never a perfect copy of the data. The two errors add up over the chain.
