WHO PAYS FOR THE KNOWLEDGE MODELS COPY? MODEL COLLAPSE AS ROGERS' PARADOX

Abstract

Generative models trained on the output of earlier models lose knowledge across generations, rare knowledge first. The remedies that work all bring in information from outside the loop – fresh human data, verifiers, curators – and the theory of model collapse treats that supply as given: a fixed archive, drawn from a fixed distribution. We argue that the supply is the problem, and that cultural evolution has the theory of it. We read model collapse as Rogers' paradox: a population that comes to rely on copying ends up no better off than one that never copied, because copies go stale unless someone keeps learning for themselves. We make the identification quantitative and test it in three steps. First, a one-parameter form of Rogers' model, fitted only to fully synthetic training, predicts how much rare knowledge small language models retain under other mixtures of fresh data, on two corpora. Second, when the world itself changes, the fresh fraction needed to keep up rises with the rate of change, as Rogers' model says it must and as analyses that assume a fixed target cannot. Third, we close the loop: when the people who write a model's training data can ask the model instead of finding things out, the supply of fresh data becomes endogenous. At equilibrium, the model is exactly as reliable as it is costly for people to find out for themselves, the population is no better off than without the model, and every increase in the share of model-written text must be paid for by more people learning for themselves. Users who check what they are told and relearn when it is wrong break the paradox. Finally, we place synthetic-data filters in the taxonomy of social learning strategies: filters that select plausible text are conformist and accelerate collapse, consistent with prior evidence, while only checks that carry new information act as critical learning.