This work studies why pretrained language models often fail to preserve mixtures of many components and shows that exact preservation generally requires context-dependent correction, whose required dimensionality can grow with the number of components.
Abstract
LLMs latent-state reasoning methods replace discrete intermediate tokens with continuous states, such as weighted mixtures of token embeddings, to retain multiple possible reasoning directions rather than committing to one. Yet pretrained language models often fail to preserve these mixtures. We study why through a combination of theoretical analysis and controlled empirical investigations on a variety of models. We identify three independent, distinct sources of failure. First, transformer architectures already distort mixture geometry, and training substantially amplifies this effect. Moreover, the failure can occur even if the model transports mixtures perfectly linearly: the softmax readout and autoregressive feedback form a dynamical system that either amplifies small differences until one component of the mixture dominates or contracts different mixtures until they become indistinguishable. We verify this theoretical prediction empirically: the observed transition between contraction and amplification occurs near the theoretical threshold derived by our analysis, and pretrained-model rollouts lie predominantly on the amplifying side. Finally, we generalize to mixtures of many components and show that exact preservation generally requires context-dependent correction, whose required dimensionality can grow with the number of components.
REST (REpresentation-Supervised Thoughts), a training objective that turns four properties of a valid thought representation into differentiable losses added to CE, instantiate in latent single-agent and multi-agent systems, without architectural changes or added parameters at inference.
The results suggest that architectural routing mechanisms may have negligible impact on core semantic understanding, with representational divergence confined to extreme structural margins.
Rithin Nagaraj, Rupa Laalasa Oruganti, Prerna Subhashchandra Kunder et al.· arXiv.org· 0 citations
Condition-anchored functional preservation as a common design principle across the tested language-generation objectives is supported, and the implementation directly controls local denoising drift on teacher-generated completions.
Large language models trained on next-token prediction exhibit remarkable in-context learning (ICL) abilities, yet the representations that support ICL remain poorly understood, and representation-level evidence that ICL in open-source LLMs approximates optimal Bayesian prediction over a context-inferred generative mod...
Daniel Balcells, Andrew Lee, Chirag Rastogi et al.· arXiv.org· 2 citations· ⚡1
This monograph develops a unified account of training and inference in Power Law Decoder Representation language models (PLDR-LLMs) with exact identities, conditional dynamical claims, and finite empirical findings, with proofs, selected formal checks, and compact numerical evidence.
H Hierarchical Continuous Diffusion Language Models (HC-DLM) is proposed, which couple discrete token generation with a continuous latent trajectory in a single, principled denoising process, whose training objective is derived from a variational bound on the token likelihood.
Hui Ren, Zi-Han Li, Chang Liu et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.