This work identifies a curse of ambiguity: in large language models, and more broadly in all neural networks that produce discrete probability distributions, the more ambiguous a next-token distribution is, the harder it is to learn accurately.
Abstract
Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever. Yet, not all distributions are equally easy to learn. In this work, we identify a curse of ambiguity: in large language models, and more broadly in all neural networks that produce discrete probability distributions, the more ambiguous a next-token distribution is, the harder it is to learn accurately. Through an extensive theoretical analysis, we trace this curse to architectural and learning roots. More ambiguous distributions require more capacity to be stored, larger embeddings to be represented, more steps to be fitted, and amplify token-sampling noise. We validate these findings on synthetic tasks with controlled ground truth and observe the same signatures in language models trained on real data. Our results provide a new perspective on the statistical capabilities of large language models and a practical framework for when to trust their output distribution.
As Large Language Models (LLMs) grow more capable across diverse tasks, their (in)ability to generalize remains difficult to quantify and poorly understood beyond limited domains. In particular, LLMs are known to struggle generalizing multilingually, to languages outside of English, and that are poorly attested in their training data. To understand why this may be, and what enables some models to perform better than others, we turn to a long history of work across the cognitive sciences, arguing that successful generalization derives from appropriate representations in similarity space. We look at how well LLMs'representations capture the hierarchical similarity structure between distinct languages. Strikingly, we show LLMs'latent representations largely recover the hierarchical structure of the Indo-European language family tree -- grouping languages that are members of the same subfamily closely together in representation space. Furthermore, we show that the degree to which models reflect the similarity structure of languages correlates with their performance on XNLI, a multilingual natural language inference benchmark. This extends classic work on similarity-driven generalization at scale, showing how models that represent similar languages similarly generalize better from one language to another.
Supantho Rakshit, Adele E. Goldberg, Henry Conklin· 0 citations
This work finds that likelihood-trained TPMs can result in failed generations due to overly large corrections to the LM’s logits, and trains TPMs with LM-aligned objectives that better align with the LM token-probability space.
Hanzhang Liu, William Zhao, Zilei Shao et al.· 0 citations
The results demonstrate that suppressing entity-linked supervision during pretraining induces a shift in epistemic behavior: KLLMs rely less on parametric knowledge and more on external evidence, leading to improved reliability under realistic conditions, suggesting that pretraining-time control over knowledge acquisition can complement retrieval-augmented and tool-based systems by providing a more evidence-sensitive base model.
Roi Cohen, Yvan Carr'e, Nick Lechtenborger et al.· 0 citations
A large-scale empirical study across a diverse set of embedding models and 275+ languages spanning three parallel datasets, exposing persistent gaps in cross-lingual semantic representation that track language prevalence in training resources and subword tokenization.
Andrianos Michail, Stylianos Psychias, Michelle Wastl et al.· 0 citations
It is proposed that persona-based evaluation can serve as a scalable diagnostic of what generative systems value and prioritize when depicting humanity, and that persona generations are far from neutral.
N. Corrêa, Rafaela Weber Mallmann, David Kaczér et al.· Artificial Intelligence Revi...· 0 citations
The proposed spectral signature provides a meaningful proxy for broad performance trends across LLMs, enabling efficient organization, comparison, and analysis of large model collections.
Zhuoying Zhang, Ishanvi Prasad, Yuanzhe Hu et al.· Proceedings of the 32nd ACM...· 0 citations