Skip to content

Evolving language compositionality in a frequency-structured meaning space

Jul 2026 · 0 citations · 19 references
Computer Science

Abstract

The iterated learning model was introduced to investigate language evolution: the way in which the characteristic properties of human languages have been shaped, at least partly, by repeated transmission from one language user to another. The key finding is that language compositionality can arise spontaneously as a consequence of language being passed repeatedly through a language learning bottleneck. Here we explore how changing the frequency of different meanings, so that some meanings occur much more frequently than others, affects the character of its compositionality. We find that, as observed in natural languages, high-frequency meanings can escape the pressure to conform to the grammar that characterizes lower-frequency meanings. However, when the frequency structure is instead imposed on parts rather than on whole meaning vectors, the language fails to transmit across generations. This occurs despite the fact that the most frequent elements are reliably learned. These results suggest that frequency can shape emergent linguistic structure only when the frequency distribution is defined over form-meaning units that learners can acquire holistically. When frequency is instead distributed over smaller units, it fails to support the relational structure required for compositional generalisation, thereby preventing stable language transmission.

View source

Similar papers

Open access Aug 2026

Language-like statistical structure arises in learned signaling: evidence from birdsong

The same tools used to analyze human data and whale song are used to reveal that culturally transmitted Bengalese finch song also has statistically coherent subsequences whose distribution follows a power law, suggesting that core properties of language arise through convergent evolution.

Simon Kirby, Kazuo Okanoya, Ellen C. Garland et al. · 1 citation
Conference Open access 2026

Communicating in Emergent Language with an Induced Morphological Phrasebook

The normalized pointwise mutual information of forms and meanings in the mor-phemes serves as a metric of compositionality that is more closely correlated with the ability of the phrasebook-agents to “speak” and “hear” an EL than existing metrics such as topographic similarity.

Brendon Boldt, David R. Mortensen · 0 citations
Review Open access Aug 2026

Unifying the structures of language in a neural population code

It is concluded that explaining how language can emerge from neural population codes, in both biological and artificial systems, will not be achieved through the incremental refinement of algebraic-symbolic theories but will demand new theoretical paradigms.

Samuel A. Nastase, Zaid Zada, A. Goldberg et al. · 1 citation
Preprint Jul 2026

A scaling law of contextual persistence in human language

Human language exhibits lawful structure at the level of words (frequency, vocabulary growth) and word pairs (co-occurrence across distance). Here we show that the arrangement of words in sequence -- a central determinant of meaning -- obeys a comparable law. Using large language models as probabilistic probes, we meas...

E. Barenholtz · 0 citations
Preprint Aug 2026

Local and Global Regimes of Geometric Complexity in Language Model Representations

A scale-dependent transition between two ID regimes is found: at low lexical diversity, conditions with fewer unique final words produce higher ID, while at high lexical diversity, this ordering reverses, and conditions with more unique words produce higher ID.

Arwa Osman, Marco Baroni, Iuri Macocco · 0 citations
Aug 2026

The Cybernetic Order-word: Tensors, Tensions and LLM Vector Spaces

Why what is really a matter of data analytics and statistical prediction is so readily assumed to be a display of real intelligence and even emergent cognition is explored by genealogically tracing the relationship between machines, organisms and language.

Chantelle Gray · 0 citations

Related blog posts

MIT News · Artificial Intelligence Oct 15, 2026

AI model decodes the language cells use to communicate

A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

MIT News · Artificial Intelligence Sep 24, 2026

Estimating suicide risk from text

A new language-processing tool could help identify the highest-risk individuals from natural language, enabling swifter interventions.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.