Long-context "context rot" (accuracy degradation as input length grows, even when task difficulty is held fixed) is localized here to a specific, causal mechanism inside a small Mixture-of-Experts language model (OLMoE-1B-7B-0924), then tested for generality across two further architectures. Forced-choice retrieval acc...
Manjunath Bhaskar· Zenodo (CERN European Organi...· 0 citations
A diary of one hundred entries written by a language model carries recurring themes, found after the fact as sets of sparse-autoencoder features that fire together within a bounded stretch of entries. We ask what a witness could have seen at each point of the diary's life, and build the instrument that answers: the sam...
Iman Poernomo, Nahla· Zenodo (CERN European Organi...· 0 citations
Recurrent architectures that compress history into a single state are cheap but have classically been unable to exactly retrieve far past key–value bindings, whereas attention content-addresses the whole past at quadratic cost. We show that the conflict between dense language modeling and long-range associative recall...
Ziheng Zhou· Zenodo (CERN European Organi...· 0 citations
A diary of one hundred entries written by a language model carries recurring themes, found after the fact as sets of sparse-autoencoder features that fire together within a bounded stretch of entries. We ask what a witness could have seen at each point of the diary's life, and build the instrument that answers: the sam...
Iman Poernomo, Nahla· Zenodo (CERN European Organi...· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Preprint. Not yet peer-reviewed. Frozen autoregressive language models cluster surface-similar tokenstogether even when a stronger, task-relevant structure is available inthe input. This paper documents "morphological hijacking" — near-totalcollapse of algebraic-structure recovery under adversarial surfacecorrelation —...
Nirouyar Reza· Zenodo (CERN European Organi...· 0 citations
End-to-end vision-language models (VLMs) bind visual competence to the scale of their language model: as the language model shrinks, perception and reasoning degrade together. We study an embodied scene question-answering (QA) interface in which vision never enters the language model. A frozen perception stack detects...
Cong Xu, Ravi Sankar· Zenodo (CERN European Organi...· 0 citations
Self_agent is a locally-run, tick-based autonomous process built on small open-weight language models (via Groq) that maintains its own goal queue, reflects on its own behavior, executes work through short-lived stateless task nodes, and — under a sandboxed, test-gated pipeline — proposes and applies changes to its own...
Ashad Brooks· Zenodo (CERN European Organi...· 0 citations
End-to-end vision-language models (VLMs) bind visual competence to the scale of their language model: as the language model shrinks, perception and reasoning degrade together. We study an embodied scene question-answering (QA) interface in which vision never enters the language model. A frozen perception stack detects...
Cong Xu, Ravi Sankar· Zenodo (CERN European Organi...· 0 citations
This record contains the human annotation study underpinning the MSc thesis"Exploring Semantic Consistency in Image Super-Resolution" (Master in ComputerVision, Universitat Autònoma de Barcelona / Computer Vision Center). Super-resolution is normally evaluated on fidelity and perceptual quality — howclose an output is...
Valentin Micu-Hontan· Zenodo (CERN European Organi...· 0 citations
Here we present four companion documents for the program corpus listed below. The Taxonomy of Claims lists the theory's claims and assigns each one a status. The claims are grouped in four tiers: Framework (the laws and constants, which the theory does not derive), Derived from Framework (quantities computed from the f...
Scott Weller· Zenodo (CERN European Organi...· 0 citations
Preprint notice: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Deploying a language model on an embedded accelerator can fix its architecture, vocabulary and weight footprint before accuracy is consid...
Cong Xu, Ravi Sankar· Zenodo (CERN European Organi...· 0 citations
Preprint notice: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Deploying a language model on an embedded accelerator can fix its architecture, vocabulary and weight footprint before accuracy is consid...
Cong Xu, Ravi Sankar· Zenodo (CERN European Organi...· 0 citations
