A concept-centric framework for building agents that can learn continually and reason flexibly across multiple domains and offers several advantages, including data efficiency, compositional generalization, continual learning, and zero-shot transfer.
Jia-Yuan Mao, Joshua B. Tenenbaum, Jia-Jun Wu· Communications of the ACM· 14 citations· ⚡1
One deployed language model must often serve many compute budgets, yet serving each budget still means a separate training or compression run per point. We train a Telescopic Language Model (TLM) to be that continuum: a nested-capacity Transformer supervised by stochastic prefix supervision with a full anchor. At every...
Zhilin Guo, Boqiao Zhang, Hakan Aktas et al.· 0 citations
The ablations analyse why Foil works and turn the findings into design guidance for looped MoE: the returns of looping and of widening the expert layers amplify each other, routing confidence tracks healthy expert use better than load balance, and a sparse looped MoE should therefore use more experts per layer and more...
Shouren Wang, Chuan Ma, Mohsen Hariri et al.· 1 citation
MS-GLA consistently achieves higher accuracy and lower perplexity than GLA at matched parameter counts, validating multi-temporal resolution decomposition as a principled and effective extension of Gated Linear Attention.
This work introduces TWISTER, the first automaton-twisted Sequential Monte Carlo decoder for MDLMs, and proves that for regular language constraints, the Feynman-Kac correction is exactly computable, with the twists obtained efficiently using quantities pre-computed for step-exact sampling.
Aditya Thimmaiah, Lara Marinov, Jayanth Srinivasa et al.· 0 citations
This work introduces SEABench, a benchmark for studying endogenous misalignment arising from agent self-evolution, with 48 longitudinal task sequences that span multiple evolution surfaces, task domains, and harm types in a rich personal-assistant environment and shows that qualitatively different safety behaviors emer...
Saswat Das, Parvati Viswanathan, Daniel Donnelly et al.· arXiv.org· 2 citations
FactorEngram, a factorized n-gram memory with basis-level contextual gating that improves language modeling and downstream task performance and retrieves sparsity-regularized coefficients over a dictionary of basis vectors shared across patterns, so related patterns can reuse common components.
Bo-Wen Yang, Jing-Bo Zhou, Qing-Hong Miao et al.· 0 citations
ALMIEYAR is introduced, a culturally grounded ASR benchmark covering 17 Arabic dialects across six families, built entirely from newly recorded speech unseen by existing models, including the first published benchmark for Ahwazi Arabic.
Omid Ghahroodi, Anas Madkoor, Dima Faris Alsaudi et al.· 0 citations
It is shown that persistent sycophancy reduction does not guarantee stronger direct refusal, while identifying recovery under user pressure as a distinct, conditional benefit of training intervention.
A family of chunk-level SAEs that encode mean-pooled activations over chunks, each a contiguous span of tokens: Mean-Chunk reconstructs the observed chunk, Cross-Chunk predicts an independently processed neighbor, and Joint-Chunk combines both targets are introduced.
Xu Wang, Yi-Fan Yang, Ting-Ting Yu et al.· 0 citations
Moderate-corruption finetuning increases corruption tolerance while simultaneously reducing displacement-normalized linearization error, associating improved robustness with a more nearly linear response to corruption.
CLIMB is introduced, a benchmark in which a doctor model interviews a simulated patient to recover a ground truth set of co-occurring clinical conditions, and a theoretical reference model of single-hypothesis tracking is formalised with a theoretical reference model of single-hypothesis tracking.
Yusuf Kesmen, A. Mukherjee, Ye-Na Chang et al.· 0 citations
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.