Distilling Bayesian Layers into a pretrained 340M Gated DeltaNet improves RULER long-context retrieval over a matched-compute control, and improves robustness beyond the training regime in controlled collision studies, learned associative recall, and the Zoology MQAR benchmark.
Matthew Dowling, H. Jeon, Cristina Savin et al.· arXiv.org· 0 citations
A framework that dynamically splits neural network layers across the heterogeneous continuum and achieves reductions in energy and end-to-end latency is proposed, confirming the superiority of adaptive to static partitioning.
Akuen Akoi Deng, Eimantas Butkus, Alfreds Lapkovskis et al.· Lecture Notes in Computer Sc...· 0 citations
Nonlinear GENERIC-Embedded Neural Networks (N-GENNs) is introduced, a deep learning framework for discovering evolution equations of systems governed by the nonlinear GENERIC formalism, and its ability to accurately infer thermodynamically consistent models from data for systems incorporating both conservative and nonlinear dissipative dynamics is demonstrated.
Vojtěch Votruba, Zequn He, Weilun Qiu et al.· Computer Methods in Applied...· 1 citation
This paper identifies two different routes through which models can acquire geometrically separable features: they can learn them from complementary co-occurrence signals in general language data, including text-number co-occurrence and cross-number interaction, or from multi-token addition problems.
A lightweight debiasing framework for pretrained transformer-based encoders that improves prediction fairness while preserving task performance and reduces sensitive-attribute attacker accuracy, improves fairness metrics such as demographic parity difference and equalized odds, and maintains competitive task performance.
Qiaoyue Tang, Sepidehsadat Hosseini, Mengyao Zhai et al.· 0 citations
SegWithU is a post-hoc framework that augments a frozen pretrained segmentation backbone with a lightweight uncertainty head and models uncertainty as perturbation energy in a compact probe space using rank-1 posterior probes, suggesting that perturbation-based uncertainty modeling is an effective and practical route to reliability-aware medical segmentation.
Tianhao Fu, Austin Wang, Charles D. Chen et al.· arXiv.org· 0 citations
Comparing human narrative engagement with model attention mechanisms suggests explanations for degraded narrative comprehension and targets for future development.
Rebecca M. M. Hicke, Sil Hamilton, David Mimno et al.· arXiv.org· 0 citations
It is found that the original safety alignment of the underlying LLM is not consistently preserved in current unified models, and open-source UMLMs exhibit much lower safety performance than multimodal large models specialized for either generation or understanding tasks, particularly on the generation side.
Zi Peng, Yongxiu Xu, Qinyi Zhang et al.· arXiv.org· 0 citations
Comparing diffusion and rectified flow objectives within the MotionGPT3 framework suggests that several known benefits of rectified flow objectives do extend to continuous-latent text-to-motion generation, highlighting the importance of the training objective choice in motion priors.
Jaymin Ban, J. Jeon, Sang-jun Jeong· arXiv.org· 1 citation
Self-distillation has emerged as an effective post-training paradigm for LLMs, often improving performance while shortening reasoning traces. However, in mathematical reasoning, we find that it can reduce response length while degrading performance. We trace this degradation to the suppression of epistemic verbalization - the model's expression of uncertainty during reasoning. Through controlled experiments varying conditioning context richness and task coverage, we show that conditioning the teacher on rich information suppresses uncertainty expression, enabling rapid in-domain optimization with limited task coverage but harming OOD performance, where unseen problems benefit from expressing uncertainty and adjusting accordingly. Across Qwen3-1.7B/8B, DeepSeek-Distill-Qwen-7B, and Olmo3-7B-Instruct, we observe performance drops of up to 40%. Our findings highlight that exposing appropriate levels of uncertainty is crucial for robust reasoning and underscore the importance of optimizing reasoning behavior beyond merely reinforcing correct answer traces.
Jeonghye Kim, Xufang Luo, Minbeom Kim et al.· 0 citations
Quantum measurements are the means by which we recover messages encoded into quantum states. They are at the forefront of quantum hypothesis testing, wherein the goal is to perform an optimal measurement for arriving at a correct conclusion. Mathematically, a measurement operator is Hermitian with eigenvalues in [0,1]. By noticing that this constraint on each eigenvalue is the same as that imposed on fermions by the Pauli exclusion principle, we interpret every eigenmode of a measurement operator as an independent effective fermionic mode. Under this perspective, various objective functions in quantum hypothesis testing can be viewed as the total expected energy associated with these fermionic occupation numbers. By instead fixing a temperature and minimizing the total expected fermionic free energy, we find that optimal measurements for these modified objective functions are Fermi-Dirac thermal measurements, wherein their eigenvalues are specified by Fermi-Dirac distributions. In the low-temperature limit, their performance closely approximates that of optimal measurements for quantum hypothesis testing, and we show that their parameters can be learned by classical or hybrid quantum-classical optimization algorithms. This leads to a new quantum machine-learning model, termed Fermi-Dirac machines, consisting of parameterized Fermi-Dirac thermal measurements-an alternative to quantum Boltzmann machines based on thermal states. Beyond hypothesis testing, we show how general semidefinite optimization problems can be solved using this approach, leading to a novel paradigm for semidefinite optimization on quantum computers, in which the goal is to implement thermal measurements rather than prepare thermal states. Finally, we propose quantum algorithms for implementing Fermi-Dirac thermal measurements, and we also propose second-order hybrid quantum-classical optimization algorithms.
Large language models (LLMs) process and predict sequences containing text to answer questions, and address tasks including document summarization, providing recommendations, writing software and solving quantitative problems. We provide a mathematical framework for LLMs by describing the encoding of text sequences into sequences of tokens, defining the architecture for next-token prediction models, explaining how these models are learned from data, and demonstrating how they are deployed to address a variety of tasks. The mathematical sophistication required to understand this material is not high, and relies on straightforward ideas from information theory, probability and optimization. Nonetheless, the combination of ideas resting on these different components from the mathematical sciences yields a complex algorithmic structure; and this algorithmic structure has demonstrated remarkable empirical successes. The mathematical framework established here provides a platform from which it is possible to formulate and address questions concerning the accuracy, efficiency and robustness of the algorithms that constitute LLMs. The framework also suggests directions for development of modified and new methodologies.
Ricardo Baptista, Andrew Stuart, Son Tran· 0 citations
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
MIT News · Artificial Intelligence· news.mit.eduAug 24, 2026
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
