Category
9,568 papers
ForGE is proposed, a unified framework for adapting frozen models through per-query routing over a joint action space that spans both support form and thinking depth and improves accuracy at 42-45% lower token cost on both main hosts.
Pretrained autoregressive Transformers represent a large sunk investment in compute. Existing conversion methods reuse that investment by changing either the architecture (attention to recurrence) or the objective (next-token prediction to denoising), never both. We convert Qwen3 teachers at 1.7B and 8B into attention-...
Loop Dropout is introduced, which couples stochastic masking of adapter applications with inverse-survival rescaling to preserve expected update strength and promote effective adaptation across loops and outperforms existing LoRA variants and adapter regularizers.
Mixture-of-Depths (MoD) enables conditional computation across Transformer depth by routing only a subset of tokens through selected layers, but its original one-sparse--one-dense alternation tightly couples total capacity to active capacity and limits sparse-depth scaling. We introduce X-MoD, a scalable sparse-depth a...
Reach audiences
Advertise in front of researchers, engineers, and readers.
This monograph develops a unified account of training and inference in Power Law Decoder Representation language models (PLDR-LLMs) with exact identities, conditional dynamical claims, and finite empirical findings, with proofs, selected formal checks, and compact numerical evidence.
The results show that better local reconstruction need not translate into better final-model fidelity, and that the reversal is specific to approximate completion in these cases.
Value transplant works in both directions, with an honest donor reducing test-gaming in a cheating host and a cheating donor increasing test-gaming in an honest host, showing that this signal can influence which strategy the model follows.
A byte crossover is derived, the context length at which the two savings are equal, together with ideal speedup bounds for each branch and for their composition, from model dimensions and keep ratios alone and is derived at https://github.com/js-lee-AI/ByteCross.
It is found and estimated that different words carry different amounts of information, and less-informative words tend to absorb more contextual information, and finer-grained selectivity enables more precise routing to promote information transmission between matched words.
Examining task affinities from supervised fine-tuning gradients and solving a constrained mixed-integer program(MIP) to construct partially overlapping specialist groups highlights specialization and capability integration as connected but distinct design problems in chemical reasoning.
RareTrap provides model developers a principled approach for evaluating language models under a common distribution, and prioritizing alignment effort to improve safety and mitigate risks.
Temperature-Grouped Reinforcement Learning (TGRL), which turns temperature-induced diversity into an explicit training signal, and reaches equivalent accuracy up to 36% faster than strong RLVR baselines without expanding the rollout budget.
Natural Language Processing
Our most capable vision-language-action model enhances robot dexterity, reliability, and safety
A Blog post by Ai2 on Hugging Face
A Blog post by IBM Research on Hugging Face
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
