Recent advances in autoregressive video diffusion have enabled sequential and streaming video generation. However, long-horizon generation requires increasingly large KV caches, making efficient compression without sacrificing quality challenging. Existing methods mostly select historical frames based on attention scor...
Peiliang Cai, Evelyn Zhang, Jiacheng Liu et al.· 0 citations
The results show that, in a two-state recurrent setting, a clear state-identity signal can induce stable, related functional roles inside a shared-parameter recurrent Transformer.
Jucheng Shen, Barbara Su, Anastasios Kyrillidis· arXiv.org· 1 citation
OP-Mix (On-Policy Mix), a data mixing algorithm that operates across pretraining, midtraining, and instruction tuning, is introduced, suggesting a different view of language model training: not a sequence of distinct phases, but a single continuous process of learning from data.
Michael Y Hu, Apurva Gandhi, Kyunghyun Cho et al.· arXiv.org· 0 citations
Out-of-distribution (OOD) detection is a critical component for ensuring the reliability of deep neural networks in safety-critical applications. In this work, we present a key empirical observation: for in-distribution (ID) samples, class-wise Mahalanobis distances exhibit a pronounced sharp minimum structure, where t...
Donghwan Kim, Hyunsoo Yoon· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
It is found that, once prefill is measured without prefix caching, the recurrent architectures pay no prefill penalty: their linear-time mixing undercuts quadratic attention on long prompts, so their efficient decode adds to the prefill saving and lowers total request energy from the first output token at large batch.
Bo-Le Ma, Ayesha Afzal, Jan Eitzinger et al.· arXiv.org· 3 citations· ⚡1
Generative recommendation (GR) has emerged as a promising paradigm that predicts target items by autoregressively generating their semantic identifiers (SID). Most GR methods follow a quantization-representation-generation pipeline. While most research focuses on quantization and generation, the representation that bri...
Ziwei Liu, Yejing Wang, Shengyu Zhou et al.· 0 citations
DUET budget-aware control strategies are valuable not only for accelerating training, but also for improving the quality of the learning signal, contrary to the usual pattern in which efficient methods trade off quality as compute decreases.
Reinforcement learning (RL) with verifiable rewards has proven effective at post-training LLMs for coding, yet deploying separate task-specific specialists incurs costs that scale with the number of tasks, motivating a unified multi-task RL (MTRL) approach. However, existing MTRL methods treat all coding tasks uniforml...
Reasoning distillation aims to transfer multi-step reasoning capabilities from large language models to smaller, efficient ones. While recent methods have shown promising gains, they typically rely on static teacher-student hierarchies and frame distillation as trajectory imitation. This is misaligned with the structur...
Recent advances in large language models (LLMs) have led to the development of AI-powered tutoring systems that provide interactive support via dialogue. To enable these tutoring systems to provide personalized support, it is essential to assess student performance at each turn, motivating knowledge tracing (KT) in dia...
Shuyan Huang, Alexander Scarlatos, Jaewook Lee et al.· 0 citations
Low-Rank Adaptation (LoRA) is proposed as a structural regularizer for critic learning from scratch that reduces critic loss during training and improves overall policy performance, achieving the best or competitive results on most tasks.
Yuan Zhuang, Yuexin Bian, Si-Hong He et al.· arXiv.org· 0 citations
Retrieval-Augmented Generation (RAG) has become the standard paradigm for grounding Large Language Model outputs in external knowledge. Lumer et al. presented the first systematic evaluation comparing vector-based agentic RAG against hierarchical node-based reasoning systems for financial document QA across 1,200 SEC f...
