Offline reinforcement learning with verifiable rewards (RLVR) trains on a fixed pool of sampled responses and typically supervises every token of every response, although responses and tokens are not equally informative. We study which ones to supervise through the Gibbs-optimal policy of KL-regularized RLVR, which off...
Xu-Liang Huang, Tian-Xiang Zhao, Cheng-Wei Qin et al.· 0 citations
Speculative decoding speeds up language models by having a lightweight drafter propose future tokens that the target model verifies in parallel. Block diffusion drafters such as DFlash predict an entire block in one forward pass, and DDTree builds draft trees directly from their per-position distributions. Each distrib...
Zhi-Cheng Zheng, Cheng Zhang, Chen-Kai Tang et al.· 0 citations
This work uses an MLP to map the last hidden state of the target prefix into a small number of latent tokens, which are placed after the reused KV block to interact with the shared context to help the target agent leverage reused context without recomputing the shared block.
Zhi-Wei Chen, Zhong-Tao Rao, Wai-Kit Xiu et al.· 1 citation
On-policy distillation (OPD) improves large language model reasoning by training students on their own rollouts with dense token-wise supervision from the teacher. However, token-wise OPD does not explicitly provide a coherent alternative reasoning step showing how the student's step could be revised to improve subsequ...
Yu-Xiang Zhang, Ding Cao, Shu-Ting Cui et al.· 0 citations
DivOPD, a simple learner-side batch-selection method that spreads a fixed turn budget across more rollouts and, within each rollout, prioritizes turns with larger cumulative teacher-student disagreement, is introduced.
Han-Yang Wang, Ze-Yuan Liu, Zheng-Yu Chen et al.· 0 citations
SAGE is a training-free inference-time framework that structures LLM strategic reasoning around three coordinated operations: anchor, adapt, and recalibrate, and distills strategically related interactions into counterfactual hypotheses about previously missing considerations, allowing past experience to recalibrate th...