Recent studies on reinforcement learning (RL) report seemingly conflicting evidence about large language model (LLM) reasoning. Training on mathematics can improve performance in other domains, yet gains in Pass@1 can coincide with lower Pass@$N$ than the base model. This raises a fundamental question: does RL expand a...
Zi-Heng Cheng, Yi-Xiao Huang, Han-Lin Zhu et al.· 1 citation
A systematic asymmetry in models using chunked KV-cache compression is uncovered: the same information can be easy to retrieve at one phase and difficult at another, revealing periodic weak spots that average benchmark scores can conceal.
Xingyu Zhu, Pu Yi, Zi-Heng Cheng et al.· 0 citations
Theoretical analysis shows that the fixed-point distributions of RWTD interpolate between off-policy reward tilting of the reference and on-policy tilting of the current model, providing a principled approach to balancing reward adaptation with retention of prior knowledge.
Austin S. Wang, Zi-Heng Cheng, Le-Xing Ying· arXiv.org· 1 citation
This work proposes a multi-mask diffusion model (MultiMDM) that preserves the masking structure towards few-step generation and derives a closed-form ELBO training objective for MultiMDM that supports continual training from pretrained MDMs.