On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard OPSD uses one fixed parameter setting at every state, but nearby settings may offer additional supervision. We find that local parameter per...
Xin Wei, Yi-Fan Ding, Yoshua Y. Li et al.· 0 citations
This work proposes SafePO, a reinforcement learning algorithm for refining violation identification while balancing explanatory reasoning and final verdicts, and develops AdaGuard, a family of 0.6B, 4B, and 8B guard models that assess agent trajectories under policies supplied at inference time.
Yun-Hao Feng, Yi-Fan Ding, Yu-Xiang Xie et al.· 0 citations
SA is proposed, a Stable Advantage Fusion framework that avoids entropy collapse and consistently outperforms fixed-coefficient GRPO+OPD fusion, improving the aggregate score by 0.70% across all six model-domain settings while achieving more stable training.
Yifan Ding, Xin Wei, Yoshua Y. Li et al.· arXiv.org· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.