On-policy distillation (OPD) is increasingly applied to frontier foundation model post-training. Prior work in this area has largely focused on algorithmic advances, yet it remains unclear how much OPD depends on its training questions and, in particular, how far this dependence can be reduced. Across two representativ...
Gengsheng Li, Mao Zheng, Ming-Yang Song et al.· 0 citations
USA is proposed, which converts per-parameter update magnitudes measured during a brief warm-up into per-coordinate perturbation radii, reducing curvature precisely on the coordinates that carry most of the merging displacement.
Qi-Yong Zhong, Mao Zheng, Ming-Yang Song et al.· 1 citation
Guided On-Policy Distillation (Guided-OPD), a simple yet effective algorithm that mixes teacher- and student-generated turns within each rollout and schedules the teacher's intervention probability along a curriculum that decays to zero is proposed.
Gengsheng Li, Mao Zheng, Ming-Yang Song et al.· arXiv.org· 24 citations· ⚡3
TrustMOPD replaces example-level teacher selection with label-free, token-level supervision allocation and closes the overall-score gap between the initial student and oracle-routed teachers when trained on single- and multi-teacher on-policy distillation.
Jie Sun, Mao Zheng, Ming-Yang Song et al.· 4 citations
DF-OPD matches and even surpasses real data, and the questions it produces track the teacher's own post-training data on three key diagnostics of training dynamics, which other real datasets do not.
Gengsheng Li, Mao Zheng, Ming-Yang Song et al.· 1 citation
Experiments on reasoning, code-generation, scientific-knowledge, scientific-knowledge, and tool-use benchmarks show that these implementations can be executed through the same verl-based backend while retaining their method-specific objectives and task-dependent performance profiles.
Jie Sun, Mao Zheng, Mingyang Song et al.· arXiv.org· 0 citations
DASH maps the gap between each local distillation signal and the sequence-level mean to an adaptive propagation gate and then uses these gates to control backward multi-step aggregation and improves over matched vanilla OPSD reruns on every benchmark at all three model scales.
Zhi-Yan Hou, Xinyu Tang, Hongyan An et al.· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
