Preprint
Jul 2026
ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation
A short-to-long OPD schedule that detects teacher-confirmed repetitive suffixes, treats the surviving prefix as each rollout's effective length, and allocates future rollout budgets to the effective lengths the policy can currently use is proposed.
Qingyu Zhang, Qianhao Yuan, Hongyu Lin et al.
· 4 citations