Does On-Policy Distillation Need to Complete Every Rollout?
On-policy distillation (OPD) transfers teacher knowledge to a student through feedback on student-generated rollouts, but generating complete rollouts and computing per-token teacher feedback are computationally expensive. Existing truncation methods save computation by stopping rollouts early. However, they do not com...
Zheng-Xin Zhang, Mao-Sen Tang, Ning Wang et al.
· 0 citations