On the Off-Policy Teacher in On-Policy Distillation
On-policy distillation (OPD) has recently emerged as a promising post-training paradigm in which the student learns from trajectories generated by its own policy under dense teacher supervision. However, OPD introduces a fundamental asymmetry: although the sampled trajectories are on-policy for the student, they are of...
Lang-Lin Huang, Hao Liu, Mononito Goswami et al.
· 0 citations