Reward-Aligned Reweighting for On-Policy Distillation
On-policy distillation (OPD) trains a student language model with dense feedback from a stronger teacher on student-generated trajectories. Yet standard OPD weights token-level distillation terms uniformly, implicitly treating local teacher preference as a proxy for correction utility. A decision's task value, however,...
Hao Xu, Junwei Su, Lan-Song Diao et al.
· 0 citations