Divergence-Routed On-Policy Distillation for Language Models
On-policy distillation (OPD) transfers knowledge from a teacher model to a student along student-generated trajectories, typically by minimizing the reverse Kullback-Leibler (RKL) divergence. However, RKL can provide weak corrective signals for teacher-supported tokens to which the student assigns very low probability....
Tian-Tian Peng, Guan-Nan Lv, Hong-Yang Li et al.
· 0 citations