TV-Regulated OPD: Direction Matters in On-Policy Distillation
It is found that retaining only the sign of token-level advantages is sufficient to achieve the performance comparable to standard OPD, and a robust TV regulated On-Policy Distillation (TV-OPD) method is proposed, benefiting from the bounded and diminished advantages.
Han Xiao, Yifan Niu, Dong-Yi Liu et al.
· 0 citations