Ruixuan Li Huazhong University of Science

TeachBack: Learning to Teach via Bidirectional Preference Optimization for On-Policy Self-Distillation

On-policy self-distillation (OPSD) enables a language model to learn from a Teacher instantiated from the same model but given access to privileged information, such as a reference solution. We find that access to sufficient privileged information allows the Teacher to generate highly accurate trajectories; however, ac...