Train Ahead, Distill Back: Bootstrapping On-Policy Self-Distillation for Large Language Models
This paper introduces Bootstrapped On-Policy Self-Distillation (B-OPSD), which temporarily trains the policy ahead to obtain a future teacher, restores the student to the original policy state, and then uses the future teacher to supervise the restarted student.