Train Ahead, Distill Back: Bootstrapping On-Policy Self-Distillation for Large Language Models
On-policy self-distillation (OPSD) improves large language models by letting a self-teacher with privileged information provide dense token-level supervision on the model's own trajectories. Yet existing methods typically construct the self-teacher from the current, initial, or slowly averaged policy state, leaving the...
Zheng Zhang, Xin-Yue Tan, Lu Li et al.
· 0 citations