Preprint
Sep 2026
What Shared Prefixes Hide: Trajectory Dropout for On-Policy Distillation
On-policy distillation (OPD) trains a student model on its own trajectories using dense token-level feedback from a stronger teacher model. Since each update is conditioned on the reasoning prefix already generated by the student, the prefix also shapes how effectively teacher feedback is converted into learning. We fi...
Zi-Zhuo Lin, Quan-Ling Liu, Yi Yang et al.
· 0 citations