Rethinking Supervision Scaling in On-Policy Distillation: From Dense Supervision to Stochastic Horizons
On-policy distillation (OPD) provides dense token-level supervision along student-generated trajectories, yet the learning benefits of longer trajectories may not scale proportionally with their computational cost. We investigate how supervision utility evolves with trajectory length. Controlled experiments reveal dimi...