Learning from Teacher Continuations at Student States
We present OLIVE (OnLine InterVEntion). At each iteration, the evolving student policy generates a new prefix, the teacher continues it autoregressively, and the student is updated using cross-entropy computed on the teacher-generated tokens. Each design choice targets a corresponding limitation of existing distillatio...
Hao-Jin Wang, Dylan Zhang, Huai-Bo Chen et al.
· 0 citations