Reward-Aligned Distillation
Reinforcement learning with verifiable rewards improves the reasoning capabilities of large language models, but remains computationally expensive and sample inefficient. On-policy distillation (OPD) provides dense token-level supervision from a stronger teacher, yet standard OPD distills every token regardless of whet...
Yilan Chen, Zian Su, Manish Prajapat et al.
· 0 citations