Open-MOPD: Diagnosing and Fixing Capability Imbalance in Multi-Teacher On-Policy Distillation
Open-MOPD, a principled framework incorporating token-share balancing, gap-aware dynamic budget allocation, and student reward refresh, systematically restore cross-domain balance, elevating headroom recovery from 35.6% to 83.4% in a single deployable student.
Huan Gao, Haohan Chi, Yong Yan et al.
· 0 citations