Preprint
Jul 2026
DUET: Dual-Teacher On-Policy Distillation via Same-Weight Disagreement for Prohibition Compliance
DUET is proposed, a token-selective on-policy distillation method for prohibition compliance that achieves 72.3-85.2% violation compliance while preserving 88-93% normal utility, dramatically outperforming teacher model and other distillation baselines.
Zihan Li, Feifei Li, Wen-Hui Que
· 0 citations