S. Kang

What Do Students Learn? A Feature-Level Analysis of Dark Knowledge

This work proposes Confusion Distillation (CD), a teacher-free self-distillation method that utilizes the model's own evolving )confusion patterns as dynamic soft targets, outperforming existing self-distillation methods like CS-KD and PS-KD while offering a computationally efficient alternative to standard KD.