SDE : Scale-Difference Evolution Knowledge Distillation
Logit-based knowledge distillation (KD) is pivotal for efficient model compression and cross-architecture learning. However, conventional methods typically rely on static, single-scale logit alignment, thereby overlooking the semantic evolution trajectory embedded in cross-scale prediction transitions. To bridge this g...