Qiang Tang

Single-Teacher View Augmentation: Enhancing Knowledge Distillation with Student-Guided Perturbations

Shift-Augmented Knowledge Distillation (SAKD) is proposed, a simple yet effective framework that leverages the student's evolving features as a dynamic condition for perturbation generation, enabling single-stage training while producing adaptive, diverse views through a parameter-free cyclic shift.