#artificial intelligence
Sep 2026
Disentangling Self-Distillation: Measuring and Modeling Acquisition and Retention
A controlled model of the same objective is proposed to explain the resulting acquisition-retention trade-offs and is formalized as a unifying framework to encompass all self-distillation methods vs classic supervised fine-tuning.
L. Zuin, Alexis Huet, Dario Rossi et al.
· arXiv.org · 0 citations