DyMD, a DMD framework that adapts both teacher supervision and critic fitting to the evolving student with no auxiliary modules at inference is proposed, and a 14B teacher is distill into a four-step 1.3B student with no auxiliary modules at inference.
On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid. We ask what happens when none of this holds, as when the strongest teacher available and the...
The method combines chunk-wise autoregressive adaptation, Source-Anchored Distribution Matching Distillation, and Long-Horizon Autoregressive Distillation to reduce train--inference mismatch, preserve source fidelity during two-step generation, and mitigate accumulated temporal drift.
Yi-Cheng Xiao, Wenxun Dai, Xinran Qin et al.· 5 citations
This paper presents Poly-OPD, a framework that can consolidate complementary strengths of heterogeneous teachers into a single compact flow-matching student, and uses a gradient compatibility diagnostic to organize its adapters.
Siming Fu, Haojun Xu, R.Z. He et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.