FocusOPD is introduced, which augments the base OPD objective with token-wise directional feature alignment restricted to deep layers and early denoising steps, and improves capability transfer in compositional generation, text rendering, and preference-based evaluation.
Abstract
Multi-teacher on-policy distillation enables rectified-flow models with diffusion-transformer architectures to combine task-specific capabilities within a single student. However, matching model outputs does not guarantee that their internal representations are aligned, and it remains unclear where and when additional feature supervision is most useful. We analyze the teacher-student feature mismatch across Transformer blocks and denoising timesteps, and find that the mismatch is concentrated in deeper Transformer blocks during early denoising and persists after OPD distillation, indicating ineffective distillation. Motivated by this observation, we introduce FocusOPD, which augments the base OPD objective with token-wise directional feature alignment restricted to deep layers and early denoising steps. Across two rectified-flow backbones and multiple OPD objectives, FocusOPD improves capability transfer in compositional generation, text rendering, and preference-based evaluation. On the primary benchmark, it improves over DiffusionOPD from 0.953 to 0.967 on GenEval, from 23.797 to 23.944 on PickScore, and from 6.052 to 6.199 on LAION-Aesthetic. Ablations show that selective deep and early supervision is more effective and efficient than alignment over the full network and trajectory.
On-policy distillation (OPD) has demonstrated two important capabilities in language models: compressing large teachers into smaller students and merging expert models into a single model. Existing diffusion OPD, however, mostly focus on the latter, with teachers and students sharing the same backbone and scale. We inv...
A likelihood-free pipeline for a mixture-of-experts dLLM, where each rollout group is routed by verifier outcome and if every student rollout fails but a teacher succeeds, the model receives dense KL supervision on the student's denoising canvases.
Wen-Tao Lu, Tian-Yu Zhu, Jesse N. Clark· 0 citations
Large language models (LLMs) typically generate text autoregressively (AR), predicting one token at a time. Block diffusion language models (dLLMs) instead generate blocks sequentially while denoising multiple tokens in parallel within each block, offering a promising way to accelerate generation. Rather than training...
Rui-Tao Liu, Qing-Hao Hu, Song Han· arXiv.org· 0 citations
ORL is introduced, a framework for on-policy residual learning in DLMs that learns from model-induced residual trajectories on the fly and introduces an entropy-contraction objective that sharpens predictions as the model revisits its own predictions.
Yi-Fan Yu, Yu-Qing Jian, Ji-Sen Li et al.· 0 citations
On-policy distillation (OPD), which uses reverse-KL supervision on student rollouts to transfer a context-conditioned model's behavior into student model parameters for closed-book inference, has shown strong results. However, we find that this success does not translate to knowledge injection: across two document-QA b...
DiffGate is introduced, an outcome-gated objective that combines GRPO with selective, bounded teacher guidance, and improves pass@8 across all four model--domain settings, demonstrating improved solution coverage under the evaluation protocol.
Karn Tiwari, V. Chordia, P. PrathoshA· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.