Skip to content
Preprint

FocusOPD: Structure-Guided On-Policy Distillation for Diffusion Models

Oct 2026 · 0 citations · 22 references
Computer Science

TL;DR

FocusOPD is introduced, which augments the base OPD objective with token-wise directional feature alignment restricted to deep layers and early denoising steps, and improves capability transfer in compositional generation, text rendering, and preference-based evaluation.

Abstract

Multi-teacher on-policy distillation enables rectified-flow models with diffusion-transformer architectures to combine task-specific capabilities within a single student. However, matching model outputs does not guarantee that their internal representations are aligned, and it remains unclear where and when additional feature supervision is most useful. We analyze the teacher-student feature mismatch across Transformer blocks and denoising timesteps, and find that the mismatch is concentrated in deeper Transformer blocks during early denoising and persists after OPD distillation, indicating ineffective distillation. Motivated by this observation, we introduce FocusOPD, which augments the base OPD objective with token-wise directional feature alignment restricted to deep layers and early denoising steps. Across two rectified-flow backbones and multiple OPD objectives, FocusOPD improves capability transfer in compositional generation, text rendering, and preference-based evaluation. On the primary benchmark, it improves over DiffusionOPD from 0.953 to 0.967 on GenEval, from 23.797 to 23.944 on PickScore, and from 6.052 to 6.199 on LAION-Aesthetic. Ablations show that selective deep and early supervision is more effective and efficient than alignment over the full network and trajectory.

View source

Similar papers

GFD-OPD: Guidance-Folded On-Policy Distillation of Diffusion Models Across Scales

On-policy distillation (OPD) has demonstrated two important capabilities in language models: compressing large teachers into smaller students and merging expert models into a single model. Existing diffusion OPD, however, mostly focus on the latter, with teachers and students sharing the same backbone and scale. We inv...

Zhen-Xing Zhang, Jia-Yan Teng, Wen-Xu Wu et al. · 0 citations

d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Models

Large language models (LLMs) typically generate text autoregressively (AR), predicting one token at a time. Block diffusion language models (dLLMs) instead generate blocks sequentially while denoising multiple tokens in parallel within each block, offering a promising way to accelerate generation. Rather than training...

Rui-Tao Liu, Qing-Hao Hu, Song Han · 0 citations
#machine learning Preprint Oct 2026

On-Policy Residual Learning for Diffusion Language Models

ORL is introduced, a framework for on-policy residual learning in DLMs that learns from model-induced residual trajectories on the fly and introduces an entropy-contraction objective that sharpens predictions as the model revisits its own predictions.

Yi-Fan Yu, Yu-Qing Jian, Ji-Sen Li et al. · 0 citations
Preprint Oct 2026

Adaptive-Policy Distillation: Knowledge Injection under Information Asymmetry

On-policy distillation (OPD), which uses reverse-KL supervision on student rollouts to transfer a context-conditioned model's behavior into student model parameters for closed-book inference, has shown strong results. However, we find that this success does not translate to knowledge injection: across two document-QA b...

Hojun Cho, K. Jeon, J. Choo · 0 citations
#machine learning Preprint Oct 2026

DiffGate: Difficulty-Gated Teacher Guidance for On-Policy Distillation

DiffGate is introduced, an outcome-gated objective that combines GRPO with selective, bounded teacher guidance, and improves pass@8 across all four model--domain settings, demonstrating improved solution coverage under the evaluation protocol.

Karn Tiwari, V. Chordia, P. PrathoshA · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.