D3D, a standalone diffusion drafter that removes inter-round dependency on target-model hidden states, combines blockwise denoising with auxiliary autoregressive supervision to achieve long accepted prefixes without target hidden states, enabling asynchronous speculative decoding.
Abstract
Target-conditioned diffusion drafters predict candidate tokens in parallel, yet each new block must wait for updated target-model hidden states. This forces drafting and verification to run sequentially, limiting speedup even though each block is generated in one pass. We introduce D3D, a standalone diffusion drafter that removes this inter-round dependency. D3D combines blockwise denoising with auxiliary autoregressive supervision to achieve long accepted prefixes without target hidden states. This independence lets D3D draft successor blocks during verification in one packed pass, enabling asynchronous speculative decoding. We progressively prune layers according to their effect on acceptance length to reduce the cost of packed drafting. Removing five of 36 layers retains over 99\% of mean acceptance length across three benchmarks with Qwen3-8B as the target. Across seven benchmarks, D3D averages 5.00$\times$ and 5.71$\times$ decoding speedup with Qwen3-8B and Qwen3-32B targets, compared with 4.03$\times$ and 3.67$\times$ for DFlash. On GSM8K with Qwen3-32B, speedup reaches 7.48$\times$. Code is publicly available at https://github.com/fudan-generative-vision/D3D.
Speculative decoding has become a standard way to accelerate LLM inference: a lightweight drafter proposes tokens that the target verifies in one parallel pass. Existing drafters face trade-offs that limits their speedups: autoregressive drafters generate tokens one at a time, so drafting cost grows with draft length,...
S. Aktas, Yusuf Erdem, Ahmed Zeer et al.· 0 citations
D-Loop is proposed, which introduces \emph{intra-block causal conditioning} within the original diffusion drafter without additional model components, and can beat DFlash and DSpark on Qwen3-4B and Qwen3-8B with obvious gains.
Ke-Cheng Chen, Yu-Yang He, Cheng Gong et al.· 0 citations
Packed-SS is introduced, a self-speculative decoding schedule that verifies and drafts in a single model call with input size linear in the speculative width, while preserving the target model's distribution.
DEdit is introduced, a diffusion-based drafter that can not only draft by conventional parallel unmasking but also iteratively edit its draft through token-to-token predictions, and ProposalMix, a training scheme that mixes draft predictions with ground-truth tokens based on first-pass confidence during training.
Long-Xuan Yu, Bingsen Chen, Peng Shi et al.· arXiv.org· 1 citation
Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model. Recent diffusion-based drafters make this paradigm particularly attractive: deep attention-based neural backbones provide strong predi...
Roy Eisenstadt, Harel Lidar, Tal Gonen et al.· 2 citations
Speculative decoding accelerates large language model inference through parallel verification of tokens proposed by a drafter. Diffusion drafters further parallelize proposal generation, but greater proposal capacity or higher mean accepted tokens (MAT) need not yield higher throughput. Because verification accepts onl...
Jia-Kun Fan, Ming-Qing Xiao, Qiang Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.