Skip to content
Preprint

One Call Ahead: Packed Self-Speculative Decoding with Diffusion Drafting

Oct 2026 · 0 citations · 54 references
Computer Science

TL;DR

Packed-SS is introduced, a self-speculative decoding schedule that verifies and drafts in a single model call with input size linear in the speculative width, while preserving the target model's distribution.

Abstract

Autoregressive (AR) language models generate tokens sequentially, so generating long outputs is costly. Speculative decoding mitigates this by using a lightweight drafter to propose multiple tokens for the target AR model to verify in parallel, while a diffusion-based drafter can produce an entire draft block in one forward pass. In models with AR and diffusion modes, one network both drafts and verifies, but drafting still waits for verification, so the standard schedule needs two sequential model calls per iteration. Existing one-call schedules avoid the wait but incur input size quadratic in the speculative width. Motivated by this efficiency bottleneck, we introduce Packed-SS, a self-speculative decoding schedule that verifies and drafts in a single model call with input size linear in the speculative width, while preserving the target model's distribution. The key idea is to let the diffusion drafter run one call ahead: while verifying the current draft, the model simultaneously constructs the next draft from an older verified prefix and keeps its unresolved suffix for the next iteration. Under a mild monotonicity assumption on the drafter, we further show that Packed-SS requires no more model calls than the standard two-call decoding in the greedy setting. Experiments on Nemotron 3B/8B and SetDLM 1.7B show higher throughput and lower mean wall time than both baselines under greedy decoding.

View source

Similar papers

Preprint Nov 2026

ChainFlow: Few-Step Latent Flow Matching for Speculative Decoding

Speculative decoding has become a standard way to accelerate LLM inference: a lightweight drafter proposes tokens that the target verifies in one parallel pass. Existing drafters face trade-offs that limits their speedups: autoregressive drafters generate tokens one at a time, so drafting cost grows with draft length,...

S. Aktas, Yusuf Erdem, Ahmed Zeer et al. · 0 citations
Sep 2026

LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

Speculative decoding accelerates autoregressive inference by verifying multiple draft tokens in a single target forward pass. However, as the context grows, existing state-of-the-art drafters become increasingly expensive, eroding the very efficiency advantage they are designed to provide. We argue that this scaling is...

Hao-Yuan He, Peng-Fei Liu, Chen Chen et al. · 1 citation
#natural language process... Preprint Oct 2026

D-Loop: Looped Diffusion Drafting for Speculative Decoding

D-Loop is proposed, which introduces \emph{intra-block causal conditioning} within the original diffusion drafter without additional model components, and can beat DFlash and DSpark on Qwen3-4B and Qwen3-8B with obvious gains.

Ke-Cheng Chen, Yu-Yang He, Cheng Gong et al. · 0 citations

DEdit: Iterative Draft Editing for Speculative Decoding

DEdit is introduced, a diffusion-based drafter that can not only draft by conventional parallel unmasking but also iteratively edit its draft through token-to-token predictions, and ProposalMix, a training scheme that mixes draft predictions with ground-truth tokens based on first-pass confidence during training.

Long-Xuan Yu, Bingsen Chen, Peng Shi et al. · 1 citation
Preprint Nov 2026

D3D: Decoupled Diffusion Drafting for Asynchronous Speculative Decoding

D3D, a standalone diffusion drafter that removes inter-round dependency on target-model hidden states, combines blockwise denoising with auxiliary autoregressive supervision to achieve long accepted prefixes without target hidden states, enabling asynchronous speculative decoding.

Bao-You Chen, Rui-Qiao Mei, Guan-Cong Lin et al. · 0 citations
#artificial intelligence Preprint Nov 2026

DForkTree: Accelerating Speculative Decoding with Fork-Aware Block Diffusion Draft Trees

Speculative decoding speeds up language models by having a lightweight drafter propose future tokens that the target model verifies in parallel. Block diffusion drafters such as DFlash predict an entire block in one forward pass, and DDTree builds draft trees directly from their per-position distributions. Each distrib...

Zhi-Cheng Zheng, Cheng Zhang, Chen-Kai Tang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.