Preprint

Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Models

Yuki Ichihara Naoto Iwase Mohammad Atif Quamar Junpei Komiyama

Aug 2026 · 0 citations · 37 references

Computer Science

TL;DR

PDC consistently improves upon the initial sample, outperforms independent generations under a computational constrained comparison, and is robust to different unmasking strategies and parameter settings, highlighting prefix-conditioned regeneration as an effective DLM-specific primitive for test-time verification.

Abstract

Diffusion Language Models (DLMs) have recently become increasingly competitive with autoregressive (AR) models, and even outperform them on certain tasks. Unlike AR models, DLMs produce output through iterative denoising without a left-to-right order. To further improve the performance of DLMs, we introduce PDC (\emph{Prefix-Denoising Consistency}), a test-time self-verification method for DLMs. PDC exploits a distinctive test-time signal in DLMs under prefix conditioned regeneration, correct trajectories are more stable and reproducible than incorrect ones. Concretely, given an initially generated sample, PDC splits the sentence at an intermediate position and regenerates the remaining tokens conditioned on the fixed prefix. Across mathematical reasoning and commonsense reasoning benchmarks, PDC consistently improves upon the initial sample, outperforms independent generations under a computational constrained comparison, and is robust to different unmasking strategies and parameter settings. These results highlight prefix-conditioned regeneration as an effective DLM-specific primitive for test-time verification.

arXiv Semantic Scholar

View source

Similar papers

Jul 2026

Trace-Based On-Policy Distillation for Masked Diffusion Language Models

Diffusion large language models (dLLMs) are a promising alternative to autoregressive generation. However, reasoning-oriented post-training for dLLMs remains challenging. Supervised fine-tuning (SFT) for dLLMs requires dense but often off-policy masked states, while reinforcement learning (RL) relies on sparse rewards...

Haolin Ren, Ziyang Huang, Chenhao Yuan et al. · arXiv.org · 0 citations

Preprint Jul 2026

CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inference

This work proposes CORA-Diff, a training-free method that preserves the original transfer rule and applies confidence-and-persistence gating only to positions that rule leaves unresolved, and shows that native confidence and persistence enable reliable residual acceptance, reducing repeated denoising computation while...

Yifan Wu, Yu-Feng Zhang, Kenli Li · 0 citations

Preprint Aug 2026

CAI-DLLM: Convergence Aware Inference for Diffusion Language Models

This work proposes CAI-DLLM, a training-free inference method that uses first-step confidence to guide denoising and reduce inference time, and evaluates CAI-DLLM on LLaDA-8B-Instruct and Dream-7B-Instruct across math, code, reasoning, commonsense, and long-context tasks.

Farhana Amin, Sabiha Afroz, D. Nikolopoulos · 0 citations

#artificial intelligence Preprint Aug 2026

Trajectory-Level Speculative Decoding for Diffusion Language Models

This work develops a trajectory-level speculative framework that constructs draft denoising trajectories via confidence-stratified tree exploration and verifies them through blockwise parallel evaluation with bidirectional attention masking, and introduces inter-block speculation, exploiting diffusion models'bidirectio...

Tian-Xiang Pan, Baitao Gong, Mo Guang et al. · 0 citations

Jul 2026

Beyond the Bidirectional Promise: Re-evaluating the Robustness of Diffusion Language Models

While highly stochastic DLM loss landscapes naturally resist gradient-based adversarial suffixes, they provide no guaranteed defense against natural noise, proving that everyday robustness is weight-dependent rather than inherently architectural.

Saurabh Yadav, B. Patro, V. Agneeswaran · arXiv.org · 0 citations

Preprint Aug 2026

Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference

DARD is proposed, a training-free framework that separates tokens into masked, candidate, and unmasked states and adaptively regulates their influence on subsequent decoding, and consistently improves the speed-quality Pareto frontier over recent revocable decoding methods.

Woo-Soon Park, Insu Lee, Minyoung Noh et al. · 1 citation