Structured sparse attention for accelerating long-context diffusion language models
Diffusion Large Language Models (DLLMs) have emerged as a compelling parallel decoding paradigm. However, the inherent full bidirectional attention at every denoising step incurs quadratic complexity, imposing severe latency bottlenecks in long-context scenarios. We investigate attention dynamics in DLLMs an...