Training-free sparse attention offers a practical acceleration solution to Diffusion Transformers (DiTs) via reducing computations without fine-tuning. It typically involves estimating the importance of query-key regions and deriving sparse masks to compute only the important candidates, which inevitably introduces app...
Cheng-Feng Han, Bao-Le Ai, Xian-Lu Bian et al.· 0 citations
PSA, a parameterized stripe attention that formalizes the observed stripe regularity, unifying diverse attention patterns for efficient mask generation, enables a single hardware-efficient CUDA kernel to process all sparse patterns, achieving FlashAttention-3-level Model FLOPs Utilization.
Xing-Yu Jia, Bao-Le Ai, Ang Wang et al.· 0 citations
This work identifies the high-sparsity trap, a unified acceleration framework for visual generation that combines a short sparse warm-up, few-step trajectory-mixed distillation, and FP8 quantization with fused kernels, and sustains attention sparsity with strong visual quality on long-sequence 720P generation across Wa...
Yu-Xi Liu, Hao-Yu Li, Ze-Kun Zhang et al.· 0 citations
We present Wan-Streamer v0.3, which reframes our native-streaming interaction model under a single organizing view: a video is a world plus an event stream. The world is the persistent context in which a video unfolds, including the environment, scene, subjects, ambient acoustic conditions, voice characteristics, and o...
Lianghua Huang, Zhigang Wu, Yupeng Shi et al.· arXiv.org· 2 citations
Diffusion Transformers (DiTs) achieve strong video generation quality, but their dense spatiotemporal self-attention scales quadratically with sequence length and quickly becomes the dominant inference bottleneck. Sparse low-rank hybrids alleviate this cost by combining a local sparse branch with a global compressed br...
Ze-Kun Zhang, Yi-Xiang Cai, Yu-Xi Liu et al.· 1 citation
Wan-Streamer v0.2 is presented, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model, which raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS.
Lianghua Huang, Zhigang Wu, Yupeng Shi et al.· arXiv.org· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
