Preprint

SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation

Junsong Chen Jincheng Yu Yitong Li Shuchen Xue Haozhe Liu Jingyu Xin Yuyang Zhao Tian Ye Zhangjie Wu Zian Wang Daquan Zhou Ping Luo Song Han Enze Xie

Jul 2026 · 0 citations · 51 references

Computer Science

TL;DR

This work introduces SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture that recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.

Abstract

We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention. To avoid quadratic attention throughout, Hybrid Linear-Softmax Attention combines gated linear attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank token interactions that pure linear attention lacks. To propagate these refreshed representations across depth, Block Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.

arXiv Semantic Scholar

View source

Similar papers

Preprint Jul 2026

MobileWan: Closing the Quality Gap for Mobile Video Diffusion

MobileWan becomes the first 5B-scale video diffusion model deployable on a commercial mobile device and proposes a learnable attention head pruning method based on binary per-head gates optimized end-to-end using a noise-biased sparsity objective and distillation-based finetuning.

Mohsen Ghafoorian, Denis Korzhenkov, Adil Karjauv et al. · 1 citation

Preprint Jul 2026

Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers

Chimera, a hybrid visual diffusion backbone with a principled scaling recipe that combines Kimi Delta Attention for long-context state tracking with O(N) complexity, interleaved Multi-head Latent Attention (MLA) for direct global interaction, and modality-aware short convolutions for local spatiotemporal context is introduced.

Chongjian Ge, Hanwen Jiang, Tianyu Wang et al. · 1 citation

Preprint Jul 2026

LPM: Industrial-Scale Generative Video Restoration

The Large Processing Model is presented, a diffusion-based generative framework for photorealistic video restoration under complex, in-the-wild degradations, demonstrating that generative restoration can be practical, scalable, and cost-effective for large-scale video processing.

Bichuan Zhu, Fulin Li, Jiachao Gong et al. · 0 citations

Preprint Jul 2026

MegaSlide-DiT: Memory-Centric Adaptation and Deformable Local Attention for Efficient Video Diffusion

High-resolution video diffusion models built on Diffusion Transformers (DiTs) deliver strong fidelity but quickly exhaust the memory budget of a single workstation. A 100 billion-plus parameter DiT easily requires over a terabyte of persistent state, while naive spatiotemporal self-attention grows quadratically in sequence length. These two walls -- parameter memory and activation memory -- prevent researchers from adapting massive generative models without large GPU clusters. We revisit this problem from a systems perspective and introduce MegaSlide-DiT, a prototype that demonstrates how a pre-trained 105B DiT can be adapted on a single H200 GPU with 1.5 TB of host RAM. Our key insight is that the GPU need not own the model state: all persistent weights, master weights and optimizer moments remain in host memory, while only transient shards are streamed to the GPU on demand. Simultaneously, we replace quadratic global attention with 3D Deformable Slide Attention (3D-DSA), a motion-adaptive local attention operator that reduces both memory and computational complexity to linear in the sequence length. We report detailed memory accounting, execution traces and evaluation results to substantiate our design. MegaSlide-DiT does not claim to train a 105B model from scratch on a single GPU, nor does it magically solve bandwidth limits; rather, it offers a pragmatic path for full-parameter adaptation of massive video diffusion models on high-end workstations.

Jiacheng Liu, J. Liu · 1 citation

Preprint Jul 2026

VideoSEMA: a scalable and efficient Mamba-like attention for video understanding

It is proved that the computationally cheaper split space-time attention is equivalent to full space-time attention and is promising to extend VideoSEMA to longer videos with a dilated/sparse temporal attention.

N. Tran, Fanghui Xue, Shuai Zhang et al. · 0 citations

Preprint Aug 2026

Token Radius Attention for Efficient Video Generation

This work proposes Token Radius Attention (TRA), a training-free framework that maps query entropy to an analytic token budget and converts it into a temporally decayed radius without explicit key ranking and achieves 1.05x speedup with competitive generation quality.

Jiayu Chen, Zhikun Jiang, Maoliang Li et al. · 0 citations