Nov 2026· Proceedings of the 34th ACM International Conference on Multimedia· pp. 15530-15538· 0 citations
Abstract
Autoregressive (AR) diffusion models offer a promising framework for sequential generation such as video synthesis by combining diffusion modeling with causal inference. Although they support streaming generation, existing AR diffusion methods struggle to scale efficiently. In this paper, we identify two key challenges in hour-scale real-time human animation. First, most forcing strategies propagate sample-level representations with mismatched diffusion states, causing inconsistent learning signals and unstable convergence. Second, historical representations grow unbounded and lack structure, preventing effective reuse of cached states and severely limiting inference efficiency. To address these challenges, we propose Neighbor Forcing, a diffusion-step-consistent AR formulation that propagates temporally adjacent frames as latent neighbors under the same noise condition. This design provides a distribution-aligned and stable learning signal throughout the AR chain. Building upon this formulation, we introduce a structured ConvKV Memory mechanism that compresses the keys and values in causal attention into a fixed-length representation, enabling constant-memory inference and effectively unbounded video generation without short-term motion-frame memory. Experiments show improved convergence, hour-scale generation quality, and inference efficiency over existing AR diffusion methods. SoulX-LiveAct enables hour-scale real-time human animation at 20 FPS on two NVIDIA H100 or H200 GPUs while attaining strong lip-sync accuracy, human animation quality, emotional expressiveness, and inference efficiency.
Glance Forcing is proposed, a lightweight distillation framework that decouples the denoising trajectory into two complementary slow and fast experts, each guided by a tailored, stage-specific objective, while keeping the pretrained teacher entirely frozen.
Zhuo-Bai Dong, Jun-Chao Yi, Hu-Jian Guo et al.· 0 citations
Advances in generative models have improved video fidelity, enabling long-horizon generation, interactive world modeling, and evolving visual environments. Autoregressive (AR) video generation extends visual sequences through causal rollouts. However, a fundamental bottleneck emerges: as the generated sequence expands,...
Harold Haodong Chen, Rong-Jin Guo, Di-Sen Lan et al.· arXiv.org· 3 citations
Autoregressive video diffusion models (AR-VDMs) enable real-time, long-horizon video generation but degrade rapidly once the inference horizon outruns the training horizon, due to compounding exposure bias. To stabilize long rollouts, recent methods heuristically adopt frame sink, keeping initial frames as a persistent...
Zheng-Yang Yu, Akio Hayakawa, Masato Ishii et al.· 0 citations
Trajectory Forcing is proposed, an AR diffusion method that exploits the denoising trajectories of previous video chunks as context and achieves the highest overall quality among all compared methods.
Bo-Han Wang, Shuo Chen, Yun-Zhi Li et al.· 0 citations
Leveraging the paradigm of autoregressive diffusion, text-conditioned interactive video world models aim to simulate temporally coherent environments guided by textual instructions. While enabling low-latency, long-duration generation is pivotal for embodied AI and simulation-based planning, current frameworks primaril...
Ze-Yu Zhang, Jin-Yuan Mao, Da-Kai An et al.· arXiv.org· 0 citations
Video world models turn actions and camera motion into long visual rollouts, making video a natural interface for interactive simulation. Such interaction requires responsive generation, yet coherent rollouts must retain visual memory of what has already been generated. Chunked video diffusion models usually keep this...