Autoregressive video generation requires denoising the current frames while writing their key-value representations as context for future predictions. However, these two roles typically share parameters, and we find that their gradients exhibit distinct patterns and systematic negative alignment, hindering the joint op...
Zi-Han Su, Junhao Zhuang, Yao-Wei Li et al.· 0 citations
EchoWM is an omnimodal world model for enterable generative media that responds to continuous navigation while jointly generating 720p video, environmental sound, music and speech, and maintains synchronized environmental sound and speech over long-horizon generation.
Mask Forcing, a Dual-Noise Masking Rollout strategy that perturbs the AR student self-rollout to mitigate mode collapse induced by reverse-KL mode seeking, improves multiple AR video diffusion distillation methods with higher visual quality efficiently, without incorporating real video data or additional post-training...
Zhuo-Ran Zhao, Sheng-Ju Qian, Tong-Tong Liang et al.· 2 citations
A large-scale synthetic data production pipeline built on Unreal Engine for generating action-conditioned, multi-view video, and describes the system architecture, the implementation decisions that emerged from production failures, and the limitations of using perceptual quality proxies for world-model data curation.
Hao-Yu Wang, Song-Chun Zhang, Hao-Ran Li et al.· 0 citations
Self Gradient Forcing (SGF), a two-pass training strategy that restores this missing memory-writing supervision within the native autoregressive training objective, using losses on future video latents to train the model to encode context into more effective causal memory.
LiveLight is presented, the first diffusion-based framework for real-time streaming video relighting with interactive 3D lighting control, which achieves state-of-the-art relighting quality while running at real-time speed, and will publicly release its models, training data, and synthetic data generator.
Yue Ma, Jiangming Wang, Yucheng Wang et al.· 1 citation
Results indicate that memory, geometric control, and rollout-aware training provide a practical foundation for generating coherent stories and continuously evolving interactive worlds.
Nan Duan, Hao-Yang Huang, Wei-Yang Jin et al.· 4 citations· ⚡1
FlexComposer is proposed, a unified framework that standardizes video compositing as a trajectory-guided conditional generation task, enabling the seamless integration of both static images and dynamic footage.
Song-Chun Zhang, S. Guo, Xianghao Kong et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
