GeoFlow is a novel framework designed to achieve efficient driving video generation by harnessing explicit geometric priors, using a Geometry-Aligned Prior (GAP) distribution as starting point, and can achieve remarkable efficiency of both training and inference.
Abstract
Generative models like Diffusion Models and Flow Matching have demonstrated remarkable capabilities in synthesizing high-fidelity driving videos, but are severely constrained by high inference latency due to the requirement of extensive sampling steps. We argue that this inefficiency stems from the prevailing reliance on a standard Gaussian source distribution, where consecutive frames are initialized as independent Gaussian noise. This paradigm disregards the rich spatiotemporal correlations inherent in driving videos, compelling the model to regenerate deterministic scene structures existing in previous frames from noise, which is both computationally redundant and prone to geometric inconsistency. To address this problem, we propose GeoFlow, a novel framework designed to achieve efficient driving video generation by harnessing explicit geometric priors. Instead of sampling from standard Gaussian noise, we leverage multi-view geometry and spatially-adaptive noise injection to construct a Geometry-Aligned Prior (GAP) distribution as starting point. This initialization bridges the gap between source distribution and data distribution, yielding a significantly straighter and shorter sampling trajectory. Extensive experiments demonstrate that GeoFlow can achieve remarkable efficiency of both training and inference: merely several hours of fine-tuning on baseline models can significantly boost few-step generation quality, while fully converged training drastically reduces number of inference steps required for state-of-the-art video generation.
This work proposes Compact Autoregressive Latent Prior (CALP), which regularizes low-dimensional latent variables with a history-conditioned autoregressive prior, achieving compact, long-range temporally coherent, and spatially structured latent representations through a more reasonable latent capacity allocation and a...
Ruosen Hao, Tian-Yu Shen, Jing-Yuan Gao et al.· IEEE Transactions on Pattern...· 0 citations
SpatialCrafter is presented, a novel two-stage framework that addresses explorable image-to-scene generation issues by introducing a global 3D proxy for high-fidelity image-to-scene generation and appearance refinement and introduces Parallel Geometry Injection and Proxy-Aware Corruption training strategies.
Chuan Fang, Lingteng Qiu, Yixun Liang et al.· 1 citation
This work repurposes pretrained video generative models as a unified and data-efficient framework for geometry estimation, formulated innovatively as a next-frames prediction task, and inherits naturally structured knowledge and richer priors from the video model, enabling more data efficient and effective learning of...
Haosen Yang, Ji-Fei Song, Zhensong Zhang et al.· 1 citation
This work proposes Real-time Adaptive Physics-Informed Diffusion (RAPID), a unified framework explicitly designed to balance high-fidelity generation with strict real-time constraints, and establishes a new state-of-the-art balance between fidelity and safety.
Zihan Yu, Huandong Wang, Jing-Tao Ding et al.· Proceedings of the 32nd ACM...· 0 citations
VGGD, a visual geometry foundation-aware 3D Gaussian Splatting framework for feed-forward surround-view driving reconstruction, which shifts geometric modeling to the frontend and adapts foundation priors to the driving camera setting and achieves the best overall rendering quality among the compared methods and improv...
Junhong Lin, Jinlong Wang, Xianda Guo et al.· 0 citations
This work presents a training-free inference framework that stabilizes reliable static structure in pretrained video predictions through multi-frame temporal context and view-conditioned routing, and demonstrates that geometry-aware inference-time refinement and view-conditioned hybrid inference can improve static-geom...
K. M. Le, H. Pham, Danh Thanh Luu et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.