Skip to content
Book Open access

Star4D: Real-Time Interactive 4D Scene Generation via Spatiotemporal Autoregressive Modeling

Nov 2026 · Proceedings of the SIGGRAPH Asia 2026 Conference Papers · 0 citations · 10 references

Abstract

Building world models with spatial consistency and real-time interactivity is a frontier challenge in computer vision. Current video generation paradigms often lack spatial persistence and sufficient visual realism, hindering seamless navigation in complex environments. To address these challenges, we propose Star4D, a novel real-time 4D scene generative model that transforms single reference videos into high-fidelity, dynamic, and interactive environments. At the core of our approach is a spatiotemporal autoregressive architecture. This architecture integrates an implicit spatiotemporal cache with explicit spatial guidance, enabling high-fidelity generation and precise camera control while maintaining real-time performance. Furthermore, we introduce Joint Distribution Matching Distillation, which leverages real-world data distributions as regularizing guidance to effectively overcome the fidelity degradation typically caused by over-reliance on synthetic data. Extensive experiments demonstrate Star4D significantly outperforms existing state-of-the-art models in spatial consistency and interaction precision, ranking first among real-time interactive methods on the WorldScore-Dynamic benchmark, and establishing a practical pipeline for navigating 4D environments instantiated from monocular videos.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.