The first multiplayer world model for highly dynamic environments governed by complex physical interactions is introduced, and how behavior changes with model and data scale is characterized, including the capabilities that emerge and the failure modes that persist.
Abstract
We introduce the first multiplayer world model for highly dynamic environments governed by complex physical interactions. Whereas single-player world models treat the other agents as part of the environment, ours conditions on the action streams of multiple agents, learning to attribute changes in the scene to the correct player and to stay coherent under arbitrary combinations of their actions. We study this problem in the game of Rocket League, where players compete and cooperate under fast, tightly coupled dynamics. Trained on 10,000 hours of gameplay collected with publicly available bots, our 5-billion-parameter latent diffusion model generates four-player matches in real time, producing 20 frames per second on a single Nvidia B200 GPU. Although trained only on short clips, its rollouts stay stable far beyond the training horizon: distributional quality holds steady out to five minutes, the longest horizon we measure, and in practice we observe rollouts continuing for hours with no sign of collapse. We systematically investigate the central design choices: the video codec, the generative objective, and the multiplayer conditioning scheme. In addition, we characterize how behavior changes with model and data scale, including the capabilities that emerge and the failure modes that persist. We further develop targeted evaluations that probe the model's physical understanding rather than visual appearance alone. To support continued research on multiplayer world models, we release our dataset, our full training and inference codebase, and a live demo.
Current video world models struggle in multiplayer environments because they entangle world state with view-dependent visual latents, leading to redundant compute, view inconsistencies, and poor scalability. We propose MASS (Multiplayer world models with Authoritative Shared State) to resolve this limitation. Inspired by multiplayer game architectures, MASS disentangles world dynamics and view rendering. A learned Logic Engine advances a global, authoritative typed state from joint actions without any hand-written transition function, acting as the sole recurrent memory and synchronization reference. From this shared state, a learned Rendering Engine generates independent and consistent views for any requested camera on demand. This explicit disentangling allows MASS to achieve superior state accuracy and lower cross-view inconsistency compared to state-of-the-art multi-view baselines on a matched multiplayer Snake benchmark. It advances predicted worlds with 1,024 concurrent players for 10,000 recurrent steps. Our results show that explicit, authoritative state modeling provides a practical foundation for scalable and consistent multi-agent world simulation.
Ziqi Cai, Si-Qi Yang, Yimu Wang et al.· 0 citations
Video world models simulate future states conditioned on current observations and user actions. Recent systems have demonstrated impressive video consistency and action controllability over long sequences. However, fairly comparing these interactive models remains challenging. In practice, a human player typically evaluates a world model by pursuing long-horizon objectives through interaction. For example, a user may turn around 360 degrees to see whether the environment remains consistent, or walk into the water and inspect whether realistic water ripples are generated. The action sequence required to achieve the same objective may vary substantially between models, making fixed action-conditioned evaluation unsuitable for cross-model comparison. To address this, we employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives. Building on this paradigm, we introduce PlayWorld, a benchmark providing 171 scenarios, each with a specified objective. To evaluate performance thoroughly, we assess models along four core dimensions: geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution. In addition, we incorporate basic ability metrics for video quality and controllability. Experiments across nine state-of-the-art world models reveal that current models remain unreliable on long-horizon interactive objectives, particularly in maintaining spatial consistency and persistent state evolution. Code and data are available at https://github.com/kxding/PlayWorld.
Kai Ding, Xi Chen, Minghong Cai et al.· 1 citation
Contemporary video games offer a plethora of interesting challenges, yet the application of imitation learning in shipped titles remains sparse. One reason is that directly mirroring a player’s setup and behavior can lead to a stale or unfair combat experience where only computer-reaction time or endurance wins the battle. To address this, we investigate a temporally shifted generative imitation learning approach to enhance player-specific adaptation and implicit dynamic difficulty adjustment (DDA). By shifting the imitation model by one iteration of a playthrough, we create an asynchronous confrontation that requires players to think strategically both forwards and backwards in time. Players must make delicate trade-offs: polishing their current build to master the run, while avoiding overtuning an agent they must face in the future. We implemented this in “Dark Zagreus”, a modification of the action roguelike Hades, replacing the final boss with an AI trained on the player’s last successful run. To evaluate its effectiveness, we conducted a two-week user study with 20 participants comparing scripted and imitation-based behaviors. While quantitative metrics showed no significant differences in overall player experience, qualitative feedback highlighted that the AI effectively mirrored individual tactics, and performance analyses revealed accuracy comparable to prior imitation learning approaches. Despite limitations such as omitted game mechanics, this work demonstrates the potential of shifted imitation learning to realize novel, engaging gameplay experiences. The source code is publicly released to foster future research on player-centric AI design.
Hui-En Lin, S. Bakkes, Johannes Pfau· International Conference on...· 0 citations
A unified geometry-aware memory framework is proposed, whose patch-memory and tiled-PRoPE components introduce no additional learnable parameters, combining explicit 3D patch retrieval with projective camera conditioning to enable geometry-consistent camera control and faithful long-horizon scene recall.
Runjia Qian, Zi-Le Wang, Ji-Hai Zhang et al.· 2 citations
Video world models are increasingly used as simulators, yet visual fidelity alone does not show that a model maintains the hidden state of the world. We examine this gap with an action-conditioned video Shell Game, a visual analog of $S_5$ state tracking that decouples visual rendering from compositing the hidden state underneath. Bidirectional and autoregressive Transformers, Mamba, and linear attention restricted to nonnegative transition eigenvalues all fit the training horizon of 5 swaps and then fall toward chance on longer swap chains (extrapolation) while still rendering plausible video with additional denoising steps providing no benefit. The pixel-based diffusion target never supervises the unseen hidden state, so the generated frames cannot carry it and the state has to live inside the architecture rather than in the tokens. For a Transformer, that architectural state is only an append-only KV cache, so the model has to re-derive the hidden arrangement from the whole history at every chunk. We find two mechanisms that do extrapolate, and both carry a state across chunks and revise it in place. Linear attention succeeds once its transition eigenvalues may be negative, and TTT with a nonlinear fast weight succeeds by updating the feature map through which it reads its own state. We further examine harder cases in dynamic world exploration tasks, and discuss the broader implications for building stateful video world models.
Joonghyuk Shin, Yicong Hong, Jaesik Park et al.· 0 citations
MiniWorld is presented, a reproducible framework for training streaming video world models from scratch that adopts a chunk-wise non-decreasing noise schedule and two-stage continued training to improve temporal modeling and stability and will facilitate future research on video world modeling.
Yian Zhao, Ruo-Chong Zheng, Hongcan Guo et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
