HappyWorld-Bench is introduced, a comprehensive benchmark that evaluates whether generated worlds remain reliable as agents interact with them, and highlights the need to evaluate world models not only by visual quality, but also by state consistency and the correctness of their responses to actions and interventions.
Abstract
Evaluating world models requires assessing both the quality of the worlds they generate and their consistency and responsiveness under exploration, interaction, and modification. We introduce HappyWorld-Bench, a comprehensive benchmark that evaluates whether generated worlds remain reliable as agents interact with them. Our design is built on a hierarchical capability framework of six world capabilities (W1-W6), from generative construction to unified world modeling, instantiated across three independent evaluation tracks: video world models, spatial world models, and embodied world models. HappyWorld-Bench comprises 1,138 video prompts, 300 spatial scenes, and 254 embodied test cases. Across all three tracks, we build and operate HappyWorld-Arena to organize human A/B comparisons and derive model-level Elo ratings, which complement newly designed automated metrics that capture behavioral correctness. We evaluate 14 video world models, 9 spatial systems, and 8 embodied candidates under this unified framework. Results reveal remaining reliability gaps across all three tracks: video models exhibit reduced consistency during extended rollouts and revisits, spatial models achieve at best 70.14% placement accuracy and 73.33% edit execution, and embodied models struggle to preserve state across multi-step actions and respond precisely to altered action conditions and physical rules. These findings highlight the need to evaluate world models not only by visual quality, but also by state consistency and the correctness of their responses to actions and interventions.
Interactive world models are increasingly capable of generating environments and acting within them, yet deliberately editing an existing executable world remains underexplored. We formulate world editing as intervening on an existing world while preserving properties that should remain unchanged, and introduce interve...
Max W.F. Ku, Nok-Kan Law, Yu-Chien Tang et al.· 0 citations
This work formalizes probabilistic alignment as a distributional criterion for world models and introduces PAWBench, a benchmark for evaluating video generators as stochastic samplers of world dynamics, and introduces PAWEval, an outcome-level protocol that converts repeated video rollouts into empirical distributions...
Yuan-Dong Pu, Le Zhuo, Sayak Paul et al.· 0 citations
PlayWorld is introduced, a benchmark providing 171 scenarios, each with a specified objective, to assess models along four core dimensions: geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution, and basic ability metrics for video quality and controllability.
Kai Ding, Xi Chen, Ming-Hong Cai et al.· 3 citations
WorldBench is presented: a comprehensive, multilingual benchmark of genuine, persona-grounded everyday workflows, where agents can act in a sandbox via structured actions, and Constrained Task Success (CTS), which combines natural language instructions and testbeds to score task completion, minimal modification, and ot...
Leonardo Ranaldi, Sherrie Shen, Jushi Kai et al.· 1 citation
TRWORLDBENCH is introduced, a benchmark for evaluating embodied world models through synchronized head, left-wrist, and right-wrist videos and uses 19 metrics to assess tri-view consistency, task alignment, physical and 3D coherence, motion quality, temporal consistency, and visual quality.
Xuan-Yi Liu, Hao-Feng Wang, Rui-Qi Li et al.· 2 citations· ⚡1
This work introduces PROWBench, comprising 170 programmatically constructed episodes and 600 proxy videos covering diverse scenes and interactions, and evaluates entity control, long-horizon memory, and adherence to the prescribed timeline and the visual realization of timestamped engine-recorded events.
Zheng-Hui Huang, Gui-Xu Lin, Yu-Ju Tsai et al.· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.