Sep 2026· Applied and Computational Engineering· 0 citations
TL;DR
This paper organizes the literature according to the state available to planning: future observations, compact latent states, geometry-structured maps, and persistent entities or relations, which clarifies where temporal prediction, persistent geometry, and semantic identity complement one another in embodied planning.
Abstract
Embodied robots need more than a description of the current image: they must estimate how the scene may change under motion, contact, and partial observation. Vision-centric world models provide this predictive layer, but they expose it through different state interfaces. This paper organizes the literature into four families according to the state available to planning: future observations, compact latent states, geometry-structured maps, and persistent entities or relations. The comparison examines how each state is formed, advanced, and queried; where inference and planning costs arise; and which errors matter in physical use. Representative methods show that observation prediction retains interpretable appearance but makes repeated rollout expensive. Latent dynamics reduce that cost while risking the loss of contact-scale variables. Geometric states support pose, clearance, and occupancy queries, although their reliability depends on calibration and timely updates. Entity-relational states preserve object identity and task relations, yet they remain vulnerable to binding errors under occlusion. Evaluation is therefore linked to the exposed state rather than to a single generic score. The resulting framework clarifies where temporal prediction, persistent geometry, and semantic identity complement one another in embodied planning.
This work studies action-conditioned predictors built on five frozen self-supervised visual backbones, finding that a world model guides action selection reliably only when the goal lies within, or slightly beyond, the trajectory it imagines during planning.
A. Alrasheed, Basim Azam, Naveed Akhtar· 1 citation
VA-Bench is introduced to evaluate the complete observe-reason-act-revise loop and tests whether general-purpose MLLMs can turn visual demonstrations and actively acquired evidence into successful embodied action.
Zhong-Bo Zhang, Jia-Yi Jin, Yi-Fan Wang et al.· 1 citation
MachEmbodied-U0, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture, is presented, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture that combines competitive...
Hao-Ran Wen, Wen-Fu Wang, Kun-Song Shi et al.· 1 citation
We introduce BIND, a new action representation for visuomotor robot policies that binds 3D robot actions to their corresponding 2D image features, yielding strong data efficiency gains and robustness to out-of-distribution object positions and camera viewpoints. The action heads of current robot policies are typically...
C. Smith, Arsh Tangri, V. Guizilini et al.· 0 citations
Autonomous driving requires more than recognizing what is present in a scene: a planner must determine how road structure, surrounding agents, and their motion states should influence a future maneuver. Existing learning-based planners can capture these influences through latent scene features and trajectory decoders,...
Zhi-Yuan Liu, Yuan-Xin Tian, Ze-Hong Ke et al.· 0 citations
Latent action world models let agents plan new behaviors at test time by predicting how actions change the environment, and joint-embedding predictive architectures (JEPAs) do so by forecasting future latent states rather than pixels. Yet nearly all such models see the world through a camera, even though robotic manipu...
F. F. Oberweger, Michael Schwingshackl, Markus Murschitz· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.