Skip to content

EWAM: Emergent Depth-Wise Specialization in a Unified Embodied Model - From Semantic Understanding through Visual Foresight to Action

Sep 2026 · arXiv.org · Vol abs/2609.39973 · 1 citation · 75 references
Computer Science

TL;DR

EWAM is an action-centric unified embodied model whose asymmetric joint attention lets action tokens read semantic, current-visual, predicted-future, and action information at every layer while the perceptual experts retain their distinct roles, suggesting that unified embodied learning can induce an ordered internal progression from semantic understanding, through visual foresight, to action formation.

Abstract

Vision-language-action (VLA) policies emphasize semantic understanding, whereas world-action models (WAMs) learn predictive representations of environment dynamics. Systems that expose a policy to both sources often still concentrate action computation on a single expert. We present EWAM, an action-centric unified embodied model whose asymmetric joint attention lets action tokens read semantic, current-visual, predicted-future, and action information at every layer while the perceptual experts retain their distinct roles. Without layer-wise supervision, EWAM develops an emergent depth-wise specialization: action queries attend mainly to vision-language features in shallow layers, to predicted future frames in intermediate layers, and to action tokens themselves in deep layers. This handoff replicates across tasks and is stable across denoising steps. Checkpoint tracking and causal interventions show that it is learned and that action generation depends on it. EWAM is pretrained in two separate regimes, one on cross-embodiment robot trajectories and one on human egocentric video. In simulation and real-robot experiments, it surpasses existing VLA, WAM, and hybrid baselines. Human egocentric data improve both cross-embodiment transfer and real-robot robustness, and subtask-phase supervision improves long-horizon completion. Together, these results suggest that unified embodied learning can induce an ordered internal progression from semantic understanding, through visual foresight, to action formation.

View source

Similar papers

Preprint Oct 2026

UniWAM: Unified World-Action Model

UniWAM is introduced, a unified architecture that integrates a physical reasoner, a world generator, and an action predictor to jointly learn semantic understanding of the physical world, visual generation, and action prediction and uncovers a log-linear scaling law of unified human-robot co-training.

Wen-Xuan Song, Jia-Yi Chen, Jing-Bo Wang et al. · 9 citations · ⚡3
Preprint Nov 2026

From Pixels to Actions--Towards Native Vision-Language-Action Policies

Vision-language-action models are typically built on a simple premise: perception is first compressed by visual encoders, and action reasoning begins only afterwards. Yet manipulation often hinges on precisely the visual signals most vulnerable to this abstraction, e.g., local geometry, boundaries, depth, and subtle st...

Wei-Chen Fan, Hai-Wen Diao, Peng-Hao Wu et al. · 0 citations
Preprint Oct 2026

Evo-World: A Lightweight Vision-Language-Action Model with Future Motion Prediction via Optical-Flow Supervision

Vision-language-action (VLA) models provide a unified framework for visual understanding, language grounding, and robot control, yet anticipating how the scene will evolve beyond the current observation can provide additional information for control. Action supervision alone does not explicitly encourage such anticipat...

Zi-Le Zhong, Yu-Xin Du, Tao Lin et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Rewiring Semantics, Dynamics, and Control: A Simple yet Effective Action-Centric Tri-Stream Transformer

Vision-Language-Action (VLA) models have emerged as a prominent framework for complex robotic manipulation, building on the strong semantic understanding of pretrained Vision-Language Models (VLMs). However, such VLM backbones offer insufficient physical dynamics priors, which limits the generalization capabilities of...

Shuang Luo, Yi-Lun Kong, Yun-Peng Qing et al. · 0 citations
Preprint Oct 2026

HiFOR: Hierarchical Foresight with Action-Guided World Grounding for Vision-Language-Action Model

Vision-Language-Action (VLA) models have demonstrated promising capabilities in robotic manipulation by integrating visual perception, language understanding, and action generation. Recent advances have further incorporated future prediction to enhance the understanding of environmental dynamics and task progression. H...

Rong-Xing Ding, Hong-Yu Qu, Ke-Hong Wu et al. · 1 citation
#artificial intelligence Preprint Nov 2026

Concertina: Embodying Manipulation and Cognition in One Diffusion Vision-Language Model

Most vision-language-action (VLA) policies decouple manipulation from cognition by bolting an external action mechanism onto a vision-language model (VLM)--either quantizing actions into discrete tokens whose errors bear no stable correspondence to physical displacements, or delegating continuous trajectory generation...

Cheng-Cheng Wang, Wei He, Wen-Hao Li et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.