Skip to content
Preprint

FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning

Oct 2026 · 0 citations · 38 references
Computer Science

TL;DR

FlowVLA is introduced, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics, designed to utilize temporally aligned 3D flow only during training.

Abstract

Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on 2D image-text priors. Therefore, they do not accurately represent the fundamental dynamics of the 3D world associated with physical actions. This over-reliance on static 2D visual cues limits the capacity of VLA models in achieving robust robot manipulation. To address these limitations, we introduce FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics. Instead of requiring explicit 3D sensor inputs or altering the inference-time architecture, our method is designed to utilize temporally aligned 3D flow only during training. This allows the model to learn the dynamic evolution of the physical scene while preserving the computational efficiency of conventional 2D VLA backbones. To bridge scene-level dynamics with robotic manipulation, we propose a modular 3D flow learning framework with dual-level flow supervision. Specifically, point-level flow guides the model to forecast the future displacements of task-critical 3D points, while TCP-level flow aggregates these motions to align with the robot's TCP trajectories. This dual-level flow supervision encourages the model to focus on semantically meaningful dynamic regions, effectively coupling scene-level motion with precise control signals. Extensive evaluations across simulation and real-world experiments demonstrate that FlowVLA consistently improves manipulation performance, zero-shot robustness, and optimizes data efficiency. FlowVLA adds only lightweight auxiliary heads during training and discards them at inference. Therefore, FlowVLA introduces no additional 3D sensor inputs, architectural modifications, or inference-time computation.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

DepthWorld: 3D World Model for Robot Manipulation

This work introduces a calibration pipeline that combines learned stereo depth with a joint factor graph, pooling all episodes collected from the same physical robot to recover its shared kinematic parameters alongside per-scene extrinsics, and trains DepthWorld, a Stable Video Diffusion-based world model that jointly...

Jai Bardhan, Josef Sivic, Vladimír Petrík · 2 citations
Sep 2026

PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics

It is shown that a flexible and expressive transformer, PointZero, outperforms prior methods on the same data and demonstrates the utility of the pre-training objective by post-training PointZero for two downstream applications: action-conditioned 3D dynamics prediction and imitation learning.

B. P. Duisterhof, Kai-Feng Zhang, Adam Hung et al. · 6 citations · ⚡1
#artificial intelligence Preprint Sep 2026

3DWay: Generalizing Robot Manipulation via 3D Consistent Waypoints

Predicting 3D consistent waypoints (3DWay) from multi-view images as generating multi-view consistent 2D waypoints followed by geometric triangulation enables explicit 3D motion specification while preserving the strong priors of pretrained VLMs.

Zi-Qin Huang, Ying-Yue Li, Chenyangguang Zhang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

WorldFlow: Joint Video and 3D Motion Prediction via Large-Scale Pretraining

WorldFlow achieves the lowest average displacement error (ADE) and final displacement error (FDE) among the evaluated learning-based methods, and is introduced as a generative world model that jointly predicts future video and query-based 3D trajectories.

Yu-Hao Lu, Jiong-Xu Chen, He-Xiang Wei et al. · 0 citations
Preprint Oct 2026

GeoMem-VLA: Internalizing 3D Scene Understanding into Temporal Memory for Vision-Language-Action Models

Geometric structure is essential for robotic manipulation and understanding the 3D world. Most VLA models rely on 2D representations of current observations, while geometry-aware approaches often require additional 3D information. However, geometry inferred independently from individual observations is often discarded...

Qin-Fang Zhang, Yi-Fan Luo, Hao-Yuan Shi et al. · 2 citations
Preprint Sep 2026

Bridge3D: Enabling Vision-Language-Action Models to See and Act in 3D

This paper proposes Bridge3D, a method that integrates both implicit and explicit 3D geometry guidance into pre-trained 2D VLA models, enabling them to see and act in 3D, and introduces two strategies that enriches visual tokens with features from 3D foundation models to improve seeing and acting in 3D.

Hao-Xuan Li, Si-Xu Yan, Liang-Hui Zhu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.