Real2Sim2Real for Vision-Language-Action Manipulation: An AMD ROCm-Based Pipeline

Qingwen Yang Xunyue Wang Ziguan Wang Zhenjiang Li Hongqiang Wang Dongdong Weng

Jul 2026 · arXiv.org · Vol abs/2607.22997 · 0 citations · 10 references

Computer Science

TL;DR

An end-to-end, fully AMD-accelerated technology stack for embodied manipulation, spanning data-center training silicon, Radeon PRO simulation/rendering GPUs, and Ryzen AI edge compute, unified by the open ROCm software stack is presented.

Abstract

Physical AI -- the integration of large vision-language-action (VLA) models with embodied agents that act in the real world -- has emerged as the next major frontier for AI, echoed by industry leaders such as Jensen Huang (``the next big thing is Physical AI, AI with a body,''GTC Paris, June 2025) and Dr. Lisa Su (`we're entering the world of Physical AI ... this is where AI enters the real world,'CES 2026). This paper presents an end-to-end, fully AMD-accelerated technology stack for embodied manipulation, spanning data-center training silicon, Radeon PRO simulation/rendering GPUs, and Ryzen AI edge compute, unified by the open ROCm software stack. We demonstrate that training and deploying VLA-based manipulation policies does not require a CUDA-locked ecosystem. Four progressive demonstrations are presented: (1) a Sim-to-Real manipulation pipeline trained with SmolVLA and deployed on a physical Franka arm; (2) a semantic, language-grounded object-selection task (`one-of-three'); (3) a Real2Sim synthetic-data generation pipeline that fuses 3D Gaussian Splatting (3DGS) reconstructions of real scenes with the Genesis physics engine; and (4) large-scale reinforcement learning for quadruped and humanoid locomotion benchmarked across multiple hardware platforms. All pipelines run natively on ROCm + PyTorch on RDNA4 (Radeon AI PRO R9700) and RDNA3.5 (Radeon PRO W7900) hardware and are reproducible on the free Radeon Cloud Platform.

DOI arXiv Semantic Scholar

View source

Similar papers

Preprint Aug 2026

StellaVLA: In-Context Structured Demonstration for Generalizable Vision-Language-Action Models

The real-robot benchmark demonstrates that StellaVLA can use both human/robot demos and human-to-robot (XR) demos as in-context structured demonstration to help VLA model adapt to OOD tasks.

Siyu Xu, Yun-Ke Wang, Zi-Jian Wang et al. · 3 citations

Efficient and Robust VLA Models for Real-Time Robotic Manipulation

This paper aims to reduce latency and the dynamic memory usage of the π 0.5 model by performing timestep distillation on the diffusion process in the π 0.5 action head to create a 3-step diffusion action head.

Shriyadita Medavaram, Lam Nguyen, Yu-Zhang Shang · 0 citations

#artificial intelligence Preprint Sep 2026

FluxVLA Engine: A One-Stop VLA Engineering Platform for Embodied Intelligence

FluxVLA Engine is presented, an open, configuration-driven platform that turns heterogeneous embodied-policy components into a reproducible data-to-deployment workflow and targets the engineering bottlenecks separating promising embodied-learning algorithms from reproducible evaluation and dependable deployment.

Yin-Hao Li, Wei-Xin Mao, Zi-Han Lan et al. · 3 citations

Preprint Sep 2026

R2S-Eval: Robot Evaluation with Real-to-Sim Calibration via Vision-Language Models

Experiments in both simulation and real-world settings demonstrate that R2S-Eval produces reliable and stable policy conclusions, achieves agreement with human preferences, substantially reduces repeated hardware-operation effort, and reveals behavior-quality differences that are not captured by binary success labels.

Yi-Di Wang, Fei-Xiang Ruan, Ruo-Qu Chen et al. · 2 citations

Jul 2026

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

TurboVLA is introduced, a new VLA paradigm that reformulates the conventional LLM-centric pathway as a direct VLA mapping, and constructs task-conditioned representations directly from visual and linguistic features, significantly reducing the computational and memory costs of VLA inference.

Heng-Yi Xie, Chen-Fei Yao, Xian Wu et al. · arXiv.org · 6 citations · ⚡2

Preprint Aug 2026

GaussVLA: Geometry-Aware Spatial Reasoning for Vision-Language-Action Model

GaussVLA is proposed, a Mamba-based VLA that incorporates two custom modules: Gaussian Spatial Tokenizer (GST) to lift frozen semantic and depth features into compact 3D Gaussian tokens, and Depth-Aware Chain-of-Thought (DA-CoT) that performs structured, non-autoregressive geometric reasoning under language and flow-ti...

MD SELIM SAROWAR, Md Tanvir Islam, Sungho Kim et al. · 1 citation