Skip to content
Preprint

Keep the Effect, Drop the Actor: Programmable Effect-to-Execution World-Action Models

Oct 2026 · 0 citations · 46 references
Computer Science

TL;DR

A robot demonstration records two things in the same frames: what happened to the objects, and how one particular arm made it happen, and how one particular arm made it happen is conditioned.

Abstract

A robot demonstration records two things in the same frames: what happened to the objects, and how one particular arm made it happen. We condition on the first. A demonstration is compiled into an effect program: the 3D keypoint trajectories of the objects that moved, two points marking where each was held, and the configuration the scene ends in, with the demonstrator removed. PEWAM, a 71.5M-parameter world-action model, generates effect, robot execution, action and terminal state as four streams with independent flow-matching times, so clamping a program and sampling the execution turns inference into programming, re-solved closed loop from the live scene. On held-out LIBERO-Goal tasks, one demonstration's program completes 40 of 90 episodes, where the same backbone given a goal image or language, and published demonstration-conditioned methods, complete at most 19; on three of Meta-World's held-out classes it exceeds the best published results, though not on the five-class mean. Because a program is a set of coordinates, a person can edit it: the placement follows a shifted terminal state and the grasp turns with rotated contact points. The same program runs on four robot arms without retraining, and after a push, re-solving completes 23 of 60 episodes where replaying the demonstration completes 6. On a Franka arm, fine-tuned on real demonstrations of other tasks, programs compiled from single human videos complete 36 of 40 trials, against 22 for the same backbone conditioned on the video's last frame as a goal image.

View source

Similar papers

Preprint Nov 2026

Localizing and Editing the Goal Representation of a Vision-Language-Action Model

Operators can state safety constraints for robots as prohibitions such as"do not put the bowl on the stove". We study whether OpenVLA follows such prohibitions and where it represents the goal it executes. The 10 tasks of LIBEROGoal share one scene, so we can label the goal that each rollout executes. We combine closed...

Sudhanshu Srivastava, F. Aqlan · 0 citations
#artificial intelligence Preprint Oct 2026

Recursive Video In-Context Learning for Agentic Robot

LLM agents that orchestrate frozen vision-language-action (VLA) policies improve across episodes through text memory, which records what the agent did but not how the task is done. A demonstration video shows it, but fits poorly into an agent's context. The full video slows every turn, fixed keyframes lose the contact...

Wen-Rui Bao, Xin-Xin Liu, Bing-Xin Xu et al. · 2 citations
#artificial intelligence Preprint Oct 2026

Let the Mind Conceive, Let the Hand Achieve: Following Multimodal Instructions for Long-Horizon Robotic Manipulation

A robot following a multi-step instruction needs to act on the right objects in the right order, even as the scene changes. It must also maintain awareness of task progress and determine when to advance. To address these challenges, we introduce Orchestra, an agent harness that separates task planning and execution man...

You-Peng Wen, Gang Wang, Tian-Yu Huang et al. · 0 citations

LIBERO-MAX: Do Robot Policies Adapt When the World Changes?

Robots must often continue a task after a target moves, the viewpoint shifts, or an obstacle appears, even though their earlier observations and committed actions reflect the previous scene. Many simulation robustness benchmarks fix external conditions at reset, leaving this temporal challenge underexamined. We introdu...

Yun-Bei Zhang, Zi-Jian Jin, Yuan-Zhe Liu et al. · 3 citations · ⚡1
Sep 2026

Watch, Recall, Act: Always-On Robots in Concurrent Embodied Streams

An always-on robot faces an endless stream that never resets: instructions arrive and lapse, the scene changes, and its own past actions reshape what it must reason about. Today's action models are built for the opposite: a fixed instruction, no mid-task intervention, single-step reasoning. In an open-ended world a rob...

Ding Yi, Pei-Wen Sun, Chen-Chu Rong et al. · 0 citations
#artificial intelligence Preprint Nov 2026

Library of Hands: Fix the Hands, Not the World Model

Library of Hands (LoH), a training-free method that keeps the world model frozen and corrects only the map from commands to applied actions, which is called the hand, is proposed.

Hasaan Ahmad, Qi-Zhen Ying, Yu-Hang Sun et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.