Skip to content

Library of Hands: Fix the Hands, Not the World Model

Nov 2026 · 0 citations · 47 references
Computer Science

TL;DR

Library of Hands (LoH), a training-free method that keeps the world model frozen and corrects only the map from commands to applied actions, which is called the hand, is proposed.

Abstract

A world model predicts how a scene changes after an action, and a planner uses it to search for actions that reach a goal. If the camera moves or the controller changes, the same command moves the robot differently: when its commands are turned by 90 degrees, a frozen planner with the LeWM world model that succeeds in 77.5% of episodes succeeds in only 7.8%. Current fixes retrain the model, need training data, or forget a set-up when it returns. We propose Library of Hands (LoH), a training-free method that keeps the world model frozen and corrects only the map from commands to applied actions, which we call the hand. LoH reads the hand through the frozen model, with no extra sensor: it finds the action that best explains the change between two camera images (a reading). It stores every hand it identifies, so a returning set-up is recognised rather than re-learnt. In closed-loop tests, LoH more than doubles success on three simulated tasks, from 28.6% to 62.3%, and loses only 0.6 percentage points when nothing changes. It also works, unchanged, with two other world models and on two of three models pretrained on real robot data, and its gain grows with the quality of its readings.

View source

Similar papers

Preprint Nov 2026

How Uncertain Is My Action? Self-Correcting Visual Planning with Frozen World Models

A frozen latent world model combined with sampling-based model predictive control is a visual planner. At deployment, the executed action can depart from the commanded one, and the frozen model has no way to notice. We show that the model can be run in reverse: for any hypothesized actuation shift, it predicts what it...

Si-Qing Li, Lu-Ming Yang, Yi-Jia Jiao et al. · 0 citations
Preprint Oct 2026

GreenWAM: A World Foundation Model for Robotics

A robot needs a world model that predicts how its actions will change the scene and a policy that chooses those actions. Models are usually built and evaluated for only one of these roles, and every robot records its actions in its own format. We ask whether one robot world model can serve both roles across different b...

G. Sidorov, В. Корнилов, Denis Grankin et al. · 1 citation
Preprint Nov 2026

Motion Belief Action: From a Visual Scene Map to Physical Decisions

A robot that works among people must judge what it cannot see: whether an object is still where it was last seen, and whether a pull on a door or drawer is worth its risk. Maps that track object change hold nothing about force, maps of physical properties answer a risk question at a fixed risk level, and vision-languag...

Dibyendu Ghosh · 0 citations
Review Oct 2026

Learning Dexterous Manipulation: Hands, Simulators, Training, and Evaluation

A dexterous hand can change an object's pose without setting it down. This survey covers 113 learning-based methods that attempt it, the hands they run on, the simulators they train in, and how they report results. The field has largely converged on one recipe: reinforcement learning on privileged state in a GPU simula...

Luai Abuelsamen · 0 citations
Preprint Nov 2026

Towards Pose-Robust Few-Shot Vision-Language-Action Models

Fine-tuning a vision-language-action (VLA) model on a few dozen demonstrations is the standard way to put a manipulation policy on a robot, yet the resulting policy silently assumes that the camera and the robot base stay exactly where they were during data collection: a bumped tripod or a base re-docked a few centimet...

Wen-Hao Li, Zhong-Yi Han, Yu-Hao Wei et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.