AntiGrounding: Executable Robot Trajectories as Visual Prompts for VLM-Guided Manipulation
TL;DR
AntiGrounding, a visual action-selection framework built around a dual geometric--visual trajectory interface, is presented, where component ablations and evaluator sensitivity characterize trajectory assessment, proposal search, orientation control, and evaluator choice.
Abstract
Natural-language instructions specify manipulation goals but leave the robot's motion underdetermined. We present AntiGrounding, a visual action-selection framework built around a dual geometric--visual trajectory interface. Each short trajectory retained after feasibility filtering remains an explicit motion plan and serves as a visual prompt for instruction-conditioned vision--language model (VLM) assessment. Structured multi-view visual question answering (VQA) scores safety, task alignment, efficiency, and physical plausibility. Weighted view fusion aggregates these scores for trajectory selection, while the scores also guide subsequent translational proposals. Separate orientation and gripper controls coordinate physical interaction. Planning proceeds in an initialized digital twin, which validates selected segments before the real robot executes the same waypoint sequences. Across eight real-world manipulation tasks, AntiGrounding with a single GPT-6 Astra evaluator achieves \AstraOverall\% overall success. Under the reported deployment protocol, $\pi_{0.5}$ achieves \PiOverall\%, and a PIVOT-style visual proposal-selection baseline with the same evaluator achieves \PivotOverall\%. Component ablations and evaluator sensitivity characterize trajectory assessment, proposal search, orientation control, and evaluator choice. Performance depends on digital-twin fidelity and physical interaction.
