Skip to content
Preprint

Plan Along the Way: Event-Triggered Foundation-Model Planning for TAMP Execution in Partially Observable Manipulation

Aug 2026 · 1 citation · 23 references
Computer Science

TL;DR

RUST TAMP is presented, a modular LLM/VLM-guided planning framework for reactive TAMP where unseen task-relevant and non-target objects may become visible during execution and the system reconstructs the visible scene state and replans using completed-action history and structured replanning event context.

Abstract

Manipulation in partially observable environments requires planning under incomplete scene information. In such settings, an initially valid plan may execute successfully yet remain insufficient for task completion. Existing foundation-model-guided task and motion planning (TAMP) systems can generate useful long-horizon task decompositions, subgoals, or constraints, but they often assume having access to a fully specified scene state or invoke model-level replanning after a subgoal, refinement, or execution attempt fails. We present ROBUST TAMP, a modular LLM/VLM-guided planning framework for reactive TAMP where unseen task-relevant and non-target objects may become visible during execution. The framework restricts the foundation-model planner to the currently visible relational scene state, validates generated task-level actions against a strict executable interface, and routes the accepted actions to scene-specific execution adapters. Object discovery is treated as a distinct replanning event and, after a stable execution horizon, the system reconstructs the visible scene state and replans using completed-action history and structured replanning event context. Evaluations are performed on six RLBench/CoppeliaSim kitchen and grill variants involving hidden objects, non-target object discovery, articulated-container interaction, and temporal manipulation procedures. We compare text-only LLM and VLM planners of different sizes under the same validation, execution, monitoring, and replanning pipeline, reporting task success, partial goal completion, discovery- and failure-triggered replanning behavior, implicit non-target-object handling, and planner inference cost.

View source

Similar papers

Preprint Oct 2026

OntoPlan: An Ontology-Grounded Scene Representation and Agentic Framework for Scalable Robot Task Planning

Large language model (LLM)-based robot task planning is promising for open-ended instruction following, but degrades on long-horizon tasks in large environments. When spatial information is conveyed to the LLM through text, the model can fail to capture spatial context, and token cost grows with environment size. Gener...

Hyeongwoo Nam, Woongje Cho, Juwon Kim et al. · 0 citations
Preprint Sep 2026

X-Planner: Event-Structured Task Planning for Embodied Intelligence

This work presents X-Planner, a planning front-end that addresses both the supervision and representation of embodied reasoning, and describes planning-text quality and downstream execution.

Howard Lu, Shalfun Li, P. Pan et al. · 0 citations
#machine learning Preprint Oct 2026

Mind the Refinement Gap: When Safe High-Level Robot Plans Produce Unsafe Executions

Language-enabled robot systems increasingly combine semantic-graph planning with temporal-logic safety monitors. We investigate a trace-completeness assumption in these systems: whether the high-level action sequence checked by a monitor represents the navigation and implicit action effects induced during execution. We...

Stabak Das, Priyesh Ranjan, Xiang-Fang Li et al. · 0 citations
Sep 2026

CommitFlow: Semantic Commitment Verification and Local Correction for Long-Horizon Robot Manipulation VLA Execution

commitFlow, a closed-loop execution framework that combines commitment monitoring with local correction while keeping the base policy frozen, and cross-policy experiments show consistent gains, pointing toward reliable long-horizon robot execution.

Zi-Xi Zhao, Yan-Song Feng, Yang Yang et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.