The results demonstrate that the synthesized skill library enables the system to transfer to novel tasks with decreasing human intervention, providing a steerable and data-efficient alternative to black-box robot learning.
Abstract
While vision-language-action models have demonstrated impressive zero-shot manipulation capabilities, they remain fundamentally black box policies that are difficult to interpret, adapt, or correct when they inevitably fail. In this work, we propose ARCHITECT, a framework that treats robot policy acquisition as an interactive program synthesis task. ARCHITECT leverages the reasoning capabilities of LLM coding agents to synthesize modular robot programs that utilize a suite of perception and control tools. Unlike end-to-end models where distribution shift leads to unpredictable, cascading failures, our modular architecture allows users to isolate failures and localize feedback at the level of abstraction required. We introduce an iterative process where a human supervisor provides natural language corrections to steer the policy. These corrections are grounded in the policy code by program execution traces and distilled into a persistent skill library, a form of long-term in-context learning which enables the agent to accumulate a repertoire of reusable, interpretable behaviors. In a benchmark evaluation on a Franka Panda robot, ARCHITECT outperforms state-of-the-art VLA models and program synthesis baselines on complex, long-horizon tasks, including articulated object manipulation and cloth folding. Our results demonstrate that the synthesized skill library enables the system to transfer to novel tasks with decreasing human intervention, providing a steerable and data-efficient alternative to black-box robot learning. Website: https://robo-architect.github.io/
End-to-end vision-language-action (VLA) and world-action models offer an elegant route to general-purpose robotics, but their reliability is bounded by validated physical coverage. When an unfamiliar object, sensor, embodiment, or contact falls outside that coverage and no validated fallback exists, correcting the failure requires new robot data, a policy update, and regression testing. This recurring burden is the retraining tax. Unlike text, embodied data must often be created by operating machines. We present Teach-and-Grow Learning (TGL), an agent-centered architecture for general robot learning. In its general form, a multimodal agent turns a few successful demonstrations into reusable Skill Blocks: closed-loop behaviors for meaningful subgoals. In a new scene, the agent grounds and composes these blocks, selects learned or geometric tools, observes the physical outcome, and revises the route when execution departs from intent. A Skill Library stores executable behavior, while structured Experience Memory carries forward success, failure, and repair. New tasks are acquired without task-specific policy retraining. Our LIBERO evaluation attains state-of-the-art performance; controlled studies expose skill induction, persistent reuse, and agent-directed adaptation. Finally, we propose the Teach-and-Grow scaling-law hypothesis: if X denotes effective reusable experience, future-task error and teaching demand should approach irreducible floors as power laws in X. The architecture therefore treats deployment as a period of continued learning, in which one task can make the next easier.
A central goal of robot learning is to enable robots to execute rich instructions specified at runtime. Large-scale language-conditioned policies have made substantial progress toward this goal, yet still struggle with temporal structure and safety constraints. Linear Temporal Logic (LTL) provides a powerful language to express complex, non-Markovian instructions. However, guiding learned manipulation policies toward LTL satisfaction remains challenging because modern policies generate short-horizon action chunks and replan in closed loop, while almost all LTL specifications are evaluated over long-horizon trajectories. In this paper, we introduce hint$^2$, a method for guiding short-horizon policies toward satisfying complex LTL specifications at inference time using hierarchical world models. Our key idea is to derive two separate guidance objectives using each world model's abstraction level. A high-level model predicts future action-induced transitions in task-relevant atomic propositions to guide progress through the LTL automaton, while a low-level dynamics model predicts immediate state evolution for accurate local safety guidance. Our results show that hint$^2$ overcomes the limitations of current LTL-guided diffusion methods, outperforms existing inference-time steering methods in CALVIN, and successfully completes instructions with complex liveness and safety constraints more elegantly than language-conditioned alternatives. Finally, we demonstrate that hint$^2$ can handle complex instructions on a real UR5e manipulator.
Moritz Zoellner, Anastasios Manganaris, A. H. Qureshi et al.· 0 citations
VLCP closes the loop where the failure actually lives, on the control code, within a single episode, and keeps the VLM frozen, which is a training-free policy with a tenfold gap between pooled success and confidence intervals in every scene family.
Dhia Naouali, Ming Wu, Claudia Wong et al.· 0 citations
Cross-modal foundation models are increasingly used for robotic task understanding and planning. However, connecting multimodal observations and natural language instructions to symbolic planners and executable robot actions remains a key challenge in embodied AI. In addressing this challenge, existing methods often rely on structured inputs and lack robustness to language ambiguity, with limited real-world validation. We propose VLA-SP (Vision-Language-Action via Symbolic Planning), a two-stage Embodied Vision-Language-Action framework. VLA-SP consists of a Multi-modal Information Processing Module and a PDDL-based Vision-Language-Action Module, enabling fully automated robotic execution from speech and vision inputs. First, our framework detects task-relevant objects using YOLO and transcribes voice instructions with Faster-Whisper. After obtaining the visual observations and language instructions, the LLM fuses these multimodal inputs and converts them into a planner-verifiable problem.pddl instance under a reusable domain.pddl. Then, Fast Downward validates the generated instance and, upon successful verification, produces a multi-step symbolic plan. If any issues are detected during verification, planner feedback is used for traceable failure diagnosis and iterative self-correction. Once a valid symbolic plan is obtained, the symbolic actions are further grounded into 6-DoF poses and executed through predefined robot skills. Finally, we conduct extensive experiments across seven planning domains in simulation and six real-world manipulation tasks on two robotic platforms, X-Arm and Franka. The experimental results demonstrate that VLA-SP enables reliable execution with high success rates across diverse manipulation tasks. In particular, the complete framework achieves an average success rate of 82% across the six real-world tasks. Results from both simulated and real-world experiments fur-ther demonstrate the strong interpretability, executability, and cross-platform applicability of our framework. Further details are available here.
Hanzhuo Zhang, Jiahao Xu, Yicheng Xu et al.· Tsinghua Science and Technol...· 0 citations
A structured taxonomy is presented that organizes existing work into three complementary paradigms that represent dominant architectural tendencies in current LLM-based embodied task planning research, and compares these paradigms along dimension of accuracy, robustness, scalability, efficiency, and sim-to-real transfer.
Zhen Zhang· Applied and Computational En...· 0 citations
Reliable planning requires converting natural-language instructions into executable symbolic specifications, yet large language models remain brittle without costly PDDL annotations and may exploit solver success in semantically unfaithful ways. We study how to learn faithful natural-language-to-PDDL formalization using only solver feedback, without human-written demonstrations. We propose a solvergrounded multi-role reinforcement learning framework where a single language model acts as an Actor, Judge, and Editor for generation, verification, and repair. The Actor proposes PDDL specifications, the Judge provides a solver-calibrated quality signal, and the Editor performs bounded diagnostic-conditioned refinement. On PlanBench, our method improves average success from 35.5% for LLM+P to 70.8%, achieves 66.3% faithful success, and reduces semantic drift to 6.4%. These results show that organizing solver feedback into generation, verification, and repair roles enables more scalable and faithful annotation-free symbolic planning
Cheng Zhang, Yikai Mao, Shan Liu et al.· 0 citations