Skip to content
Open access

Multi-Step Obstruction Reasoning for Target-Oriented Grasp Sequence Generation in Cluttered Scenes

Sep 2026 · Robotics · Vol 15, pp. 181 · 0 citations · 6 references

TL;DR

A novel DAG-based obstruction reasoning framework coupled with an integrated diagnostic evaluation protocol is introduced, which model scene-level physical dependencies as Directed Acyclic Graphs (DAGs), explicitly converting graph constraints into topologically feasible sequence permutations to drive VLM fine-tuning.

Abstract

Retrieving target objects in severe clutter requires multi-step reasoning to establish valid obstacle removal sequences. While recent Vision–Language Models (VLMs) have advanced instruction-driven clutter grasping, existing paradigms lack explicit construction of graph-constrained grasp sequences encompassing canonical trajectories and valid topological permutations to guide model fine-tuning and evaluation. In addition, comprehensive evaluation requires accounting for the full space of topologically valid clearing sequences while systematically disentangling high-level topological planning errors from low-level physical execution failures. To fulfill these requirements, we introduce a novel DAG-based obstruction reasoning framework coupled with an integrated diagnostic evaluation protocol. Specifically, we model scene-level physical dependencies as Directed Acyclic Graphs (DAGs), explicitly converting graph constraints into topologically feasible sequence permutations to drive VLM fine-tuning. For diagnostic evaluation, we establish a three-part offline protocol comprising Strict Exact Match (EM), Graph-Feasible Accuracy (GFA), and Multi-Reference Normalized Sequence Edit Distance (MR-NSED), paired with online simulation testing. Extensive experiments demonstrate that our topological fine-tuning significantly improves multi-path reasoning performance, outperforming strong foundation model baselines including GPT-4o, GPT-4o-mini, and Llama-3.2-90B-Vision-Instruct, while our diagnostic protocol provides a faithful mechanism to systematically isolate reasoning logic from manipulation mechanics in complex physical clutter.

Read PDF

Similar papers

Preprint Oct 2026

SCOPE: Scene-graph Construction Optimized by Plan Error-correction for Robot Long-term Planning Datasets

Task and Motion Planning (TAMP) stands as a fundamental challenge in robotics, essential for enabling autonomous agents to execute complex, long-horizon tasks in diverse environments. While Large Language Models (LLMs) introduce a new paradigm for TAMP by leveraging extensive common sense, their inherent reasoning limi...

Chao-Yue Li, Ze-Yu Gao, Yan-Feng Lu et al. · 0 citations
Sep 2026

An Information-Space Perspective to Scene Graph Sufficiency for Robotic Task Planning

This paper provides a formalization of what constitutes a sufficient scene graph for planning by modeling planning over scene graphs within an information-spaces framework through the definition of scene graph transition systems and relevant action semantics for navigation and manipulation.

Basak Sakçak, Francesco Verdoja · 0 citations
Sep 2026

DualManip: Agentic Dynamic Manipulation via Dual-Path Semantic Reasoning and Geometric Adaptation

Vision-language models (VLMs) enable open-vocabulary reasoning for robot manipulation, but their high inference latency limits responsiveness in dynamic scenes. Many scene changes, however, alter object geometry without invalidating task intent. We present DualManip, a dual-path framework that decouples infrequent sema...

Cheng-Xi Li, Yan Di, Ying-Yue Li et al. · 0 citations
#artificial intelligence Preprint Nov 2026

CPNav: Calibrated Vision-Language Navigation for Object-Goal Search in Unseen Environments

CPNav is introduced, a navigation framework that combines collision-aware planning with statistically calibrated VLM action scoring for map-free object-goal navigation and provides finite-sample guarantees at a user-specified risk level.

Qi-Zhao Chen, Yuan-Hong Zeng, Shoh Nishino et al. · 0 citations
Sep 2026

Search, Ground, Plan: Functional Sufficiency for Task and Motion Planning under Incomplete Scene Knowledge

GRAB-TAMP is introduced, an FM-based TAMP framework that searches for scene entities required for task completion, grounds functional roles to valid physical objects, and plans only after a complete joint assignment establishes functional sufficiency.

N. Vijayakumar, Nav Singhal, G. Varma et al. · 0 citations
#artificial intelligence Conference Open access Oct 2026

Embodied Zero-Shot Task Planning via Hardware-Aware LLM–VLM Integration for Grounded Robotic Manipulation

Large language models can translate natural-language instructions into plausible manipulation plans, but robotic execution requires physically grounded decisions that respect robot embodiment, workspace geometry, reachability, collision constraints, and grasp feasibility. This paper presents a hardware-aware large lang...

Lonnie Tankersley, Noorbakhsh Amiri Golilarz, Soroush Korivand · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.