Conditional Flow Matching for Grasp Pose Generation in Multi-Workpiece Scenes

Abstract

While most robotic research focuses on household tasks such as bus table arrangement and cloth folding, numerous manipulation tasks remain challenging for industrial applications, particularly the grasping and transportation of scattered workpieces. In this paper, we propose SGDIFF, a vision-guided grasping architecture that integrates a pre-trained vision-language model (VLM) with flow matching-based diffusion model. Given an RGB-D image of a tabletop scene, the VLM first detects each workpiece, outputs its bounding box, and assigns a unique ID. A point cloud is then generated for each detected instance. Subsequently, flow matching model iteratively refines the initially noisy gripper pose to a stable and collision-free grasp for each target workpiece. The proposed method eliminates the need for object-specific models and enables efficient multi-object grasping in cluttered industrial environments. The experiments validates the effectiveness of combining semantic understanding with precise pose refinement for robust industrial automation.