Jul 2026· 2026 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM)· pp. 1-7· 0 citations· 37 references
Computer Science
TL;DR
A unified monocular vision-based grasping framework that targets both soft and rigid objects within a single control pipeline, using only RGB input and a position-controlled gripper, and is validated in real-world pick-and-place experiments.
Abstract
Grasping in unstructured environments requires handling objects with widely different mechanical properties, from soft and deformable items to rigid everyday objects. Most existing approaches address these categories separately and often rely on tactile sensing, object-specific models, or specialized grippers. In this paper, we present a unified monocular vision-based grasping framework that targets both soft and rigid objects within a single control pipeline, using only RGB input and a position-controlled gripper. The proposed system combines open-vocabulary object detection, image segmentation, boundary-aware point assignment, real-time point tracking, and monocular depth estimation to recover object motion and geometry from visual observations. A key component of the framework is a language-based stiffness estimation model that infers an object’s expected compliance from its semantic description and provides an object-level prior for selecting the grasping strategy before contact. For deformable objects, grasp adaptation is governed by a Procrustes-based dissimilarity measure computed from tracked keypoints, which acts as a visual proxy for deformation. For rigid objects, the gripper width is regulated through the scaling of tracked point distances. We validate the proposed method in real-world pick-and-place experiments on a Franka Emika Research 3 arm using objects with substantially different mechanical properties.
A deep learning-based grasp estimation model designed to enable robotic manipulation with articulated objects that incorporates the attention-based semantic and geometric feature fusion (ASGF) module improved the grasp success rate in the evaluated setting.
Dongwoo Lee, Yeongmin Kim, Seong-Bo Jo et al.· IEEE Access· 0 citations
This work proposes a real-world bimanual grasping framework that includes a multimodal dataset capturing joint angles, visual observations and force signals; a Denoising Diffusion Probabilistic Model (DDPM)-based module that generates joint-level grasp configurations from segmented point clouds; and an execution strategy that integrates motion planning with online grasp refinement to ensure physical stability and feasibility.
Ziming Li, Mingxuan Wu, Jiaqi Zhang et al.· 0 citations
A novel 7-DoF grasping pose generation framework that integrates sparse attention and null convolution is introduced, which enhances the model’s ability to capture fine-grained features from point clouds, significantly improving the accuracy of parallel gripping pose estimation.
Hui Zhang, Yue Wang, Kang An et al.· Signal, Image and Video Proc...· 0 citations
This work proposes PartialBiGrasp, a dual-arm grasp generation framework that operates directly on partial point cloud observations that learns geometric features implicitly through convolutional occupancy networks, enabling local reasoning about graspability, collision-free contact regions, and object thickness.
Robotic grasping of small, low-feature objects requires highly precise grasp pose estimation to ensure reliable in-gripper alignment. Vision-based approaches (e.g., YOLO-derived detectors) can localize the object but often fail to recover the exact grasp position under occlusion or partial views, while tactile-only methods lack global context; moreover, existing work rarely evaluates, in a task-specific and systematic manner, which fusion configurations are most suitable for this setting. Motivated by this gap, this paper investigates in-hand pose estimation of a USB stick that is already held within a robotic gripper; initial grasp detection is outside the scope of this work. A deep-learning regression model based on YOLOv11 as a feature extractor was developed to estimate the grasp position using inputs from an RGB eye-in-hand camera and a tactile sensor. Within the defined experimental setup, the early RGB–tactile fusion model selected via NAS achieved the lowest summed lateral error of 1.02 mm, compared to tactile-only (1.35 mm) and RGB-only (1.31 mm) models trained under identical protocols. These results reflect performance under the dataset’s image-level partitioning and indicate improved lateral localization within the evaluated setup. The study does not claim a universally optimal fusion strategy, but provides a controlled task-specific analysis showing that structured model selection can improve multimodal grasp-pose regression within the evaluated setup.
A. Altenbuchner, Bsher Karbouj, Fabian Dilly et al.· IEEE Access· 0 citations