DeformGrasp: A Human-Guided and Synthetic Data Framework for Learning-Based Deformable Object Grasping Using Anthropomorphic Robot Hands

Abstract

Object grasping and manipulation are fundamental operations for daily activities. Individuals with impaired grasping ability could be assisted and benefited by using prosthetic anthropomorphic robot hands (ARHs). However, controlling a high-degree-of-freedom ARH in unstructured environments using model-based methods is challenging because object-specific interaction models must be developed in advance and accessed in real time. This research presents an imitation learning framework for developing a model-free behavioral cloning (BC) policy that generates grasp strategies for three-dimensional deformable objects. The proposed four-stage framework includes human-guided grasp synthesis, synthetic grasp augmentation, BC policy training, and BC policy evaluation. Human demonstrations are captured using a user-interaction glove, an NVIDIA Isaac SimTM-based simulation environment, and an NI LabVIEW interface to transfer finger-motion and feedback data between the wearable and simulation environments. Fifty four (54) successful human-guided demonstrations were performed for cylindrical and cuboid deformable objects with variations in size, positional offset from the palm, and Young’s modulus. The human-guided demonstrations formed the basis to generate 200 additional successful grasps through contact-driven synthetic augmentation to form a 254-demonstration dataset for BC policy training and evaluation using an 85% and 15% split, respectively. The trained BC policy was assessed using offline action-error metrics and in-simulation deployment on held-out object configurations, achieving stable grasp execution rates of 68% (15 of 22) for cylindrical and 56% (9 of 16) for cuboid objects. These results demonstrate that contact-driven augmentation and BC can support ARH grasping of three-dimensional deformable objects despite an initial small-number human-demonstration dataset.