Bimanual Tactile-Augmented Teleoperation for Contact-Rich Robotic Manipulation: A Pilot Evaluation
TL;DR
BiTAT, a bimanual tactile-augmented teleoperation system for collecting multimodal human demonstrations and learning manipulation policies, and a multimodal Diffusion Policy that fuses visual, tactile, and proprioceptive features through a Transformer encoder are presented.
Abstract
Recent advances in robotics have highlighted the importance of multimodal perception for dexterous manipulation in contact-rich environments. Here we present BiTAT, a bimanual tactile-augmented teleoperation system for collecting multimodal human demonstrations and learning manipulation policies. The system integrates custom capacitive tactile sensors into parallel grippers and displays the resulting contact-deformation images to the operator. We evaluated the system in four controlled laboratory tasks: USB removal/insertion, bottle cap unscrewing, cucumber peeling, and toothpaste squeezing. In a pilot repeated-measures study with eight laboratory participants, visual tactile feedback was associated with success-rate increases of 12.5–32.5 percentage points and shorter completion times among successful trials. We further propose a multimodal Diffusion Policy that fuses visual, tactile, and proprioceptive features through a Transformer encoder. In two fixed-layout autonomous tasks, the complete model achieved higher observed success rates than the vision-only baseline, including a 45-percentage-point difference in the 50-demonstration toothpaste-squeezing setting. Together, these results demonstrate the feasibility of the proposed hardware–policy pipeline and suggest that tactile augmentation benefits both human teleoperation and learned manipulation policies in contact-rich tasks.