Tactile-Guided In-Contact Robotic Insertion via Physically Consistent Force-Array Representation Learning

Abstract

Insertion tasks involve delicate, contact-rich manipulations commonly performed by robots in industrial and domestic environments. Vision-based insertion policies have shown progress, but they often become unreliable once contact is established, as the robot, object, or fixture can occlude the visual observations needed to perceive subtle alignment errors. In this work, we focus on the post-contact stage of insertion and propose a tactile-only framework for in-contact search, alignment, and insertion. Our method learns a physically consistent force-array latent representation from high-resolution tactile force arrays, capturing contact patterns and alignment cues in a compact, task-relevant space. Using this representation as policy observations, we train a diffusion policy from post-contact human demonstrations to generate insertion actions without relying on visual feedback during in-contact execution. Extensive real-world experiments show that the proposed tactile-only diffusion-policy framework can learn generalizable in-contact insertion behaviors from demonstrations collected on simple geometric insertion tasks. On diverse in-distribution tasks, both the full tactile-representation-based policy and a force/torque-only variant achieve over 81% success, substantially outperforming other baselines. While the resultant-wrench-only variant performs slightly better on in-distribution tasks, the full method generalizes better when transferred zero-shot to realistic connector insertion scenarios, including 2-prong, 3-prong, USB, and HDMI connectors, achieving 65% success our project repository.