Skip to content
Open access

TransDex: Interaction Geometry Enables Robust Visuo-Tactile Dexterous Manipulation of Transparent Objects

Nov 2026 · IEEE Robotics and Automation Letters · Vol 11, pp. 12671-12678 · 0 citations · 23 references

Abstract

This paper presents TransDex, an interaction-geometry-aware 3D visuo-tactile fusion motor policy for dexterous manipulation of transparent objects. Dexterous manipulation enables complex tasks but is fundamentally challenged by hand–object interaction geometric ambiguity caused by self-occlusion, severe depth noise, and depth information loss when manipulating transparent objects. To overcome these challenges, we first propose a self-supervised point cloud reconstruction pre-training approach based on Transformer, which accurately recovers the 3D structure of objects from interaction point clouds of dexterous hands, even when random noise and large-scale masking are added. Building on this, TransDex is constructed in which perceptual encoding adopts a fine-grained hierarchical scheme and multi-round attention mechanisms adaptively fuse features of the robotic arm and dexterous hand to enable differentiated motion prediction. Experimental results conducted on a real robotic system demonstrate that TransDex outperforms existing baseline methods, which successfully validate the generalization capabilities of the proposed method.

Read PDF

Similar papers

Sep 2026

OccluDex: Hierarchical 3D Visuo-Tactile Representation Learning for Egocentric Dexterous Manipulation under Self-Occlusion

OccluDex is presented, a hierarchical 3D visuo-tactile representation learning framework that integrates global geometric structure with local contact information for robust manipulation under dynamic self-occlusion during hand-object interaction and could enable humanoid egocentric object manipulation for seen and uns...

Zi-Heng Xu, Yue-Yuan Chen, Xin-Yuan He et al. · 0 citations
Preprint Sep 2026

TacBPM: A Tactile-conditioned Behavior Prior Model for Dexterous Reorientation

Dexterous in-hand manipulation requires policies that coordinate high-DoF hand joints through intermittent, contact-rich interaction. Beyond target-orientation tracking, such policies must discover finger gaits that preserve object stability while adapting to geometry, anisotropy, pose, contact, and sensing changes. We...

Jie Yin, Wan-Li Xing, Ze-Yuan Zhao et al. · 0 citations
Sep 2026

UniDex-ViTac: Learning Unified Visuo-Tactile Dexterous Manipulation Policy from Human Video Data

Human videos provide demonstrations of dexterous manipulation but lack robot-executable actions and tactile measurements. We present UniDex-ViTac, a framework that uses human-video-guided simulation to generate robot demonstrations paired with fingertip contact observations for training a deployable visuo-tactile polic...

Hyesung Lee, Si-Hwan Heo, Sungwook Yang · 0 citations
Preprint Oct 2026

DeTac: Dual-branch Decoupling of Texture and Force Signals in Vision-Based Tactile Sensors

Vision-based tactile sensors enable fine-grained perception for dexterous robotic manipulation, but current marker-based designs tightly couple surface texture and contact force signals within a single observation. Existing data-driven decoupling methods rely on serial mask-based inpainting, which suffers from mask-siz...

Zhi-Rui Zhang, Yun-Kai Tang, Jia-Hang Li et al. · 0 citations
Conference Oct 2026

A unified visuo-tactile pipeline architecture for dexterous in-hand manipulation

Fine-grained manipulation by-hand is still one of the major challenges for robotics, with recent progress-whether in visuo-tactile sensing or in representation learning or in sim-to-real transfer-being mostly parallel in time and in unrelated hardware/learning setups. There is no single architectural standard that woul...

Yuan Huang, Yi-Qian Liu · 0 citations

TacSushi: Tactile-Grounded World-Action Modeling for Dexterous Sushi Manipulation

Comparisons support complementary benefits of feature-wise gated tactile fusion and training-only predictive supervision of TacSushi, a tactile-grounded, Cosmos3-based world-action policy that learns from recorded future consequences while acting on current observations.

Hao-Di Hu, Kaen Kogashi, T. Koike-Akino · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.