Nov 2026· IEEE Robotics and Automation Letters· Vol 11, pp. 12671-12678· 0 citations· 23 references
Abstract
This paper presents TransDex, an interaction-geometry-aware 3D visuo-tactile fusion motor policy for dexterous manipulation of transparent objects. Dexterous manipulation enables complex tasks but is fundamentally challenged by hand–object interaction geometric ambiguity caused by self-occlusion, severe depth noise, and depth information loss when manipulating transparent objects. To overcome these challenges, we first propose a self-supervised point cloud reconstruction pre-training approach based on Transformer, which accurately recovers the 3D structure of objects from interaction point clouds of dexterous hands, even when random noise and large-scale masking are added. Building on this, TransDex is constructed in which perceptual encoding adopts a fine-grained hierarchical scheme and multi-round attention mechanisms adaptively fuse features of the robotic arm and dexterous hand to enable differentiated motion prediction. Experimental results conducted on a real robotic system demonstrate that TransDex outperforms existing baseline methods, which successfully validate the generalization capabilities of the proposed method.
OccluDex is presented, a hierarchical 3D visuo-tactile representation learning framework that integrates global geometric structure with local contact information for robust manipulation under dynamic self-occlusion during hand-object interaction and could enable humanoid egocentric object manipulation for seen and uns...
Zi-Heng Xu, Yue-Yuan Chen, Xin-Yuan He et al.· arXiv.org· 0 citations
Dexterous in-hand manipulation requires policies that coordinate high-DoF hand joints through intermittent, contact-rich interaction. Beyond target-orientation tracking, such policies must discover finger gaits that preserve object stability while adapting to geometry, anisotropy, pose, contact, and sensing changes. We...
Jie Yin, Wan-Li Xing, Ze-Yuan Zhao et al.· 0 citations
Human videos provide demonstrations of dexterous manipulation but lack robot-executable actions and tactile measurements. We present UniDex-ViTac, a framework that uses human-video-guided simulation to generate robot demonstrations paired with fingertip contact observations for training a deployable visuo-tactile polic...
Vision-based tactile sensors enable fine-grained perception for dexterous robotic manipulation, but current marker-based designs tightly couple surface texture and contact force signals within a single observation. Existing data-driven decoupling methods rely on serial mask-based inpainting, which suffers from mask-siz...
Zhi-Rui Zhang, Yun-Kai Tang, Jia-Hang Li et al.· 0 citations
Fine-grained manipulation by-hand is still one of the major challenges for robotics, with recent progress-whether in visuo-tactile sensing or in representation learning or in sim-to-real transfer-being mostly parallel in time and in unrelated hardware/learning setups. There is no single architectural standard that woul...
Yuan Huang, Yi-Qian Liu· International Conference on...· 0 citations
Comparisons support complementary benefits of feature-wise gated tactile fusion and training-only predictive supervision of TacSushi, a tactile-grounded, Cosmos3-based world-action policy that learns from recorded future consequences while acting on current observations.
Hao-Di Hu, Kaen Kogashi, T. Koike-Akino· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.