Skip to content
Open access

Rob-ACT: Toward Practical Robust Deployment of Robotic Imitation Learning Under Distribution Shifts

2026 · IEEE Transactions on Automation Science and Engineering · Vol 23, pp. 18368-18380 · 0 citations · 34 references

Abstract

For robot manipulation tasks, the performance of imitation learning methods often degrades when deployed under out-of-distribution (OOD) conditions. These uncertainties may arise from object-state and environmental shifts, as well as visual observation disturbances and sensor degradation. To address this, we propose the Robust Action Chunking with Transformer (Rob-ACT) framework with two synergistic core modules. We propose the Dynamic Geometric Encoding (DGE). It takes a task benchmark image as a unified anchor and leverages RAFT-generated optical flow to encode benchmark-relative geometric features of targets. We also design an Adaptive Gating Fusion (AGF) to complement DGE. It contains two dedicated streams: a multi-view camera gating stream for adapting to self-observation reliability fluctuations, and an optical flow reliability weighting stream for preserving task-relevant benchmark-relative geometric representations from DGE. Experiments are conducted on clothing manipulation and tool stowage tasks under controlled OOD conditions. Rob-ACT achieves a higher average success rate than ACT, Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation (InterACT), and Diffusion Policy (DP) by 11.4, 20.4, and 35.6 percentage points, respectively. These results demonstrate improved robustness under the evaluated distribution shifts within manipulation tasks. Note to Practitioners—This work addresses the performance degradation of robotic imitation learning under common OOD disturbances in real-world deployment, and is applicable to industrial manipulation and daily service scenarios. Conventional imitation learning methods rely on aligned training-deployment distributions, failing drastically under pose variations, background changes, occlusion and sensor noise, which demands costly re-demonstration and on-site recalibration. We propose Rob-ACT, an ACT-based robust imitation learning framework with two collaborative modules. DGE encodes benchmark-relative geometric variation features via benchmark-referenced optical flow, reducing the dependence of the policy on absolute object appearance and position cues. AGF dynamically weights multi-view and flow features. Evaluated on clothing manipulation and tool stowage tasks under OOD settings, Rob-ACT outperforms mainstream baselines in success rate and data efficiency, lowering deployment data demand and maintenance costs. The framework currently requires task benchmarks and is validated on quasi-static planar tasks. Future work will focus on developing automatic benchmark generation and 3D encoding for complex dynamic manipulation.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.