FreeTrack6d: a training-free framework for 6D pose tracking and robotic grasping of moving objects on conveyor belts
TL;DR
FreeTrack6D, a training-free unified segmentation and 6D pose tracking framework integrating frame-wise mask generation, multi-hypothesis pose refinement, motion prediction, observation gating, and visual-servo-based robot control, forms a closed-loop training-free dynamic grasping framework.
Abstract
For robotic dynamic grasping of moving objects in conveyor-belt scenarios, accurate and robust 6D pose estimation and tracking are essential for reliable grasping. However, existing deep-learning-based methods usually rely on large amounts of supervised data for specific objects or categories, which limits their generalization, deployment efficiency, and flexibility for rapid object changeover in industrial applications. To address these challenges, this paper proposes FreeTrack6D, a training-free unified segmentation and 6D pose tracking framework. Relying only on the CAD model of the target object, FreeTrack6D can be applied to dynamic tracking and grasping of unseen objects without additional object-specific training. Specifically, an adaptive multi-cue mask generation module is first introduced to generate frame-wise target masks in real time, which provides target-region constraints for initial pose registration and subsequent pose refinement. This helps reduce the influence of background interference and target-region misalignment caused by rapid motion. Based on the generated mask, RGB-D observations, and the CAD model, FoundationPose is used for initial 6D pose registration and subsequent pose refinement. To improve tracking robustness under large inter-frame motion and rotational variations, a Kalman-guided multi-hypothesis refinement strategy is further designed, where multiple candidate poses predicted from historical motion states are refined and selected according to mask consistency. In addition, a Pose Consistency-aware Association and Gating mechanism is developed to reject abnormal detections, protect the filter state, and trigger re-initialization when consecutive mismatches occur. By integrating frame-wise mask generation, multi-hypothesis pose refinement, motion prediction, observation gating, and visual-servo-based robot control, FreeTrack6D forms a closed-loop training-free dynamic grasping framework.