OPAL-Grasp: Object-Part Guided Alignment for Language-Driven Robotic Grasp Detection

Abstract

Language-Driven Grasp Detection aims to generate executable robotic grasp rectangles based on user instructions described in natural language and visual inputs. User commands typically specify not only the target object but also fine-grained constraints such as specific parts or functional regions. Most existing methods implicitly treat language as a whole rather than as structured semantic descriptions, limiting the ability of language to guide precise grasp generation. In this paper, we propose OPAL-Grasp, a novel end-to-end framework that explicitly learns object-level and part-level semantics to strengthen perceptual understanding. We design an Object-Part Guidance Module that uses joint object- and part-level segmentation supervision to learn fine-grained vision–language representations, with the object branch serving as auxiliary supervision and only part-aware features propagated to the grasp prediction module. A Grasp Imbalance-Aware Loss is introduced to reweight sparse yet critical grasp regions, thereby improving optimization stability and detection accuracy. Extensive experiments on large-scale language-driven grasp benchmarks and real-world robotic setups demonstrate that OPAL-Grasp achieves state-of-the-art performance and exhibits strong robustness and generalization across challenging scenes.