Semantic-Aware Coarse-to-Fine 3D Part Assembly via Multi-Modal Priors

Abstract

3D part assembly aims to reconstruct complex objects from disordered components into physically valid structures. However, existing approaches relying on geometric features struggle with ambiguities among geometrically similar parts. Specifically, the components sharing closely resembling shapes but serving distinct functional roles are frequently misclassified, leading to structurally invalid assemblies. To address this issue, we propose a semantic-aware coarse-to-fine assembly framework. Our method operates through two stages: semantic-aware global topology initialization and text-guided fine contact alignment. In the coarse stage, we introduce a semantic affinity grouping mechanism that leverages semantic priors to distinguish functional roles among geometrically similar parts, thereby establishing a topologically consistent global pose. In the fine stage, we design a text-guided semantic injection module that aligns local contact surfaces with functional descriptions via cross-modal attention, effectively resolving local physical conflicts and ensuring precise connectivity. Extensive experiments on the PartNet dataset demonstrate the superiority of our method in resolving geometric ambiguities, achieving significant improvements over existing baselines in both part accuracy and physical connectivity.