LiDAR Point Cloud Oversegmentation via SAM-based Knowledge Distillation
TL;DR
A SAM-guided framework for point cloud oversegmentation that significantly improves boundary recall and maintains high oracle accuracy while maintaining high oracle accuracy, and generalizes well to unseen datasets without retraining, showing strong cross-dataset inference capability.
Abstract
Abstract. Large-scale LiDAR point clouds provide rich geometric information, yet learning effective structural representations remains challenging due to the misalignment between semantic categories and geometric structures. To address this issue, we propose a SAM-guided framework for point cloud oversegmentation. We transfer grouping knowledge from 2D vision by constructing a large-scale oversegmentation dataset using the Segment Anything Model (SAM) on bird’s-eye-view projections. Based on these grouping priors, a structure-aware point cloud encoder is learned via a distillation objective that enforces intra-region compactness and inter-region separation in the embedding space. The proposed approach does not rely on semantic supervision and directly learns generalizable structural representations. Experiments on various benchmark datasets (STPLS3D, Toronto-3D, DALES, and S3DIS) demonstrate that the proposed method achieves competitive performance. In particular, it significantly improves boundary recall (e.g., 92.21% on STPLS3D and 93.47% on Toronto-3D) while maintaining high oracle accuracy (up to 97.62%). Moreover, the model generalizes well to unseen datasets without retraining, showing strong cross-dataset inference capability.