Boundary Cues for Improved 3D Semantic Segmentation

PointPDF V2: A Unified Framework for Continual Open-World 3D Semantic Segmentation.

Leveraging NeRFs to Obtain Consensus in Semantic Segmentation Performed by Image Segmenters

Automated Geometric Correction of OpenStreetMap Buildings via Context- and Boundary-Aware Segmentation

Abstract. OpenStreetMap (OSM) is a representative open geospatial platform that provides free access to major spatial objects, including buildings worldwide, constructed through crowdsourcing-based manual digitization. However, subjective differences among contributors and the absence of unified quality control standards have led to the accumulation of positional offsets and boundary shape errors in building polygons. To address this issue, studies using deep learning-based semantic segmentation for OSM quality improvement have been conducted. Nevertheless, Transformer-based segmentation models exhibit an under-segmentation tendency that merges adjacent buildings into a single object, along with limitations in precise boundary delineation. To overcome these challenges, this study proposes a two-stage framework that integrates SegFormer, which excels in global context recognition, with SAM 2, which is capable of precise boundary segmentation. In the first stage, SegFormer semantically segments building regions from a true orthoimage, and in the second stage, SAM 2 infers object-level precise boundaries using the bounding boxes of OSM polygons as box prompts. The two results are combined into a prior probability map, enabling uncertain boundary regions to be re-evaluated in an unsupervised manner. In experiments conducted over the Suseo-dong area in Gangnam-gu, Seoul, the proposed method achieved a BIoU of 70.40%, an improvement of 23.85 percentage points over OSM building data, with consistent performance gains across all evaluation metrics. This framework offers scalability applicable to any region worldwide without additional label construction, provided that high-resolution true orthoimagery and OSM data are available.

LiDAR Point Cloud Oversegmentation via SAM-based Knowledge Distillation

Abstract. Large-scale LiDAR point clouds provide rich geometric information, yet learning effective structural representations remains challenging due to the misalignment between semantic categories and geometric structures. To address this issue, we propose a SAM-guided framework for point cloud oversegmentation. We transfer grouping knowledge from 2D vision by constructing a large-scale oversegmentation dataset using the Segment Anything Model (SAM) on bird’s-eye-view projections. Based on these grouping priors, a structure-aware point cloud encoder is learned via a distillation objective that enforces intra-region compactness and inter-region separation in the embedding space. The proposed approach does not rely on semantic supervision and directly learns generalizable structural representations. Experiments on various benchmark datasets (STPLS3D, Toronto-3D, DALES, and S3DIS) demonstrate that the proposed method achieves competitive performance. In particular, it significantly improves boundary recall (e.g., 92.21% on STPLS3D and 93.47% on Toronto-3D) while maintaining high oracle accuracy (up to 97.62%). Moreover, the model generalizes well to unseen datasets without retraining, showing strong cross-dataset inference capability.

Text-Guided Semantic Segmentation Method for Indoor 3D Point Clouds

Abstract. Point cloud semantic segmentation of indoor environments is a fundamental task in 3D scene understanding. However, existing methods mainly rely on geometric structures and color information, which are prone to error results in scenarios involving occlusion, sparse sampling, and geometrically similar structures. To address this issue, this paper proposes a text-knowledge-guided method for the point cloud semantic segmentation of indoor 3D scene. Built upon RandLA-Net as the baseline, the proposed method first constructs the textual semantic prototypes using multi-template prompts, and further enhances the stability of semantic anchors through periodic prototype refreshing. Then, a cross-modal semantic feature alignment mechanism is introduced at both the shallow and the high-level feature stages. Through feature alignment, bidirectional semantic interaction, and gated fusion, textual priors are progressively injected into the point cloud feature learning process. Finally, the model is jointly trained with a point-wise classification loss, a text-prototype alignment constraint, and a boundary optimization constraint to improve the semantic feature discrimination and segmentation boundary quality. Experimental results on the S3DIS dataset demonstrate that the proposed method achieves 86.8% OA, 81.6% mAcc, and 67.2% mIoU, exhibiting more stable segmentation performance in complex indoor scenes. As a consequence, these results indicate that incorporating textual semantic priors can effectively enhance high-level semantic representations of point clouds, providing a feasible solution for indoor 3D scene understand.

Emergent 3D Instance Segmentation from Self-Supervised Point Transformers

Unsupervised 3D instance segmentation of outdoor LiDAR scans has traditionally relied on handcrafted geometric priors such as density-based clustering, motion cues, or projected 2D detections. In this work, we investigate whether a frozen, self-supervised point transformer already contains the structural information required to isolate object instances without any handcrafted geometric prior. Using this transformer purely as a feature extractor, we probe its internal representations across the SemanticKITTI, nuScenes, and Waymo Perception datasets. Our analysis yields four core insights: (1) the instance signal concentrates in the attention queries and keys rather than in the values or final output features; (2) output features semantically collapse, merging adjacent same-class objects that the queries and keys keep distinct; (3) this instance signal is bimodal in depth, strongest at the shallowest and deepest encoder stages; and (4) this signal is driven predominantly by the rotary position encoding (RoPE), whose removal collapses its advantage. We put these findings into our method TokenGraph3D, a training-free segmenter that groups points via connected components on a key-similarity graph, using neither density-based clustering nor proximity priors. Under identical prior-free conditions, we substantially outperform output-feature baselines, making the emergent 3D instance structure visible.