PointPDF V2: A Unified Framework for Continual Open-World 3D Semantic Segmentation.

Abstract

3D semantic segmentation has achieved remarkable advances with powerful architectures and large-scale datasets. However, most existing approaches perform under closed-set assumptions, limiting their applicability in dynamic real-world environments where novel categories frequently emerge. This motivates the task of open-world semantic segmentation (OWSS), in which a model must not only identify unknown objects at inference time but also incorporate them with previously learned classes once annotated. To address this challenge, we propose PointPDF V2, a unified framework that integrates open-set recognition (OSS) and incremental learning (IL) into a cohesive pipeline. Our design consists of three components: a multistage pseudo-labeling (MPL) module that generates pseudo-labels for unknown classes by leveraging confidence and geometric information; a distribution-aware confidence estimation (DCE) module that models class-wise semantic distributions to separate known and unknown categories; and a weighted knowledge distillation (WKD) module that balances old-class retention with novel-class adaptation in IL. In addition to the standard OWSS setting, we introduce a more challenging continual OWSS (COWSS) protocol in 3D, where models must simultaneously preserve the known-class performance, acquire new knowledge, and still identify the remaining unknowns across sequential updates. Extensive experiments on both indoor and outdoor benchmarks show that PointPDF V2 consistently outperforms state-of-the-art baselines in OSS, IL, and C-OWSS. We shall release our code and models upon publication of this work.