CGSNet: Category Prior-Guided Self-Supervised Semantic Segmentation for Remote Sensing Images

Abstract

Multimodal fusion methods have shown great potential in remote sensing image analysis, but existing approaches rely heavily on massive amounts of annotated data. This is not only costly and time-consuming but also prone to subjective bias. To address this issue, we propose a category-prior-based self-supervised framework, CGSNet, which uses category prior maps extracted from multispectral images as supervisory signals for end-to-end training. An adaptive confidence-weighted pseudo-label generation mechanism is designed to alleviate noise and errors in prior maps by replacing binary labels with continuous confidence maps, enabling the learning of uncertain interclass features. In addition, a multispectral feature-guided refinement strategy utilizes color and texture information to calibrate class transition regions and enhance the discriminative power of pseudo-labels in complex scenes. A dynamic mask selection strategy further enhances the model’s robustness and generalization capabilities through progressive learning. Experiments demonstrate that CGSNet achieves state-of-the-art performance without the need for human annotation, achieving an Mean Intersection over Union (mIoU) score of 78.46% on the Gaofen image dataset (GID) (vegetation) dataset and 79.58% on the Zurich (vegetation) dataset—12.22% and 15.07% higher than existing methods, respectively—while exhibiting strong cross-dataset zero-shot generalization capabilities. The code will be available at https://github.com/NUAALISILab