Pixel relationship-constrained weakly supervised semantic segmentation for remote sensing images: enhancing CAM integrity and pseudo-label quality
TL;DR
This work advances WSSS for remote sensing images by addressing key limitations of CAM-based pseudo-labels generation, offering a cost-effective alternative to fully supervised approaches and facilitating broader applications in geographic information science and earth observation.
Abstract
ABSTRACT Remote sensing image semantic segmentation plays a pivotal role in converting complex image data into quantifiable geographic spatial information, underpinning applications such as disaster assessment, urban planning, and agricultural resource investigation. Fully supervised semantic segmentation methods rely heavily on labour-intensive pixel-level annotations, prompting a shift towards weakly supervised semantic segmentation (WSSS) that utilizes image-level annotations. However, remote sensing images are characterized by dense, intricate targets and the absence of distinct backgrounds, leading to challenges, such as sparse activation of local regions, incomplete localization in class activation maps (CAMs), and noisy, rough boundaries in pseudo-labels generated from image-level supervision. To address these issues, we propose a deep learning method with pixel relationship constraints for WSSS in remote sensing images. Specifically, we design an image reconstruction (IR) loss function to provide pixel-level supervision, enhancing the completeness of CAMs; a pixel relationship constraint (PRC) module to strengthen the global correlation of target regions and improve detailed information extraction; and an intersection optimization strategy (IOS) based on the Segment Anything Model (SAM) to refine pseudo-labels and segmentation results by mitigating noise. Here, we show that our method achieves mean Intersection over Union (mIoU) values of 63.85%, 71.20%, and 40.96% on the Vaihingen, Potsdam, and iSAID datasets, respectively, reaching 89.25%, 89.02%, and 65.66% of the performance of fully supervised methods. This work advances WSSS for remote sensing images by addressing key limitations of CAM-based pseudo-labels generation, offering a cost-effective alternative to fully supervised approaches and facilitating broader applications in geographic information science and earth observation. The code is available at https://github.com/CHENDL-SHEN/PRCCAM.