Jul 2026· ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences· 0 citations· 17 references
Abstract
Abstract. Semantic segmentation of remote sensing imagery (RSI) is essential for urban mapping, land-use monitoring, and many other domains. However, pixel-level annotation is expensive, making weakly supervised semantic segmentation (WSSS) that relies on image-level labels an attractive alternative. Pre-trained models provide strong priors from large-scale learned representations, making them beneficial for WSSS. However, when kept frozen, they often produce sparse and misaligned class activation maps (CAMs) due to domain gaps and static inference. We propose a lightweight and efficient framework that integrates CLIP and DINO foundation models to address three challenges: (i) semantic misalignment between generic text prompts and RSI-specific visuals; (ii) static CAM quality; and (iii) incomplete object coverage. Our design includes: (1) a Textual Prototype-Aware Enrichment (TPE) module that builds an RS-specific knowledge base using large language model (LLM)-generated descriptions to enrich text prompts; (2) a Unified Semantic Relation Mining (USR) module that fuses learnable adapter features with CLIP attention and DINO affinity for online CAM refinement; and (3) a Visual Prototype-Aware Enrichment (VPE) module, which maintains momentum visual prototypes to complete regions and sharpen boundaries. By freezing the CLIP and DINO backbones and optimizing only lightweight adapter and decoder modules, the proposed framework reduces the number of trainable parameters while achieving competitive performance. Experimental on iSAID and ISPRS Potsdam datasets demonstrate the effectiveness of the proposed framework, achieving 38.01% mIoU on iSAID dataset and 47.01% mIoU with 66.89% overall accuracy on Potsdam dataset.
Pixel-level annotation of remote sensing imagery is costly, motivating weakly supervised semantic segmentation (WSSS) using only image-level labels. However, class activation maps (CAMs) often highlight only discriminative sub-regions and fail to separate adjacent land-cover regions, particularly in remote sensing scenes characterized by densely co-occurring land-cover classes and substantial variations in object scale. To address these limitations, we propose a three-stage framework that integrates complementary priors from Contrastive Language–Image Pre-training (CLIP), Self-Distillation with No Labels version 2 (DINOv2), and the Segment Anything Model (SAM). First, a lightweight CLIP adapter aligns vision–language priors with remote sensing imagery, while sigmoid-based multi-label decoupled distillation replaces class-competitive distillation with independent class-wise supervision, producing more complete CAMs. Second, DINOv2-guided feature clustering decomposes large merged regions before SAM prompt generation, while Spatial–Semantic Constraints are used to construct confidence-guided point-and-box prompts and reject excessively expanded or semantically inconsistent masks, thereby generating reliable pseudo-labels. Finally, a compact segmentation network is initialized with the weights learned in Stage 1 and retrained using the refined pseudo-labels generated in Stage 2, eliminating the need for foundation models during inference. Experiments on the Potsdam, LoveDA, and DeepGlobe datasets show that the proposed method achieves mean intersection over union (mIoU) scores of 53.16%, 52.66%, and 62.98%, respectively, outperforming state-of-the-art WSSS baselines by 6.55, 1.16, and 1.27 percentage points, respectively. These results demonstrate the effectiveness and generalizability of the proposed framework across diverse remote sensing scenarios under image-level supervision.
Referring remote sensing image segmentation (RRSIS) aims to segment target ground objects in high-resolution remote sensing imagery according to textual descriptions. Due to complex backgrounds and large-scale variations, accurately aligning linguistic semantics with spatial regions remains a challenging problem. Existing RRSIS methods are predominantly segmentation-centric, where textual descriptions mainly serve as conditional guidance for pixel-wise prediction rather than explicitly enforcing semantic consistency between visual regions and linguistic representations. This limitation often leads to suboptimal fine-grained alignment and insufficient suppression of background-dominant regions. To address these challenges, we propose a novel RRSIS framework centered on structure-constrained semantic contrastive learning (CL), termed S2CLNet. Departing from the segmentation-driven paradigm, instead of relying solely on segmentation supervision, S2CL introduces region-level structural constraints to perform fine-grained CL in the joint embedding space. This design encourages semantically corresponding visual regions and linguistic representations to be more closely aligned while effectively separating target regions from irrelevant background areas. Furthermore, we develop a dynamic modality balancing module (DMBM) to enhance cross-modal interaction under complex remote sensing scenarios. The DMBM jointly models bidirectional cross-modal attention (BCA) and dynamically adjusts the relative contributions of visual and linguistic modalities according to scene complexity and linguistic specificity, thereby facilitating more adaptive and robust multimodal feature integration. Extensive experiments on three public benchmarks, RefSegRS, RRSIS-D, and RISBench, demonstrate that the proposed method achieves superior performance compared with state-of-the-art approaches. The code will be publicly available at https://github.com/45degreesl
Sen Lei, Shuai Li, Xin-Yu Xiao et al.· IEEE Transactions on Geoscie...· 0 citations
ABSTRACT Remote sensing image semantic segmentation plays a pivotal role in converting complex image data into quantifiable geographic spatial information, underpinning applications such as disaster assessment, urban planning, and agricultural resource investigation. Fully supervised semantic segmentation methods rely heavily on labour-intensive pixel-level annotations, prompting a shift towards weakly supervised semantic segmentation (WSSS) that utilizes image-level annotations. However, remote sensing images are characterized by dense, intricate targets and the absence of distinct backgrounds, leading to challenges, such as sparse activation of local regions, incomplete localization in class activation maps (CAMs), and noisy, rough boundaries in pseudo-labels generated from image-level supervision. To address these issues, we propose a deep learning method with pixel relationship constraints for WSSS in remote sensing images. Specifically, we design an image reconstruction (IR) loss function to provide pixel-level supervision, enhancing the completeness of CAMs; a pixel relationship constraint (PRC) module to strengthen the global correlation of target regions and improve detailed information extraction; and an intersection optimization strategy (IOS) based on the Segment Anything Model (SAM) to refine pseudo-labels and segmentation results by mitigating noise. Here, we show that our method achieves mean Intersection over Union (mIoU) values of 63.85%, 71.20%, and 40.96% on the Vaihingen, Potsdam, and iSAID datasets, respectively, reaching 89.25%, 89.02%, and 65.66% of the performance of fully supervised methods. This work advances WSSS for remote sensing images by addressing key limitations of CAM-based pseudo-labels generation, offering a cost-effective alternative to fully supervised approaches and facilitating broader applications in geographic information science and earth observation. The code is available at https://github.com/CHENDL-SHEN/PRCCAM.
Jiaming Fan, Dali Chen, Yang Liu et al.· International Journal of Rem...· 0 citations
Multimodal fusion methods have shown great potential in remote sensing image analysis, but existing approaches rely heavily on massive amounts of annotated data. This is not only costly and time-consuming but also prone to subjective bias. To address this issue, we propose a category-prior-based self-supervised framework, CGSNet, which uses category prior maps extracted from multispectral images as supervisory signals for end-to-end training. An adaptive confidence-weighted pseudo-label generation mechanism is designed to alleviate noise and errors in prior maps by replacing binary labels with continuous confidence maps, enabling the learning of uncertain interclass features. In addition, a multispectral feature-guided refinement strategy utilizes color and texture information to calibrate class transition regions and enhance the discriminative power of pseudo-labels in complex scenes. A dynamic mask selection strategy further enhances the model’s robustness and generalization capabilities through progressive learning. Experiments demonstrate that CGSNet achieves state-of-the-art performance without the need for human annotation, achieving an Mean Intersection over Union (mIoU) score of 78.46% on the Gaofen image dataset (GID) (vegetation) dataset and 79.58% on the Zurich (vegetation) dataset—12.22% and 15.07% higher than existing methods, respectively—while exhibiting strong cross-dataset zero-shot generalization capabilities. The code will be available at https://github.com/NUAALISILab
Jiahang Liu, Jian Cui, Mao-yin Guo et al.· IEEE Transactions on Geoscie...· 0 citations
High-precision, pixel-level annotations are indispensable for remote sensing semantic segmentation and related tasks, yet producing such labels manually is prohibitively expensive. Although recent generative models can synthesize realistic remote sensing data, existing approaches typically either rely heavily on preexisting ground-truth masks as conditioning inputs or lack precise control over the spatial layout of the generated content. To address this gap, we propose Graph2Scene, a novel framework for the joint generation of remote sensing images and pixel-level labels driven by scene graphs. This framework establishes a flexible control mechanism that utilizes scene graphs derived from existing semantic labels during training to learn semantic priors, while enabling users to explicitly define object quantities, categories, and topological relationships for customized generation during inference. Graph2Scene adopts a two-stage cascade: Graph2Mask encodes the scene graph into textual prompts and employs an image-level low-rank adaptation (LoRA) to finetune a FLUX model for label generation; Mask2Scene uses a class-level LoRA strategy to learn fine-grained visual features and generates remote sensing images conditioned on the label. Experiments on our developed non-agricultural conversion process (NACP) dataset and the public LoveDA dataset show that Graph2Scene effectively produces structurally coherent and visually realistic remote sensing images together with accurate pixel-level annotations. The code will be made available at https://github.com/GeoRSAI/Graph2Scene
Shaoxuan Zhao, Xiaoguang Zhou, Dongyang Hou et al.· IEEE Geoscience and Remote S...· 0 citations
Referring Remote Sensing Image Segmentation (RRSIS) has achieved significant progress through the integration of VLMs and the Segment Anything Model (SAM). However, this progress largely relies on strong pre-trained capabilities, while leaving two fundamental limitations insufficiently addressed: (1) Architectural Weak-Coupling, where the unidirectional flow forces reliance on coarse VLM prompts and wastes SAM's pixel-level structural guidance, causing localization drift; and (2) Object-Centric Semantic Bias, where models overemphasize dominant object semantics while remaining insensitive to spatial reasoning crucial for RRSIS. Motivated by these observations, we propose CROSS, a tightly integrated paradigm for RRSIS. First, we introduce Linguistic-Guided Cascaded Distillation (LGCD) to bridge the architectural gap, which distills SAM's geometric affinities as soft regularizers into VLM intermediate layers, injecting dense structural priors to refine localization. Second, Perspective-Spatial Contrastive Learning (PSCL) imposes cross-anchored constraints by mining mask-filtered deceptive distractors and spatial-linguistic counterfactuals as hard negatives, explicitly shattering semantic shortcuts to enforce genuine logical consistency. Extensive experiments on RRSIS benchmarks demonstrate that CROSS achieves state-of-the-art performance and maintains precise localization even under severe spatial description perturbations, standing as a robust new paradigm for RRSIS.
Tingzhang Luo, Ruizhong Liu, Yichao Liu et al.· 1 citation