Jul 2026· The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences· Vol XLIX-B2-2026, pp. 665-672· 0 citations· 11 references
TL;DR
The proposed GeoRGMAE, a geospatially guided masked autoencoder pretraining strategy for building segmentation, introduces three masking strategies that prioritize semantically relevant building regions under the varying urban densities and suggests that incorporating geospatial priors into masked image modelling (MIM) can improve representation learning for downstream building segmentation tasks.
Abstract
Abstract. Accurate building segmentation from high-resolution aerial imagery is essential for various urban applications such as digital twins, geographic information system (GIS), and flood risk modelling. However, conventional supervised deep learning approaches require large amounts of pixel-level annotations, which are costly and time-consuming to obtain for large remote sensing datasets. To address this limitation, self-supervised learning (SSL) has recently emerged as an effective paradigm for learning visual representations from unlabeled data. In particular, masked autoencoders (MAE) have demonstrated strong performance by reconstructing masked image patches during pretraining. Nevertheless, conventional MAE frameworks rely on random masking strategies that ignore the spatial structure and semantic importance of regions in high-resolution remote sensing imagery. In this study, we propose GeoRGMAE, a geospatially guided masked autoencoder pretraining strategy for building segmentation. Unlike standard MAE, which rely on random masking, our approach leverages building footprint annotations available during pretraining to guide the masking process while preserving the original reconstruction objective. We introduce three masking strategies -core, balanced, and density-aware masking- that prioritize semantically relevant building regions under the varying urban densities. The core strategy focuses on building interiors, the balanced strategy distributes masking between buildings and background, and the density-aware adapts masking based on scene-level building density. Experiments on the Roof3D and WHU Building datasets demonstrate consistent, though modest, improvements over standard MAE pretraining, with the most effective masking strategy depending on dataset characteristics. These findings suggest that incorporating geospatial priors into masked image modelling (MIM) can improve representation learning for downstream building segmentation tasks.
Multimodal fusion methods have shown great potential in remote sensing image analysis, but existing approaches rely heavily on massive amounts of annotated data. This is not only costly and time-consuming but also prone to subjective bias. To address this issue, we propose a category-prior-based self-supervised framework, CGSNet, which uses category prior maps extracted from multispectral images as supervisory signals for end-to-end training. An adaptive confidence-weighted pseudo-label generation mechanism is designed to alleviate noise and errors in prior maps by replacing binary labels with continuous confidence maps, enabling the learning of uncertain interclass features. In addition, a multispectral feature-guided refinement strategy utilizes color and texture information to calibrate class transition regions and enhance the discriminative power of pseudo-labels in complex scenes. A dynamic mask selection strategy further enhances the model’s robustness and generalization capabilities through progressive learning. Experiments demonstrate that CGSNet achieves state-of-the-art performance without the need for human annotation, achieving an Mean Intersection over Union (mIoU) score of 78.46% on the Gaofen image dataset (GID) (vegetation) dataset and 79.58% on the Zurich (vegetation) dataset—12.22% and 15.07% higher than existing methods, respectively—while exhibiting strong cross-dataset zero-shot generalization capabilities. The code will be available at https://github.com/NUAALISILab
Jiahang Liu, Jian Cui, Mao-yin Guo et al.· IEEE Transactions on Geoscie...· 0 citations
The first systematic zero-shot evaluation of SAM 2 for aerial building segmentation is presented, establishing SAM 2 as a viable tool for rapid building mapping while highlighting where domain adaptation remains necessary.
Bingning Xiong, Mingyu Ou· Journal of image processing...· 0 citations
Abstract. Semantic segmentation of remote sensing imagery (RSI) is essential for urban mapping, land-use monitoring, and many other domains. However, pixel-level annotation is expensive, making weakly supervised semantic segmentation (WSSS) that relies on image-level labels an attractive alternative. Pre-trained models provide strong priors from large-scale learned representations, making them beneficial for WSSS. However, when kept frozen, they often produce sparse and misaligned class activation maps (CAMs) due to domain gaps and static inference. We propose a lightweight and efficient framework that integrates CLIP and DINO foundation models to address three challenges: (i) semantic misalignment between generic text prompts and RSI-specific visuals; (ii) static CAM quality; and (iii) incomplete object coverage. Our design includes: (1) a Textual Prototype-Aware Enrichment (TPE) module that builds an RS-specific knowledge base using large language model (LLM)-generated descriptions to enrich text prompts; (2) a Unified Semantic Relation Mining (USR) module that fuses learnable adapter features with CLIP attention and DINO affinity for online CAM refinement; and (3) a Visual Prototype-Aware Enrichment (VPE) module, which maintains momentum visual prototypes to complete regions and sharpen boundaries. By freezing the CLIP and DINO backbones and optimizing only lightweight adapter and decoder modules, the proposed framework reduces the number of trainable parameters while achieving competitive performance. Experimental on iSAID and ISPRS Potsdam datasets demonstrate the effectiveness of the proposed framework, achieving 38.01% mIoU on iSAID dataset and 47.01% mIoU with 66.89% overall accuracy on Potsdam dataset.
Xin Li, Nicola Genzano, M. Gianinetto et al.· ISPRS Annals of the Photogra...· 0 citations
Accurate and up-to-date knowledge of land use and land cover represents one of the central challenges in spatial planning and landscape sciences. In this context, the present work introduces GANCIU (Geospatial Analysis with Neural Classification and Image Understanding), an original hybrid pipeline for the automatic extraction of man-made infrastructure from high-resolution satellite imagery. The primary methodological contribution lies in the sequential integration of four technologically heterogeneous components: a per-pixel Random Forest classifier, a guided image modulation step, edge detection via the Mumford–Shah variational functional solved through the Ambrosio–Tortorelli approximation, and final object delineation via the Segment Anything Model (SAM). Each component does not operate independently but conditions and informs the next: The RF probability map guides the modulation, which in turn directs the sensitivity of the variational step exclusively towards regions of interest; the AT edges provide spatial prompts to SAM, for which its masks are finally filtered by the RF probability in an adaptive manner through a Gaussian Mixture Model. This progressive conditioning scheme constitutes the architectural core of GANCIU and distinguishes it from approaches that combine classification and segmentation in parallel or in purely sequential fashion with each stage conditioning the next but without any reverse correction between them. The Random Forest classifier was trained on 44 manually annotated scenes, geographically disjoint from the twelve independent scenes used for quantitative validation. This validation, based on an instance matching protocol (precision, recall, F1 score, and IoU), confirms the contribution of the full pipeline over a Random-Forest-only baseline: Pooled false positives fall by close to two orders of magnitude (from 8320 to 209), while true positives rise nearly twentyfold (from 5 to 95), with a mean IoU of 0.742 ± 0.060 on correctly matched objects. Notably, the entire pipeline—including SAM-based segmentation—runs end-to-end on a modest, GPU-free consumer laptop (four logical CPU cores, under 16 GB RAM), demonstrating that competitive infrastructure-extraction performance does not require specialised computing hardware.
A. Ganciu, Giovanna Ricci, Margherita Solci· Journal of Imaging· 0 citations
Pixel-level annotation of remote sensing imagery is costly, motivating weakly supervised semantic segmentation (WSSS) using only image-level labels. However, class activation maps (CAMs) often highlight only discriminative sub-regions and fail to separate adjacent land-cover regions, particularly in remote sensing scenes characterized by densely co-occurring land-cover classes and substantial variations in object scale. To address these limitations, we propose a three-stage framework that integrates complementary priors from Contrastive Language–Image Pre-training (CLIP), Self-Distillation with No Labels version 2 (DINOv2), and the Segment Anything Model (SAM). First, a lightweight CLIP adapter aligns vision–language priors with remote sensing imagery, while sigmoid-based multi-label decoupled distillation replaces class-competitive distillation with independent class-wise supervision, producing more complete CAMs. Second, DINOv2-guided feature clustering decomposes large merged regions before SAM prompt generation, while Spatial–Semantic Constraints are used to construct confidence-guided point-and-box prompts and reject excessively expanded or semantically inconsistent masks, thereby generating reliable pseudo-labels. Finally, a compact segmentation network is initialized with the weights learned in Stage 1 and retrained using the refined pseudo-labels generated in Stage 2, eliminating the need for foundation models during inference. Experiments on the Potsdam, LoveDA, and DeepGlobe datasets show that the proposed method achieves mean intersection over union (mIoU) scores of 53.16%, 52.66%, and 62.98%, respectively, outperforming state-of-the-art WSSS baselines by 6.55, 1.16, and 1.27 percentage points, respectively. These results demonstrate the effectiveness and generalizability of the proposed framework across diverse remote sensing scenarios under image-level supervision.
Accurate segmentation of water bodies in satellite imagery is essential for environmental monitoring, resource management, and disaster response. This study introduces a novel deep learning framework that leverages image embeddings from a pre-trained SegFormer-B4 encoder (MiT-B4) without fine-tuning, combined with a U-Net decoder, to achieve precise water body segmentation in Sentinel-2 satellite imagery. Unlike conventional methods that rely on full fine-tuning, this approach significantly reduces computational cost and training time while maintaining high segmentation accuracy. The extracted transformer-based embeddings capture both local and global spatial features, serving as input to a lightweight U-Net decoder that efficiently reconstructs segmentation masks. Experimental results demonstrate that the proposed method outperforms the fine-tuned SegFormer-B4 model in training and testing scenarios, achieving superior Intersection over Union (IoU), F1 Score, Precision, and Recall. Despite the absence of fine-tuning, the SegFormer-B4 encoder effectively extracts meaningful spatial representations, enabling accurate segmentation with minimal computational overhead. This work highlights the advantages of integrating pre-trained transformer embeddings with a dedicated segmentation decoder, offering a scalable and efficient solution for water body mapping in remote sensing applications.
V. Patel· Signal, Image and Video Proc...· 1 citation