2026· IEEE Transactions on Geoscience and Remote Sensing· Vol 64, pp. 5407118-5407118· 0 citations· 50 references
Abstract
Remote sensing open-vocabulary scene classification aims to recognize unseen scene categories beyond the predefined training set by leveraging external knowledge and vision-language models. Prompt learning has emerged as an effective paradigm to adapt pretrained vision-language models to open-vocabulary remote sensing image processing tasks by dynamically encoding task-specific knowledge into textual prompts. However, most existing approaches mainly rely on category names derived from training data, which lack fine-grained semantic descriptions and attribute-level associations that are crucial for complex remote sensing scenes. This limitation often degrades generalization when encountering novel classes and diverse geospatial domains. To address this problem, we propose KnowProKD, a knowledge-guided framework that incorporates diverse external knowledge and constraint-based regularization to enhance base-to-novel generalization. Specifically, KnowProKD consists of three components: 1) leveraging large language models (LLMs) to generate diverse attribute and description knowledge for constructing knowledge-aware prompts, improving cross-modal semantic alignment; 2) introducing knowledge-based constraints to regularize training and enhance robustness to novel category recognition; and 3) designing a dual-branch logits calibration mechanism that exploits attribute and description prompts to balance performance across base and novel classes. Extensive experiments on eight challenging remote sensing scene classification benchmarks demonstrate that KnowProKD achieves favorable performance compared with recent prompt-learning baselines and generalizes well to previously unseen scenarios. Furthermore, we apply KnowProKD to cross-regional urban land-use mapping and statistical analysis across representative cities from six continents, validating its practical effectiveness in real-world remote sensing applications.
For open-set domain adaptation (OSDA) in remote sensing scene classification, it is essential to establish precise semantic boundaries for different scenes. Existing vision–language models usually achieve OSDA with fixed templates based on category labels. However, such templates lead to coarse category representations, which make it difficult to describe the diverse scene information within the same category and cause confusion among similar scenes. Furthermore, the fragmented and interleaved background increases scene complexity and interferes with interdomain category knowledge transfer. Inspired by the above-mentioned issues, this article proposes a novel large language model-guided structural alignment (LLMSA) method. Specifically, an LLM-driven semantic generation module is introduced to establish diverse and scalable fine-grained attribute descriptions, which effectively handles the intraclass diversity of scenes by leveraging comprehensive representation capability of LLMs. Under the guidance of scalable attributes, the structural alignment module is employed to mine the relative relationship of structural elements, which alleviates the interference from complex backgrounds and effectively suppresses negative knowledge transfer. Experiments on six cross-domain scenarios with three widely used public datasets demonstrate that LLMSA delineates the semantic boundaries clearly and achieves a favorable balance between classification accuracy on known category and unknown category recognition.
Yang Zhao, Ge-Fei Zhang, Jia-Qi Liang et al.· IEEE Journal of Selected Top...· 0 citations
Open-vocabulary semantic segmentation (OVSS) of remote sensing faces severe performance degradation when encountering unseen scene distributions caused by geographic, sensor, and resolution variations. Existing vision–language approaches provide strong semantic priors but lack scene-invariant structural representations required for dense prediction. In this work, we focus on open-vocabulary scene generalization semantic segmentation (OVSGSS), achieving dense inference guided by joint modeling of semantic alignment and structural consistency. To this end, we construct USGMS-100K, a large-scale multisensor dataset for self-supervised pretraining, and develop a structure-aware remote sensing image encoder (RSIE) via masked reconstruction to learn scene-invariant representations. Building upon this encoder, we propose a semantic–structural collaborative framework (namely RS-OVSGSeg) that integrates language-derived semantic priors with structural priors through a semantic–structural cost map enhancement (SSCME) module and a dual-prior guided decoder (DPGD). Extensive cross-scene evaluations on five public datasets demonstrate that proposed method achieves state-of-the-art performance in open-vocabulary cross-scene segmentation, while maintaining a favorable balance between accuracy and computational efficiency. The results highlight the importance of explicitly modeling structural invariance for robust open-vocabulary scene generalization in remote sensing. The USGMS-100K dataset, RSIE weight and code are publicly available at https://github.com/HuangWBill/RS-OVSGSeg.
Wu-Biao Huang, Hu-Chen Li, Shuai Zhang et al.· IEEE Transactions on Geoscie...· 0 citations
UniRS-Instruct is presented, a high-quality, diversified, and unified multimodal instruction-following dataset for RSI understanding that unifies diverse tasks, including image captioning, visual question answering, visual grounding, and region-level captioning, into a consistent format.
Lin-Rui Xu, Yuhan Wang, Ling Zhao et al.· IEEE Journal of Selected Top...· 0 citations
Wetlands are fragile yet vital ecosystems that require precise remote sensing (RS) mapping. Hyperspectral image (HSI) provides high spectral resolution for fine-scale wetland classification, but cross-scene application suffers from domain shifts and a lack of model interpretability. To address these issues, this article proposes a knowledge-driven hierarchical language-aware domain generalization network (KHLDGNet) for interpretable cross-scene wetland HSI classification. Unlike existing vision–language methods that merely use category names or simple attributes as auxiliary text, the core innovation of our framework is the construction of a structured, four-level geographic knowledge hierarchy that encompasses conceptual, rule, instance, and causal. This hierarchy transforms a geographical cognitive framework into computable semantic constraints, guiding the model from superficial pattern matching to deep mechanistic understanding. A MixStyle module in the visual encoder implicitly diversifies feature-level styles to counter spectral and textural variations. Visual and textual features are aligned in a shared semantic space via supervised contrastive learning, unifying generalization and interpretability. Experiments on three real-world wetland datasets demonstrate that KHLDGNet significantly outperforms state-of-the-art domain generalization (DG) methods. Visualization and case studies confirm its semantic explanation capability, contributing to explainable geographic AI (XGeoAI). Code available at https://github.com/SYFYN0317/KHLDGNet
Yining Feng, Zhenhua Mu, Yin Zheng et al.· IEEE Transactions on Geoscie...· 0 citations
Cross-scene remote sensing (RS) image classification is crucial for urban analysis and environmental monitoring, but often suffers from distribution shifts caused by different imaging conditions. While unsupervised domain adaptation (UDA) can alleviate cross-scene discrepancies, its closed-set assumption is violated when unknown categories emerge in the target domain, thereby motivating the open-set domain adaptation (OSDA) setting. Existing OSDA methods remain limited in semantic modeling, since conventional visual approaches lack high-level semantic guidance and recent vision–language model (VLM)-based methods predominantly rely on positive prompts that encode only class-belonging semantics, providing limited cues for unknown-sample rejection. To address these issues, we propose category-level positive–negative prompting (CLPNP) for open-set cross-scene RS image classification. CLPNP integrates learnable positive prompts and multiple groups of negative prompts into a frozen VLM to jointly model belonging and nonbelonging semantics, thereby forming a category-level binary semantic discrimination mechanism. Built upon this prompt design, a positive–negative semantic-guided known–unknown separation (PNKUS) strategy is proposed, which establishes reliable binary semantic boundaries from labeled source data and transfers this discriminative structure to the target domain through uncertainty regularization, together with structured regularization on positive and negative prompts to ensure semantic diversity and separability. In addition, a threshold-free open-set inference mechanism is introduced by directly comparing positive and negative semantic responses for each known category. Extensive experiments on 18 cross-scene transfer tasks under two benchmark settings demonstrate that CLPNP consistently outperforms state-of-the-art OSDA methods and achieves a favorable balance between known-category recognition and unknown-category rejection.
Xin Zhao, Yuanyuan Ye, Jun Lin et al.· IEEE Transactions on Geoscie...· 0 citations
Existing class-incremental learning (CIL) methods for remote sensing (RS) scene classification often tend to be training-intensive or rely on static visual features that may inadequately capture the complex interclass similarity and intraclass diversity inherent in RS imagery. Moreover, directly reusing features from models pretrained on natural images (e.g., CLIP) often suffers from a domain gap and limited semantic richness when transferred to the RS domain. In this article, we propose a simple yet effective training-free CIL framework for RS scene classification that leverages multimodal semantic information to build more discriminative category representations. Our framework treats pretrained models as frozen feature extractors to guarantee zero forgetting of the representation space. To enhance semantic discriminability, we employ a large-language model (LLM) to generate rich candidate textual descriptions for each class and introduce an image-guided description selection (IGDS) strategy to align semantic information with visual characteristics. Classification is performed using a distance-based metric without any additional training. Extensive experimental results demonstrate that our framework achieves leading performance and superior stability across different session sequences, surpassing both training-based and training-free baseline methods. Our code is available at https://github.com/WenliangDu/ZFCIL-RS
Wen-Liang Du, Ji-Cun He, Jiaqi Zhao et al.· IEEE Transactions on Geoscie...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
