Enhancing Scene Generalization for Open-Vocabulary Remote Sensing Segmentation via Semantic–Structural Collaboration

Abstract

Open-vocabulary semantic segmentation (OVSS) of remote sensing faces severe performance degradation when encountering unseen scene distributions caused by geographic, sensor, and resolution variations. Existing vision–language approaches provide strong semantic priors but lack scene-invariant structural representations required for dense prediction. In this work, we focus on open-vocabulary scene generalization semantic segmentation (OVSGSS), achieving dense inference guided by joint modeling of semantic alignment and structural consistency. To this end, we construct USGMS-100K, a large-scale multisensor dataset for self-supervised pretraining, and develop a structure-aware remote sensing image encoder (RSIE) via masked reconstruction to learn scene-invariant representations. Building upon this encoder, we propose a semantic–structural collaborative framework (namely RS-OVSGSeg) that integrates language-derived semantic priors with structural priors through a semantic–structural cost map enhancement (SSCME) module and a dual-prior guided decoder (DPGD). Extensive cross-scene evaluations on five public datasets demonstrate that proposed method achieves state-of-the-art performance in open-vocabulary cross-scene segmentation, while maintaining a favorable balance between accuracy and computational efficiency. The results highlight the importance of explicitly modeling structural invariance for robust open-vocabulary scene generalization in remote sensing. The USGMS-100K dataset, RSIE weight and code are publicly available at https://github.com/HuangWBill/RS-OVSGSeg.