Text-Guided Dual Refinement for Domain Generalized Semantic Segmentation in Remote Sensing

Abstract

Recently, domain generalized remote sensing semantic segmentation (DG-RSSS) methods leverage vision foundation models (VFMs) with a parameter-efficient fine-tuning (PEFT) strategy to achieve remarkable progress. Although VFMs offer robust representations under distribution shifts between different remote sensing scenes, they still have some limitations, including inaccurate segmentation between similar classes and boundary regions. To address these limitations, this work proposes a text-guided dual refinement (TGDR) approach for DG-RSSS, which contains a text-guided discriminative refinement (TDR) module and a text-guided mask features refinement (TMR) module. In particular, first, the proposed TDR module generates class-discriminated features by using class-related learnable tokens and interclass similarity to refine the original features from the frozen backbone, where the tokens are initialized with class texts. Second, the proposed TMR module injects class-specific semantics into the phase component of mask features by using class-related learnable tokens to enhance the correlation between the semantics of classes and scene contents in mask features for refining the prediction of class boundaries in scene contents. Extensive experiments demonstrate that the proposed TGDR approach achieves superior performance across multiple DG-RSSS benchmarks, e.g., achieving 65.6%, 54.0%, 59.0%, 45.2%, 48.8%, and 79.3% mIoU on the Potsdam–to–Vaihingen (P2V), Vaihingen–to–Potsdam (V2P), Rural–to–Urban (R2U), Urban–to–Rural (U2R), Aerial–to–Satellite II (A2S), and Satellite II–to–Aerial (S2A) benchmarks.