Remote Sensing Image-Text Retrieval via Hierarchical and Targeting Semantic Alignment

Abstract

Remote sensing image-text retrieval (RSITR) can facilitate visual language mining in spatial scenarios, which holds significant importance for aviation and aerospace applications. Existing RSITR methods often lack joint coarse-grained and fine-grained alignment, or their unified learning strategies are inefficient. Additionally, there is no analysis of remote sensing characteristics in these fine-grained feature learning. To address these issues, this paper proposes Hierarchical and Targeting Semantic Alignment (HTSA) for remote sensing image-text retrieval, which simultaneously considers the construction of global and local features. In coarse-grained semantic alignment, cross-modal classification learning and inter-modal matching learning are combined. For fine-grained semantic learning, targeted semantic alignment is introduced to highlight essential objects, which are overlooked by current RSITR methods, but crucial for remote sensing scenarios. Comprehensive experiments demonstrate the superiority of the proposed framework, while different ablation studies discuss the functions of specific components.