KPG-RSIC: Knowledge Prior-Guided for Remote Sensing Image Captioning
Abstract
Remote sensing image captioning (RSIC) plays a significant role in environmental surveillance and disaster evaluation. However, existing methods often struggle with the complex semantics of remote sensing imagery and lack sufficient domain-specific prior knowledge, leading to generic descriptions and semantic deviations. To address this, we propose KPG-RSIC, a novel knowledge prior-guided framework for RSIC. Unlike previous approaches that passively receive visual inputs, our method explicitly leverages linguistic knowledge from a pretrained large language model (LLM) to guide the generation process. In particular, we first construct a semantic prior knowledge base (SPKB) by extracting and encoding high-level semantic descriptions from the image using an LLM. We then introduce a decoupled dual cross-attention (DDCA) mechanism within the Transformer decoder, which sequentially performs prior cross-attention (PCA) and visual cross-attention (VCA). More importantly, this “prior-first, visual-second” paradigm enables the model to retrieve relevant concepts from the knowledge base and then ground them into visual features. Experiments on three datasets demonstrate the advancement of KPG-RSIC over strong baselines, validating the effectiveness of leveraging explicit semantic prior knowledge for enhancing caption accuracy and richness.