Fine-Grained Remote Sensing Imagery Generation Driven by Expert Knowledge and Hierarchical Captions
TL;DR
A novel hierarchical caption encoding mechanism that efficiently injects decoupled hierarchical caption segments into the U-Net's cross-attention layers enables the model to exert hierarchical and decoupled attentional control over the global scene, spatial layout, and geographical element details during the denoising process.
Abstract
Abstract. Current diffusion models struggle to achieve fine-grained remote sensing imagery (RSI) generation. This limitation fundamentally stems from their reliance on "flattened" text prompts, which overlook the inherent hierarchical structure of RSI. This paper proposes a fine-grained RSI generation method driven by expert knowledge and hierarchical captions. We first deconstruct RSI into a hierarchical "element-relation-scene" caption and employ an automatic caption optimization mechanism, grounded in spatial relation knowledge, to ensure high fidelity. Critically, we introduce a novel hierarchical caption encoding mechanism that efficiently injects decoupled hierarchical caption segments into the U-Net's cross-attention layers. This design enables the model to exert hierarchical and decoupled attentional control over the global scene, spatial layout, and geographical element details during the denoising process. Experiments demonstrate that, when combined with efficient fine-tuning algorithms such as LoRA, our method significantly outperforms traditional single-level captions across all six evaluation metrics, exemplified by the FID metric decreasing from 228.43 to 205.59 and the GSHPS metric increasing from 0.86 to 0.92. This research provides a new paradigm for controllable remote sensing scene generation, establishing an effective link between hierarchical semantic understanding and the progressive generation process of diffusion models.