ST-AVIT: A Semantic Prior-Guided and Thermal Structure-Constrained Model for Aerial Visible-to-Infrared Image Translation
Abstract
Visible-to-infrared (V2IR) image translation offers a practical approach to alleviating the high acquisition cost of infrared data and the scarcity of paired cross-modal samples. However, insufficient consideration of land-cover-dependent infrared responses can cause inaccurate target intensities, blurred boundaries, and response mixing across heterogeneous regions. To address these problems, we propose ST-AVIT, a semantic prior-guided and thermal structure-constrained model for aerial V2IR image translation. The model concatenates dataset-adaptive semantic maps with visible images along the channel dimension, providing explicit spatial and category-boundary information to distinguish adjacent land-cover regions. A physics-inspired temperature–emissivity–semantic (TeS) radiation network is pretrained on real infrared images using the corresponding visible-image-derived semantic label maps and frozen as a teacher. It constrains feature consistency between generated and reference infrared images, providing additional supervision for regional infrared response modeling without requiring the teacher during inference. Experiments on the aerial V2IR Image Dataset (AVIID)-3 dataset, together with supplementary validation on the ground-level multi-scenario multi-modality dataset (M3FD) dataset, show that ST-AVIT outperforms eight competing adversarial and diffusion-based methods across all four evaluated metrics. These results indicate improvements in pixel-level reconstruction, structural similarity, perceptual quality, and distributional agreement with real infrared imagery on both datasets. Qualitative comparisons show better preservation of object–background contrast, local boundaries, small targets, and regional infrared intensity patterns. Image-level and category-level ablation studies demonstrate that semantic input conditioning and TeS regularization jointly improve structural fidelity and regional infrared response modeling.