SDFNet: Semantic Driven Frequency-Domain Infrared and Visible Image Fusion Network

Abstract

Infrared and visible image fusion aims to integrate complementary thermal target information and visible structural details for multimodal perception and remote sensing applications. However, existing deep learning methods often underutilize semantic information during fusion and mainly perform cross-modal interaction in the spatial domain, limiting the exploitation of complementary representations. To address these issues, we propose SDFNet, a semantic driven frequency-domain infrared and visible image fusion network based on an encoder–decoder architecture. Guided by semantic priors extracted from a pre-trained segmentation model, SDFNet introduces a frequency-domain feature aggregation module to enhance cross-modal feature interaction and semantic information preservation. Extensive experiments on the MSRS dataset demonstrate the effectiveness of the proposed method. On the 361-pair MSRS test set, SDFNet ranks among the top two on five of the six evaluated fusion metrics, achieving EN, AG, SF, SCD, and Qabf scores of 6.73, 3.78, 11.59, 1.74, and 0.69, respectively, while obtaining a competitive VIF score of 1.02. In the downstream semantic utility evaluation, SDFNet achieves the highest mIoU of 78.76 among the compared methods, further demonstrating its ability to preserve task-relevant semantic information.