ESPE-Fusion: Infrared–Visible Image Fusion with Efficient Semantic Prior Encoding and Global Semantic Modulation
Abstract
Semantic-prior-guided infrared-visible image fusion can improve target representation and downstream utility, but foundation visual encoders introduce considerable inference overhead. This paper proposes ESPE-Fusion, a compact local-global semantic-guided fusion framework. During training, the high-level semantics of CLIP and the spatial-relation representations of DINO form a Teacher Semantic Prior that supervises an Efficient Semantic Prior Encoder (ESPE). During inference, the teacher encoders are removed, and shared-weight ESPE produces a patch-level semantic prior and a global semantic descriptor for each modality. The patch prior provides local spatial-semantic guidance, while the global descriptor drives Global Semantic Modulation to adapt high-level bimodal features. Experiments on MSRS and LLVIP demonstrate that ESPE-Fusion preserves fusion quality and pedestrian-related semantic information while reducing the overhead of semantic-prior extraction. The results show that complementary local and global semantic guidance can be retained in a compact, deployment-oriented fusion framework.