SDFMod: Semantics-guided Dynamic Frequency Modulation for Multi-modality Image Fusion

Abstract

Multi-modality image fusion (MMIF) seeks to generate images that are suitable for human viewing while also supporting downstream visual understanding. Existing approaches usually learn fusion and segmentation together, either through cascaded task supervision or by exchanging intermediate features. In most cases, however, the two tasks are coupled directly in the spatial domain, even though they depend on different forms of representation. Fusion focuses on preserving local structure and integrating complementary cross-modal details, whereas segmentation depends more on regional coherence and object-level semantics. Since these signals are largely associated with different frequency bands, direct interaction in the spatial domain often cannot effectively translate high-level semantic cues into guidance for low-level fusion. This mismatch can lead to blurred boundaries, local artifacts, and limited overall performance. To address this issue, we propose SDFMod, a semantics-guided dynamic frequency modulation framework for multi-modality image fusion. Rather than enforcing task interaction in the spatial domain, SDFMod models the collaboration between fusion and segmentation in the frequency domain, where semantic structures and perceptual details can be more explicitly coordinated. The framework consists of three progressive components. The Task-aware Frequency Separator (TFS) decouples features into task-relevant frequency components using dynamic kernel decomposition. The Frequency Filter (FreFil) then uses modality-specific frequency priors to extract complementary information. Finally, the Dynamic Frequency Prompt (DFP) adaptively balances semantic representation and perceptual detail based on task goals. Extensive experiments show that frequency-domain reformulation is a promising direction for task-oriented fusion.