Infrared and visible image fusion (IVIF) targets to integrate thermal saliency and rich textures into a single image that is not only visually appealing but also beneficial to downstream vision tasks. However, conventional methods relying on heuristic visual criteria struggle to guarantee task utility. Conversely, task-driven fusion paradigms typically employ fixed-weight scalarization, which suffers from potentially conflicting objectives among heterogeneous tasks, leading to rigid compromises and sub-optimal generalization in multi-task scenarios. To overcome these bottlenecks, this paper proposes a symbiotic evolutionary learning framework for task-adaptive IVIF, termed EvoFuse. Rather than relying on static loss weights, we formulate the fusion-perception correlation from a multi-objective perspective. To structurally instantiate this formulation, we first develop a re-parameterizable fusion architecture that accommodates multi-branch representational capacity during training, yet analytically folds into an ultra-compact single-branch model for efficient inference. To navigate the conflicting multi-task objectives, we introduce an evolutionary search mechanism that dynamically evolves task-aware loss-weight configurations. This enables a mutual adaptation process where the fusion network and task models are jointly updated under a Pareto-inspired non-degradation criterion. Furthermore, a novel saliency discriminative loss is designed to explicitly emphasize semantically crucial regions. Extensive experiments across eleven datasets covering fusion and downstream perception tasks demonstrate that the proposed method achieves competitive or better results in most evaluated metrics, while maintaining efficient inference under the considered task settings.
Jinyuan Liu, Bowei Zhang, Ludan Sun et al.· IEEE Transactions on Pattern...· 0 citations
Infrared and visible image fusion is pivotal for robust visual perception across all weather conditions and scenes. Although deep learning-based methods have made notable progress, most either assume pre-aligned inputs or rely on implicit feature-space alignment, which fails to fundamentally address the amplification of registration errors and the loss of semantic structure in the fused results. To this end, we propose a universal representation and end-to-end framework for jointly registering and fusing unaligned infrared-visible image pairs, dubbed URMIF. Each image is mapped into modality-invariant (homogeneous) and modality-specific (heterogeneous) features: the invariant "structural skeleton" encodes geometry and semantics to stabilize alignment, while the specific "texture carrier" preserves thermal saliency and visible details to enable complementary fusion. Therefore, we propose a bi-directionally coupled registration-fusion module. This module performs hierarchical deformation estimation from coarse to fine, effectively mitigating visual mismatches caused by complex parallax in real-world scenes. Within this framework, the fusion component acts as the "evaluator" of registration, providing feedback regularization to update the deformation and suppress error accumulation. Furthermore, we introduce a dominant-plane prior as a scene-level constraint, seeding stable global and patch-wise homographies and reconciling cross-modal detail conflicts, to reinforce geometric consistency and semantic reliability. We also release a large-scale dataset comprising 1,500+ unaligned infrared/visible pairs with registration ground truth, spanning diverse illumination conditions and fields of view. Based on this dataset and additional benchmarks, extensive experiments validate that our framework achieves robust alignment and high-quality fusion on misaligned inputs, markedly reducing artifacts and improving the performance of downstream tasks such as detection and segmentation. Code and benchmark are available at https://github.com/ZengxiZhang/URMIF.
Jinyuan Liu, Zengxi Zhang, Jiahao Zhang et al.· IEEE Transactions on Pattern...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
