TAPGFusion: Anatomy-Aware Triple-Attention and MRI-Conditioned Prior Learning for Multimodal Medical Image Fusion

Abstract

Multimodal medical image fusion combines anatomical and functional information from different imaging modalities. However, existing methods often struggle to preserve fine anatomical structures while incorporating complementary functional information. To address this problem, we propose TAPGFusion, an anatomy-aware multimodal medical image fusion network. It employs a Multi-Scale Encoder to capture fine local details and broad anatomical structures through parallel convolutions with different receptive fields. A Detail-Enhanced Attention Block further refines the extracted features through channel, spatial, and pixel attention. In addition, a Physiological Prior-Guided Attention Block dynamically balances anatomical and functional features using MRI-conditioned prior information and edge constraints. The main contribution of TAPGFusion is a unified framework that jointly addresses multi-scale feature representation, fine-grained feature selection, and spatially adaptive anatomical–functional fusion. Extensive experiments on three public medical imaging datasets demonstrate the effectiveness and robustness of the proposed method. TAPGFusion achieves CC values above 0.83 and SSIM values above 0.78 on the CT–MRI, PET–MRI, and SPECT–MRI fusion tasks. These results indicate that the proposed method effectively preserves anatomical structures while integrating complementary functional information.