Bidirectional cross-modal attention with adaptive gating for multimodal Alzheimer’s disease classification

Abstract

Alzheimer’s disease (AD) is a neurodegenerative disorder, and mild cognitive impairment (MCI) represents a transitional stage between AD and cognitively normal (CN) individuals. Early diagnosis is clinically important for delaying disease progression. To address the limitations of single-modal approaches and the insufficient modeling of complex cross-modal interactions in existing multimodal fusion methods, this paper proposes a multimodal deep learning (DL) classification framework integrating structural magnetic resonance imaging (sMRI) and clinical features. The framework employs a 3D ResNet-34 to extract imaging features and a multilayer perceptron to encode clinical data. A bidirectional cross-modal attention mechanism enhances associations between imaging and clinical modalities, followed by an adaptive gated fusion module that dynamically integrates concatenated global multimodal features with cross-modal interaction features. To evaluate the robustness of the proposed framework, all experiments were repeated using five different random seeds, and the results are reported as mean ± standard deviation. Experimental results demonstrate competitive performance across multiple classification tasks, achieving accuracies of 95.67% ± 1.70% for three-class classification (CN vs MCI vs AD) and 93.64% ± 2.36% for four-class classification (CN vs early MCI (EMCI) vs late MCI (LMCI) vs AD). For binary classification (AD vs CN, AD vs MCI, MCI vs CN, and EMCI vs LMCI), the method achieves accuracies of 96.83% ± 1.53%, 95.33% ± 1.00%, 95.83% ± 1.39%, and 93.61% ± 2.35%, respectively. The proposed framework provides an effective solution for multimodal DL-based computer-aided diagnosis of AD.