The proposed framework demonstrates robust diagnostic performance and provides reliable biomarkers for neurological assessment, and the identified regions and connectivity patterns offer insights into ASD neural mechanisms and potential diagnostic biomarkers.
Abstract
Multimodal neuroimaging fusion has gained considerable attention in diagnosing autism spectrum disorder (ASD) due to its ability to integrate complementary information across different modalities. However, most existing approaches rely on simple feature concatenation or late fusion strategies without achieving effective cross-modal coupling, failing to capture the intrinsic relationships between functional and structural information. To address these limitations, this study develops the deep multimodal dual-level coupling (DMDC) framework integrating brain functional and structural information to improve model discrimination capability and interpretability. Specifically, the pyramid-inverted graph-attention network is first proposed to dynamically update graph topological structures with the Top-k node filtering algorithm for function-structure network coupling. Second, the skeleton-based white matter projection method maps functional magnetic resonance imaging (fMRI) signals onto diffusion tensor imaging (DTI) derived white matter skeletons, followed by the multi-layered hierarchical convolutional network for representation extraction. Finally, the weighted feature integration mechanism combines both components, and the neural network with cross-entropy loss optimization is employed for ASD identification. Extensive experiments show DMDC outperforms state-of-the-art methods. Key discriminative regions identified include the hippocampus, anterior cingulate cortex, amygdala, and frontal gyri. Both hyperconnectivity and hypoconnectivity were observed in ASD, especially in prefrontal cortex, amygdala, and hippocampus, which were critical for social and emotional processing. The proposed framework demonstrates robust diagnostic performance and provides reliable biomarkers for neurological assessment. The identified regions and connectivity patterns offer insights into ASD neural mechanisms and potential diagnostic biomarkers.
The integration of diffusion tensor imaging (DTI) and functional magnetic resonance imaging (fMRI) for computer-aided diagnosis presents significant challenges, primarily due to their diverse spatial-temporal representations. Furthermore, current multimodal fusion techniques frequently encounter challenges in differentiating between shared and individualized information across various modalities, which restricts their capacity to identify nuanced structural and functional abnormalities linked to the progression of neurodegenerative conditions. In response to these challenges, we introduce a framework known as modal characteristic optimization and spatiotemporal correlation features fusion (MCO-SCFF) aimed at diagnosing mild cognitive impairment (MCI) and Alzheimer’s disease (AD). MCO-SCFF utilizes specialized encoders to derive both shared and modality-specific representations from DTI and fMRI data, facilitating efficient intra- and inter-modality fusion. Furthermore, a brain region spatiotemporal features aggregator (BR-SFA) is proposed to consolidate spatiotemporal features across various brain regions through the implementation of a graph-based architecture. Extensive experiments on the ADNI and PPMI datasets demonstrate the effectiveness of the proposed framework. In particular, on the ADNI dataset, MCO-SCFF achieves 84.8±2.6\% accuracy, 93.6±7.3\% sensitivity, and 82.9±6.8\% specificity on the challenging MCI vs AD task. On the PPMI dataset, it achieves 85.6±5.3\% accuracy, 86.4±4.8\% sensitivity, and 85.9±7.6\% specificity, showing robust and competitive performance for multimodal neurodegenerative disease diagnosis. Our code will be released to the community.
Wang Yuxiunan, Rumeng Wu· Engineering Research Express· 0 citations
Brain disorders involve abnormalities spanning multiple biological and phenotypic scales that require the integration of multiple data sources, including neuroimaging and structured clinical data, to achieve accurate diagnosis. However, effectively integrating heterogeneous imaging modalities with tabular clinical data remains challenging due to fundamental differences in data structure, dimensionality, and semantics. Existing fusion strategies often restrict fusion to a single network layer, hindering the progressive integration of complementary features across multiple layers and limiting the sufficiency and flexibility of interactions between modalities. To address these limitations, we propose an Adaptive Tabular-guided Layerwise Alignment and Synthesis Network (ATLAS-Net) for effective multimodal neuroimaging and tabular data integration. Specifically, ATLAS-Net introduces an adaptive layerwise alignment (ALA) module that progressively aligns and fuses features from imaging branches through attention-driven interactions at multiple network layers, enabling continuous and sufficient cross-modality information exchange. In addition, we design a tabular-adaptive parameterization (TAP) module, which incorporates structured clinical variables as conditional priors to dynamically generate network parameters and modulate intermediate feature channels, thereby allowing tabular information to directly guide image feature extraction throughout the network hierarchy. Extensive experiments on 8 brain disorders demonstrate that ATLAS-Net consistently outperforms state-of-the-arts in classification performance, cross-dataset generalization, and robustness to realistic spatial noise perturbations. The code is available at https://github.com/liangchuang11/ATLAS-Net.git.
Chuang Liang, T. Adalı, Jing Sui et al.· Proceedings of the 32nd ACM...· 0 citations
Experimental evaluation on the proposed multi-head Attention-driven Multimodal Feature Integration Network framework demonstrates that the proposed MAtMFIN framework consistently outperforms state-of-the-art transfer learning and Deep Learning models, including a hybrid Convolutional Neural Network (CNN)-Long Short-Term Memory (LSTM) model.
A. R., Radha Senthilkumar· Discover Artificial Intellig...· 0 citations
Alzheimer’s disease (AD) is a neurodegenerative disorder, and mild cognitive impairment (MCI) represents a transitional stage between AD and cognitively normal (CN) individuals. Early diagnosis is clinically important for delaying disease progression. To address the limitations of single-modal approaches and the insufficient modeling of complex cross-modal interactions in existing multimodal fusion methods, this paper proposes a multimodal deep learning (DL) classification framework integrating structural magnetic resonance imaging (sMRI) and clinical features. The framework employs a 3D ResNet-34 to extract imaging features and a multilayer perceptron to encode clinical data. A bidirectional cross-modal attention mechanism enhances associations between imaging and clinical modalities, followed by an adaptive gated fusion module that dynamically integrates concatenated global multimodal features with cross-modal interaction features. To evaluate the robustness of the proposed framework, all experiments were repeated using five different random seeds, and the results are reported as mean ± standard deviation. Experimental results demonstrate competitive performance across multiple classification tasks, achieving accuracies of 95.67% ± 1.70% for three-class classification (CN vs MCI vs AD) and 93.64% ± 2.36% for four-class classification (CN vs early MCI (EMCI) vs late MCI (LMCI) vs AD). For binary classification (AD vs CN, AD vs MCI, MCI vs CN, and EMCI vs LMCI), the method achieves accuracies of 96.83% ± 1.53%, 95.33% ± 1.00%, 95.83% ± 1.39%, and 93.61% ± 2.35%, respectively. The proposed framework provides an effective solution for multimodal DL-based computer-aided diagnosis of AD.
Xiao-Li Yang, Chen-Chen Wang, Xiao Li et al.· Biomedical engineering and p...· 0 citations
Accurate diagnosis and progression prediction of Alzheimer's disease (AD) remain challenging due to the heterogeneous nature of the disease, which involves structural brain degeneration, electrophysiological dysfunction, and molecular dysregulation. Most existing deep learning approaches rely on a single modality or limited multimodal combinations, thereby failing to capture the complex cross-domain interactions underlying AD progression. Furthermore, the scarcity of large-scale datasets containing synchronized neuroimaging, electrophysiological, and genomic measurements restricts the development of comprehensive multimodal diagnostic systems. To address these challenges, this study proposes NeuroOmics-Net, a multimodal deep learning framework for Alzheimer's disease analysis that integrates structural magnetic resonance imaging (sMRI), electroencephalography (EEG), and gene expression data. The proposed framework combines a Hierarchical Multi-View Encoder (HME) for modality-specific feature extraction, a Cross-Omics Attention Fusion (CAF) module for adaptive integration of complementary biomarkers, and a Disease Progression Graph Learning (DPGL) module for modeling progression-related relationships across biological domains. To facilitate cross-modal integration from independent cohorts, Regularized Canonical Correlation Analysis (RCCA) is employed to align heterogeneous feature representations within a shared latent space. Experiments were conducted using publicly available datasets from ADNI, PhysioNet, and GEO repositories comprising 1120 diagnosis-aligned samples. The proposed framework achieved 94.3% classification accuracy and an AUC of 0.975 for distinguishing normal controls (NC), mild cognitive impairment (MCI), and Alzheimer's disease subjects, while attaining 93.7% accuracy for predicting conversion from stable mild cognitive impairment (sMCI) to progressive mild cognitive impairment (pMCI). However, a fairness sensitivity analysis using stratified demographic reweighting revealed accuracy ranging from 90.8% (low-education, high-comorbidity proxy subgroup) to 96.1% (low-risk, high-reserve proxy subgroup), a demographic parity gap of 5.3 percentage points, indicating that overall accuracy reflects a performance ceiling in a relatively homogeneous research cohort rather than a realistic estimate for demographically diverse clinical populations. Comparative evaluations demonstrated consistent improvements over state-of-the-art unimodal and multimodal deep learning models. Interpretability analysis further identified clinically relevant biomarkers, including hippocampal and entorhinal atrophy, theta-alpha EEG alterations, and APOE-associated molecular pathways. Because sMRI, EEG, and gene expression data were sourced from separate, unpaired cohorts with no subjects possessing all three synchronized measurements, all reported cross-modal associations reflect population-level statistical correspondence across diagnosis-matched groups rather than within-subject physiological coupling; no claim of intra-individual causal cross-modal interaction is made. These findings demonstrate that NeuroOmics-Net provides an effective computer-aided framework for multimodal biomedical data processing and Alzheimer's disease analysis. By integrating neuroimaging, electrophysiological, and genomic information, the proposed approach enables accurate diagnosis, progression prediction, and biologically interpretable decision support for clinical and translational applications.
R. Kavitha, K. Premalatha· Comput. Biol. Medicine· 1 citation
Functional brain network analysis is pivotal for understanding the neurological basis of Autism Spectrum Disorder (ASD). However, functional connectivity (FC) suffers from inherent noise, making it challenging to accurately capture its global and local information while mitigating noise interference. To address this limitation, we propose a Region-Aware Graph Transformer with Global-Local Dual Attention Mask (RAGT-DM). Specifically, the global attention mask retains nodes with larger attention scores, which captures long-range global topological patterns while reducing noise interference. Meanwhile, the local attention mask selects adjacent brain nodes based on Euclidean distance to characterize local structural connections, strengthening the model’s perception of local information. Experiments on the ABIDE I and ABIDE II datasets demonstrate that RAGT-DM achieves 73.04% and 71.49% accuracy, outperforming state-of-the-art methods.
Yu-Jin Wang, Jia-Yu Lu, Zhen-Guo Zhang et al.· Journal of Physics, Conferen...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
