TIER-MoE is introduced, a risk-guided subspace mixture-of-experts model that defines sample-specific modality reliability as the prediction loss its unimodal predictor is expected to incur, and demonstrates its superiority over state-of-the-art methods in predictive performance and probability calibration.
Abstract
The promise of multimodal fusion lies in combining complementary sources of evidence, yet more evidence does not always yield a better prediction. Recent multimodal models have advanced fusion through richer cross-modal interaction and sample-adaptive fusion. However, the influence assigned to a modality during fusion does not reveal whether that source is unreliable, redundant, or poorly matched to a specialized expert. To address this limitation, we introduce TIER-MoE, a risk-guided subspace mixture-of-experts model that defines sample-specific modality reliability as the prediction loss its unimodal predictor is expected to incur. This risk is learned from out-of-fold predictions generated by models that were not trained on the corresponding sample. TIER-MoE combines the estimated risk with expert-specific subspace compatibility for sparse modality-expert routing, while an always-active shared path preserves multimodal complementarity. We evaluate TIER-MoE on four public multimodal biomedical datasets spanning Alzheimer's disease status, skin-lesion malignancy, and retinal classification. Results demonstrate its superiority over state-of-the-art methods in predictive performance and probability calibration, with consistent improvements in Macro-F1 and Brier score and strong zero-shot generalization to an external cohort.
Multimodal deep learning for clinical decision support frequently fails in real-world deployments due to severe data missingness and sensor noise. Standard fusion and Mixture-of-Experts (MoE) architectures assume all modalities are uniformly informative, causing routing collapse and miscalibrated confidence when faced with corrupted inputs. My doctoral research addresses this fragility by shifting the paradigm from generative imputation to uncertainty-guided abstention. As a foundational step, I introduce QA-MoE (Quality-Aware MoE), which explicitly decouples reliability estimation from the routing process. By quantifying epistemic uncertainty, the model dynamically filters noisy modalities before fusion, ensuring sparse and stable inference. Building on this static robustness, my ongoing and future research extends to longitudinal and structured clinical data. I am developing PathFlow-EM, which integrates Expectation-Maximization to handle hidden variable distributions in unevenly sampled time-series, and Uncertainty-Guided Dynamic Topological Fusion (U-DTF), which uses topological data analysis to gate features based on evolving patient graphs. Ultimately, this research trajectory aims to deliver robust, calibration-aware architectures that form the computational basis for reliable Patient Digital Twins.
Lin-Peng Sun· Proceedings of the Thirty-Fi...· 0 citations
Clinical prediction increasingly relies on multi-modal inputs, where reliability and efficiency are crucial for real-world deployment. However, mainstream fusion and MoE gating typically treat all available modalities as uniformly beneficial and allow noisy or weakly informative modalities to perturb routing, leading to instability, routing collapse, and miscalibrated confidence under missingness and shift. We propose QA-MoE, a Quality-Aware and stable multimodal Mixture-of-Experts that decouples reliability estimation from routing to enable robust, sparse, and interpretable fusion. QA-MoE adopts a modular architecture where each modality is initially encoded into a shared embedding space. To deal with structurally missing data, we employ a completion pathway that maintains a consistent interface. Unlike standard approaches, QA-MoE separates reliability estimation from the routing process. We propose an Evidential Quality Scorer to measure epistemic uncertainty, which then guides a Stability-Enhanced Subset Selector to filter out noisy modalities on the fly. Additionally, we include a Ternary Expert Aggregation mechanism acting as a specialized branch to stabilize predictions when data missingness is severe. Evaluations on clinical benchmarks (ADNI for Alzheimer’s staging and MIMIC-IV for Length-of-Stay) demonstrate that QA-MoE outperforms strong multimodal baselines, improving reliability while cutting down unnecessary computation. This indicates that QA-MoE offers a robust solution for multimodal decision support, especially in clinical settings prone to noise and missing data.
Lin-Peng Sun, Victor S. Sheng· Proceedings of the Thirty-Fi...· 0 citations
REM-Fuse is introduced, an evidential deep learning framework in which per-source Dirichlet uncertainty adaptively weights each source through dual-channel weighting, asymmetric cross-scale enhancement, and Dempster–Shafer-inspired evidence accumulation that provides a compact reliability-aware fusion strategy for cancer prognosis.
Yalu Huang, Yushuai Yuan, Wenbin Ye et al.· Mathematics· 0 citations
Skin cancer is a prevalent and potentially fatal disease that emphasizes the need for accurate and interpretable diagnostic tools to improve patient outcomes. Although DL has advanced automated skin lesion analysis, most models rely solely on dermoscopic images and neglect the complementary clinical metadata. In this study, we propose FuseMD-XNet, a multimodal transformer-based framework that integrates dermoscopic images with structured patient metadata. The model employs intermediate fusion through feature concatenation and cosine similarity alignment, followed by adaptive certainty-guided fusion to dynamically weigh the modality contributions based on confidence estimates. To ensure transparency, the FuseMD-XNet incorporates multilevel explainability using ShapleyCAM, FinerCAM, and SHAP methods. The efficacy of FuseMD-XNet was validated on the PAD-UFES-20 dataset, where it achieved an overall mean diagnostic accuracy of 94.4±0.8% and a mean AUC of 95.9±0.5% across all lesion classes. The highest class-specific performance was observed for basal cell carcinoma (BCC), with an accuracy of 98.4±0.4% and an AUC of 98.7±0.3%, whereas melanoma achieved an accuracy of 97.9±1.8% and an AUC of 98.2±0.5%. On the ISIC 2019 dataset, FuseMD-XNet demonstrated strong generalization performance with an overall mean accuracy of 93.0±0.8% and a mean AUC of 94.6±0.6%, whereas melanoma achieved a class-specific accuracy of 94.7±1.6% and an AUC of 96.3±1.1%. Additionally, an integrated risk stratification module enabled personalized assessments validated by feature importance analysis. These results demonstrate the potential of FuseMD-XNet to improve the classification accuracy and interpretability of skin cancer.
Akbar Kushanoor, Sanjay K. Sahay· Medical Image Analysis· 0 citations
