It is shown that Monte Carlo dropout, applied to a multi-task chest-radiograph classifier, provides an epistemic uncertainty signal that tracks generalisation across training-set scales and flags confident yet error-prone predictions.
Abstract
Machine learning models for medical image analysis typically lack a reliable measure of confidence, limiting their use in ambiguous or atypical cases. Here we show that Monte Carlo dropout, applied to a multi-task chest-radiograph classifier (eight thoracic findings, 137,593 training images), provides an epistemic uncertainty signal that tracks generalisation across training-set scales and flags confident yet error-prone predictions. Adding this signal to the point prediction raised error-detection AUROC from 0.74 to 0.77 ($\Delta$AUROC +0.023, 95% CI [+0.014, +0.033]). In a controlled 2x2 factorial experiment, a clinical-decision-support agent exploited this uncertainty only when it was delivered as a binary error-risk flag rather than as raw scores, cutting confident misdiagnoses on unreliable findings from 8.5% to 2.7%. Epistemic uncertainty estimation thus carries decision-relevant information beyond point predictions, but its value for downstream agents depends on how it is communicated.
A rigorous empirical framework is presented for comparing three uncertainty quantification approaches on two clinical prediction tasks, in-hospital mortality and 30-day readmission, using 74,829 ICU admissions from the MIMIC-IV database to support a more demanding evaluation standard for UQ in clinical machine learning.
Isaac Tosin Adisa, Francis Mawutor Amuyao, Ezekiel Olaoluwa Joaquim· International journal of re...· 0 citations
The Expected Gradients Reconstruction Uncertainty Estimate (egRUE), which incorporates prediction explanations into its uncertainty computation and decomposes uncertainty into feature-wise contributions, and proves theoretical properties of egRUE and shows that it improves reliability and interpretability compared to existing methods.
Li-Rong Wang, J. Duell, Xinran Xu et al.· 0 citations
Artificial intelligence (AI) is rapidly integrating into clinical radiology. As primary diagnosticians, radiologists increasingly interpret AI-generated analyses and are expected to oversee the monitoring and governance of deployed AI systems. Although AI literacy among radiologists is improving, several technical aspects of AI remain insufficiently accessible. One such concept is uncertainty quantification (UQ), which estimates the reliability of AI predictions and can signal when outputs should be interpreted with caution. This review introduces key UQ concepts relevant to radiology, distinguishing between aleatoric uncertainty and epistemic uncertainty arising from data variability and knowledge gaps. We summarize commonly used UQ approaches in current research and practice. Furthermore, through a narrative review of selected recent AI imaging studies, we illustrate how UQ methods are applied in practice and highlight methodological trends, findings, and limitations. Although UQ has the potential to improve the safety and interpretability of AI-assisted screening, challenges remain, including calibration, threshold selection, computational cost, and the need for prospective clinical validation.
Fernando Vega Lara, Lisa Koopmans, Christian Roest et al.· Abdominal Radiology· 0 citations
Automated chest X-ray diagnosis fails most critically when models produce confident yet incorrect predictions, suppressing clinical oversight at the point of decision-making. Existing uncertainty estimation methods define epistemic uncertainty primarily as a property of model parameters, overlooking whether predictions remain consistent with clinically similar cases. We propose that the divergence between a model’s prediction and the empirical label distribution of neighbouring training cases provides a practical reliability signal—one that correlates with, and can serve as a proxy for, epistemic uncertainty, while acknowledging that it may also reflect additional sources of discrepancy including label noise, representation error, and local population variability. To operationalize this perspective, we introduce a retrieval-based framework that constructs a fixed embedding-space memory of training cases and estimates a non-parametric label distribution over nearest neighbours. This enables direct comparison between model predictions and local case-level structure without introducing additional trainable parameters or modifying the diagnostic backbone. Experiments on ChestX-ray14 demonstrate improved detection of misclassified predictions relative to deep ensembles, with approximately 2.9 percentage-point gains in uncertainty-based error identification and an approximately 23% reduction in calibration error. Under zero-shot transfer to PadChest and CheXpert, the proposed approach exhibits smaller degradation in uncertainty estimation quality, with larger improvements observed for rare pathological conditions. These findings suggest that case-level consistency provides a meaningful practical signal for reliability estimation in medical imaging, whose relationship to the classical epistemic–aleatoric decomposition warrants further theoretical investigation across broader clinical settings.
Muhannad Faleh Alanazi, B. Z. Shakhreet, Hattan Ali A. Asiri et al.· Scientific Reports· 0 citations
Machine learning models are increasingly deployed in decision-critical environments such as healthcare, finance, and autonomous systems. However, most conventional models generate deterministic predictions without quantifying uncertainty, which can lead to overconfident mispredictions when data are noisy, incomplete, or outside the training distribution. This limitation exposes a critical gap between predictive accuracy and decision reliability in real-world Al systems. To address this challenge, this study proposes an uncertainty-aware machine learning framework that integrates probabilistic modeling techniques into conventional predictive architectures to jointly estimate epistemic and aleatoric uncertainty. The proposed framework enables models to produce predictive distributions rather than single point predictions, allowing systems to quantify confidence and identify high-risk predictions. Experiments were conducted on multiple benchmark datasets representing both classification and regression tasks under varying levels of noise and data incompleteness. The experimental results demonstrate that the proposed framework achieves predictive performance comparable to deterministic baselines while significantly improving reliability and uncertainty calibration. In classification tasks, the model maintained competitive accuracy and F1-scores while providing well-calibrated confidence estimates, whereas in regression experiments the approach reduced prediction risk by identifying high-error cases through increased uncertainty variance. Robustness tests further show that the framework effectively signals degraded prediction reliability when encountering noisy or incomplete inputs. These findings indicate that incorporating uncertainty estimation enhances trustworthiness and robustness without sacrificing predictive performance. The study highlights uncertainty modeling as a critical component for developing reliable and responsible Al systems capable of supporting risk-sensitive decision-making in real-world data-driven environments
Evi Yulianingsih, E. Noche, V. Yadav et al.· Journal of Data Science· 0 citations
Although Artificial Intelligence (AI) has shown strong predictive performance in medicine, its role in providing cognitive support remains unclear. We developed a knowledge-based expert system using a Bayesian Network (BN) to explicitly model an expert neurosurgeon's clinical reasoning for predicting outcomes of Gamma Knife radiosurgery. The BN structure was constructed from elicited expert knowledge and encoded as a directed acyclic (DAG) graph that represents clinically relevant variables and their interrelationships. Model parameters were estimated using a clinical dataset to predict patient overall survival (OS). In parallel, physicians independently estimated outcomes for the same patients, enabling a direct comparison of predictive performance and error characteristics. The BN captured clinically plausible reasoning patterns and produced less skewed prediction errors than physicians, suggesting partial mitigation of individual cognitive bias. However, both the BN and physicians exhibited a heavy-tailed error distribution, systematically failing to predict patients with unexpectedly long-term survival. This convergence of error patterns indicates that both human and model-based predictions are constrained by unobserved or unmodeled factors, limiting their ability to capture exceptional outcomes. These findings highlight a limitation of predictive paradigms based on current expert knowledge and suggest the need for iterative refinement of knowledge-based models. Incorporating additional expert knowledge and updating the underlying structure may help identify previously unmodeled relationships and reduce unexplained variance. Taken together, our results position knowledge-based BNs not only as interpretable predictive tools, but also as evolving frameworks for integrating clinical knowledge and probing the shared limits of human-AI decision-making, highlighting their potential as computational cognitive neuroprosthetic systems that aim to augment physicians' clinical reasoning under uncertainty.
Yusuke Uchiyama, Soichiro Fujiki, Kensaku Nomoto et al.· Frontiers in Neuroscience· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
