Skip to content

Performance vs Consistency: Evaluating a Foundation Model in Lung-RADS Screening

Sep 2026 · 0 citations · 16 references
Computer Science

TL;DR

MedGemma, a medical general-purpose foundation model derived from Gemini and its fine-tuned version adapted for lung cancer detection and diagnosis is compared against radiologists performing Lung-RADS v2022 assessment on the NLST dataset, supporting the role of fine-tuned foundation models potential complementary tools for clinical decision support.

Abstract

Foundation models have recently demonstrated strong capabilities across a wide range of medical imaging tasks. However, their performance in structured clinical interpretation settings remains insufficiently explored. In lung cancer screening, interpretative variability persists despite standardized frameworks such as Lung-RADS. In this study, we evaluate MedGemma, a medical general-purpose foundation model derived from Gemini and its fine-tuned version adapted for lung cancer detection and diagnosis, compared against radiologists performing Lung-RADS v2022 assessment on the NLST dataset. Twelve radiologists independently evaluated each case in a multi-reader design, enabling quantification of inter-reader variability. Radiologists achieved a mean AUC of 0.90, with substantial variability across readers (range: 0.80-0.94). The native foundation model achieved an AUC of 0.70, failing to reach clinically relevant performance. In contrast, fine-tuning significantly improved performance to an AUC of 0.83, placing the model within the lower range of individual radiologists performance. These findings highlight a trade-off between peak accuracy and prediction consistency. Unlike radiologists, under fixed conditions, the model produces deterministic outputs, removing inter-run variability under identical inputs, in contrast to inter-reader variability observed among radiologists. This supports the role of fine-tuned foundation models potential complementary tools for clinical decision support, particularly in settings with limited expertise. However, evaluation is performed on a case-enriched cohort from NLST and does not account for real-world prevalence or external validation, limiting direct clinical generalization.

View source

Similar papers

Review Open access Sep 2026

Foundation Models in Medical Imaging: A Systematic Review, Taxonomy, and Future Research Agenda

This systematic review evaluated forty-two studies published between 2024 and 2026 across eleven databases using PRISMA 2020 guidelines and AI-specific quality frameworks and demonstrated high organ segmentation accuracy with Dice coefficients exceeding 0.90, whereas performance declined for infiltrative tumours, and e...

Roaa Ghanim · 0 citations
Open access Sep 2026

Performance-interpretability trade-offs and generalization in deep learning for pneumonia detection: A benchmarking study

A reproducible, end-to-end methodology that jointly optimizes performance and interpretability, offering practical guidance for selecting and deploying explainable deep learning models in clinical pneumonia screening is contributed.

Francisco A. Gómez-Vela, Aurelio López-Fernández, F. Divina et al. · 0 citations
Review Open access 2026

A COMPARATIVE STUDY OF AI MODELS FOR LUNG CANCER PREDICTION AND CLINICAL DECISION SUPPORT

Artificial-intelligence (AI) research on lung cancer in 2026 shows rapid growth across lightweight convolutional networks, CNN–transformer hybrids, capsule-inspired models, segmentation-driven pipelines, radiomics, multimodal fusion and reader-assistance systems. Model selection is now more than accuracy maximisation:...

A. Sirajdeen, S. Rajakumaran · 0 citations
Review Oct 2026

A Concept-informed Interpretable Artificial Intelligence Framework for Breast Ultrasound Diagnosis

Most existing artificial intelligence systems for breast ultrasound rely primarily on image-driven prediction, with limited incorporation of the knowledge from the Breast Imaging Reporting and Data System (BI-RADS) and the intermediate assessment processes underlying radiologists'workflow, thereby constraining their in...

Sheng-Zhou Zhong, Zi-Yi Wang, Lian-Fu Li et al. · 0 citations
Open access Aug 2026

Incremental Diagnostic Value of Clinical Information for Large Language Models Across Multiple Organs: Retrospective Study

The addition of clinical information was associated with a numeric trend toward higher diagnostic accuracy overall, but this trend was heterogeneous across models and disease types, and no statistically significant improvement was demonstrated after adjustment for multiple comparisons.

Jin-Qi Zhang, Xiao-Yi Wang, Yanfeng Zhao et al. · 0 citations
Sep 2026

Multicohort Evaluation of Acquisition Confounding in Single-Site Lung-CT Malignancy Classification with Frozen Foundation-Model Features.

The near-perfect in-site AUC is therefore most plausibly explained by acquisition-related confounding, which extends beyond reconstruction protocol to scanner model and age, and is not evidence of malignancy discrimination.

Ming-Hsun Lee, Wei-Yang Lin, Yen-Jung Chiu · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.