Data-efficient Neuroradiology MRI Imaging Protocol Prediction Using Open-weights Large Language Models.
TL;DR
A locally deployable SOP-grounded open-weight LLM can support MRI protocol selection while preserving data privacy and needing minimal labeled data, and support prospective evaluation in human-in-the-loop clinical workflows.
Abstract
Purpose
To evaluate the feasibility of a locally deployable large language model (LLM) system for automated MRI protocol selection addressing data privacy, annotation burden, and scalability limitations.
Methods
This retrospective study included 598 German-language MRI order entries from three neuroradiology domains (brain, head/neck, spine) between June 2018 and January 2023. A radiologist labeled entries for 27 protocol classes based on institutional standard operating procedures (SOP). An SOP-grounded AI system using MedGemma 27B was developed to predict the MRI protocol from the order entry. The system was optimized using Stochastic Introspective Mini-Batch Ascent (SIMBA), a self-reflective prompt optimization algorithm, and compared with a hierarchical system that first classified the body region and then the MRI protocol. Data efficiency was evaluated using training subsets of 10-119 examples across 3 optimization runs per subset size.
Results
The flat zero-shot model achieved 73.07% accuracy in the three-domain setting on the held-out dataset (n = 479). In the hierarchical model, prompt optimization yielded 73.90% ± 3.24% with 30 labeled examples and a maximum of 74.46% ± 3.82% but did not outperform the flat approach. Prompt optimization benefited the hierarchical system, whereas the flat model already performed strongly without labeled examples for optimization. Performance on brain and head/neck cases remained broadly stable after expansion from 22 to 27 protocol classes, i.e., including spine.
Conclusion
A locally deployable SOP-grounded open-weight LLM can support MRI protocol selection while preserving data privacy and needing minimal labeled data. In this dataset, hierarchical routing and prompt optimization did not improve overall performance over the flat baseline, although they altered optimization behavior and error profile. These findings support prospective evaluation in human-in-the-loop clinical workflows.