Skip to content

Large Language Models in Clinical Neurology: A Scoping Review.

Feb 2027 · Neurology Clinical Practice · Vol 17 1, pp. e200687 · 0 citations · 43 references
Medicine

Abstract

Purpose

OF REVIEW Large language models (LLMs) are increasingly being evaluated for clinical neurology applications, yet their real-world utility, safety, and deployment readiness remain unclear. This review maps the literature on LLMs applied to text-based and text-derived neurology tasks, characterizing current use cases, evaluation strategies, performance metrics, and reported safety concerns. We identify areas with relatively robust evidence, highlight important evidence gaps, and outline key methodological and clinical barriers that must be addressed before responsible clinical adoption. RECENT

Findings

Following  Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR), we searched 4 databases (January 2022 to February 2026) and included 32 studies (2023-2026) spanning 7 neurology subspecialties; 9 were simulation or feasibility studies, 17 analyzed retrospective clinical data, and 6 reported prospective evaluation. Proprietary models predominated; 5 studies used retrieval-augmented generation and 1 used a multiagent framework. Performance was highest for constrained tasks, including binary diagnostic classification (area under the curve 0.75-0.94), information extraction (F1 0.89-0.90), patient-education question answering (accuracy 68%-97%), and ischemic stroke thrombectomy decision support (area under the curve [AUC] 0.92) and lowest for open-ended case-based reasoning (accuracy 42%-54%). Recurrent safety signals included hallucinated and fabricated citations, poor calibration with overconfidence (for example, 100% sensitivity with 27% specificity for epilepsy diagnosis), and a systematic bias toward intervention. On adapted Quality Assessment of Diagnostic Accuracy Studies Using Artificial Intelligence (QUADAS-AI) charting, overall risk of bias was low in 1 study (3%), moderate in 16 (50%), and high in 15 (47%), driven chiefly by absent external validation (31/32, 97%). Benchmarked against high-reliability and 6 Sigma standards, reported error rates remain orders of magnitude too high for autonomous use. SUMMARY Evidence for LLMs in neurology is early, heterogeneous, and limited by scarce external and prospective validation. Current data support only narrow, clinician-supervised uses such as documentation and education, while cautioning against autonomous diagnostic or therapeutic use. Rigorous, prospective, externally validated studies with standardized reporting and explicit safety and ethical evaluation are needed before broader adoption. REGISTRATION PROSPERO CRD420251082465.

Read PDF

Similar papers

#artificial intelligence Preprint Jul 2024

The Llama 3 Herd of Models

It is found that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks, and performs competitively with the state-of-the-art on image, video, and speech recognition tasks.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri et al. · 21,082 citations · ⚡3,832
#computer vision Review Sep 2017

Agile Software Development Methods: Review and Analysis

This publication proposes a definition and a classification of agile software development approaches and analyses ten software development methods that can be characterized as being "agile" against the defined criterion.

P. Abrahamsson, O. Salo, Jussi Ronkainen et al. · 727 citations · ⚡54
#computer vision Jun 2008

The impact of agile practices on communication in software development

The study shows that agile practices improve both informal and formal communication, but indicates that, in larger development situations involving multiple external stakeholders, a mismatch of adequate communication mechanisms can sometimes even hinder the communication.

M. Pikkarainen, Jukka Haikara, O. Salo et al. · 401 citations · ⚡48
#machine learning Review Open access Oct 2014

Software development in startup companies: A systematic mapping study

The results indicate that software engineering work practices are chosen opportunistically, adapted and configured to provide value under the constrains imposed by the startup context.

Nicolò Paternoster, Carmine Giardino, M. Unterkalmsteiner et al. · 394 citations · ⚡54

Trajectory Balance: Improved Credit Assignment in GFlowNets

It is proved that any global minimizer of the trajectory balance objective can define a policy that samples exactly from the target distribution, and empirically demonstrate the benefits of the trajectories balance objective for GFlowNet convergence, diversity of generated samples, and robustness to long action sequenc...

Esmeralda S. Whitammer, Moksh Jain, Emmanuel Bengio et al. · 302 citations · ⚡60

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.