Findings highlight key trade-offs among symbolic, generative, and instruction-tuned approaches for CGA and 4Ms phenotyping, suggesting that hybrid heuristic-LLM methods offer interpretability and stability, whereas large language models provide greater adaptability for complex clinical narratives.
Abstract
Current Natural Language Processing (NLP) algorithms for detecting geriatric conditions are largely limited to domain-specific models that fail to capture the interdependent, multidimensional nature of comprehensive geriatric assessment. This study aimed to develop and evaluate a comprehensive, scalable, and robust information extraction framework to identify Comprehensive Geriatric Assessment (CGA) and Age-Friendly Health Systems (AFHS) 4Ms-related data elements from unstructured electronic health record (EHR) text across multiple health systems. Using a team science approach grounded in the TRUST framework, we annotated pooled clinical notes from four health systems to produce a gold-standard dataset of 41 CGA- and 4Ms-related geriatric care data elements. Three information extraction approaches were implemented and evaluated: an in-context learning generative large language model (GPT-4o), a hybrid heuristic-LLM model (MedAgingIE), and an instruction-tuned open-source lightweight model (Qwen2-7B-Instruct). Performance was assessed on a blinded test set using macro- and micro-averaged metrics. GPT-4o achieved a macro F1-score of 0.56 and micro F1-score of 0.87; MedAgingIE achieved 0.55 and 0.92; and Qwen2-7B-Instruct achieved 0.30 and 0.81, respectively. MedAgingIE demonstrated the strongest consistency between precision and recall, while GPT-4o showed superior sensitivity for diverse, context-rich geriatric concepts. These findings highlight key trade-offs among symbolic, generative, and instruction-tuned approaches for CGA and 4Ms phenotyping, suggesting that hybrid heuristic-LLM methods offer interpretability and stability, whereas large language models provide greater adaptability for complex clinical narratives.
Empirical evidence is provided on how retrieval, embedding, and prompting choices influence clinical information extraction performance and practical guidance is offered for researchers and healthcare informatics practitioners seeking to optimise retrieval-augmented large language model systems for extracting clinicall...
Dinithi S. Vithanage, Quang Vinh Duong, Chao Deng et al.· Journal of Healthcare Inform...· 0 citations
This approach combines semantic understanding of clinical narratives with structural modeling of patient-disease-treatment relationships and successfully validates synthetic EHR data utility for privacy-preserving healthcare AI development while addressing critical requirements necessary for clinical decision support s...
U. Luke, P. Asuquo, Victor Anaga et al.· E3S Web of Conferences· 0 citations
A novel framework that leverages the language understanding and code generation ability of Large Language Models (LLMs) to build an information retrieval system with Visualization-oriented Natural-language-based Inter-faces (V-NLI).
Xin Gao, Zheng-Ye Zhu, Xin-Yu Ma et al.· Tsinghua Science and Technol...· 0 citations
The benefits of multimodal data integration are task-dependent and healthcare LLMs should examine clinical data modalities according to specific tasks for efficient integration, and provide practical guidance for designing efficient clinical decision support systems.
Cheng Peng, Mengxian Lyu, Ziyi Chen et al.· JAMIA Journal of the America...· 0 citations
The results show that it is feasible to make better predictions and gain valuable insights by merging these two types of data and that integrating unstructured data allows for a more holistic view of patient health, leading to earlier detection, personalized interventions, and improved decision-making in clinical setti...
A schema-driven, zero-shot LLM workflow can support reproducible extraction of clinically relevant information from narrative notes and provides a transferable methodology for institutional adaptation and validation prior to clinical use.
Diya Saha, J. Edgcomb· JMIR Mental Health· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Senior Research Scientist Jim Glass discusses how his MIT-IBM Computing Research Lab collaboration connects perception and language, and addresses the shortcoming of current models.
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Enforcing physics-based constraints prevents AI models from generating unreliable predictions for complex nonlinear systems in weather, climate, and turbulent flows.
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.