Skip to content
Open access

Evidence-based clinical question answering using retrieval-augmented generation (RAG) and open-source large language models

Oct 2026 · Scientific Reports · 0 citations

TL;DR

It is shown that, within a structured, single-textbook corpus, embedding model choice has significantly greater impact on RAG system reliability and performance than chunking strategy, and that open-source, evidence-driven architectures offer a transparent alternative for textbook-based clinical QA settings.

Abstract

Large language models (LLMs) are being investigated for clinical decision support tools, but can produce unreliable outputs, including hallucinated information, overgeneralized answers, and responses not clearly derived from authoritative medical sources. To ground responses in high-quality medical texts, researchers developed Retrieval-Augmented Generation (RAG). In this work, we show that, within a structured, single-textbook corpus, embedding model choice has significantly greater impact on RAG system reliability and performance than chunking strategy. The system compares twelve embedding models—general-purpose and biomedical-specific—under token-level and character-level chunking, prompting the LLM to answer from retrieved snippets. We evaluate retrieval effectiveness using rank-based metrics and generation quality using standard measures across 232 questions. General-purpose embeddings, particularly Jina-Embeddings-v3 and BGE-base-en-v1.5, outperformed biomedical-specific encoders in both retrieval and generation. Across both strategies, higher retrieval precision translated directly into improved lexical and semantic alignment with gold references. Under our operational definition requiring explicit evidence spans, 35.6% of answered questions lacked supporting evidence; our best configuration (BGE-base-en-v1.5, token-level) achieved the lowest rate at 26.1%—an evidence-grounding measure, not a guarantee of correctness. These findings highlight the central role of retrieval quality in clinical RAG systems, showing that open-source, evidence-driven architectures offer a transparent alternative for textbook-based clinical QA settings.

Read PDF

Similar papers

#artificial intelligence Review Oct 2026

DocTalk: Enhancing Medical Knowledge Retrieval via Fine-tuned Large Language Models with Chain-of-Thought Reasoning for Robust Clinical Decision Support

Objective: The rapid advancement of large language models (LLMs) in healthcare has opened new possibilities for AI-assisted clinical decision-making; however, safe and effective deployment remains challenging because of hallucination and domain misalignment. This study introduces DocTalk, a doctor-assistive question-an...

Asad Gohar, Md Ariful Islam Mozumder, Moon-Il Joo et al. · 0 citations
Review Open access Sep 2026

From Fine-Tuning to Grounding: Retrieval-Augmented Generation for Biomedical LLMs in Research and Clinical Data Infrastructures

Large language models (LLMs) are increasingly explored for biomedical research and clinical medicine, yet their use remains limited by hallucinations, missing provenance, domain drift, privacy constraints, and uncertain regulatory pathways. Retrieval-augmented generation (RAG) offers a pragmatic alternative to full mod...

M. Enayati, Vishnu Priya, E. Prochaska et al. · 0 citations
Conference Open access Sep 2026

MERGE-Med: Multi-Expert Retrieval-Grounded Engine for Medical Report Summarization Using Hierarchical RAG and Contrastive Learning

MERGE-Med is a hybrid multi-expert architecture that consists of a mixture-of-experts (MoE) layer with three domain-specialized LLaMA-3 8B models for diagnostic reasoning, treatment protocol generation, and patient-friendly communication and a contrastive fine-tuning framework validated by an ensembled DeBERTa-v3-large...

D. Tejaswi, P. R. Reddy, K. Shailaja et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.