Skip to content
Conference Open access

MERGE-Med: Multi-Expert Retrieval-Grounded Engine for Medical Report Summarization Using Hierarchical RAG and Contrastive Learning

Sep 2026 · Conference Proceedings in Science and Management · 0 citations · 7 references

TL;DR

MERGE-Med is a hybrid multi-expert architecture that consists of a mixture-of-experts (MoE) layer with three domain-specialized LLaMA-3 8B models for diagnostic reasoning, treatment protocol generation, and patient-friendly communication and a contrastive fine-tuning framework validated by an ensembled DeBERTa-v3-large fact-checking layer that achieves more than 95% consistency.

Abstract

Electronic health records (EHRs) have created large volumes of clinical data that require automated and accurate summarization that patients can understand. Current large language models (LLMs) are prone to hallucination and show limited task specialization and inadequate clinical grounding. MERGE-Med is a hybrid multi-expert architecture that consists of (i) a mixture-of-experts (MoE) layer with three domain-specialized LLaMA-3 8B models for diagnostic reasoning, treatment protocol generation, and patient-friendly communication; (ii) a hierarchical retrieval-augmented generation (RAG) pipeline that combines FAISS dense retrieval, BM25 sparse retrieval, cross-encoder reranking, and Unified Medical Language System (UMLS) knowledge-graph traversal over 35 million PubMed abstracts, 50,000 clinical guidelines, and 15,000 pharmaceutical records; and (iii) a contrastive fine-tuning framework validated by an ensembled DeBERTa-v3-large fact-checking layer that achieves more than 95% consistency. MERGE-Med achieved accuracies of 92.45%, 82.34%, and 98.76% on MedQA, MedMCQA, and Clinical Knowledge, respectively, improvements of 5.18, 6.44, and 1.78 percentage points (pp) over the baseline LLaMA-3 8B. The hallucination rate decreased by 67% (from 18.3% to 6.1%), ROUGE-L improved by 22.3%, and in an evaluation by board-certified physicians of 100 de-identified reports, 92% of summaries were judged appropriate for patients, with an inference latency of 2.3 s.

Read PDF

Similar papers

Open access Oct 2026

Evidence-based clinical question answering using retrieval-augmented generation (RAG) and open-source large language models

It is shown that, within a structured, single-textbook corpus, embedding model choice has significantly greater impact on RAG system reliability and performance than chunking strategy, and that open-source, evidence-driven architectures offer a transparent alternative for textbook-based clinical QA settings.

Ayesha Qasim, Jesús Jaime Moreno Escobar, C. Maldonado-Sifuentes et al. · 0 citations
Open access Jan 2026

Ensemble multi-retrieval methodologies with reasoning model decision in multi-step biomedical QA

This work presents a retrieval-augmented generation (RAG) and decision-making framework that integrates diverse retrieval strategies, including Query2Doc-based, Rationale-based, and Web-augmented retrieval, and employs a dedicated Decision Maker model to select or directly generate the most accurate and well-reasoned a...

Jongmyung Jung, Hyeon Hwang, Yein Park et al. · 0 citations
Preprint Aug 2026

MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis

This work introduces MedReaMM, a benchmark specifically designed to evaluate models'ability to synthesize heterogeneous clinical evidence consisting of detailed patient histories alongside multiple medical images into accurate differential diagnoses under a complete-information paradigm.

Lai Wei, Yu-Chao Chen, Zhenbiao Cao et al. · 0 citations
#natural language process... Preprint Oct 2026

CliQ-RRG: Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation

Radiology reports are essential for diagnostic reasoning and patient care, yet their manual preparation is time-consuming and cognitively demanding. Automatic radiology report generation (RRG) offers a scalable alternative, but existing models often produce lengthy, unstructured narratives that overlook diagnostic cues...

Chandravardhan Singh Raghaw, Shahid Shafi Dar, A. Sharma et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.