This work presents a method to generate multilingual reasoning traces based on medical knowledge extracted from Wikipedia, and produces 500k traces in English, Italian, and Spanish, using a retrieval-augmented generation approach over medical information from Wikipedia.
Abstract
Large Language Models (LLMs) with reasoning capabilities have recently demonstrated strong potential in medical Question Answering (QA). Existing approaches are largely English-focused and primarily rely on distillation from general-purpose LLMs, raising concerns about the reliability of their medical knowledge. In this work, we present a method to generate multilingual reasoning traces based on medical knowledge extracted from Wikipedia. We produce 500k traces in English, Italian, and Spanish, using a retrieval-augmented generation approach over medical information from Wikipedia. The traces are generated to solve medical questions drawn from MedQA and MedMCQA, which we extend to Italian and Spanish. We test our pipeline in both in-domain and out-of-domain settings across Medical QA benchmarks, and demonstrate that our reasoning traces improve performance both when utilized via in-context learning (few-shot) and supervised fine-tuning, yielding state-of-the-art results among 8B-parameter LLMs. We believe that these resources can support the development of more transparent clinical decision-support tools in multilingual settings. We release the full suite of resources: reasoning traces, translated QA datasets, Medical-Wikipedia, and fine-tuned models.
This review explores the transformative impact of Large Language Models on medical question-answering robots and addresses critical challenges faced by LLM-driven systems, such as limited interpretability, restricted applicability across medical domains, and serious concerns about patient privacy.
A comprehensive survey of various benchmark datasets used in medical LLM tasks, focusing on various aspects of medical knowledge such as electronic health records, doctor–patient dialogues, medical question answering, and medical image captioning, is presented.
L. K. Yan, Qian Niu, Ming Li et al.· Medicine Advances· 34 citations· ⚡1
Medical question answering (QA) plays a crucial role in clinical decision support, yet robust performance requires models to effectively distinguish relevant evidence from topically similar distractors within retrieved contexts. Existing Vietnamese medical QA benchmarks, however, focus exclusively on zero-shot evaluati...
Nhan Phuoc Thanh Tran, P. Huynh, Trương Quốc Tuấn Trương et al.· International Conference on...· 0 citations
This work presents MedProb, a lightweight probing framework that predicts multiple-choice Med-VQA answers from frozen VLM representations without free-text generation, and shows the probe can be extended to open-ended generation via a rejection-sampling scoring procedure.
A decision-support QA framework that integrates structured retrieval over a diabetes knowledge graph with retrieval-augmented generation (RAG) and large language model (LLM)-based answer synthesis and achieves disease entity recognition, drug entity recognition, and intent classification accuracy.
This work proposes Knowledge Graph for Question Generation (KG4QG), a novel framework that integrates knowledge graphs with large language models to address the challenges of MHQG, and demonstrates the effectiveness of combining structured knowledge and pre-trained language models for complex question generation tasks.
Yllias Chali, Al Hasib Mahamud· International Conference on...· 0 citations