Retrieval-Augmented Fine-Tuning with Reasoning Distillation for Vietnamese Medical Question Answering
Abstract
Medical question answering (QA) plays a crucial role in clinical decision support, yet robust performance requires models to effectively distinguish relevant evidence from topically similar distractors within retrieved contexts. Existing Vietnamese medical QA benchmarks, however, focus exclusively on zero-shot evaluation, leaving this capability underexplored. In this work, we propose a retrieval-augmented fine-tuning (RAFT) pipeline for Vietnamese medical QA that explicitly teaches models to filter distractors and extract clinically relevant evidence. Our approach distills Chain-of-Thought (CoT) rationales from a large language model into a smaller model via Low-Rank Adaptation (LoRA), using oracle-and-distractor contexts constructed from VMHQA. To enable precise evaluation of reading comprehension independent of retrieval quality, we further introduce a fixed oracle-context protocol that isolates context utilization ability. Experimental results show that our fine-tuned LLaMA-3-8B achieves 93.64% accuracy and 93.56% macro-F1, outperforming the provided-context baseline by 18.76% and surpassing zeroshot LLaMA-3-70B. Evaluated under these optimal evidence conditions, our model demonstrates reasoning capabilities highly competitive with prior results on VMHQA, including fine-tuned GPT-4o (90.10%). Qualitative analysis further indicates that CoT distillation reduces context-induced hallucinations and improves robustness to distractors.