Why Retrieval Doesn't Cure Everything: A Review of Hallucination in Retrieval-Augmented Generation

Abstract

responses depending on domain, retriever quality, and model family. This paper reviews the literature on why RAG systems continue to hallucinate even when correct evidence is available in context, organizes the reported causes into a five-part taxonomy (retrieval failure, conflicting evidence, unfaithful generation, over-reliance on parametric memory, and stale source content), and surveys the detection methods (NLI-based faithfulness scoring, LLM-as-judge, internal-state probing, evidence-graph consistency) and mitigation strategies (corrective retrieval, self-reflective generation, verification loops, faithfulness fine-tuning) proposed against each cause. We synthesize benchmark statistics across RAGTruth, RAGBench, and HalluRAG, and close with open challenges: the absence of a standardized evaluation protocol, the compounding of hallucination in multi-hop and agentic RAG, and the persistent gap between embedding-based detectors and reliable detection under real, model-generated hallucinations.