Numerical Hallucinations in Retrieval-Augmented Generation: Detection and Analysis
With the rise in the usage of Retrieval-Augmented Generation (RAG) systems to improve the factual accuracy of the large language models (LLM), there still exists a concern regarding these systems producing hallucinating outputs not grounded in the retrieved documents. Although prior work has studied general hallucination detection, the specific challenge of numerical fabrication remains unquantified. This research study analyzes 500 RAG outputs using GPT-3.5-turbo on MS MARCO queries to address this specific challenge and found that 38.2% of failures involve numerical fabrication. The evaluation consists of four detection methods that span different paradigms: embedding-based (Semantic Similarity), metric-based (BERTScore), LLM-based (GPT-4o-mini), and a number-aware heuristic. The results show that all standard methods struggle with numerical hallucinations, notably GPT-4o-mini achieves only 25.7% recall on numerical failures despite being a state-of-the-art LLM judge. In contrast, the simple number-aware heuristic of this research study achieves a 100% recall on numerical failures with F1=0.616, significantly outperforming all baselines (McNemar's test, p<0.001). These findings highlight numerical fabrication as a critical gap in current hallucination detection approaches and recommend the need for specialized, number-aware methods in RAG systems.