RAG-Enhanced ForenGemma-1B: A Systematic Evaluation of Retrieval Strategies Using DeepEval
Abstract
Digital forensics demands high reliability from AI systems due to legal and investigative consequences. Large Language Models (LLMs) show incredible performance in generative tasks, but suffer from hallucinations and knowledge staleness. This study systematically evaluates six Retrieval-Augmented Generation (RAG) strategies; Native RAG, Query Transformation, Semantic Chunking, Re-ranking, Adaptive RAG, and GraphRAG, applied to ForenGemma-1B, a custom 1B-parameter forensic LLM. Using the ForenSQA dataset (11,591 bilingual QA pairs), we assess accuracy, faithfulness, and contextual relevance. Results show that retrieval augmentation improves forensic LLM performance in distinct ways, introducing trade-offs between retrieval precision, evidence grounding, and answer relevancy, indicating that increased retrieval sophistication does not necessarily improve overall response quality in forensic LLMs. This work establishes best practices for deploying RAG in high-stakes digital forensic applications.