Comparative Analysis of Context-Extension and Question-Matching RAG Architectures: A Case Study on German Regulatory Documentation

Abstract

Retrieval-Augmented Generation (RAG) grounds Large Language Models (LLMs) in domain-specific knowledge, yet systematic comparisons of RAG architectures in non-English regulatory domains remain limited. We present a comparative study of context-window extension and Question-to-question Inverted Index Matching (QuIM-RAG), evaluated on the German Core energy market data register (Marktstammdatenregister). Both approaches are implemented in a fully local, privacy-compliant setup using open-source LLMs and assessed through parameter optimization, architectural comparison, and expert validation. QuIM-RAG achieves slightly higher factual correctness (F1: 0.56 vs. 0.54) in automatic evaluation. Expert evaluation shows that 65% of responses are factually correct and only 3% are incorrect. Our results further demonstrate that automated LLM-based metrics systematically underestimate performance and provide less interpretable outcomes compared to domain experts. The study contributes a systematic comparison of RAG strategies for German regulatory contexts, introduces an evaluation framework tailored to accuracy-critical domains, and outlines guidelines for privacy-compliant deployment of RAG systems in high-stakes settings.