Veritas: Dynamic Evaluation Framework for Rag Systems' Defense Against Prompt Injection Attacks

Abstract

Prompt injection poses a significant security risk to Retrieval-Augmented Generation (RAG) systems, enabling adversaries to embed malicious instructions within retrieved documents and hijack model behavior to exfiltrate sensitive information or execute unauthorized actions. This work presents a modular dynamic evaluation environment for systematically testing and comparing defense mechanisms against prompt injection attacks in RAG architectures. The framework simulates diverse injection scenarios targeting the retrieval pipeline, integrates optional mitigation strategies such as input filtering, prompt rewriting, and retrieval-aware defenses, and automatically logs model behavior to assess attack success. By varying retrieval parameters and quantifying defense robustness across diverse attack verticals and RAG configurations, the system enables reproducible and scalable evaluation of prompt injection resilience. The results highlight strengths and weaknesses of existing defenses in RAGspecific threat models and establish a foundation for standardized benchmarking of defense mechanisms in knowledge-grounded generative AI systems.