EviReqRAG: An Evidence-Based Retrieval-Augmented Generation Framework for Software Requirements and Security Classification

Abstract

Requirements classification is one of the key steps in requirements engineering, as it supports early analysis, traceability, validation, prioritization, and security-conscious decision making. However, automating this classification is challenging due to the nature of software requirements, which are often short, ambiguous, domain-specific, and semantically overlapping. Although successful attempts have been made to develop supervised and transformer-based classifiers, these approaches usually require task-specific training and offer limited explainability regarding the grounds for their classification decisions. To address this problem, we introduce EviReqRAG, an evidence-based framework for software requirements classification based on retrieval-augmented generation (RAG). Instead of making classification decisions solely on the basis of a single requirement, EviReqRAG retrieves semantically and lexically similar requirements as evidence and provides them to large language models (LLMs) through structured prompts. EviReqRAG combines sparse, dense, and hybrid retrieval strategies and utilizes cross-encoder re-ranking to improve the suitability of the retrieved evidence before passing it to the generative model. The framework is applied to three requirements engineering tasks: binary classification of functional and non-functional requirements (NFRs), multi-class classification of NFRs, and security requirements classification (SRC). Experiments are performed on the PROMISE, PROMISE_exp, Promise+, and SecReq datasets using DeepSeek-v4-pro and Claude-Opus-4-8 models. The results show that EviReqRAG achieves 0.95 accuracy and 0.94 F1-score for binary classification, 0.89 macro-F1 and 0.92 balanced accuracy (BACC) for multi-class classification, and 0.96 accuracy and 0.94 F1-score for security classification.