Analysis of Threshold Similarity Effect on Dense Retrieval in Retrieval-Augmented Generation Systems Using the SQuAD Dataset

Abstract

Retrieval-Augmented Generation (RAG), which combines text generation and document retrieval, is one of the more useful methods for maintaining big language model outputs based on actual, verifiable sources rather than the model's own stored knowledge. However, what is initially extracted from the retrieval stage has a significant impact on how well a RAG system responds to a query. Using the Stanford Question Answering Dataset (SQuAD) as the testbed, this study especially examines how the similarity threshold employed during dense retrieval affects both retrieval quality and, later on in the pipeline, the quality of the final answer. The multi-qa-mpnet-base-dot-v1 embedding model was used to encapsulate context passages and questions, which were then compared using cosine similarity. The answer generator was LLaMA 3.1 8B, which was accessed using the Groq API. Fifty questions selected from the SQuAD validation set were used to test eleven threshold values, ranging from 0.0 to 1.0 in steps of 0.1. The quality of retrieval was evaluated by the measures Recall, Precision, and Mean Reciprocal Rank (MRR). The generated answers were assessed by means of Exact Match (EM) and F1 Measure. The baseline performance (at 0.0 threshold) had F1 at 0.827 and EM at 0.72, while for the 0.3 threshold it became F1 at 0.848 and EM at 0.78. Once past the 0.5 threshold, all metrics showed a sharp drop. Overall, the conclusions that can be drawn from this experiment are that threshold-based filtering of context significantly improves the performance of a RAG model.