Skip to content
Review

When Is Additional Retrieval Worthwhile? A Cost-Aware Study of Evidence Gain and Answer Reliability in RAG

Oct 2026 · 0 citations · 11 references
Computer Science

TL;DR

A cost-aware, two-action retrieval policy that chooses between a dense-retrieval-plus-reranking baseline and a hybrid lexical--dense retriever is studied, exposing an important distinction between retrieving relevant evidence efficiently and improving the reliability of the generated answer.

Abstract

Retrieval-augmented generation (RAG) can benefit from more sophisticated retrieval, but extra retrieval may be unnecessary for many questions and gains in evidence retrieval may not translate into faithful or correct answers. We study a cost-aware, two-action retrieval policy that chooses between a dense-retrieval-plus-reranking baseline and a hybrid lexical--dense retriever. Rather than classifying baseline retrieval as simply successful or unsuccessful, we estimate the question-specific incremental evidence-retrieval benefit of hybrid retrieval using features available before the routing decision. A regularized linear model and decision threshold are selected using document-grouped cross-validation and frozen prior to held-out evaluation. On 60 previously unseen QASPER training-split articles (188 questions, 169 evidence-evaluable), the policy triggers hybrid retrieval on 46 questions (24.5\%) and obtains the same evidence Hit@10 as always-hybrid retrieval (78.70\%, versus 76.92\% for always-keep). Paired, paper-cluster bootstrap intervals do not establish a reliable utility advantage. We further compare generated answers from a 4-bit Qwen3-4B model under the three retrieval policies. Provisional answer-level assessments show no clear correctness or faithfulness improvement after partial review, despite the evidence-retrieval result. The generation assessments remain model-assisted and lack independent human validation. These results expose an important distinction between retrieving relevant evidence efficiently and improving the reliability of the generated answer. The work is presented as a reproducible empirical study, not a claim of statistically established answer-quality superiority.

View source

Similar papers

#natural language process... Preprint Oct 2026

Finding the Right Balance: Relevance and Diversity in LLM Retrieval

It is concluded that diversification should be used selectively, based on observable redundancy and evidence requirements, and introduces a query-adaptive rule that diversifies only when the effective number of distinct documents in the nearest-neighbor top-$k$ selection falls below the query's evidence requirement.

Guillaume Brouillette, Faustin Kagabo, Usef Faghihi et al. · 0 citations

STAR: Structure-Aware Adaptive Retrieval for RAG

STAR is presented, a structure-aware adaptive retrieval framework for RAG that treats this mismatch as a problem of diagnosing evidence sufficiency and benefits from a control signal that preserves structurally distinct insufficiency patterns rather than collapsing them into a single scalar confidence estimate.

Yeowon Jeon, Chong-kwon Kim, Y. Choi · 0 citations
Preprint Oct 2026

Scan or Index? large Evidence Coverage, Answer Quality, and Cost in Long-Document RAG

Scanning every passage in a long document seems thorough, but does it preserve more useful evidence or produce better answers? We compare 11 retrieval configurations spanning lexical and dense search, hybrid reranking, and full-document scanning with Jev, OpenAI Decisions, and Cloudflare Clef. Our evaluation uses 396 Q...

Juan R. Terven, Diana-Margarita Córdova-Esparza, J. Romero-González et al. · 1 citation
#artificial intelligence Conference Open access Oct 2026

Joint Constraint Aggregation for Qualifier-Sensitive Retrieval-Augmented QA

Retrieval-augmented QA can fail even when the correct answer is in the retrieved evidence. This can happen when a competing answer satisfies some, but not all, of the question's requirements and is mentioned frequently in the retrieved documents, which can make it appear to be the best-supported answer. To test whether...

Jennifer Hsia, Zachary Chase Lipton, Aarti Singh · 0 citations
#natural language process... Preprint Oct 2026

Beyond Retrieval Failure: Understanding Why LLMs Miss Evidence in Multi-Hop RAG

Multi-hop retrieval-augmented generation (RAG) requires an agent to retrieve evidence for a sequence of dependent sub-questions. When a wrong answer lacks a required passage, the passage may be inaccessible to the retriever, or the agent may have failed to issue a query that would retrieve it. Standard retrieval evalua...

Jia-Qi Wei, Wen-Xuan Huang, Cheng-Dong Xu et al. · 0 citations
Preprint Nov 2026

Probabilistic Relevance Decisions for Semantic Retrieval

Dense similarity supplies reusable corpus representations, while probabilistic relevance decisions offer query-dependent document scoring. We compare dense-only BGE-M3 with TypeSafe Jev Noul and OpenAI Decisions Predicate scores on the official BEIR SciFact test benchmark. The study evaluates 300 official test queries...

Bahadır Tatar, Cevat Şener Recruspace Llc, Middle East Technical University · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.