Jun 2026· Knowledge· Vol 6, pp. 13· 0 citations· 88 references
TL;DR
This review focuses specifically on Retrieval-Augmented Generation and fine-tuning, identifying them as the two dominant paradigms for integrating external knowledge: RAG for retrieval-based context injection and fine-tuning for parametric knowledge adaptation.
Abstract
Large Language Models (LLMs) have emerged as transformative tools across various domains, exhibiting remarkable capabilities in natural language processing and generation. However, their reliance on static pre-training data limits their ability to access up-to-date and domain-specific information. The existing research often treats augmentation strategies in isolation, and limited efforts have been made to systematically compare them through the lens of information integrity. This review focuses specifically on Retrieval-Augmented Generation (RAG) and fine-tuning, identifying them as the two dominant paradigms for integrating external knowledge: RAG for retrieval-based context injection and fine-tuning for parametric knowledge adaptation. While existing surveys predominantly focus on performance metrics like accuracy or latency, this paper addresses the critical gap of data fidelity—the preservation of truthfulness, integrity, and fairness during augmentation. We systematically synthesize empirical findings from diverse methodologies to determine how each approach mitigates hallucinations and bias. By comparing the trade-offs between retrieval-based context injection and parametric knowledge adaptation, this survey provides unique value to readers by providing a structured taxonomy, a unified evaluation framework, and actionable insights to guide future research and practical deployment of robust, high-fidelity LLMs.
This systematic review examines recent progress in the pretraining and adaptation of LLMs for Low-Resource Languages (LRLs) and focuses on the ethics in AI practice, the development of corpora through communities, and interdisciplinary research collaboration among computational linguists, social scientists, and digital humanists.
Ismail Hossain, Mridul Banik, Fahmid Al Farid et al.· Computer Modeling in Enginee...· 0 citations
The results demonstrate that suppressing entity-linked supervision during pretraining induces a shift in epistemic behavior: KLLMs rely less on parametric knowledge and more on external evidence, leading to improved reliability under realistic conditions, suggesting that pretraining-time control over knowledge acquisition can complement retrieval-augmented and tool-based systems by providing a more evidence-sensitive base model.
Roi Cohen, Yvan Carr'e, Nick Lechtenborger et al.· 0 citations
As the context window of Large Language Models (LLMs) continues to expand, the data required to effectively train and evaluate these capabilities remains underexplored. With existing research primarily focuses on architectural optimization, there is a need for a systematic, data-centric review. This survey bridges this gap by investigating the data foundations of Long-Context Language Models (LCMs). We begin by examining current data strategies alongside their strengths and limitations, mapping the required data to desired model capabilities. Building on this, we explore how targeted training data designs drive core, often interconnected skills such as retrieval, reasoning, and aggregation. Furthermore, we analyze the evaluation landscape, illustrating how selecting appropriate benchmarks is crucial for probing capability boundaries and guiding effective model selection. Finally, we synthesize actionable guidelines for data construction and outline critical future directions to propel the advancement of long-context language models, including quantifying data quality, establishing scaling laws for length distributions, and developing dynamic evaluation frameworks.
Zechen Sun, Yu-Yang Sun, Zhao-yu Su et al.· Transactions of the Associat...· 0 citations
Whether contemporary LLMs can reproduce the research outcomes of a fully documented human study: a 1991 article that identified dermatophytosis (ringworm) in historical fine art was evaluated.
: As small-scale, open-source Large Language Models (LLMs) proliferate for on-device and privacy-centric applications, understanding the trade-offs between their utility and behavioural reliability becomes critical. This study evaluates a suite of instruction-tuned LLMs, Gemma, Llama, and Qwen ( ≤4 B parameters), treating the model family, and scale as the primary units of analysis. Retrieval-Augmented Generation (RAG) is employed as a controlled experimental condition to assess utility gains on the Natural Questions (NQ) benchmark, while utilizing native, non-augmented configurations to establish a fairness baseline via the Bias Benchmark for QA (BBQ). The findings reveal that while RAG significantly enhances utility, often doubling Exact Match (EM) scores, these gains are non-uniform and architecture-dependent, with certain families exhibiting greater "retrieval-readiness" than others. Paradoxically, the fairness analysis shows that providing explicit context in disambiguated settings can increase stereotype engagement rather than suppressing it. These results suggest a fundamental disconnect between a model's capacity for factual accuracy and its ability to maintain social fairness, highlighting the need for multi-dimensional evaluation frameworks for small-scale systems.
M.J.F. Valdez, Arghir-Nicolae Moldovan· Proceedings of the 15th Inte...· 0 citations