Enhancing hybrid information retrieval with hyperdimensional computing: a framework for efficient and robust knowledge-intensive NLP

Abstract

Retrieval-Augmented Generation (RAG) has emerged as a practical framework for augmenting large language models (LLMs) with external knowledge sources. However, standard RAG pipelines often remain computationally intensive and exhibit sensitivity to input noise, particularly in real-world scenarios where queries can be ambiguous, noisy, or semantically varied. This thesis proposes HD-RAG, a hybrid retrieval system that integrates Hyperdimensional Computing (HDC) as a symbolic filtering layer prior to dense reranking within the RAG pipeline, with the aim of improving retrieval robustness and computational efficiency while maintaining semantic relevance.The HD-RAG framework encodes each document as both a high-dimensional binary vector—via a learnable projection from SBERT embeddings—and a dense vector for downstream reranking. Retrieval proceeds in two stages: symbolic filtering using Hamming distance to rapidly select candidate passages, followed by dense reranking based on cosine similarity. Experiments on a 100,000-passage subset of the MS MARCO ranking dataset show that HD-RAG achieves performance comparable to strong SBERT-only baselines on clean queries, and demonstrates improved robustness under some types of input noise, particularly synonym and adversarial perturbations. For other noise types, such as character-level typos or combined noise, the hybrid approach offers only marginal gains or, in some cases, slight degradations.Fusion of symbolic and dense scores was found to provide a small but consistent improvement in retrieval effectiveness, with the optimal configuration achieving an MRR@10 of 0.9025 on the evaluated subset. However, the anticipated gains in computational efficiency were not fully realized: symbolic preprocessing introduces additional latency at higher candidate pool sizes, partially offsetting expected speed-ups. The main advantage of HD-RAG, as demonstrated by these experiments, lies in its more stable performance across a range of noisy input scenarios, rather than in absolute retrieval speed or accuracy. All code, trained models, and experimental logs are available upon request to support reproducibility and future research in robust neural retrieval systems.