Jul 2026· 2026 4th International Conference on Sustainable Computing and Smart Systems (ICSCSS)· pp. 1389-1394· 0 citations· 16 references
Abstract
Retrieval-Augmented Generation (RAG) has established itself as a compelling strategy for grounding large language model outputs in documentary evidence. However, production deployments continue to rely almost exclusively on homogeneous text corpora, even as enterprise repositories grow increasingly heterogeneous blending technical schematics, radiological images, annotated diagrams, and unstructured prose within the same archival system. This mismatch between system design and data reality motivates the present work. We propose a Multimodal RAG framework that unifies text and image retrieval through four tightly coupled components: a dual-stream embedding engine, a learned four-class AI query router, an adaptive confidence threshold, and a session-aware context store. Text is encoded with the allmpnet-base-v2 Sentence Transformer; images are embedded in the same 768-dimensional space via a domain-adapted CLIP ViT-L/14 model fine-tuned on approximately 120,000 technical and clinical text-image pairs. Concatenating 768-d vectors yields a 1,536-d composite query that drives a single approximate nearest-neighbor (ANN) search simultaneously across both modalities. A fine-tuned DistilBERT router assigns each query to one of four retrieval pathways text-only, image-only, hybrid, or conversational at 91.3% accuracy. Experiments were run on three enterprise corpora (engineering manuals, clinical case summaries, and legal paperwork) with the results being a mean F1@5 of 0.90, an increase of 22% over a dense text-only baseline, and a reduction of 49% in the number of hallucinations. The median first-token latency is 1.34s, which meets the interactive-use target deployment-contexts. These results show that it is possible to implement modality-aware retrieval in a simple and realistic setting with real-world organizational constraints and that the results are reproducible.
Large Language Models tend to hallucinate when answering domain-specific ques tions from scientific documents without prior fine-tuning. Currently, methods such as Retrieval-Augmented Generation partially solve this problem but face different challenges: limited context knowledge, difference between sparse and dense retrieval, and retrieval noise. This paper presents an Advanced Multimodal Retrieval-Augmented Generation system that aims to solve those challenges and im prove the accuracy of information extraction. The proposed architecture introduces a multimodal ingestion pipeline that leverages an open-source Vision-Language Model (Qwen2-VL-2B-Instruct) to generate textual summaries of tables and fig ures. The retrieval phase integrates HNSW-based semantic search with GIN-based lexical search, unified through Reciprocal Rank Fusion and refined using Cross Encoder reranking to minimize retrieval noise. To ensure conversational coherence across multi-turn interactions, a Query Condenser module is employed. Evaluation is conducted by independently assessing the ingestion, retrieval and generation stages using the MMLongBench benchmark, a BeIR-format synthetic dataset and the DeepEval framework. Moreover, results demonstrate a 157% improvement in retrieval quality over a Naive-RAG baseline, with only 50 ms additional la tency, while Qwen2-VL-2B-Instruct achieved results comparable to cloud-based models in BERTScore. These findings validate that open-source optimized SLMs, paired with advanced retrieval strategies, can provide competitive performance for document understanding without relying on cloud-based models.
Due to the surge in digital document creation, intelligent systems that can effectively retrieve accurate information from unstructured PDF content have become a necessity. Current keyword-based search techniques lack semantic meaning, while large language models (LLMs), when utilized individually, produce hallucinated results when they are not contextually aware of document content. In this paper, we introduce a novel real-time question answering system for PDF content using a lightweight approach to Retrieval-Augmented Generation (RAG). Our system combines a HuggingFace sentence transformer model for semantic embeddings with a FAISS vector similarity search for efficient retrieval, leveraging a Groq-hosted LLaMA 3.3 70B model for context-grounded answer generation. The system is built using LangChain for orchestration and Streamlit for web-based deployment, and requires no GPU, no fine-tuning, and no pre-indexed corpus. Evaluation across five complementary metrics—Exact Match, Token-level F1, BLEU, ROUGE, and semantic similarity—demonstrates strong alignment between generated and reference answers, with a peak semantic similarity of 0.8709 and a mean Token F1 of 0.4701, confirming the viability of this approach for real-world document intelligence.
Mukesh Lakshmi Sai Medikonda, Kalva Vishnu Teja, K. Greeshma et al.· International Conference Com...· 0 citations
Modern Multimodal Retrieval-Augmented Generation (M-RAG) systems are fundamentally limited by the binary connectivity paradigm of traditional simple graphs, which fails to capture the intricate, high-order correlations among heterogeneous entities, such as the N-ary relationships between a visual chart, its scattered textual descriptions, and underlying numerical data. Furthermore, existing refinement strategies often rely on exhaustive, full-page reconstruction to align cross-modal information, leading to prohibitive computational redundancy and the introduction of contextual noise in long-form document processing. In this paper, we propose Hyper-M2RAG, a novel framework that redefines multimodal document retrieval through High-order Hypergraph Representation Learning. We first formalize the document structure as a Multimodal Hypergraph, utilizing hyperedges as unified semantic containers to encapsulate multi-way associations across text, images, and tables, thereby transcending point-to-point modeling. To mitigate semantic fragmentation caused by physical pagination, we introduce an Anchor-driven Incremental Refinement mechanism. Rather than performing a global sweep, our approach identifies boundary-crossing anchor nodes and reconstructs their local hyper-topology using one-hop neighborhood contexts. This targeted refinement effectively bridges cross-page knowledge gaps with minimal computational footprints. Extensive evaluations on multimodal benchmarking datasets demonstrate that Hyper-M2RAG significantly outperforms state-of-the-art methods in both retrieval precision and generation coherence. Our code is available at https://github.com/ShenAoChen2001/MMHRAG.
Shenao Chen, Yidan Xu, Xiangmin Han et al.· 0 citations
Experimental results show that SAC-RAG reduces token consumption by 38%–58% at the cost of only a 1–2 percentage point EM drop, with EM actually improving after compression for reasoning-type questions, achieving the optimal quality–efficiency trade-off in terms of token consumption.
Deyu Zhang, Hongqiang Yu, Jinze Huo et al.· IEEE Access· 0 citations
The Document Question Answering (DocQA) task necessitates the synergistic interpretation of visual and textual information embedded within documents. Although Retrieval-Augmented Generation (RAG) has enhanced the capabilities of Large Vision-Language Models (LVLMs), existing approaches still encounter significant bottlenecks when processing large-scale documents: the inability to capture long-range contextual dependencies within non-textual modalities, the difficulty in facilitating interaction and mutual complementation between different modalities, and the inefficient integration of heterogeneous modal information. To address these challenges, we introduce MMHRAG, a novel Multi-Modal Hierarchical Retrieval-Augmented Generation framework that achieves cross-modal interaction on DocQA for the first time. First, we construct a cross-modal hierarchical retrieval tree via a bottom-up recursive clustering and summarization mechanism. A key innovation of our approach is the structural injection of visual information, where image semantics are integrated as high-level abstract summaries of textual segments, thereby bridging the semantic gap between modalities. Furthermore, we design a Summarizing Agent to resolve logical conflicts, information redundancy, and granularity discrepancies among retrieved cross-modal evidence. Extensive experiments across multiple multi-modal long-document benchmarks demonstrate that MMHRAG significantly outperforms state-of-the-art baselines, achieving superior accuracy and consistency in complex reasoning tasks.
Jiayuan Wang, Jie Lian, Fu Zhao et al.· Proceedings of the 32nd ACM...· 0 citations
Document digitization and knowledge extraction remain challenging when dealing with heterogeneous PDF repositories comprising both machine-readable text and degraded, scanned visual artifacts. Traditional Optical Character Recognition (OCR) systems enforce rigid linear pipelines, while conventional Retrieval-Augmented Generation (RAG) models suffer from hallucination when context is sparse or noisy. In this paper, we propose DocuMind-AI, an end-to-end autonomous agentic architecture for document processing and intelligent question answering orchestrated via LangGraph. The framework introduces a dynamic routing agent that assesses character density metrics to intelligently dispatch inputs between fast native text extractors and multimodal vision large language models. Extracted text is normalized into structured Markdown through an automated cleansing agent and ingested into a dual-engine hybrid retrieval index combining BM25 lexical search with dense vector embeddings via Reciprocal Rank Fusion (RRF). Furthermore, a Corrective RAG (CRAG) self-reflection loop audits retrieved document chunks for semantic relevance, triggering automated query reformulation when retrieval confidence is low, and performs secondary hallucination auditing on synthesized answers. Empirical benchmarks demonstrate that our adaptive routing reduces multimodal API overhead by 68.4% on mixed corpora while achieving a 94.2% answer grounding accuracy, outperforming conventional single-engine RAG pipelines in both precision and computational efficiency.
Keywords— Agentic AI; Optical Character Recognition; Corrective RAG; LangGraph; Multimodal LLM; Hybrid Search.
Yashas .b.s· International Journal of Cre...· 0 citations