DualG-MRAG is proposed, a Dual-tier framework that introduces a decoupled architecture comprising Macro-reasoning and Micro-matching Graphs for Multimodal RAG to suppress retrieval noise by isolating global structural reasoning from fine-grained evidence matching, and introduces a dynamic programming decoding mechanism that extracts explicit reasoning paths directly from the GNN's forward pass.
Abstract
While Multimodal Retrieval-Augmented Generation (MM-RAG) has shown promising results, it still struggles with complex multi-hop reasoning tasks. Existing methods primarily focus on independent instance-level matching, which often fails to capture explicit relationships across modalities and documents. Although Graph-enhanced methods introduce structural modeling, they face a fundamental challenge in multimodal scenarios: incorporating fine-grained visual features leads to rapid graph expansion and retrieval noise, whereas coarse-grained representations cause the discarding of critical local evidence. To address this dilemma, we propose DualG-MRAG, a Dual-tier framework that introduces a decoupled architecture comprising Macro-reasoning and Micro-matching Graphs for Multimodal RAG. Specifically, to suppress retrieval noise by isolating global structural reasoning from fine-grained evidence matching, we construct a Macro Graph for global topological routing and a Micro Graph for precise local verification. Subsequently, to enable dynamic relevance propagation across heterogeneous evidence sources, we formulate retrieval as a query-driven message passing process via a GNN Retriever. Furthermore, to provide the generative model with coherent structural guidance, we introduce a dynamic programming decoding mechanism that extracts explicit reasoning paths directly from the GNN's forward pass, replacing the standard input of isolated document chunks. Extensive experiments demonstrate that DualG-MRAG outperforms baselines in both evidence recall and complex QA accuracy.
CrossModalQA is introduced, an open-domain benchmark for evaluating multimodal retrieval and reasoning over heterogeneous corpora and it is revealed that complete cross-modal retrieval contributes more to answer accuracy than generator scaling, while multi-image retrieval and reasoning remain the primary bottlenecks li...
Jia-Cheng Cai, Zi-Jin Hong, Zheng Yuan et al.· 0 citations
This paper formalizes the document structure as a Multimodal Hypergraph, utilizing hyperedges as unified semantic containers to encapsulate multi-way associations across text, images, and tables, thereby transcending point-to-point modeling and introducing an Anchor-driven Incremental Refinement mechanism.
Shenao Chen, Yidan Xu, Xiangmin Han et al.· 0 citations
Retrieval-Augmented Generation (RAG) has become a fundamental paradigm for enhancing Large Language Models (LLMs) with external knowledge. However, while recent structure-augmented approaches organize documents into graphs to improve information access, their retrieval strategies remain largely static, relying on simil...
Jinghong Lei, Wang Kun, Zhigang Chen et al.· Proceedings of the 32nd ACM...· 0 citations
Retrieval-Augmented Generation (RAG) has established itself as a compelling strategy for grounding large language model outputs in documentary evidence. However, production deployments continue to rely almost exclusively on homogeneous text corpora, even as enterprise repositories grow increasingly heterogeneous blendi...
E.Vijayakumar, G. A.· 2026 4th International Confe...· 0 citations
MCoRe, a multi-entry complementary retrieval framework with reflection-guided iteration for multi-hop QA that enables multi-entry complementary retrieval by indexing entry units at multiple semantic resolutions with explicit links to chunk evidence, and fusing cross-resolution hits via chunk-level voting to form a comp...
Ju-Xiang Zeng, Zhuohui Gao, Zhe Hou et al.· Proceedings of the 32nd ACM...· 0 citations
This study proposes STaR, a novel retriever fine-tuning framework that integrates BM25 similarity graph-based soft labeling with a triplet similarity learning strategy based on Sentence-BERT (SBERT), and introduces a triplet-aware SBERT training architecture that explicitly models relative semantic distances between qu...
Jiali Jiang, Chih-Yung Chang, Youxi Li et al.· Multimedia Systems· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.