GEM is presented, a generative embedding model that augments retrieval through its own knowledge by explicitly reasoning about user intent and relevance criteria, and its generative nature allows test-time compute scaling via prompting to further enhance retrieval performance.
Abstract
Modern LLMs excel at reasoning and instruction following, enabling users to express complex and diverse information needs. However, conventional retrievers largely rely on surface-level matching between queries and documents, resulting in a growing gap between how users express their needs and how retrievers interpret them. In this paper, we present GEM, a generative embedding model that augments retrieval through its own knowledge by explicitly reasoning about user intent and relevance criteria. GEM unifies generation and embedding within a single model: it first reasons over the query, then appends an embedding token to encode the enriched context for retrieval. Evaluated on reasoning-intensive and instruction-following retrieval tasks, GEM demonstrates the effectiveness of its reasoning-augmented retrieval, outperforming its non-reasoning variant and matching baselines using substantially larger models. Furthermore, GEM's generative nature allows test-time compute scaling via prompting to further enhance retrieval performance. Our code is available at: https://anonymous.4open.science/r/GEM.
Retrieval-augmented generation (RAG) has attracted significant attention for enhancing large language models (LLMs) in domain-specific and knowledge-intensive tasks by utilizing external documents retrieved by retrievers. However, LLMs often struggle to determine which retrieved documents are relevant and how they relate to one another. We argue that this difficulty arises from a semantic gap between retrievers and LLMs due to differences in their training objectives and architectures. Existing methods either align retrievers and LLMs through costly fine-tuning or feedback signals, but they still do not explicitly strengthen document relationship modeling during generation. This paper proposes ConRAG, a novel enhanced RAG framework to establish an information connection between retrievers and LLMs in RAG, thereby enhancing relationship modeling of LLMs. Specifically, ConRAG employs a lightweight Con-Former model, placed between a retriever and an LLM to capture and transmit semantic information. Then, a semantics injection strategy is employed to integrate the semantic information into the LLM's generation. Accordingly, we employ three tasks for feature modeling and alignment: two relationship modeling tasks to consolidate the local and global perceptions of document relevance and one generative alignment task to facilitate the interpretation of LLM. Notably, ConRAG is suitable for low-resource scenarios where LLMs and retrievers are frozen. Further analysis shows that retriever-derived information helps the LLM better identify relevant evidence and model relationships among documents, leading to more effective generation. The source code is available at https://github.com/yefd/ConRAG.
Fu-Da Ye, Shuang-Yin Li, Yong-Qi Zhang et al.· ACM Transactions on Knowledg...· 0 citations
Universal multimodal embedding (UME) learns unified representations across modalities, enabling a single model to support diverse retrieval tasks. Recent methods use Chain-of-Thought (CoT) reasoning to better interpret multimodal inputs before generating embeddings for complex retrieval tasks and further optimize this reasoning process through GRPO with retrieval-based rewards. However, two limitations hinder corpus-scale deployment. GRPO assigns all CoT tokens the same advantage, without identifying input-supported claims or evidence that distinguishes the positive from negatives. Moreover, generating a complete CoT before each embedding introduces substantial latency, even when a partial trace already provides sufficient retrieval evidence. To address these limitations, we propose Reason What Matters (ReWAM), a retrieval-grounded reasoning framework that uses retrieval feedback to guide both credit assignment and reasoning computation. Specifically, we introduce Retrieval-aware Self-Distillation (RASD), which constructs privileged guidance from input-supported evidence that distinguishes the positive item from retrieved hard negatives. An on-policy self-teacher uses this guidance to refine trajectory-level feedback into token-specific supervision for retrieval-relevant reasoning. We further develop Retrieval-adaptive Inference (RAI), which uses a retrieval confidence head to estimate the remaining retrieval utility of a partial CoT. It stops unproductive traces early and accelerates useful continuations with speculative decoding. Extensive experiments on MMEB-V2 and MRMR demonstrate that ReWAM achieves state-of-the-art retrieval performance while delivering up to 5x the inference throughput of competitive explicit-CoT UME methods. These results bridge the gap between retrieval quality and inference efficiency, making reasoning-enhanced UME practical for large-scale deployment.
Mingzhou Jiang, Pei-Xi Wu, Hang Cheng et al.· 0 citations
Traditional search engine returns ranked lists for humans to interpret. Retrieval Augmented Generation pipelines go further, feeding retrieved context directly into large language models to synthesize knowledge rather than simply surface it. This study addresses a focused question: When the generative layer is held constant, how do distinct retrieval architectures perform across diverse query types? We fix the LLM and chunking strategy while independently varying the retrieval backend and embedding model across three configurations: Memory Resident Specialist (MRS), Industrial Semantic Index (ISI), and Agile Hybrid Scaler (AHS) evaluated over a six-tier query taxonomy. Our results demonstrate that retrieval quality acts as a rigid performance ceiling for generative correctness. Notably, the "Efficiency Paradox" emerged: the AHS configuration, leveraging lightweight MiniLM embeddings with Pinecone cloud indexing, achieved parity with proprietary high-dimensional configurations on complex logical reasoning tasks. However, all three architectures share a critical weakness for conversational queries. Multi-turn contexts revealed a systemic failure stemming from the absence of anaphora resolution in naive RAG pipelines. These findings suggest that for RAG applications, strategic database indexing is a more critical performance lever than embedding parameter scale: architectural retrieval decisions matter more than model size.
Assmaa Moutaoukkil, Ali El Mezouary, A. Idarrou et al.· EPJ Web of Conferences· 0 citations
Document question answering over scanned pages requires two coupled abilities: (i) canonicalizing complex layouts into a faithful textual structure, and (ii) selecting and reasoning over query-relevant evidence from that structure. Most existing pipelines decouple OCR from retrieval-augmented reasoning and optimize OCR for global reconstruction, which often misaligns with evidence needs and causes brittle grounding in multi-page settings. We propose UniDocVLM, an end-to-end framework that unifies OCR and visual RAG within a single vision-language model: the model first generates a structured parse of retrieved pages, then activates question-relevant evidence from the parse to support grounded reasoning and answering. To train UniDocVLM under heterogeneous supervision, we introduce a unified JR-GRPO reinforcement learning recipe with lightweight, verifiable rewards, including format, layout-aware OCR, evidence-consistency, and answer-correctness signals, and route them to the corresponding parts of the output to improve credit assignment and reduce interference. Experiments on multi-page document QA benchmarks show that UniDocVLM yields more reliable evidence grounding and improves downstream accuracy under complex layouts.
Zong-Sheng Cao, Anran Liu, Jun Xie et al.· Proceedings of the 32nd ACM...· 0 citations
This work proposes SHIFT, a retrieval training framework based on LLMs that transfers LLMs into reasoning-efficient retrievers with residual projection and task-oriented bidirectional attention aggregation in the latent space, and alleviates the mismatch between contrastive learning and implicit reasoning using fine-grained next-token-prediction-based reconstruction.
Yuxiao Luo, Da Li, Mingjie Zhang et al.· arXiv.org· 0 citations
Although multi-turn inference remains more expensive than single-call retrieval, VecTree-RAG provides a structure-aware and traceable architecture for scientific literature question answering.
