RATIO (Retrieval Across Typed Ideation Operations), a large-scale benchmark in which relevance is defined by three operations which are name ideation moves, provides a scalable training and evaluation framework for retrieval components that support literature-grounded ideation, opening up new research avenues on scientific inspiration retrieval.
Abstract
Retrieved scientific literature can serve as inspiration for both human and AI scientists. Inspiration can take different forms: prior work may directly suggest how to address a problem, or surface directions at different levels of abstraction---zooming out to a more general view or zooming in to a concrete realization. We introduce RATIO (Retrieval Across Typed Ideation Operations), a large-scale benchmark in which relevance is defined by three operations which we name ideation moves: Address retrieves potential approaches for stated problems, Broaden retrieves more general formulations, and Specify retrieves concrete instantiations. RATIO is constructed from millions of full-text scientific papers across CS literature via a general recipe that extends discourse-marker distant supervision---previously used only for classification---to corpus-scale retrieval, combined with extensive LLM and human vetting. Experiments show that operation-specific fine-tuning substantially boosts retrievers and LLM reranking helps further, but much room for improvement remains. Researchers blind to condition found the fine-tuned retriever's candidates more useful as inspirations. RATIO provides a scalable training and evaluation framework for retrieval components that support literature-grounded ideation, opening up new research avenues in scientific inspiration retrieval. We release full data, code, models, and annotations.
This work introduces MUSES, a million-instance benchmark for prospective intellectual-roots retrieval over a fixed 2.33M-paper corpus, and releases CiteRoots, both CiteRoots layers, and a distilled open companion judge for future work on prospective retrieval and intellectual roots.
The results demonstrate that retrieval quality acts as a rigid performance ceiling for generative correctness and suggest that for RAG applications, strategic database indexing is a more critical performance lever than embedding parameter scale: architectural retrieval decisions matter more than model size.
Assmaa Moutaoukkil, A. El Mezouary, A. Idarrou et al.· EPJ Web of Conferences· 0 citations
We introduce SciRet, a compute-aware empirical study of retrieval-augmented generation for scientific question answering over CORD-19. Rather than proposing a new model, we evaluate a fixed scientific RAG pipeline across three corpus scales: 1,034 chunks (1K papers), 5,160 chunks (5K papers), and 15,480 chunks (15K pap...
Kaysarul Anas Apurba, Mahade Hasan, Rofiqul Alam Shehab et al.· 0 citations
GEM is presented, a generative embedding model that augments retrieval through its own knowledge by explicitly reasoning about user intent and relevance criteria, and its generative nature allows test-time compute scaling via prompting to further enhance retrieval performance.
An expert-validated benchmark for multi-aspect, full-paper retrieval that evaluates whether retrievers can consistently recover the same paper from queries targeting its motivation, method, and experimental findings, and proposes MAPLE-Synth, a retrieval-based in-context learning pipeline that leverages OpenReview disc...
Yiyang Wei, Fang Guo, Qiji Zhou et al.· 0 citations
This work connects papers in a graph whose edges are typed by these four facets, derived from facet items and citation signals, and distil it into a generative retriever whose identifiers are the papers'own facet text.
Italo Luis da Silva, Hanqi Yan, Yu-Jing Wang et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 19, 2026
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.