Beyond text: quantifying the impact of figure-guided retrieval in biomedical RAG systems

Abstract

Retrieval-Augmented Generation (RAG) has been established as a reliable paradigm for grounding large language model (LLM) outputs in factual evidence, acting as a safeguard in high-stakes domains such as biomedicine, where hallucination carries significant consequences. However, existing biomedical RAG systems rely predominantly on text-only corpora, overlooking the rich semantic information encoded in figures, charts, and other visual evidence. In this work, we investigate whether incorporating figure-based retrieval methods into a hybrid RAG pipeline meaningfully improves generation quality and retrieval effectiveness in the biomedical domain. We propose Figure-First Hybrid RAG (FFHRAG), a multi-stage retrieval architecture integrating multi-channel figure retrieval with hybrid text retrieval. Our architecture employs a figure-first discovery mechanism that retrieves relevant visual evidence, and utilizes it as a guiding signal for subsequent text selection. This strategy eliminates the need for complex embedding alignment, offering a low-overhead system that avoids cross-modal embedding noise at the final text chunk selection stage. The system is evaluated zero-shot on summary-type questions derived from BioASQ-13b, against a corpus spanning over 1.1 million PubMed Central text chunks and 82,000 figure-captions. A progressive, multi-seed six-variant ablation study demonstrates that while the text-only baseline components establish a strong retrieval foundation—BM25 retriever yielding a Recall@10 of 0.7322 ± 0.0154 and the Cross-Encoder reranking securing an MRR of 0.8331 ± 0.0018, our proposed framework introduces isolated, targeted enhancements. Specifically, the localized PMCID-level boosting module refines retrieval ordering to an MRR of 0.8358 ± 0.0018, while prompt-level figure caption injection substantially enriches downstream generation quality, driving ROUGE-1 from 0.4068 ± 0.0037 to 0.4137 ± 0.0036 and ROUGE-L from 0.2623 ± 0.0018 to 0.2678 ± 0.0016 over the strong text-only baseline. Our findings suggest that figures represent an underutilized retrieval modality in biomedical RAG systems, and that even lightweight figure-aware boosting mechanisms can produce measurable gains in answer quality without requiring multimodal fusion at the embedding level.