DocAttriBench (DAB) is introduced, a large-scale benchmark for fine-grained, element-level source attribution in Document VQA, grounding answers to specific layout elements such as text blocks, tables, and images.
Abstract
Answer grounding in document visual question answering remains an open challenge: most benchmarks lack grounding annotations or provide limited-quality labels, while constructing grounded datasets still requires costly manual effort. We introduce DocAttriBench (DAB), a large-scale benchmark for fine-grained, element-level source attribution in Document VQA, grounding answers to specific layout elements such as text blocks, tables, and images. To build DAB, we propose a Mask-based Perplexity-Derived Attribution method (MAPPET) that combines document layout and language modeling to identify the most informative element for each answer. MAPPET measures the increase in perplexity after masking candidate elements and attributes the answer to the element contributing most to model confidence. Applying MAPPET to multiple existing Document VQA datasets yields DAB, with 237k documents and 296k question-answer pairs with element-level grounding. We benchmark grounding-capable multimodal LLMs on DAB, evaluating answer accuracy, attribution accuracy, and overall answer quality. Results show that while larger models generally achieve higher answer accuracy, even the strongest models often fail to localize the supporting elements. DAB provides a scalable benchmark for developing grounded, verifiable, and trustworthy Document VQA models. Dataset and code are available at https://aimagelab.github.io/DocAttriBench/.
Existing long-document QA benchmarks rarely combine large-scale human-authored questions with detailed visual evidence annotations, limiting diagnosis of evidence localization and interpretation failures. We present VELD-QA, a large-scale benchmark for Visual Evidence grounding and question answering in Long Documents....
Zhivar Sourati, Marianne Menglin Liu, Rongguang Wang et al.· 0 citations
Textual graph question answering (GraphQA) and graph-based retrieval-augmented generation (GraphRAG) have gained increasing attention as a way to ground large language models (LLMs) on structured knowledge. Despite extensive efforts, existing methods still face two key gaps. The first is an architectural gap: existing...
Kai-Feng Jin, Yu-Chen Yan, Zhichen Zeng et al.· 0 citations
In grounded remote sensing visual question answering, a model produces a textual answer and identifies relevant objects in the image as grounding evidence. Existing evaluations measure answer correctness and localization quality, but do not directly assess answer-grounding consistency. We propose the Answer-Evidence Su...
Ping Guo, Zhi-Qi Huang, Xin-Ran Li et al.· 0 citations
Visual Question Answering (VQA), a multimodal AI task combining computer vision and language understanding, answers text-based questions about visual content. Although VQA has developed rapidly, most existing approaches assume a single correct answer, whereas real-world images often require several valid answers simult...
Nida Hasanati, Taufik Djatna, I. S. Sitanggang et al.· Applied Information System a...· 0 citations
Visual citations show where an answer is supported, but not how the cited content contributes to it. In scientific document question answering, a model may read a value directly, compress several observations, or combine evidence to infer a conclusion. We study typed visual provenance, where a system generates an answe...
Jing-Xuan Wei, Zhen-Yang Wang, Zheng Guo et al.· 0 citations
Vision-language models achieve remarkable results on various visual question-answering benchmarks. However, they struggle with ambiguous questions where the target entity is unclear. To address this issue, it is essential to construct a benchmark dataset that pairs ambiguous questions with entity-grounded clarifying su...
Gyu-Min Park, Seong-Bae Park· Proceedings of the 35th ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.