Skip to content
Conference Open access

ArtHopQA: Diagnosing Hidden-Chain Recovery in Artwork Question Answering

2026 · Volume 1 · 0 citations · 33 references

Abstract

Many artwork questions begin with vision but cannot be answered by visual recognition alone. A model may need to connect an artwork to its creator, depicted subject, collection, or location, and then follow multiple facts to reach the answer. We introduce A RT H OP QA, a multimodal multi-hop question answering benchmark for artwork understanding, containing 4,885 image-grounded questions paired with supporting fact chains. Experiments with open and API large vision-language models show a large gap: even the strongest open model reaches only 36.87% accuracy when given the image and title, while gold-path performance reaches 97.77% to 99.80% when the supporting fact chain is provided. Text-only language models exhibit a similar gap, showing that the difficulty is not confined to visual input. Further analysis reveals high-prior geographic guesses, inconsistent answers under image-title conflict, and limited recovery of intermediate entities even when final answers are correct.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.