Exemplar2VQA is introduced, a scalable exemplar-driven visual question answering generation framework that rapidly synthesizes large-scale spatial QA pairs in simulated environments via multi-agent coding and is established as a scalable and powerful paradigm for bridging the sim-to-real gap in Embodied AI.
Abstract
Advancing spatial intelligence in Multimodal Large Language Models (MLLMs) is bottlenecked by the scarcity of complex, scalable 3D question-answer (QA) data. While manual annotation is labor-intensive, directly utilizing LLMs to synthesize these QA pairs often fails due to their inherent deficiencies in spatial and geometric computation. We introduce Exemplar2VQA, a scalable exemplar-driven visual question answering generation framework that rapidly synthesizes large-scale spatial QA pairs in simulated environments via multi-agent coding. By equipping collaborative agents with a meticulously designed library of geometric utilities, Exemplar2VQA bypasses LLMs'spatial reasoning flaws through deterministic code execution. Crucially, the framework exhibits remarkable versatility: taking diverse static object-centric spatial query templates as exemplars, it seamlessly and autonomously scales them into massive, high-fidelity synthetic datasets. Fine-tuning Qwen2.5-VL (3B/7B) exclusively on Exemplar2VQA-generated synthetic indoor data yields significant performance improvements across various diverse benchmarks. Furthermore, its effectiveness is not limited to in-domain indoor datasets but also robustly extends to outdoor and mixed-scene benchmarks. These results establish Exemplar2VQA as a scalable and powerful paradigm for bridging the sim-to-real gap in Embodied AI. Our code is at https://github.com/yingjiayu12/Exemplar2VQA
AVA, a training-free framework that wraps a frozen RS MLLM with three modules that improves average accuracy by up to 2.40 points over the strongest same-setting baselines, and raises the visual-dependency score from 0.28 to 0.49, is presented.
Jing-Wen Huang, Yi-Ming Fang, Han-Xiao Wu et al.· 0 citations
The deployment of large language models (LLMs) in enterprise environments faces two fundamental challenges: single-search strategies are unable to effectively handle the structural heterogeneity of operational queries, and LLM generation without external grounding is prone to producing hallucinations on structured doma...
The inference efficiency of Multimodal Large Language Models (MLLMs) is severely constrained by massive visual token sequences induced by high-resolution inputs, with computational cost scaling quadratically. Existing approaches primarily focus on downstream token compression, while overlooking a fundamental upstream i...
Zhi-Qiang Xia, Yang Li, Xin-Yuan Zhang et al.· arXiv.org· 0 citations
Inference-time steering in vision-language models often entangles query-relevant reasoning dynamics with low-level perceptual noise, causing collateral degradation on correct predictions. We propose Question-Contrastive Visual Steering (QCVS) to isolate task-specific intervention directions via double-differential acti...
This work investigates whether comparable performance can be retained with a compact workflow exposing only essential perceptual capabilities, and achieves comparable or better accuracy than the specialized-tool agent while reducing inference cost by around 70%.
Yi Wang, Tian-Zi Wang, Peter Bell· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.