Evaluating human-AI co-design requires testing whether coding agents can translate real users'multimodal requests into working application revisions. We introduce Co-Design Bench, a reference-free benchmark of 273 tasks curated from 37,463 deduplicated queries across 26,314 real user coding sessions. Its two tracks, Mu...
Xi-Lin Wei, Sheng-Yuan Ding, Xinyu Fang et al.· 0 citations
BlenderWorld is introduced, the first comprehensive benchmark dedicated to evaluating frontier models on realistic 3D creation tasks, and reveals a critical bottleneck: while current models can parse broad instructions, they still profoundly struggle with fine-grained 3D spatial coherence and accurate structural reprod...
Xinyu Fang, Yu-Chi Wang, Wei-Hao Yu et al.· 0 citations
Beyond basic tool use and coding skills, effectively incorporating visual references into agentic generation is equally essential for applying AI agents to real-world productivity tasks. We study this crucial capability through webpage development, a practical and readily evaluable setting in which existing benchmarks...
Yu-Chi Wang, Rongyao Fang, Wei-Hao Yu et al.· 0 citations
A 4B model that uses its own dense metric predictions as object-grounded evidence for language generation and leads the evaluated methods in instance-level metric depth estimation and overall accuracy on both relative and metric reasoning tracks, while broadly preserving general VQA performance and improving spatial un...
Jiang-Ning Wei, Yuan Yao, Miao-Miao Cui et al.· 0 citations
Experiments demonstrate strong 3D perception and driving scene understanding while largely preserving general vision-language capability and comprehensive evaluations across open-loop, pseudo-closed-loop, and closed-loop settings further show highly competitive motion-planning performance.
Xin Zhou, Zongchuang Zhao, Zhibo Yang et al.· 9 citations· ⚡2
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
