Evaluating human-AI co-design requires testing whether coding agents can translate real users'multimodal requests into working application revisions. We introduce Co-Design Bench, a reference-free benchmark of 273 tasks curated from 37,463 deduplicated queries across 26,314 real user coding sessions. Its two tracks, Mu...
Xi-Lin Wei, Sheng-Yuan Ding, Xinyu Fang et al.· 0 citations
BlenderWorld is introduced, the first comprehensive benchmark dedicated to evaluating frontier models on realistic 3D creation tasks, and reveals a critical bottleneck: while current models can parse broad instructions, they still profoundly struggle with fine-grained 3D spatial coherence and accurate structural reprod...
Xinyu Fang, Yu-Chi Wang, Wei-Hao Yu et al.· 2 citations
Beyond basic tool use and coding skills, effectively incorporating visual references into agentic generation is equally essential for applying AI agents to real-world productivity tasks. We study this crucial capability through webpage development, a practical and readily evaluable setting in which existing benchmarks...
Yu-Chi Wang, Rongyao Fang, Wei-Hao Yu et al.· 0 citations
This work introduces UniME-R1, an embedder-adviser framework that learns to reason over initially retrieved candidates and generate Retrieval-Centric Chain-of-Thought (RC-CoT), and aligns the adviser with retrieval outcomes through supervised learning and retrieval-oriented reinforcement learning.
Ze-Long Sun, Jun Wang, Kaicheng Yang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.