Evaluating agents by outcomes alone can obscure the capabilities that produce them. This problem is especially pronounced in evolving environments, where outcomes reflect a closed-loop interaction between agent behavior and changing external conditions. We introduce LiveMACEBench, a process-aware benchmark that uses li...
Jun Zhao, Lei-Ming Fu, Yan Wen et al.· 0 citations
EvoIn is an agent fine-tuning framework that bridges evolution and internalization, and consistently enables agents to learn stronger decision-making procedures, raising the pass rate by 10.9 points in-domain and by 9.2 points out-of-domain.
Shi-Han Dou, Shao-Hua Liu, Zhong-Hang Lu et al.· 0 citations
SciHorizon-eLab is introduced, an agentic protocol-to-task compiler that formulates scientific embodied task construction as a compilation problem and constructs \BenchName, a ready-to-use benchmark comprising 300 certified tasks across diverse laboratory operations.
Mao-Kai Qin, Chuan Qin, Qi Zhang et al.· 0 citations
ExplorationBench turns the wicked problem of evaluating scientific exploration into a concrete and tractable framework built on verifiable Alien Worlds, and finds that the strongest systems can acquire and apply unfamiliar rules, while performance varies substantially across trajectories and continued exploration can s...
Ming Zhang, Zhen Xiang, Pei-Zhong Gao et al.· 0 citations
Results show that persistent strategy experience provides effective guidance for automated fine-tuning on unseen tasks, and propose Strategy Accumulation and Guided Execution (SAGE), a two-stage framework that makes automated fine-tuning search cumulative.
Haoran Zhao, Wei Du, Dingwen Yang et al.· 0 citations
ReDraft (Reference-Driven Revision and Fine-Tuning) shows that revising the model's own rollout rather than directly imitating an expert trajectory can reconcile cold-start acquisition with prior-capability retention.
Zhi-Hao Zhang, Ming-Qi Wu, Qiaole Dong et al.· 0 citations
SPIEval is introduced, a human-curated benchmark grounded in five cognitive capabilities that exposes fundamental limitations of current LLM-based mobile assistants and motivates future research in this direction.
Junjie Ye, Zhuohui Sheng, Shao-Hua Liu et al.· 0 citations
SWE-Bench Pro Verified offers a more trustworthy benchmark for assessing software engineering agents, which combines anti-hacking safeguards that eliminate major leakage channels without disrupting normal agent functionality, with task refinement that minimally corrects inconsistencies within flawed instances.
Pujun Zheng, Zi-Xin Shang, Shufan Jiang et al.· 1 citation
The fundamental goal of agentic visual reasoning is to improve the success rate of multimodal large language models (MLLMs) on complex tasks, rather than merely equipping them with a sophisticated yet inefficient reasoning paradigm. In this work, we rethink agentic visual reasoning through two key dimensions of tool us...
Qixun Wang, Yang Shi, Le-Tian Cheng et al.· arXiv.org· 0 citations
A novel textual representation of fault trees is proposed, and a benchmark for multi-turn dialogue systems that emphasizes robust interaction in complex environments is constructed, evaluating a model's ability to assist in malfunction localization.
Yuhui Wang, Zhi-Xiong Yang, Ming Zhang et al.· arXiv.org· 0 citations
AgentCompass is introduced, an open-source, lightweight, and extensible infrastructure for evaluating LLM-based agents that organizes the evaluation process around three independent components, thereby enabling flexible configurations without requiring the reimplementation of complex execution logic.
CAFE (Coupled Agent--Feedback Evolution), a framework in which a shared-parameter model alternates between search-agent and critic roles, is introduced, suggesting that a self-improving search agent needs feedback that co-evolves with the policy it guides.
Bo-Yang Liu, Sen-Jie Jin, Pei-Xin Wang et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.