Coding agents must deliver correct software while respecting users'expertise, preferences, and working conditions. Existing benchmarks increasingly incorporate interaction, yet personalized requirement fulfillment remains underexplored. We introduce PersonaCodeBench, which turns user characteristics into verifiable obl...
Hai-Xin Wang, Jian Hou, Hong-Yuan Shen et al.· 0 citations
PersonaChatBench is introduced, a framework for evaluating task competence and responsiveness to diverse user needs that provides a basis for evaluating consistency across users and guiding the development of assistants that better accommodate diverse needs in everyday applications.
Zhe-Yuan Deng, Tao-Ran Li, Jing-Wen Zhang et al.· 0 citations
Large language models increasingly collaborate with humans and peer AI agents whose contributions may be incomplete, inconsistent, or incorrect. Effective collaboration requires calibrated trust, whereby decisions to rely on, verify, or assist a partner adapt rationally as empirical evidence accumulates. We investigate...
PersonalityScienceBench evaluates how agents collaborate with diverse researchers alongside whether they complete the task, and across eighteen modelconfigurations, task performance, fact recovery, and self-repetition reveal different strengths.
Si-Yang Wu, Wei-Zhi Zhang, Chen Yang et al.· 0 citations
Computer-use agents (CUAs) are commonly evaluated against fixed task specifications, but users may omit requirements, add constraints, or require approval before consequential actions. Completing the benchmark instruction does not guarantee serving the person who asked. We introduce PersonaCUABench, which holds 74 task...
Hao-Ming Li, De-Fu Cao, Xu-Ming Hu et al.· 2 citations
The evaluation of 20 frontier model arms demonstrates that high-fidelity user simulation is already attainable: leading models achieve up to 84.7% full-pass adherence under unprompted conditions.
Jin-Tao Huang, Yi-Fan Wang, Hong-Yuan Shen et al.· 1 citation
The two-electron oxygen reduction reaction (2e− ORR) presents a promising route for the on-site production of hydrogen peroxide (H2O2), offering a green alternative to energy-consuming anthraquinone process. However, the high selectivity toward the competing 4e− ORR over the desired 2e− pathway leads to low Faradaic ef...
Xiao Huang, Zi-Hao Zhan, Hao Niu et al.· Journal of Electrochemistry· 0 citations
Electrocatalytic water splitting using renewable electricity is widely regarded as a promising route to sustainable hydrogen production. However, its practical application is hindered by the sluggish kinetics of both the hydrogen evolution reaction (HER) and the oxygen evolution reaction (OER). A critical challenge rem...
Yu'an Li, Hui-Ying Li, Jin-Tao Huang et al.· Chemical Society Reviews· 0 citations
MatrAIx is introduced, a population-scale simulated-user evaluation infrastructure for testing AI systems and digital products with heterogeneous users and provides an end-to-end infrastructure for evaluating AI systems and digital products with diverse simulated human users.
Xiaomin Li, Yuexing Hao, Jian Hou et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.