PersonaChatBench is introduced, a framework for evaluating task competence and responsiveness to diverse user needs that provides a basis for evaluating consistency across users and guiding the development of assistants that better accommodate diverse needs in everyday applications.
Zhe-Yuan Deng, Tao-Ran Li, Jing-Wen Zhang et al.· 0 citations
PersonalityScienceBench evaluates how agents collaborate with diverse researchers alongside whether they complete the task, and across eighteen modelconfigurations, task performance, fact recovery, and self-repetition reveal different strengths.
Si-Yang Wu, Wei-Zhi Zhang, Chen Yang et al.· 0 citations
We introduce AgentPersonaBench (APB), a benchmark evaluating whether persona conditioning faithfully steers downstream agent behavior. While language models are increasingly deployed for persona-driven user simulation, existing benchmarks primarily evaluate conversational styling or self-reports rather than authentic b...
Jin-Tao Huang, Yi-Fan Wang, Hong-Yuan Shen et al.· 0 citations
MatrAIx is introduced, a population-scale simulated-user evaluation infrastructure for testing AI systems and digital products with heterogeneous users and provides an end-to-end infrastructure for evaluating AI systems and digital products with diverse simulated human users.
Xiaomin Li, Yuexing Hao, Jian Hou et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.