Long-horizon, semi-verifiable agentic tasks often rely on rubric-based evaluation, but the evidence needed to assess these tasks can be spread across the environment state and the execution trace. A judge may overlook this evidence or accept unsupported claims, including an agent's own reports of success. When rubric v...
Reza Sanayei, Razvan-Gabriel Dumitru, Sun-Jie Hou et al.· 0 citations
Environments are the foundation of recursive self-improvement: they provide the problems agents work on and the feedback used to evaluate progress. Yet constructing challenging research environments with reliable evaluation still depends on domain experts, limiting their scale and disciplinary coverage. We introduce RS...
Ren-Xiong Wang, Darvin Yi, Abril Herrlein et al.· 0 citations
CRAFT is introduced, a method that converts any rubric based evaluation dataset into a model specific diagnosis of weak capabilities, and yields both a sharper picture of what a model cannot do and measurably better models after finetuning on that diagnosis.
Vipul Gupta, Zihao Wang, Razvan-Gabriel Dumitru et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.