By providing a shared vocabulary, Act*ONOMY aims to help agent researchers, designers, and deployment engineers interpret agent behavior more effectively and consistently, supporting further study of oversight and control.
Jie Gao, Kaiser Sun, Jen-Tse Huang et al.· arXiv.org· 10 citations
Generative models increasingly expose a reasoning effort setting that lets users scale inference-time compute, trading higher cost for potentially improved performance. Users must choose an effort level before knowing whether extra reasoning will help, and even a successful run does not reveal whether the effort purcha...
Kaiser Sun, Yongchan Kwon, Zainudin Hasan et al.· 0 citations
Evaluating four agents and three model-level interventions, it is found that higher task accuracy does not necessarily correspond to greater epistemic humility: some high-accuracy configurations recognize conflicts during execution but do not communicate unresolved uncertainty in their incorrect final answers.
Kaiser Sun, Bernal Jimenez Gutierrez, Hong-Jun Liu et al.· Conference on Empirical Meth...· 1 citation
This work introduces PrimeScientist, which jointly determines research direction and resource investment across successive research attempts, and proposes an adaptive MCTS-based allocation policy that balances exploration and exploitation using experimental feedback and remaining resources.
The Last Translation Benchmark is introduced, a collection of human-authored and peer-reviewed examples that break leading machine translation models and a new evaluation approach: each example comes with handcrafted verification rules describing concrete failure cases on that example, therefore allowing reliable and a...
Vilém Zouhar, Niyati Bafna, Mukund Choudhary et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.