LLM-as-a-Judge has become a standard for evaluating large language models, yet most approaches implicitly assume that a single judge prompt functions as a stable evaluation instrument. Prior work has primarily explored deliberate prompt modifications such as scoring rubrics, role descriptions, or chain-of-thought reaso...
Ilija Subasic, Brett Levert, Andrew Rabinovich et al.· 0 citations
SWE-Clar, a benchmark grounded in real, high-value client commissions to evaluate requirement clarification and code implementation in this setting, comprises 100 tasks based on real software development jobs on Upwork, each with at least US\$4,000 in total payments.
Jia-Lei Li, Ding-Xian Wang, Xiao-Yan Zhao et al.· 0 citations
This work introduces Reinforcement Learning with Decomposed Subtasks (RLDS), whose core is Subtask-Decomposed Advantage Estimation (SDAE), a replacement for the scalar GRPO advantage that splits trajectory reward into per-subtask shares on a fixed taxonomy, computes a group-relative advantage per subtask, and distribut...
Mattie Terzolo, Mikolaj Sacha, Ayan Sinha et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
