SecProbe is introduced, a framework for adaptive evaluation that combines Item Response Theory (IRT) with on-demand synthesis of repository-scale vulnerability-repair tasks and achieves comparable agent ability estimates while requiring agents to solve up to 29.5\% fewer tasks.
Xiao-Nan Luo, Yue Huang, Ke-Han Guo et al.· 0 citations
How often models reward-hack without instructions to do so, how effective and detectable their methods are when hacking is allowed, and how they adapt when an LLM review panel returns its decision and reasons are studied.
Yue Huang, Zhangchen Xu, Yu-Chen Ma et al.· 1 citation
This paper proposes specification-grounded alignment, a new alignment paradigm that treats provider-authored model specifications as the primary alignment target rather than abstract principles or static benchmarks, and introduces SpecAlign, a framework that synthesizes alignment data directly from specification docume...
MemoHarness is introduced, an adaptive harness optimization framework that learns from its own executions and improves over the fixed harnesses it is compared against and shows selective transfer to unseen suites and base models.
Yue Huang, Wenjie Wang, Han Bao et al.· arXiv.org· 4 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.