Auto-research agents now run machine-learning development unattended for hours, revising data pipelines, models, and validation from their own feedback, yet on most competitions they still finish below strong human competitors. Outcome-based benchmarks record this gap but not its cause, because they grade the final sub...
Harbor Adapters are developed that port more than 80 benchmarks to evaluate arbitrary agents, and validated through rigorous code review and parity experiments, and Harbor-Index is introduced, a curated set of 82 difficult, diverse, and high-quality tasks spanning 29 benchmarks to support more reliable and comprehensiv...
Lin Shi, Hao-Wei Lin, Zi-Xuan Zhu et al.· 1 citation
This work conceptualizes multi-agent reasoning as an evolutionary pipeline of candidate generation, peer communication and terminal selection, wherein consensus without quality control can exhibit patterns of memetic drift.
Jia-Hao Ji, Si-Jie Li, Jia-Bei Cheng et al.· 1 citation
This work introduces TraceML, which pairs human and agent work on the same competitions under one version-level schema, and releases the corpus, the schema, the labelers, and the extraction pipeline at https://huggingface.co/datasets/jerryyan/TraceML.
J. Yan, Weiwei Sun, Si-Jie Li et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
