It is found that generated harnesses remain substantially behind mature human-engineered references on code and on search and research, while matching or exceeding the selected references on writing and machine-learning experimentation, with large variation in execution cost.
Yu-Hao Wu, Jingyuan Zhang, Jia-Jun Shi et al.· 4 citations
Results show that looping can improve Transformers even under budget matching, offering a practical recipe that turns depth reuse into measurable gains.
Shaowen Wang, Ge Zhang, Kai-Rong Luo et al.· 4 citations
This work introduces ASPIRE, a benchmark for vague-goal-driven self-evolution and shows that vague goals redirect search effort toward goal interpretation, and evaluates the resulting systems on a hidden, expert-authored set of 520 items spanning six goals.
Yu-Hao Wu, Jingyuan Zhang, Jia-Jun Shi et al.· 0 citations
These findings show that recognizing successful actions is insufficient; agents must also transform feedback into executable and transferable policies, and provide a unified framework for diagnosing this process and identifying the bottlenecks that prevent agents from translating interaction experience into reliable se...
Jia-Jun Shi, Siyang Tao, Yu-Hao Wu et al.· 1 citation
Together, the perplexity analysis indicates improved continuation predictability, while the controlled pre-training experiments suggest that this augmentation can improve model performance without changing the standard pre-training objective.
Haoran Que, Jia-Jun Shi, Ting Huang et al.· 0 citations
This work systematically study AI products with demonstrated adoption, together with their product workflows and users, to identify real-world tasks for which AI has established practical demand across diverse professional domains and establishes StartupBench as an empirical measure of progress toward E2E completions o...
Li-Ya Zhu, Xin Ma, Tao Liu et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.