The first controlled longitudinal study that isolates the scaffolding’s contribution to agent quality over time is conducted, revealing that despite continuous development activity and growing codebase complexity of the scaffoldings, there is no statistically significant improvement in SWE-bench benchmark score across releases for a given fixed LLM version.
This paper empirically study the development and release evolution of five major open-source agent harnesses, revealing extreme release velocities exceeding two releases per day and thousands of issues within months, and performs the first controlled longitudinal study that isolates the agent harness contribution.
O. Sghaier, Hao Li, Bram Adams et al.· 2 citations
This work compares six tool architectures that hold the underlying information and actions similar while varying how they are organized and exposed to the model, across three actors and a total of 11,700 trajectories to show that, even when tools provide similar capabilities, tool architecture changes agent behavior.
Xiangzhe Xu, H. Saghir, Qianhui Wu et al.· 1 citation
This survey aims to clarify the conceptual boundaries of self-evolving coding agents and provide a foundation for designing more adaptive, reliable, and software-aware agentic systems.
H. Zhou, Haichuan Hu, Ye Shang et al.· 0 citations
Over the past months, the way developers direct agentic AI coding tools has moved up several levels of abstraction, from phrasing prompts to engineering context to configuring the harness around the model. In June 2026, practitioners began to describe a further level called loop engineering: Instead of prompting an agent interactively, developers design systems that prompt agents for them. These systems start agent runs on a schedule or on repository events and stop them when a machine-checkable condition holds. The term spread rapidly, accompanied by bold claims and vocal skepticism, but its adoption in software projects has not been measured. We present an exploratory review of the emerging gray literature, which largely agrees on what a well-engineered loop contains: triggered agent runs bounded by machine-checkable stop conditions, persistent state files, verifier sub-agents, token budgets, and defined points of escalation to humans. From this review, we derive a research agenda for the empirical study of loop engineering in open-source projects, analyze which of its aspects are traceable from repository data, and report an exploratory mining study of 36,710 software repositories. We confirmed the operation of autonomous agent loops in 217 of the 256 repositories our heuristics matched. The repositories commit the configuration around these loops, but almost none commits the state files the discourse prescribes, and the loops'runtime state remains outside version control. We conclude by outlining a planned controlled study of agent autonomy levels and their effect on effort and outcomes.
Jai Lal Lulla, Vahram Nersesyan, Seyedmoein Mohsenimofidi et al.· 0 citations
Together, these results recast the evolved harness as a legible compensation layer, shaped jointly by the language's engineering demands and the model's behavioral gaps, rather than an opaque benchmark-tuned scaffold.
Siqi Yang, Qianlan Yang, Yu-Xiong Wang et al.· 1 citation
Agent Gym is introduced, a modular, domain-agnostic framework that wraps any existing LLM-based agent in a continuous evaluation-and-evolution loop and introduces the Spec-to-Note Gap, an autoencoder-inspired view of agentic system transparency.
Pouya Ghiasnezhad Omran, Michael Zimmermann, Duncan Cambridge et al.· 0 citations