An empirical study of whether complete software artifacts generated by LLM coding agents can be executed in a clean environment using only the code, dependency specifications, and instructions the agent provides suggests that coding-agent evaluation should treat clean-environment executability as a first-class metric alongside functional correctness.
These findings suggest that while AI agents excel at rigorous boundary testing, they lack the “environmental awareness” needed to write stable, hermetic tests.
Preet Jhanglani, Zeel Desai, Vidhi Kansara et al.· Annual International Compute...· 0 citations
CAT is introduced, a paradigm in which the agent writes Playwright code to drive the browser, gathers feedback, and autonomously explores web applications to uncover bugs, revealing a clear gap between current VLM capabilities and the demands of real-world testing in AI web development.
Bin Hong, Zhen-Chao Zhang, Ji-Yuan He et al.· 0 citations
Vero is introduced, the first benchmark to evaluate joint implementation and proof synthesis at the repository level and an audit mechanism where agents are allowed to formally prove unsatisfiability of provided specification or incorrectness of reference code, which surfaces and corrects latent code and specification...
Zhe Ye, Hantao Lou, Yuechun Sun et al.· 0 citations
The AgentCodeReview system is presented, a multi-agent system that is able to conduct explainable code review and automated bug repair by leveraging software engineering agents with different code review tasks and its utility and extensibility to the field of explainable AI in software quality assurance are demonstrate...
B. N, T. Manasa· International journal of com...· 0 citations
This paper reviews recent empirical literature to ask what the developer's job is shifting from typing code to directing agents that type code a change often summarized as a move from code generation to code orchestration.
P. N. Nesarajan, P. Thenmozhi, S. A. et al.· International Journal of Inn...· 0 citations
It is observed that generated code often omits basic input validation or memory-safety checks, which can lead to overflows, resource exhaustion, or other reliability/security issues, and even the largest models frequently make simple mistakes.
Rodrigo Pato Nogueira, Marco Vieira, João R. Campos· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
