A comprehensive overview of the existing tools and frameworks for implementing MAS in software engineering and a set of lessons learned and challenges that can help researchers and practitioners to select a suitable MAS framework according to their needs are provided.
Abstract
The proliferation of Generative Artificial Intelligence (Gen AI) powered by large language models (LLMs) has transformed the software development process, introducing new paradigms for code generation, debugging, testing, and maintenance. While early applications focused on leveraging single, independent LLMs to assist developers with isolated tasks, recent advances have shifted toward multi-agent systems (MAS) that orchestrate multiple LLM-based agents working collaboratively toward common objectives. Despite their promising potential, using MAS encompasses a set of challenges for developers who have to carefully select the right technology, devise proper coordination rules, and design specific roles for the involved agents. In this paper, we provide a comprehensive overview of the existing tools and frameworks for implementing MAS in software engineering. First, we conducted a quantitative analysis of the most relevant open source MAS frameworks by evaluating their documentation, features, and capabilities from the developers'perspective. Second, we performed a qualitative evaluation of a subset of the selected frameworks by implementing a common use case: the summarization of README.MD files. The findings show that the selected frameworks provide a good coverage of fundamental components of MAS, though advanced features such as telemetry of agents are still missing. In addition, the empirical evaluation shows that there is no significant difference in terms of ROUGE scores considering the summarization task. Finally, we provide a set of lessons learned and challenges that can help researchers and practitioners to select a suitable MAS framework according to their needs.
TestAgent, a multi-agent tool implemented as a VS Code extension that automates the generation of high-quality unit tests for Java projects using repository-level Code Knowledge Graphs, demonstrates its practical utility for regression testing and bug discovery.
Ye Shang, Quanjun Zhang, Zheng Zhan et al.· SIGSOFT FSE Companion· 0 citations
Large Language Model (LLM)–based agents are rapidly evolving from passive assistants into autonomous, tool-using, and collaborative systems capable of executing complex, long-horizon tasks across web, software, and physical environments. However, the current literature remains fragmented, with inconsistent terminology, ad hoc architectures, and limited evaluation standards, making it difficult to compare systems or deploy them reliably in real-world settings. This paper presents a unified, taxonomy-driven, and deployment-oriented survey of agentic AI systems, synthesizing recent advances through a modular reference architecture and a four-dimensional taxonomy that characterizes agents along the axes of autonomy, tool use, collaboration, and safety–governance. We systematically analyze representative single-agent, tool-augmented, and multi-agent frameworks within this taxonomy, highlighting design trade-offs, capability scaling patterns, and recurring failure modes. Beyond architectural analysis, we review emerging evaluation methodologies that move beyond static benchmarks to assess agent behavior, robustness, grounding, and operational cost in interactive environments. Importantly, the survey emphasizes practical considerations for enterprise and safety-critical deployment, including access control, human-in-the-loop oversight, and policy enforcement. By unifying conceptual foundations with empirical trends and deployment constraints, this work provides a structured roadmap for researchers and practitioners to design, evaluate, and govern next-generation LLM-based agentic systems.
Sparsh Bajoria, Shreyanshu Ranjan, Adhitya M et al.· Cognitive Computation· 0 citations
Automating software development is an important aspect of software engineering. Generative Artificial Intelligence (GenAI) Agents that control multiple tools and use feedback with some autonomy to iteratively solve problems are emerging as promising tools for increased software production. To achieve autonomous software maintenance with minimal human involvement, such agents must not only solve human-posed problems but also decide which problems to address. We report our experiences developing a GenAI agent that autonomously formulates, prioritizes, and completes a limited set of code improvement tasks enterprise-wide, and we evaluate this agent from two perspectives: first, developing a methodology to estimate effort savings from such autonomous agents by comparing time spent on agent-introduced tasks and agent maintenance efforts against manual execution; and second, comparing the impact of manual versus agent-driven code improvements on future codebase maintenance. Our results demonstrate significant effort savings, even after accounting for effort in agent development, and a Difference-in-Differences (DiD) analysis found that automated improvements had an effect on the reduction of future maintenance effort that was as strong as or stronger than manual improvements confirming the feasibility of autonomous maintenance. While the relaxed completion and timeliness needs of perfective maintenance make it an ideal candidate for autonomy, a degree of autonomy may be achievable in other areas of software development as well.
Large Language Models (LLMs) have revolutionized software development, from analyzing code and generating suggestions to detecting bugs and errors, and even creating entire programs. Despite these advances, existing AI-driven code review solutions still provide a one-size-fits-all approach to code review with overall feedback and suggestions, often of a non-specific nature. This restriction promotes modular architectures which would be able to provide specific and direct code quality reports. This paper presents the AgentCodeReview system, a multi-agent system that is able to conduct explainable code review and automated bug repair by leveraging software engineering agents with different code review tasks. There would be five independent entities, each one to be able to review code, analyze security, evaluate performance, document it and be able to automatically fix bugs. They run parallelly under the guidance of a centralized orchestration layer that collects the results from the analytical agents, calculates software quality scores and creates comprehensive HTML and PDF reports. Moreover, a Streamlit-based web interface was created that allows the interactive visualization of the results of the analysis and interactive entry of the input values. A set of twenty python programs was created to test the framework for effectiveness, consisting of a variety of runtime errors, security flaws, performance issues, documentation issues and a mixture of these types of errors. Two metrics, namely execution time and qualitative assessment were used to compare the proposed multi-agent framework with a single-agent framework as baseline. Experimental results demonstrated the benchmark execution success rate was 95%, while the multi-agent architecture provided more structured, explainable and domain specific feedback than the single agent. The extra computational cost of the coordinated analyses was acceptable for software quality assessment tasks because of the resulting interpretability and modularity. Through implementation and experiments, the results demonstrate AgentCodeReview's utility and extensibility to the field of explainable AI in software quality assurance. The proposed architecture can be expanded to other programming languages, integrated into the industrial development flow, and enhanced with the advanced LLMs for scalable intelligent code review.
Bharath Kumar N, T L Manasa· International journal of com...· 0 citations
Recently, the emergence of Large Language Models (LLMs) has spurred a surge of research into automated unit test generation, yielding impressive performance and reducing manual effort. However, existing LLM-based approaches still suffer from two major limitations: (1) they follow rigid, procedural workflows that underutilize the autonomous reasoning potential of LLMs, making it difficult to dynamically adapt testing strategies based on real-time feedback; and (2) they rely on rule-based context extraction that is not tailored to test generation, failing to capture fine-grained code dependencies and test-specific knowledge required for deriving test requirements. In this paper, we propose TestAgent, an LLM-based test generation approach that addresses the above limitations by emulating human testing practices via a multi-agent collaboration mechanism. Particularly, TestAgent designs three specialized agents, namely a requirement planner, a test generator, and a test reviewer, to simulate how developers understand, construct, and validate unit tests. To unleash the autonomous capabilities of LLMs, we equip TestAgent with a set of tool APIs that can be invoked dynamically in an on-demand and adaptive manner. To further support repository-level reasoning, TestAgent constructs a test-specialized knowledge graph via static analysis, which captures code entities and their dependencies across the project and persistently stores testing artifacts (e.g., test reports and failure analyses) produced during generation. Experimental results show that TestAgent achieves 97.46% execution rate, 92.34% line coverage, 90.24% branch coverage, and 83.69% mutation score on six Java projects, outperforming LLM-based baselines across all metrics and achieving substantially higher mutation scores than search-based tools.
Quanjun Zhang, Ye Shang, Siqi Gu et al.· 0 citations
Large language models (LLMs) have evolved from standalone generative systems into agentic AI systems capable of planning, reasoning, tool use, and multi-agent collaboration. Enterprises are increasingly adopting AI agents to automate and orchestrate complex workflows, from IT operations to employee productivity. While early deployments focused on proof-of-concept prototypes, the past year has marked a clear shift toward production-grade enterprise AI agents. This transition has been enabled by a wave of new technologies, including multi-agent orchestration, memory and state management, skill-based and modular agent architectures, and deeper integration with enterprise data and workflow platforms, which together make scalable, reliable agent systems feasible in practice. At the same time, moving agents into production introduces new technical and organizational challenges, such as rigorous evaluation and benchmarking, security and governance, and system design for long-running, autonomous operation. Building on the success of our two prior highly attended editions: ''Agentic AI for Enterprise'' workshop at KDD 2025 and ''Enterprise RAG'' workshop at CIKM 2024, this workshop aims to bring together researchers and practitioners to examine how enterprise AI agents can successfully move from prototypes to production. We focus on three pillars: 1) Agent architectures and systems; 2) Enterprise applications and deployments; 3) Evaluation and governance.
Min Du, Anbang Xu, Jasmine Jaksic et al.· Proceedings of the 32nd ACM...· 0 citations