Results show that combining the two repair scopes achieves higher accuracy and lower token consumption than graph-level repair alone, and Ablation results show that combining the two repair scopes achieves higher accuracy and lower token consumption than graph-level repair alone.
Abstract
LLM-based multi-agent system (MAS) applications can produce useful intermediate results while leaving task requirements unsatisfied. Repairing such executions requires identifying the work still needed and the completed results that can support it. We present \tool, a contribution-guided runtime repair framework for MAS applications. \tool derives expected contributions and acceptance conditions from task requirements and method specifications, and connects their dependencies in a contribution graph. Runtime checks locate unmet conditions and establish which results remain usable. These results guide action-level correction when available inputs suffice, or graph-level recovery when prerequisites or unfinished work need revision. Both repair scopes retain validated contributions and check repaired outputs before subsequent work uses them. We evaluate \tool on 30 applications across six MAS frameworks and 9,000 tasks. It detects runtime failures with 88.28\% precision and 79.18\% recall, and achieves 84.76\% task accuracy, improving over Native execution by 26.74 percentage points and the strongest of four recovery baselines by 21.28 points. Estimated deployment token consumption is 17.92\% above Native, but 24.19\% below Reflexion and 13.05\% below Iterative Rollback. Ablation results show that combining the two repair scopes achieves higher accuracy and lower token consumption than graph-level repair alone.
This work presents AgentEvolver, a system for developing capabilities during task execution while keeping the foundation model fixed, and evaluates task outcomes on SWE-bench Pro Public and examines capability changes in six application cases.
Wen-Tao Zhang, Fu-Chao Yang, Yi-Lei Zhao et al.· 0 citations
The EIR is introduced, a lower bound on the information that must remain accessible for correct completion under specified task and access conditions and LACUNA is developed, a framework that generates tasks with known dependencies and varies information demand, retention, and recovery separately from the difficulty of...
N. Mehrotra, Ashish Tiwari, Priyanshu Gupta et al.· arXiv.org· 0 citations
Does recovery advice help an agent finish a task after a tool fails? AgentFaultBench evaluates LangGraph and CrewAI agents performing specified tasks in an isolated GitLab installation. We inject tool errors and check both the resulting files and issues and the required reads, verification and final response. The promp...
Agent-evolution methods use execution feedback to improve prompts, skills, tools, and orchestration, but their implementations often rebuild the same experimental machinery. We present RSIHub, an open-source framework for building, comparing, and extending these methods on shared infrastructure. Recipes compose reusabl...
Zi-Mu Wang, Xiaobo Wang, Si-Yu Ye et al.· 1 citation
Repository-level coding benchmarks largely assume that tasks and environments remain unchanged throughout execution. In software development, code, dependencies, runtime state, and requirements can change while an agent is still working. We introduce SWE-Shift, a benchmark for evaluating coding agents under such mid-ex...
Victor Shea-Jay Huang, Rong-Qing Li, Zhangyue Yin et al.· 0 citations
Improving an LLM agent requires knowing what to change when it fails. Similar failures can require different fixes, and resolving one problem can expose another. We introduce Trace-to-Architecture (T2A), a framework that uses execution trajectories to determine what to change in an agent's harness and when those change...
Wen-Feng Wang, Jian-Rui Shi, Min-Gang Guo et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.