Self-Evolving AI Agents With Dual Memory for Automated Software Testing and Bug Localization
TL;DR
LS-CM (Long-Short Collaborative Memory), a dual-memory architecture that couples a Retrieval-Augmented Generation (RAG)-based short-term memory for code-base context with a Reinforcement Learning (RL)driven long-term memory that distills strategic patterns from historical test failures and patch iterations, is proposed.
Abstract
Large Language Model (LLM)-based autonomous agents have shown significant promise in automating software engineering tasks, yet existing systems still suffer from two fundamental limitations: i) the lack of persistent experiential knowledge across debugging sessions, which forces agents to repeat exploratory mistakes, and ii) the static nature of prompt structures, which prevents agents from adapting their reasoning strategy to recurring failure modes. To address these gaps, we propose LS-CM (Long-Short Collaborative Memory), a dual-memory architecture that couples a Retrieval-Augmented Generation (RAG)-based short-term memory for code-base context with a Reinforcement Learning (RL)-driven long-term memory that distills strategic patterns from historical test failures and patch iterations. The core innovation is a self-evolving context engineering module that automatically refines the agent’s internal prompt structure based on execution feedback signals, treating the prompt as a configurable policy object that a gradient-free controller selects among pre-generated variants, rather than a fixed artifact. We evaluate LS-CM on two industry-standard benchmarks: SWE-bench, for end-to-end issue resolution on real-world GitHub repositories, and Defects4J, for bug localization and program repair in Java projects. Across multiple LLM backbones, LS-CM resolves 33.7% of SWE-bench Lite issues, a modest gain over the strongest reproduced baseline (Agentless, 32.0%), and improves Top-1 fault localization accuracy on Defects4J from 42.6% to 47.6% while reducing editing churn by 31%. We find that LS-CM’s most robust advantages lie in debugging efficiency, cross-episode strategy transfer, and run-to-run stability rather than in a single headline resolved-rate number. Ablation studies confirm that short-term retrieval, long-term policy learning, and self-evolving prompts each contribute to the gains. Our work demonstrates that explicitly modeling memory and prompt adaptation as first-class learning objects substantially enhances agent productivity on real-world software engineering tasks.