Automated skill evolution enables Large Language Model (LLM) agents to continuously improve without expensive retraining. However, existing approaches typically treat skill evolution as a sequence of local updates, overlooking relationships among skills and often producing overfitted skill updates that fail to generalize across tasks. We propose GSE, a globalized skill evolution framework that jointly optimizes skill compatibility and skill generalization. To preserve consistency across the skill bank, GSE maintains a Skill Relation Graph (SRG) that explicitly models and co-evolves inter-skill relationships. To improve generalization, GSE performs cluster-based skill consolidation to abstract reusable capabilities from local updates and employs replay-driven verification to prevent overfitting and behavioral regressions. We evaluate GSE on two representative software engineering tasks: bug-revealing test generation and false-positive bug report filtering. Across two state-of-the-art coding agents, OpenHands and mini-SWE-agent, GSE consistently achieves the best precision, recall, and F1-score. Compared with existing evolution techniques, GSE improves precision and recall by 6.1%~34.1% and 31.8%~180.0% for test generation, and by 15.4%~96.4% and 13.1%~19.8% for false-positive filtering. Deployment on an internal industrial agent further yields a 61.4% improvement in F1-score, demonstrating the effectiveness and generalizability of GSE for evolving effective skills.
Chen Yang, Jiashuo Tian, Ziqi Wang et al.· 1 citation
Test assertions are critical elements of unit tests, serving as checkpoints to validate expected behavior and ensure software correctness. Numerous techniques have been proposed to automate assertion generation, with recent progress notably driven by large language models (LLMs). Despite the promise, existing approaches such as ChatAssert suffer from modest accuracy, heavy reliance on oversampling, and vulnerability to model randomness due to one-shot prompting. To address these limitations, we propose AssertMate, a novel agent-based assertion generation framework that enhances the quality and reliability of LLM-generated assertions through three key components: (1) actual value construction that identifies assertion targets via static analysis and type-aware heuristics; (2) multi-perspective expected value prediction using code generation, retrieval-augmented generation (RAG), and chain-of-thought (CoT) reasoning agents; and (3) an LLM-as-a-Judge collaboration mechanism to select the most appropriate assertion. Evaluation on the Defects4J benchmark demonstrates that AssertMate significantly outperforms state-of-the-art techniques in compilation success and pass rates, along with substantially higher bug detection capabilities. Integration with EvoSuite further validates AssertMate's practicality, yielding superior mutation coverage and kill counts. Ablation studies reveal that each of the three components makes a significant and complementary contribution to the overall performance. This work affirms the great potential of aggregating diverse perspectives to enhance the effectiveness of LLM-based assertion generation.
Dong Wang, Qiaoyu Han, Lin Yang et al.· 0 citations
Software testing critically depends on test oracles, yet existing test oracles are often incomplete and insufficient for detecting bugs where implementations deviate from their specifications. Meanwhile, despite advances in test oracle construction, existing techniques typically rely on coarse-grained failure signals or require substantial manual effort, and thus remain inadequate for detecting specification-violation bugs. To address these limitations, we propose JavaOracle, a specification-driven approach that leverages large language models (LLMs) to reason over specifications and systematically enhance test oracles. Specifically, JavaOracle consists of three stages. First, it leverages LLMs to analyze specifications and derive additional test oracles that are not covered by existing tests, while integrating a root-cause-guided repair workflow to ensure that the enhanced test cases are syntactically and semantically valid. Second, it designs a multi-agent debate workflow to distinguish previously unknown specification-violation bugs from assertion failures caused by LLM hallucinations, thereby mitigating the impact of hallucinations. Third, unlike existing approaches that stop at bug detection, JavaOracle further automates test case minimization and bug report generation, producing submission-ready reports without manual effort. We evaluate JavaOracle on 3,961 test cases for Java SE API from the OpenJDK. Experimental results on the latest OpenJDK standard library show that JavaOracle substantially outperforms state-of-the-art baselines, including Fuzz4All, ChatAssert, and Randoop. Cumulatively, JavaOracle discovers 45 previously unknown bugs, which have already been confirmed/fixed by developers, with many persisting since their initial implementation. In an entire pipeline running, JavaOracle reduces execution failures to an average of 6 reports per run, achieving 88.9% precision. In contrast, baseline approaches required exhaustive manual inspection to identify only 0, 3, and 2 real bugs, respectively. Further analysis shows that test cases enhanced by JavaOracle achieve high validity, with an execution pass rate of 79.1%, compared to 35.4%, 92.2% (55.5% test cases unchanged), and 77.6% for the baselines. Ablation studies further demonstrate the effectiveness of JavaOracle components, while the automated pipeline significantly reduces manual analysis effort.
Ruifeng Fu, Yingquan Zhao, Meng Wang et al.· SIGSOFT FSE Companion· 0 citations
It is shown that fuzzing-generated crashes can meaningfully improve agent robustness and enable accurate safety monitoring with strong cross-method generalization, and the benefits of combining complementary fuzzing strategies and adopting multi-level diversity analysis to achieve more comprehensive and practical RL testing.
Zhibin Kang, Hanmo You, Dong Wang et al.· 0 citations