This work proposes a novel Reinforcement Learning from AI Feedback (RLAIF) framework, finding that specifications can be iteratively refined by leveraging feedback from the LLM's own outputs, creating a self-improving loop.
Abstract
The use of Large Language Models (LLMs) for code generation has advanced rapidly, yet current systems often struggle with complex algorithmic logic and edge cases. Standard methods for improving code generation, such as iterative self-refinement, often cause repetitive failure cycles, as models cannot independently escape their own logical blind spots. To address this gap, we propose a novel Reinforcement Learning from AI Feedback (RLAIF) framework. Our key insight is that specifications can be iteratively refined by leveraging feedback from the LLM's own outputs, creating a self-improving loop. This approach contrasts with prior methods that treat specifications as static inputs. Rather than updating the parameters of the base model, a proprietary LLM (the Worker) is accessed via API, while a smaller, trainable Student agent learns to guide generation by modifying natural language specifications. A Tutor agent supervises these refinements, ensuring they are precise and actionable, and provides a reward signal driven by failures identified by a synthetic test generator. Crucially, the Worker, Tutor, and Test Generator roles are all instantiated using the same proprietary model, isolating the learning capacity entirely within the Student agent. We evaluate our approach on N = 200 problems using a Qwen 2.5-14B-Instruct Student and a GPT-4o-mini model fulfilling the Worker, Tutor, and Test Generator roles on the LiveCodeBench and HumanEvalPlus benchmarks. Our method achieves relative pass@1 improvements of 3.2% on LiveCodeBench (64.0% vs. 62.0%) and 1.1% on HumanEvalPlus (96.0% vs. 95.0%).
SCOPE is presented, a prover-initialized subgoal critic for code generation that adapts a Lean-oriented prover model to produce three parseable feedback fields for downstream code generation: subgoals, gap analysis, and a robustness checklist.
Yueke Zhang, Yifan Zhang, Zihan Fang et al.· 0 citations
This work introduces a search-based approach that identifies and evolves a set of natural language transformation rules with strong downstream effects on coding performance, and proposes DUALFIX, a staged repair pipeline that combines the evolved transformation rules with execution-feedback repair, addressing both specification-level and implementation-level failures.
Amal Akli, Melissa Akli, Cedric Richter et al.· 0 citations
Large Language Model (LLM)-based autonomous agents have shown significant promise in automating software engineering tasks, yet existing systems still suffer from two fundamental limitations: i) the lack of persistent experiential knowledge across debugging sessions, which forces agents to repeat exploratory mistakes, and ii) the static nature of prompt structures, which prevents agents from adapting their reasoning strategy to recurring failure modes. To address these gaps, we propose LS-CM (Long-Short Collaborative Memory), a dual-memory architecture that couples a Retrieval-Augmented Generation (RAG)-based short-term memory for code-base context with a Reinforcement Learning (RL)-driven long-term memory that distills strategic patterns from historical test failures and patch iterations. The core innovation is a self-evolving context engineering module that automatically refines the agent’s internal prompt structure based on execution feedback signals, treating the prompt as a configurable policy object that a gradient-free controller selects among pre-generated variants, rather than a fixed artifact. We evaluate LS-CM on two industry-standard benchmarks: SWE-bench, for end-to-end issue resolution on real-world GitHub repositories, and Defects4J, for bug localization and program repair in Java projects. Across multiple LLM backbones, LS-CM resolves 33.7% of SWE-bench Lite issues, a modest gain over the strongest reproduced baseline (Agentless, 32.0%), and improves Top-1 fault localization accuracy on Defects4J from 42.6% to 47.6% while reducing editing churn by 31%. We find that LS-CM’s most robust advantages lie in debugging efficiency, cross-episode strategy transfer, and run-to-run stability rather than in a single headline resolved-rate number. Ablation studies confirm that short-term retrieval, long-term policy learning, and self-evolving prompts each contribute to the gains. Our work demonstrates that explicitly modeling memory and prompt adaptation as first-class learning objects substantially enhances agent productivity on real-world software engineering tasks.
Tianjun Mo, Changhao Zhang, Jitong Zou et al.· IEEE Access· 0 citations
Large Language Models (LLMs) show promise for synthesizing software directly from natural-language problem descriptions. However, LLM-based code synthesis remains unreliable: models may hallucinate features, generated tests and code may diverge, and repairs often require manual effort. We present a test-driven pipeline that extracts functional requirements (FR) from a problem description, resolves dependencies, maps them into a modular Model-View-Controller (MVC) structure, and generates tests before code, followed by bounded, execution-driven refinement. We evaluate execution reliability with Pass@1 and refine@k, and assess the faithfulness of FR extraction using NLI-based entailment.
Wasay Mohammed Abdul, Ragib Shahariar Ayon, Shibbir Ahmed et al.· SIGSOFT FSE Companion· 0 citations
Large Language Models (LLMs) are predominantly assessed based on their common sense reasoning, language comprehension, and logical reasoning abilities. While models trained in specialized domains like mathematics or coding have demonstrated remarkable advancements in logical reasoning, there remains a significant gap in evaluating their code generation capabilities. Existing benchmark datasets fall short in pinpointing specific strengths and weaknesses, impeding targeted enhancements in models’ reasoning abilities to synthesize code.
To bridge this gap, this thesis introduces two novel contributions: CodeEval and CodeQual. CodeEval is an innovative, pedagogical benchmarking method that mirrors the evaluation processes encountered in academic programming courses. It comprises a multi-dimensional benchmark dataset of 602 hand-crafted problems designed to rigorously evaluate LLMs across 24 distinct aspects of Python programming, covering three proficiency levels—beginner, intermediate, and advanced—and includes both class-based and function-based problem types with detailed problem specifications and comprehensive test suites achieving 99.1% coverage. To facilitate widespread adoption, we developed RunCodeEval, an open-source execution framework that provides researchers with a ready-to-use evaluation pipeline. Our evaluation of 15 state-of-the-art LLMs revealed consistent performance degradation with increasing complexity (validated statistically, Cohen’s d = 0.790) and universal struggles with advanced concepts like concurrency.
Code quality is inherently subjective, encompassing dimensions like readability, efficiency, and adherence to language idioms that traditional static metrics fail to capture adequately. While large language models can assess these subjective qualities, lightweight models offer practical advantages: seamless CI/CD pipeline integration, lower operational costs, and full control over model behavior. We investigate whether such models can learn to assess code quality by training on synthetic LLM annotations. We introduce CodeQual, a dataset of 5,819 code samples derived from five established sources spanning diverse domains—competitive programming, pedagogical problems, software engineering, and general benchmarks—scored by LLMs across five quality dimensions, with 655 human-annotated samples for evaluation. Our fine-tuned model, CodeQualBERT, not only matches LLM performance but exceeds inter-human agreement on all five dimensions, achieving 16–100% improvement over the inter-human agreement baseline.
Together, these contributions provide a comprehensive framework for evaluating and improving LLMs in software engineering contexts, encompassing both functional correctness assessment and subjective code quality evaluation.
Large Language Models (LLMs) have achieved remarkable progress in code generation, yet ensuring correctness in complex, repository-level tasks remains challenging. Existing approaches often use generated tests as static post-hoc validators, which limits their ability to guide implementation and may introduce misleading feedback when the tests themselves are incomplete or incorrect. In this paper, we introduce TDD-Agent, which operationalizes the test-driven development paradigm for code generation. TDD-Agent first prompts the model to generate executable tests, encouraging it to clarify expected behaviors before implementation, and then performs iterative dual-track refinement over both the generated code and tests using execution feedback. We first isolate the effect of test-first reasoning through a prompt variant TDD-prompt on LiveCodeBench, where it consistently improves upon reasoning-based prompting baselines. Building on this finding, we evaluate the full TDD-Agent framework on RepoEval, a repository-level benchmark, and show that it consistently outperforms retrieval-based and agent-based baselines. Additional analyses show that iterative refinement improves not only code correctness but also the effectiveness of the generated tests, yielding higher pass rates, coverage, and mutation scores, suggesting that tests can serve as evolving reasoning artifacts rather than fixed validators. Our source code is available at https://anonymous.4open.science/r/TDD-Agent-Framework-6370/.
Hong Yu, Ke-Shen Li, Jiakun Li et al.· 0 citations