Skip to content

Investigating Software Aging in LLM-Generated Software Systems across Generation-and-Execution Environments

Aug 2026 · 0 citations · 38 references
Computer Science

TL;DR

It is found that functional correctness alone is insufficient to assess the operational reliability of LLM-generated software before deployment in continuously running environments, and aging trends can also emerge in manually developed implementations.

Abstract

Large Language Models (LLMs) are increasingly used to generate executable software systems from natural language specifications, accelerating development and reducing manual implementation effort. Although recent studies have investigated the functional correctness, security, maintainability, and robustness of LLM-generated code, little is known about the long-term reliability of such systems under sustained execution. In this paper, we experimentally investigate software aging symptoms in LLM-generated service-based applications across different programming languages. Using backend scenarios derived from BaxBench, we generated applications targeting JavaScript, Python, and Rust through LLM-based generation platforms, validated them with BaxBench-derived tests, and subjected them to 48-hour workload executions. We monitored memory usage, response time, and throughput and analyzed them using the Mann--Kendall test and Sen's slope estimator. We further complemented the runtime evaluation with static analysis of the generated source code and an exploratory comparison with human-written implementations of related backend scenarios. The results show that memory usage is the most consistent indicator of potential software aging, with statistically significant upward trends in most application-language combinations, while response time and throughput exhibit more heterogeneous behavior. Static analysis identified plausible code-level aging mechanisms, and the comparison with human-written systems showed that aging trends can also emerge in manually developed implementations. These findings indicate that functional correctness alone is insufficient to assess the operational reliability of LLM-generated software before deployment in continuously running environments.

View source

Similar papers

#software testing Review Aug 2026

Model-Based Agentic Software Engineering

MAGE explains how externalized knowledge, bounded action, independent evaluation, and retained human authority can compose into a governed engineering environment, and proposes tests of when that environment turns commodity intelligence into durable engineering progress.

James C. Davis, Kelechi G. Kalu, Huiyun Peng et al. · 1 citation
#software testing Preprint Aug 2026

Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code

LLMs are increasingly used for code generation, yet they frequently hallucinate non-existent software packages, creating exploitable entry points into the software supply chain. We make four contributions to this problem. First, we show that prior evaluation methodologies systematically inflate hallucination rates by misclassifying standard-library modules as hallucinations in some languages. For Python, the overestimation reaches 9.4 percentage points. Second, we evaluate seven inference-time defenses for mitigating package hallucinations, including five guided decoding strategies (Greedy, Contrastive, DoLa, Nudging, and Active Layer-Contrastive Decoding), an iterative self-refinement approach (Self-Refine), and a Retrieval-Augmented Generation (RAG)-based defense.. Across eight models spanning five families and four programming languages (Python, JavaScript, Ruby, Rust), RAG reduces the package hallucination rate (PHR) in 18 of 32 model--language configurations. Third, we introduce Package Utility (PU) to assess whether defenses preserve valid and task-relevant recommendations. Among strategies evaluated, Greedy decoding provides the strongest average mitigation--utility trade-off. Fourth, we stress-test all strategies under adversarial prompts seeded with fabricated package names and find that PHR surges by up to 45 percentage points relative to standard prompts, with Ruby consistently the most vulnerable language (80.9--95.2\%). Under adversarial conditions, RAG and Self-Refine outperform all decoding-only strategies, indicating that robust defense requires either external grounding or iterative self-verification when prompts are actively hostile. Our results recast package hallucination as both a measurement problem and a decoding-time control problem, and they demonstrate that the choice of defense must be matched to the threat model and recommendation utility.

Albérick Euraste Djiré, Iyiola E. Olatunji, Melissa Tessa et al. · 1 citation
#software testing Open access Aug 2026

HawkEye: Web Vulnerability Analysis and Security Audit Tool

HawkEye is introduced, a modular, web-based vulnerability auditing platform designed to streamline security analysis by integrating multiple scanning tools within a unified dashboard and illustrates how consolidated reporting improves vulnerability prioritization for development teams.

D. R. Patil, Varad Salgare, Devaj Arya et al. · 0 citations
#software testing Review Open access Sep 2026

Application for Tracking Bugs in Software Development: Improving Project Management Efficiency

By streamlining workflows and fostering collaboration, this platform offers a scalable, cost- effective solution for SMEs and contributes to software engineering by demonstrating how integrated technologies can modernize development processes in resource limited contexts, with potential for broader adoption in Albania and beyond.

Griselda Belba TRUPJA, Agim Kasaj · 0 citations

Prospective evaluation of hearing aid fitting adjusted toward one-third functional gain in patients with sensorineural hearing loss.

OBJECTIVE To prospectively evaluate whether modifying DSL version 5-based hearing aid (HA) fittings by adjusting gain on HA fitting software so that measured functional gain (FG) approached a one-third gain (1/3G) target could provide appropriate fitting outcomes in patients with sensorineural hearing loss. METHODS Twenty-four patients (48 ears) with bilateral sensorineural hearing loss underwent initial HA fitting using the DSL version 5 prescription formula. FG was measured at 250-4000 Hz, and HA gain was adjusted on HA fitting software so that FG approached the target 1/3 G. Speech discrimination scores at 65 and 80 dB SPL were evaluated after a two-week trial period using the 67-S Japanese monosyllable word list. Based on speech discrimination test results, ears were classified as well-fitting or non-well-fitting. FG values were compared between the two groups. RESULTS Twenty-one patients (42 ears) completed the study. Thirty-one ears (73%) were classified as well-fitting. Although HA gain was adjusted toward the target 1/3 G, measured FG values at 250 and 500 Hz remained lower than the target values. In well-fitting ears, low-frequency FG values were lower than the target 1/3 G, whereas FG at 2000 Hz was close to the target value. In contrast, non-well-fitting ears showed low-frequency FG values closer to the target 1/3 G, whereas FG values at 2000 and 4000 Hz remained below the target values. CONCLUSIONS Although HAs adjusted toward a 1/3 G target did not achieve the intended FG values, particularly at low frequencies, relatively favorable fitting outcomes were obtained in approximately three-quarters of the ears. In well-fitting ears, low-frequency FG remained below the target 1/3 G, whereas FG in the mid-frequency range around 2000 Hz was close to the target value. These findings provide a basis for future prospective studies to clarify how these FG characteristics should be applied to optimize HA adjustment.

Unknown authors · 0 citations

Related blog posts

MIT News · Artificial Intelligence Aug 17, 2026

Q&A: Rethinking how innovation happens

In his latest book, Professor Eugene Fitzgerald examines the forces that turn breakthroughs into value — and why innovation resists simple formulas.