Skip to content
Open access

THE POTENTIAL OF AI AGENTS IN THE FIELD OF RELIABILITY AND QUALITY

2025 · Reliability and Quality of Complex Systems · 0 citations

TL;DR

The proposed hybrid approach demonstrates practical potential, but its implementation in real-world development will require improving the semantics of automatically generated contracts, expanding training sets, and creating automated error correction procedures integrated into the generation cycle.

Abstract

Background. Automated generation of formal specifications remains a key task in formal software verification to improve its reliability, since manual code annotation is labor-intensive and error-prone. The aim of the study is to evaluate the quality of generated annotations and identify practical limitations of the approach when using small-class models. Materials and methods. The experiment used the local models Qwen2.5-coder, CodeLlama, and DeepSeek-coder; syntactic checking of annotations was performed using SpotBugs, and formal verification was performed using OpenJML together with the Z3 SMT solver. Results. On a set of 11 OpenJML examples, medium-sized models demonstrated the ability to generate syntactically correct JML annotations; however, the proportion of fully verified specifications remained low. The main limitations were identified in the area of semantic accuracy of annotations, the inability to take side effects into account, and, in some cases, format mismatches between analysis tools. Conclusions. The proposed hybrid approach demonstrates practical potential, but its implementation in real-world development will require improving the semantics of automatically generated contracts, expanding training sets, and creating automated error correction procedures integrated into the generation cycle.

Read PDF

Similar papers

Open access 2025

INTEGRATION OF LOGIC-ORIENTED VERIFICATION WITH LARGE LANGUAGE MODELS INTO SOFTWARE DEVELOPMENT

Experimental evaluation shows that increasing the proportion of successful verifications requires additional model training, a broader context, and the integration of mutation and candidate selection operations, while practical limitations of the proposed hybrid approach when using small-class models are identified.

P. Mel'nikov, Andrey Tyugashev · 0 citations
#artificial intelligence Review Sep 2026

Is manual software optimization a thing of the past?

This work investigates whether LLM-based agents can autonomously achieve substantial performance improvements in scientific software, including mature implementations that have already been extensively optimized by human developers.

Pavlin G. Poličar, Martin Špendl, Tomaž Hočevar · 0 citations
#artificial intelligence Preprint Sep 2026

A Study of the Reliability of Agentic AI-Generated Programs

This project verified that generating robust and reliable software using agentic AI requires careful practice and human supervision and demonstrated that using agentic AI workflow for software development (with its prompts and skills) can become a specification of the code that leads to cost-effective sustainability of...

Ayesha Shafique, Barton P. Miller, E. Heymann · 0 citations
Book Open access Oct 2026

Understanding the Code and Architectural Smells Generated by LLMs and Autonomous Agents

A systematic audit of technical debt in AI-generated software is presented, revealing that AI does not eliminate flaws but rather introduces a distinct "machine signature" of defects.

Yue-Cai Zhu, Nikolaos Tsantalis, Peter C. Rigby · 0 citations
Preprint Oct 2026

Towards Agentic Proof Harness Engineering

Formal verification of real-world projects requires substantial manual effort, from specifying correctness properties to constructing the infrastructure needed to check them. We envision verification agents that assist developers throughout this process. Among the many tasks involved, one stands out as both foundationa...

R. Shanmuganathan, Cedric Richter, Heike Wehrheim · 0 citations
Sep 2026

LLM-enabled Behavior Driven Development Workflow for Formally Verified Hardware Designs

This work proposes an integrated view on the use of LLMs for EDA and establishes an LLM-enabled behavior driven hardware development workflow, introducing and defining Formal Verification Gherkin Scenarios (FV Gherkin Scenarios), unlocking CNL specifications as the foundation for formally verified hardware designs via...

Luca Müller, Qian Liu, Rolf Drechsler · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.