Artificial Intelligence-Driven Software Test Automation: A Comprehensive Survey
TL;DR
This paper focuses on two key tasks—AI-driven test case generation and defect detection—and provides a systematic review of the technological evolution from traditional automation to intelligent testing, contrasting traditional deep learning and large language model approaches in defect detection.
Abstract
Software testing accounts for a significant proportion of resource consumption within the software development lifecycle. Traditional automated testing methods rely on predefined scripts and deterministic logic, and have inherent limitations when addressing the dynamic complexities of modern software systems. Artificial intelligence, particularly large language models, offers new technical avenues for test automation.This paper focuses on two key tasks—AI-driven test case generation and defect detection—and provides a systematic review of the technological evolution from traditional automation to intelligent testing. It analyses methods such as prompt engineering, retrieval-augmented generation, model fine-tuning and multi-agent systems,while contrasting traditional deep learning and large language model approaches in defect detection. The paper also examines key challenges, including the hallucination problem, evaluation criteria, interpretability and generalisation capabilities.The fundamental contribution of large language models lies not in replacing human testers, but in driving the transformation of test automation from 'execution automation' to 'decision support'—a distinction that provides important guidance for the design, evaluation and deployment of AI testing tools.