Automated Testing Tools and Software Reliability: A Systematic Mapping
Abstract
Modern software systems operate in increasingly complex and heterogeneous environments, intensifying challenges related to reliability. Automated software testing (AST) helps address these demands, yet existing evidence on how widely used tools support reliability-oriented attributes remains dispersed. A consolidated view is needed to understand their contributions to coverage, fault detection, robustness, and execution reliability. This study conducts a systematic mapping of 184 primary works to identify the most frequently cited AST tools and to examine empirical evidence of their reliability-related capabilities. The analysis synthesizes findings on structural coverage, fault revelation, robustness under stress, and cross-platform consistency, revealing complementary strengths and gaps in current tool support. The results offer a structured perspective on how automation contributes to reliability evaluation, supporting software practitioners in selecting tools aligned with reliability goals and assisting researchers in identifying opportunities for advancing reliability-aware testing. In addition to the empirical synthesis, this study produces an evidence-based artifact in the form of a reliability-oriented attribute matrix, explicitly linking automated testing tools to traceable primary-study evidence and supporting informed, practical tool selection.
