Explainable AI-Driven Metrics for Transparent Software Quality Prediction

Abstract

Accurate software quality prediction is critical for early defect identification and effective allocation of testing resources. Although machine learning (ML) and deep learning (DL) models have significantly improved defect prediction performance, their opaque nature limits transparency, interpretability, and practitioner trust. This study presents an Explainable Artificial Intelligence (XAI) driven framework for developing composite, transparent software quality metrics that integrate predictive accuracy with multi-level interpretability. The framework has been implemented and empirically validated using multiple real-world datasets. The framework combines static code metrics, process metrics, and developer activity metrics to construct robust defect prediction models using Random Forest, Gradient Boosted Trees (e.g., via the XGBoost implementation), Support Vector Machines (SVM), Logistic Regression, and attention-based Neural Networks. To enhance transparency, model agnostic explanation techniques, including SHAP and LIME, are integrated with in-model attention mechanisms to provide global and local explanations of predictions. Experiments conducted on PROMISE, open-source, and industrial datasets demonstrate that the proposed composite explainable metric achieves superior performance on the PROMISE (NASA-JM1) dataset, with the Gradient Boosted Trees model reaching an Accuracy of 0.85 and an AUC of 0.88, and consistently outperforms traditional and opaque baselines across the Eclipse JDT, Apache Commons, and Industrial datasets. Quantitative faithfulness analysis shows strong alignment between SHAP explanations and model behavior (Pearson’s r= 0.81). A controlled user study involving software developers reports a 14% improvement in decision accuracy and significantly higher confidence levels (p< 0.01) when using explainable metrics. The framework further incorporates interactive visual dashboards and textual summaries to support actionable decision-making during quality assurance processes. By bridging the gap between predictive performance and interpretability, this research demonstrates that explainability not only enhances trust but also improves the effectiveness of practical defect triage. The methodology provides a scalable and extensible foundation for transparent AI-driven software engineering tools.