Predicting Backup Non-Compliance Using KPI-Derived Labels and Machine Learning: A Proof-of-Concept Study

Abstract

Ensuring backup compliance is essential for data protection and operational resilience in enterprise IT environments. This study proposes a KPI-aware machine learning approach combining weak supervision, rule-based labeling, and supervised learning to estimate KPI-derived backup non-compliance status. The approach was evaluated on 439 device instances with 33 features using an 80:20 stratified split, producing a test set of 88 instances comprising 86 non-compliant and only 2 compliant proxy labels, indicating extreme class imbalance. Proxy labels were generated from predefined KPI rules without manual annotation or independent compliance audits. Logistic Regression, Random Forest, and XGBoost were evaluated using AUC-PR, F1-score, recall, Brier score, and calibration analysis. Random Forest achieved the strongest comparative results, with an AUC-PR of 0.998, F1-score of 0.994, recall of 1.000, and the lowest Brier score of 0.014. However, calibration remains uncertain because only two compliant instances were available. Therefore, these results indicate preliminary feasibility for reproducing or estimating KPI-derived proxy labels and should not be interpreted as independent validation of real-world non-compliance detection. The approach provides a scalable basis for backup compliance monitoring and preliminary risk prioritization in enterprise IT systems.