COMPARATIVE EVALUATION OF CLASS-IMBALANCE CORRECTION TECHNIQUES FOR SOFTWARE DEFECT PREDICTION
TL;DR
This study compared no correction, random oversampling, random undersampling, SMOTE, ADASYN, and class-weighted learning across logistic regression, decision tree, random forest, support vector machine, and neural network classifiers to find accuracy alone is unsuitable for selecting defect predictors.
Abstract
Class imbalance can make software defect predictors appear successful while missing defective modules. This study compared no correction, random oversampling, random undersampling, SMOTE, ADASYN, and class-weighted learning across logistic regression, decision tree, random forest, support vector machine, and neural network classifiers. KC1 and PC1 NASA/PROMISE datasets (3,218 modules; 403 defective) were evaluated by stratified five-fold cross-validation. Imputation, scaling, and correction were fitted only within training folds. Precision, sensitivity, specificity, F1-score, balanced accuracy, ROCAUC, PR-AUC, and confusion matrices were reported. Across classifiers, baseline balanced accuracy was 0.580; corrected means ranged from 0.696 to 0.720. Correction increased sensitivity but generally reduced precision and specificity. ROS achieved the highest mean F1-score (0.388), while SMOTE achieved the highest mean PR-AUC (0.378). A Friedman comparison indicated heterogeneity among techniques, followed by Holm-adjusted paired Wilcoxon tests. No approach dominated every classifier or metric. Leakage-safe correction and multi-metric assessment are essential; accuracy alone is unsuitable for selecting defect predictors.