Toward Reliable Machine Learning Model Selection: A Standardized Multi-Metric Evaluation Framework For Loan Approval

Abstract

The growing need for accurate, consistent, and reliable loan approval systems The use of machine learning in credit decision-making is increasingly important for financial institutions, but comparative research still often focuses on Accuracy or a limited number of classification metrics, so the trade-off between predictive performance and computational efficiency is not fully described. This study aims to develop a Standardized Multi-Metric Evaluation Framework (MMEF) to support the selection of more objective and reproducible machine learning models in the case of loan approval. The research method uses a standardized experimental pipeline with consistent preprocessing, class balancing using the Synthetic Minority Over-sampling Technique (SMOTE), identical data sharing, model optimization, and multi-metric evaluation. Five algorithms, namely Logistic Regression, Support Vector Machine (SVM), Random Forest, XGBoost, and CatBoost, are compared using a loan approval dataset consisting of 45,000 records and 13 predictor features. The evaluation includes Accuracy, Precision, Recall, F1-score, ROC-AUC, training time, and Overall Score. The results show that XGBoost provides the best overall performance with Accuracy 87.86%, Precision 66.77%, Recall 90.30%, F1-score 76.77%, ROC-AUC 96.27%, and Overall Score 0.827503. CatBoost has the fastest training time of 1.01 seconds, while SVM obtained the highest Recall of 92.40% with a much longer training time. These results indicate that model selection is not sufficient based on a single metric. MMEF provides a more systematic evaluation basis to identify models that have a balance of performance and efficiency in loan approval experiments.