The Fairness Illusion? A Cross-Dataset Audit of Accuracy and Demographic Bias in Credit Scoring Based on Machine Learning

Abstract

Machine learning has transformed consumer credit scoring, delivering substantial gains in predictive accuracy over traditional scorecards—but whether those gains come at a cost to fairness has remained contested. The dominant assumption in the literature is that more complex, accurate models amplify bias by encoding historical patterns of disadvantage more effectively. This paper challenges that assumption with direct empirical evidence. We evaluate four model families—logistic regression, random forest, XGBoost, and a multilayer perceptron—across two real-world datasets: the UCI Credit Card Default Dataset and the 2024 US Home Mortgage Disclosure Act national loan-level data, comprising over six million mortgage applications. Using repeated cross-validation, we report predictive performance alongside two primary fairness metrics—demographic parity difference and equalized odds difference—supplemented by false positive rate difference and calibration difference, with confidence intervals across 15 estimation folds. On the UCI data, where demographic disparities are modest, model choice has negligible effect on fairness outcomes. On the HMDA mortgage data, where racial disparities are large and legally consequential, the expected accuracy–fairness tradeoff does not hold; more accurate models produce significantly fairer outcomes on equalized odds within the models, data, and fairness criteria examined here, with logistic regression occupying the worst position simultaneously on all dimensions. Persistent demographic parity disparity among the more complex models is consistent with feature-level bias that no model architecture can resolve. The findings have direct implications for the less-discriminatory-alternatives framework under US fair lending law and for the high-risk classification of credit scoring AI under the EU AI Act.