FIFT: Feature Importance-Guided Fairness Testing for machine learning software

Machine Learning for Credit Approval: Enhancing Decision Accuracy and Explainability

Machine learning is widely used to improve predictive accuracy in complex domains like credit scoring, but many models (e.g., deep neural networks) remain opaque. This lack of interpretability is problematic in regulated domains (banking, finance) where transparency is required. Rule-based learning methods, such as Learning Classifier Systems (LCS), offer a trade-off between accuracy and explainability. We introduce a novel Ranked Attribute Selection with Midpoint Filtering (RASF) framework that extends LCS (EXTRACS) to enhance feature selection and rule validation for credit approval. RASF first ranks features by mutual information, then employs rank-guided randomized selection to diversify rule conditions, and finally filters new rules by midpoint-based Euclidean distance to the current instance. We evaluate RASF-enhanced LCS on public loan approval datasets, comparing against a baseline LCS and logistic regression. Results show that RASF improves predictive accuracy by about 3.6 – 4.64% over the base LCS, while producing an inherently interpretable rule set. By bridging accuracy and transparency, RASF-LCS supports explainable AI in credit scoring.

BiasQD-DT: Creating Interpretable Trade-Offs in Classifier Bias with Quality-Diversity Optimization via Decision Trees

The implementation of Artificial Intelligence (AI) in sensitive domains requires that models are fair and clearly understandable for stakeholders to trust in the decisions made. While Explainable AI (XAI) methods approximate the behavior of models, Interpretable AI (IAI) focuses on inherently transparent representations such as decision trees. However, interpretability alone is not enough, as models may encode biases from different sources, including biases present in data, leading to unfair outcomes. This introduces a fundamental trade-off between predictive performance and fairness. In this paper, we propose a novel framework that integrates evolved decision trees generated from random seed-based vectors and Quality Diversity (QD) optimization to jointly address decision interpretability, predictive performance, and fairness. Using evolutionary search, the method generates a repertoire of high-performing, interpretable models that span different regions of the fairness-accuracy space. This enables a systematic characterization of trade-offs and provides decision-makers with multiple transparent alternatives. Experimental results show that the proposed approach effectively discovers diverse decision tree models with competitive overall accuracy while achieving improved fairness across standard benchmarks.

Moving Towards a Fairer Future: Reproducing Kamiran and Calders’ Findings on Anti-Discrimination Techniques

Resampling strategies for machine learning-based effort overrun risk detection: A controlled factorial study with cost-sensitive evaluation

The Synthetic Minority Oversampling Technique (SMOTE) has become the default preprocessing step for handling class imbalance in software effort-risk prediction, yet its effectiveness in this domain has not been rigorously tested. This study addresses that gap through a controlled factorial experiment on the publicly available Software Development Effort Dataset Annotated with Expert Estimates dataset, comprising 4,329 software issues from Apache projects drawn from an initial repository of 23,186 records. Six resampling strategies are compared across four classifier families under both standard and cost-sensitive evaluation metrics that weight missed high-risk issues more heavily than false alarms. A secondary contribution is the analysis of how resampling interacts with the extreme class imbalance characteristic of real-world effort data (2.2% minority rate), a regime substantially more severe than those examined in prior investigations. Results are interpreted through SHAP-based feature attribution to determine whether oversampling alters which features the models rely on. The findings reveal two overarching results: near-perfect performance under a full feature set is largely attributable to target leakage rather than a genuine predictive signal, and, under deployment, valid early-warning features degrade cost-sensitive performance relative to no resampling when SMOTE is used. Cost-sensitive weighting emerges as the more reliable alternative, preserving both performance and feature attribution structure. These findings challenge the uncritical adoption of SMOTE in software analytics and carry direct implications for the design of reproducible, interpretable risk-detection pipelines.

Bounding the Fairness of a Classifier Using Population-level Statistics

Background: Classifiers increasingly affect people’s lives, necessitating their audit for fairness and accuracy on diverse populations. However, direct auditing is often not possible, due to a lack of access to the classifier or to suitable individual-level validation data. Objectives: This work aims to assess the fairness and accuracy of black-box classifiers using only population-level statistics, without requiring access to the classifier or individual predictions. Specifically, it introduces a method to lower-bound the discrepancy of a classifier: a quantity that jointly captures inaccuracy and unfairness. Methods: We define a novel measure of unfairness based on the equalized odds fairness criterion, quantifying the fraction of the population on which a classifier deviates from ideal fair behavior. Using this measure, we develop a computationally efficient procedure for calculating the tightest possible lower bound on the classifier’s discrepancy, using only aggregated rates of positive predictions and true positives across protected sub-populations. Results: Empirical evaluations confirm the tightness of the proposed lower bound in practical settings. The method is demonstrated on several use cases, including estimating the reliability of voting polls and assessing the fairness of patient identification from internet search data. The code and data are available at https://github.com/sivansabato/bfa2. Conclusions: This work provides a practical and interpretable framework for auditing classifiers using population-level statistics. The proposed approach enables stakeholders to identify fairness and accuracy concerns in settings where traditional auditing is not feasible.

The Price of Fairness in Active Learning: Fundamental Limits and Optimal Label Acquisition

When a classifier is trained by actively choosing which examples to label, should its labeling budget depend on which notion of fairness it must satisfy? We answer this with the first information-theoretic lower bounds for group-fair active learning, and the answer is a sharp hierarchy across the three standard notions. Demographic Parity (DP), which equalizes the positive prediction rate across groups, can be verified from unlabeled data alone, so it adds no labels beyond ordinary learning. Equal Opportunity (EO), which equalizes each group's true positive rate, is fundamentally harder: estimating a rate conditioned on the true label forces the learner to collect labeled positives from every group, an irreducible cost that no query strategy can avoid. Equalized Odds (EqOdds), which additionally equalizes false positive rates, costs no more than EO whenever positives are the minority, the typical regime. This exposes a separation at the heart of fair learning: active querying yields the well-known exponential savings for accuracy, but no speedup whatsoever for fairness verification. Concretely, certifying EO to fairness tolerance γ requires Ω(d/ε2 + k/(ε2 p+)) labels, where d is the model's Vapnik--Chervonenkis (VC) dimension, k the number of groups, ε the target error, and p+ = Pr[Y=1] the positive rate; the fairness term is independent of ε, so it is a floor that active learning cannot lower. We give matching algorithms (ConstrainedERM-DP for DP, FairStratified for EO and EqOdds) that are tight up to logarithmic factors. The cost is largest exactly when positives are rare: at p+ = 0.01, as in fraud detection, EO demands roughly 90× more labels than DP. Experiments on Folktables, COMPAS, and Adult confirm the predicted scaling and yield 36 to 39% label savings over passive fair-learning baselines.