Search-Based Test Case Generation for Unfairness Detection in Multi-Layer Perceptron-Based Systems: A Genetic Algorithm Approach and Empirical Evaluation

Abstract

Fairness in machine learning (ML) and deep learning (DL) systems is becoming increasingly important because algorithmic decisions start to have a considerable influence on the governance of some high-stakes domains, making unfairness detection and testing crucial. Despite notable progress, existing automated fairness testing approaches remain limited by static fairness definitions, single-objective optimization, and restricted adaptability across models and datasets. This study addresses these limitations by proposing a Genetic Algorithm–based Test Case Generation (GATCG) framework that combines knowledge-based reasoning with multi-objective evolutionary optimization to reveal unfair behavior dynamically. Our approach comprises a Fairness Knowledge Base (FKB) that formally encodes fairness concepts through logical rules and semantic networks, thereby enabling explicit analysis of the relationships between protected attributes, fairness metrics, and constraints. This knowledge-driven foundation is tightly integrated with a multi-objective genetic algorithm that evolves test cases via selection, crossover, and mutation while simultaneously optimizing disparate impact, disparate treatment, and disparate mistreatment. By combining structured fairness reasoning with evolutionary search, the proposed approach extends beyond single-metric detection to include automatic and adaptable fairness testing. Experiments conducted on benchmark datasets (Adult Income and German Credit) using a Multi-Layer Perceptron (MLP) classifier as the underlying model demonstrate that GATCG achieves lower discrimination sample search time and higher detection success rates compared to baseline approaches (ExpGA, AEQUITAS, SG and ADF). Prior to introducing the uniqueness-tracking mechanism, GATCG demonstrated strong effectiveness by reducing discriminatory sample search time (DSS) by 35%, increasing the number of discovered discriminatory samples (DSN) by 22%, expanding test sample generation (TSN) by 8%, and improving the success rate (SUR) by 9% compared to baseline approaches. To provide a more accurate assessment of the approach, a uniqueness-tracking mechanism and a one-per-generation rule were subsequently incorporated to eliminate duplicate discriminatory discoveries. Under this enhanced evaluation setting, GATCG achieved overall success rates (SUR) of 61.85% on the Adult Income dataset and 68.17% on the German Credit dataset. The results indicate that GATCG efficiently directs the search toward unique discrimination-prone regions of the input space, leading to improved detection speed, more effective identification of biased samples, higher-quality test generation, and strong discriminatory sample discovery performance. Finally, we conduct a controlled experiment that demonstrates GATCG’s superiority over THEMIS and manual testing with respect to unfairness detection effectiveness, usability, coverage, and testing efficiency.