GRoFA: Noise-Gated Adapters for Jointly Fair and Robust Face Embeddings

Abstract

Face recognition is a fundamental technology in social interaction systems. However, face recognition systems exhibit demographic accuracy disparities that worsen under image degradation. Existing methods address bias mitigation and noise robustness independently; as a result, debiased embeddings remain fragile under real-world corruption, while noise-robust models can amplify per-group fairness gaps—neither approach alone ensures equitable performance across varying image quality. We propose GRoFA (Gated Robust Fair Adapter), a parameter-efficient adapter that jointly addresses both challenges through noise-conditional adaptation: a severity-supervised gate softly modulates the adapter’s contribution to a frozen vision encoder on a per-input basis. Experiments on FairFace and UTKFace against nine baselines show GRoFA attains 47/66 and 48/66 first-place wins (62/62 baseline wins; Wilcoxon p < 10− 9; Cohen’s d = 1.05/0.72) with only \(0.92\%\) trainable parameters, and transfers to CLIP and DINOv2. Gains concentrate in the noise-corrupted regime (the method’s design target); on the clean-only fairness regime GRoFA narrows FairFace Race Gap (0.052 → 0.042) and UTKFace Gender Gap (0.101 → 0.072, \(-29\%\)), trading a controlled HSIC-vs-texture Pareto cost on FairFace age.