FAIR-SHEPHERD: Fairness beyond Statistical Parity toward Structural Alignment

Abstract

All fairness algorithms unavoidably rely on normative assumptions about fair treatment, yet these assumptions often remain implicit. We argue that these assumptions should be formalized as explicit, auditable policies and introduce FAIR-SHEPHERD, a structural policy-based framework for transparent fairness in real-world settings with noisy labels and shifting distributions. FAIR-SHEPHERD uses gradients as attribution signals, encoded in a Structural Fairness Specification (SFS) that defines vertical coherence and orthogonality to vulnerable proxies. We introduce SFS metrics: Vertical Coherence Score (VCS) to measure directional coherence across adjacent normative slices, and Horizontal Leakage Score (HLS), augmented by a signed directional variant, to detect gradient alignment with policy-declared vulnerable or proxy directions. To enforce these policies, we develop S-agnostic tools that optimize policy-aligned gradient objectives using lattice-defined proxies. We demonstrate that outcome-based auditing is brittle to measurement error: under 10% label noise on Adult, Worst-Group AUC for ERM drops by 0.159. In contrast, our gradient-based structural metrics provide a label-agnostic audit of the decision logic, remaining stable even when evaluation labels are corrupted. Specialized fairness baselines including ARL, JTT, and GoG retain substantial structural leakage on COMPAS, with HLS values from about 0.47 to 0.71. Some also reduce EOD relative to ERM, which shows that outcome and structural criteria can diverge. Using our S-agnostic Gradient Penalty tools, we reduce policy-specified structural leakage by roughly 85% on COMPAS (0.478 → 0.073) and over 94% on Adult (0.138 → 0.008) while retaining competitive AUC and Worst-Group AUC. We have released our source code to facilitate further research here.