The Accuracy Trap: Structural Scarcity Amplifies Relative Inequality in Algorithmic Allocation
Abstract
Algorithmic systems increasingly rank individuals for access to scarce public resources, from child welfare interventions to cancer treatment referrals. The prevailing fairness frame treats disparity as a property of biased data or deficient models, with remedies through calibration and debiasing. Under structural scarcity, where demand exceeds supply by an order of magnitude, allocation becomes a rationing problem, and the statistical properties of ranking diverge sharply from those of classification. We derive a scaling law $D \propto \exp(t \cdot \rho \cdot \Delta)$, in which relative disparity between two groups separated by a structural gap $\Delta$ grows in the product of the scarcity-induced threshold $t$ and rank-discrimination fidelity $\rho$. Scarcity and accuracy interact multiplicatively, producing exponentially larger between-group disparities. We term this dynamic the Accuracy Trap. We validate this Accuracy Trap through Monte Carlo simulation and two independent public-sector systems in Canadian child welfare and U.S. cancer care. Debiasing alone cannot dissolve the trap.