Mitigating human review bottlenecks in AI evaluation pipelines: A queueing theory approach

Abstract

Human review stages in AI-assisted evaluation pipelines can become throughput bottlenecks when automated upstream processes generate cases faster than reviewers can assess them—a problem acute when information availability varies across client types, a form of structural information asymmetry whose operational consequences remain underexplored. We quantify this bottleneck and identify the efficiency gain that explainable-AI (XAI) support must provide to restore stability. We develop an M/G/1 queueing model of a technology-valuation pipeline, motivated by an anonymised case in which the human pricing-decision stage requires up to 72 h for information-scarce small firms versus about 5 h for information-rich mid-size firms. With small firms at 70% of the portfolio, the two-point mixture yields a mean service time of 51.9 h (standard deviation 30.7); being a discrete mixture rather than exponential, it requires M/G/1 rather than M/M/1. Using the Pollaczek-Khinchine formula, we derive a critical-α curve giving the minimum XAI gain needed for stability. Three scenarios are evaluated analytically and cross-checked against simulation (agreement within 3%). An interview-grounded, conservative estimate approximately doubles stable capacity; a full-deployment scenario suggests up to 8.6-fold, but this rests on assumed reductions and is an upper-range illustration. Future work should calibrate the framework empirically, for instance through controlled measurement of Stage-5 review times under varying levels of XAI assistance.