Controlled, Not Correct A Computer Software Assurance framework for domain-specific regulatory AI agents

Abstract

Current discourse on artificial intelligence in regulatory affairs is organised around a single variable: model accuracy. Benchmarks assert accuracy thresholds, vendors claim accuracy figures, and adopters ask whether the model is correct. This paper argues that accuracy is the wrong variable, and that the regulated question — the one that has governed every other tool in a quality management system for fifty years — is whether the tool's output is subject to adequate control proportionate to its intended use and risk. The paper derives a three-layer human-in-the-loop control architecture from existing regulatory instruments (FDA Computer Software Assurance draft guidance 2022; ISO 13485:2016 Clause 4.1.6; MDCG 2019-11 rev.1; EU AI Act Article 10) rather than proposing a new framework. It then quantifies the architecture using a defect-escape model, and reports a result with direct consequences for how regulatory AI should be evaluated: A model at the 3σ baseline of human expert cognition (93.3% accuracy, 66,800 DPMO), placed under three mature control layers, yields a residual defect rate of 401 DPMO — an effective process yield of 99.960%. This is functionally equivalent to accuracy thresholds currently asserted as unreachable by AI systems. The threshold is reachable. It was never a property of the model. The same model under a single control layer yields 13,360 DPMO — worse than an unvalidated 99% model. The number of control layers, not the accuracy of the model, is the dominant term.