A Preregistered Comparison of Acceptance-Gate Architectures for an LLM That Rewrites Its Own Code

Abstract

When a language model is allowed to revise a program repeatedly and a gate decides which revisions to keep, what the gate checks determines what the model does. We report a preregistered experiment in which an LLM optimiser rewrote a small posture-risk classifier for 20 revisions per trajectory under four acceptance architectures: (A) the optimiser's own judgement, (B) a visible score that must not fall, (C) B plus a hidden holdout, and (D) C plus an absolute behavioural envelope, a check that the program's effective decision thresholds lie inside a physically specified range on a fixed sweep of synthetic inputs, with canary cases. A seeded simulator defined the ground truth, so every proposal, accepted or rejected, was scored on exact hidden truth (balanced accuracy on 2,000 frames) that the optimiser never saw. The design, hypotheses and decision rules were frozen at a public git tag before the first confirmatory trajectory ran. In the primary design (4 arms x 10 seeds, Claude Sonnet 5) the visible gate produced a Goodhart curve in 10 of 10 seeds: the visible score rose to 100% in 7 of 10 while hidden truth stayed flat or fell (proxy-truth gap +0.59 points per revision, Hodges-Lehmann 95% CI +0.34 to +0.84). The hidden holdout was protective but not productive. The behavioural envelope was the only architecture under which hidden truth rose (7 of 10 seeds with Sonnet 5; 10 of 10 with Claude Opus 5; exploratory D - B on final truth +7.3 points, CI +3.6 to +11.2), and it changed what the optimiser proposed, not only what it accepted. Three secondary results sharpen the picture: a parametric hill-climber that cannot write code ended with higher hidden truth than the LLM under the visible gate, because it cannot memorise; the stronger model was no safer under the weak gate and more productive under the strong one (tier x gate interaction +5.3 points, CI +0.8 to +9.1); and removing the optimiser's cross-revision notes made gaming faster and honest fixes rarer. All 120 trajectories, 2,400 revisions, prompts, policies and gate decisions are archived (doi:10.5281/zenodo.23223679); code is at https://github.com/nickjlamb/rsi-loop.