Side channels in LLM secret defense: a pre-registered replication of the Paradox of Active Defense across five target models

Abstract

Large language models deployed with a secret in their system prompt defend it only by following an instruction. A pilot study (88 tracked adversarial assessments against ten models and seven prompt designs, March to April 2026) found that the defense leaves traces while it holds: the model recites the secret in its reasoning, its "random" word lists avoid the secret with visible strain, it once retracted a secret word it had just written into a list of example codes, in its reasoning it swapped a near-miss out of a list, and a conditional-release rule was followed by more disclosures (3 of 8 Gemini Pro assessments against 2 of 9, a difference the pilot could not establish). The pilot's evidence was small (one reasoning session, one verified retraction, one to ten runs per condition) and judged by a single non-blind pipeline. This study re-runs the design at fixed n = 20 in each of 15 cells (five targets by three prompt variants, 300 runs) with a two-model attacker, a programmatic breach oracle, deterministic side-channel detectors, a blind offline tribunal, and a human audit with a measured false-negative rate, and tests four pre-registered predictions of the Paradox of Active Defense, the machine analogue of Ironic Process Theory.