AI-Generated Image Detectors Exploit Global Artifacts: Evidence from Inpainting Exchange

Abstract

Modern diffusion-based inpainting enables realistic, localized image manipulation, raising critical challenges for reliable detection. We show that state-of-the-art AI-generated image detectors largely fail to identify \emph{what} was synthesized and instead exploit a \emph{global} artifact introduced as a side effect of the generation pipeline: latent-diffusion inpainting passes the entire image through a VAE encoder--decoder, inducing a subtle but pervasive spectral shift even in unedited regions. We turn this observation into a simple, model-free diagnostic, \emph{Inpainting Exchange} (INP-X), that restores the original pixels outside the edited region while preserving all synthesized content. To study the effect at scale, we build a benchmark of $30$K matched real, standard-inpainted, and exchanged triplets ($90$K images) spanning four datasets and three inpainters. INP-X requires no model access and is a single compositing step, yet it collapses pretrained detectors, commercial APIs, and bias-aware methods toward chance (commercial APIs: $>$$90\%\!\to\!\sim$$55\%$; the bias-aware DDA: $97.6\%\!\to\!58.4\%$). We give formal propositions linking the effect to high-frequency attenuation from the VAE information bottleneck, and we causally isolate it with a factorial battery and a variance decomposition that attributes $97\%$ of the background spectral shift to the autoencoder round-trip alone. A perceptual study ($n{=}153$) shows the shift is also perceptible to humans, whose accuracy falls from $63.8\%$ to $56.2\%$ ($p{=}0.002$): significantly reduced, though not eliminated. Finally, we show that mechanism-driven mitigations (mixed-distribution and localization-supervised training) improve robustness, but detecting INP-X edits remains an open problem. Our benchmark and code are available at: https://github.com/emirhanbilgic/INP-X