Interpretation-Aware Image Manipulation Detection

Abstract

Image manipulation has been a common practice since the early days of photography, and has evolved into a sophisticated threat with the rise of powerful image editing tools and generative AI. It can be used to spread misinformation, manipulate public opinion, and cause harm to individuals and communities. In this article, we study the problem of determining whether an image is misleading, i.e. edited in a misleading way. We introduce a risk framework to assess the intrinsic interpretive shift (ITS) of an ordered image pair, which evaluates whether the final edited image changes the interpretation supported by a source reference image. We show our risk framework can be operationalized with the Content Provenance and Authenticity (C2PA) provenance framework. We estimate ITS with a single-call vision-language implementation of our risk framework, which first checks for meaningful visual differences, then tests whether the differences induce interpretive shift, and finally determines whether the shift is fully explained by permissible transformations rather than content-level changes. The system outputs a binary label (potentially misleading or not) together with interpretive clues and a short justification. To test the approach and show that ITS is a measurable proxy for misleading edits, we built a benchmark of 418 ordered pairs, drawn from real-world fact-check cases and meaning-preserving transformations. The best-performing model configuration (gpt-5.4) achieves 88.2% precision, 94.2% recall, and 96.5% balanced accuracy. The code is available at https://github.com/liyu0x/ITS_detection/.