Sep 2026· IEEE Transactions on Image Processing· Vol 35, pp. 10588-10599· 0 citations· 54 references
MedicineComputer Science
Abstract
Out-of-context (OOC) misinformation, where genuine images are paired with misleading text, poses substantial societal risks due to its deceptive narratives. Prior efforts primarily assess image–text consistency but often lack explainable judgments, limiting their usefulness for forensic validation. Recently, Multimodal Large Language Models (MLLMs) show promise with vast world knowledge and strong reasoning capabilities, yet their judgments in OOC detection remain unreliable. On one hand, their static internal knowledge quickly becomes outdated in fast-moving news domains, and compensating with externally retrieved evidence often introduces noise and conflicts with their internal priors. On the other hand, the absence of task-specific cognitive calibration leads to hallucinations, whereas instruction tuning introduces such specialization only at the cost of vast curated data, rendering it impractical for dynamic, real-world scenarios. In this paper, we propose $\text {IN}^{3}$ SIGHT, a cognitive framework that emulates expert forensic reasoning through three synergistic stages: Epistemic Inspection, which calibrates the epistemic state of an MLLM by task-aware prompt optimization and discrete confidence stratification, yielding a stable epistemic anchor without model tuning; Semantic Investigation, which audits multimodal evidence by enforcing event-level identity and deriving a relevance-aware external assessment under conditional visual anchoring, distilling retrieved evidence into claim-aligned snippets and summaries; and Dialectical Introspection, which conducts multi-fact reasoning by dynamically adjusting the scope of consulted evidence according to evidential certainty and produces the final consistency verdict. Experiments show that $\text {IN}^{3}$ SIGHT markedly improves zero-shot MLLM performance by 15.0 percentage points without parameter updates and achieves competitiveness with or even surpasses instruction-tuned methods. This indicates that imposing structured cognitive constraints and evidence-grounded reasoning on MLLMs offers a principled pathway toward reliable misinformation forensics.
Tampered Text Detection (TTD) is essential for safeguarding document authenticity in security-critical workflows. Existing expert models are effective at capturing subtle manipulation traces but often generalize poorly across diverse document domains, while Multimodal Large Language Models (MLLMs) offer stronger semant...
This paper proposes a novel text-based perturbation method, Natural Textual Unlearnable Examples (NTUE), which can effectively mislead LVLMs by subtly manipulating the image label, establishing a shortcut between an identity of interest and a separate target, without introducing factual misinformation or affecting read...
Chen-Yu Yi, Jia-Wen Zong, Sami Hormi et al.· 0 citations
Hallucinations in Large Language Models (LLMs) remain a major challenge, as they can produce misleading, factually incorrect, or unverifiable information, thereby undermining their trustworthiness and reliability. However, reliable ground-truth labels remain a key challenge in evaluating hallucination detectors. Recent...
Rounak Sen, Divya Kumari, Deeksha Varshney· 0 citations
Multimodal large language models (LLMs) are increasingly used for structured extraction from financial documents, but their behavior under visual ambiguity is poorly understood. We study CARLAR (Courtesy Amount Recognition and Legal Amount Recognition) influence: a model returns the same amount for both cheque fields e...
A causal stage-intervention protocol that overwrites individual stages while holding the downstream task fixed is introduced, finding that grounding is the dominant source of downstream error, with roughly four times the causal impact of corrupting visual observations.
Shu-Zhi Gong, F. Sun, Yuansan Liu· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.