A Literature Review on Multimodal Misinformation Detection with External Knowledge Augmentation
TL;DR
When external knowledge is useful and what controls are required for reliable and interpretable multimodal misinformation detection are clarified, with particular emphasis on three unresolved issues: noisy or irrelevant knowledge, leakage caused by label-bearing descriptions, and the gap between semantic alignment and genuine fact verification.
Abstract
Social media misinformation is increasingly presented through combinations of headlines, body text, images, comments, interaction traces, and source metadata. This shift complicates detection because a misleading post may arise not from an explicitly false sentence, but from an authentic image reused in another event, a weakly related headline, a satirical publishing context, or missing background information. Recent multimodal and vision-language models improve the joint representation of textual and visual signals, yet most of them still operate mainly on evidence contained in the sample. They therefore have limited access to factual context, entity relations, or information about the platform and source. External knowledge augmentation offers one way to address this gap by adding knowledge-graph facts, retrieved evidence, event descriptions, entity links, or source-context information. This review examines how such knowledge is represented and combined with multimodal features, with attention to commonly used datasets, fusion strategies, and evaluation practices. Particular emphasis is placed on three unresolved issues: noisy or irrelevant knowledge, leakage caused by label-bearing descriptions, and the gap between semantic alignment and genuine fact verification. The discussion is intended to clarify when external knowledge is useful and what controls are required for reliable and interpretable multimodal misinformation detection.