Fusion of Frequency-Domain Features and Sequential Dependency for Multimodal Rumor Detection

Abstract

With the rapid growth of social media, multimodal content combining text and images has become a major medium for rumor dissemination. However, existing multimodal rumor detection methods primarily focus on semantic modeling in the spatial domain, making it difficult to simultaneously capture latent image manipulation traces and structural dependencies among image regions. Moreover, frequency-domain features are susceptible to noise during feature extraction, which limits the model’s ability to identify forged content effectively. To address these challenges, this paper proposes a multimodal rumor detection framework that integrates frequency-domain features with sequential dependency modeling. For textual representation, a pretrained BERT encoder is employed to extract contextual semantic features. For visual representation, a spatial–frequency dual-branch architecture is designed. Specifically, the spatial branch combines ResNet34 and a Bi-GRU network to model sequential dependencies among image regions, thereby capturing long-range contextual relationships. The frequency-domain branch employs the Discrete Cosine Transform (DCT) and a residual convolutional network to extract potential frequency-domain anomalies, while a gating mechanism is introduced to suppress noise and enhance the representation of forgery-related features. Furthermore, a cross-modal cross-attention mechanism is employed to achieve bidirectional semantic alignment and model semantic inconsistencies between textual and visual modalities. Finally, an adaptive gated fusion strategy dynamically balances the contributions of different modalities, thereby enhancing the discriminative capability of the fused representation. Experimental results on two benchmark datasets demonstrate that the proposed method consistently outperforms existing baseline methods. On the Weibo dataset, the proposed method achieves an Accuracy of 95.41 ± 0.48% and a Macro-F1 of 95.36 ± 0.50%, while on the Pheme dataset, it achieves an Accuracy of 87.63 ± 0.58% and a Macro-F1 of 85.26 ± 0.82%.