HDST-CAF: Adaptive CNN Transformer Frequency Fusion for Deepfake Detection Under Visual Degradation

Abstract

Digital media integrity is seriously threatened by deepfake technology, especially under real-world visual degradations such as poor resolution, noise, greyscale conversion, and compression artefacts. Unlike prior hybrid deepfake detectors that focus primarily on clean benchmark datasets, the proposed tri-stream framework is designed to improve robustness under multiple concurrent visual degradations through adaptive cross-attention fusion of spatial, semantic, and frequency-domain features. We propose HDST-CAF, a tri-stream architecture combining a lightweight Vision Transformer for global semantic inconsistencies, a multiscale CNN Feature Pyramid Network for local texture artefacts, and a Frequency Domain Analysis stream for GAN/diffusion fingerprints invisible in the spatial domain. The three streams are fused via bidirectional cross-attention with adaptive gating to dynamically weight complementary spatial and spectral cues conditioned on input degradation severity, trained with a joint main and auxiliary classification loss. On a custom multi-condition dataset of 506 videos (five degradation categories, approximately 5,060 extracted faces via YOLOv11), HDST-CAF achieves 92% overall accuracy, outperforming an identically evaluated ResNet18 baseline (89%) under most conditions, with 97% on greyscale and 96% on expression manipulation. Per-category accuracy ranges from 58% to 97%, with face swapping (74%) and low resolution (58%) remaining the most difficult cases. The proposed cross-attention tri-stream fusion offers a practical framework for deepfake forensics under real-world degradation, achieving a three-point improvement over the ResNet18 baseline in overall accuracy.