A novel temporal attention regularization loss for robust and explainable deepfake detection
TL;DR
Results indicate that ASTAR improves classification performance, temporal attention coherence, and interpretability under the evaluated experimental settings, while further validation using multi-source training and additional unseen datasets remains necessary for broader generalization claims.
Abstract
Deepfakes pose a growing threat to digital media integrity, as many detection methods rely mainly on spatial artifacts and insufficiently model the temporal attention dynamics that distinguish authentic from manipulated videos. This paper proposes Adaptive Selective Temporal Attention Regularization (ASTAR) Loss, a feature-guided temporal regularization loss for robust and explainable video-based deepfake detection. ASTAR stabilizes temporal attention by penalizing unnecessary attention shifts between visually similar consecutive frames while relaxing the penalty during legitimate temporal variations caused by head motion, facial expression changes, blinking, or illumination fluctuation. The proposed framework integrates ResNeXt50 for spatial feature extraction, Bidirectional Long Short-Term Memory (BiLSTM) for temporal modeling, and Multi-Head Attention for selective temporal feature aggregation, with ASTAR applied as an auxiliary training objective alongside the primary Cross-Entropy classification loss. Unlike uniform attention-smoothing approaches, ASTAR adaptively balances temporal coherence and motion flexibility, improving attention stability without suppressing natural video dynamics. Experiments on FaceForensics++, DFDC, and Celeb-DF demonstrate strong intra-dataset performance, achieving 98.49% accuracy and 100.00% AUC on FaceForensics++, 98.18% accuracy and 99.92% AUC on DFDC, and 98.29% accuracy and 99.97% AUC on Celeb-DF. In single-source cross-dataset evaluation, where the model was trained on FaceForensics + + and evaluated on unseen datasets without fine-tuning, ASTAR achieved 93.83% AUC on DFDC and 94.60% AUC on Celeb-DF. These results indicate that ASTAR improves classification performance, temporal attention coherence, and interpretability under the evaluated experimental settings, while further validation using multi-source training and additional unseen datasets remains necessary for broader generalization claims.