Synthetic Face-Swap Forgeries Detection System: An Ensemble Deep Learning Approach with Explainable AI

Abstract

The growth of Generative Adversarial Networks (GANs), diffusion models, and encoder-decoder architectures has enabled hyper-realistic facial forgeries (deepfakes/face-swaps), posing risks to digital trust, privacy, and information integrity. Since single-backbone detectors cannot capture the diverse artifact signatures of modern forgery methods, this paper presents an enhanced Synthetic Face-Swap Forgeries Detection System that replaces a prior single hybrid Vision Transformer (ViT)-Convolutional Neural Network (CNN) model with a three-model ensemble: (i) EfficientNet-B3, a CNN specialized in local texture artifacts; (ii) FasterViT-2-224, a hierarchical Vision Transformer capturing global semantic inconsistencies; and (iii) EfficientFormerV2-S1, a hybrid CNN-Transformer balancing efficiency and representational depth. Predictions are combined via validation-guided weighted probability fusion using Receiver Operating Characteristic (ROC)-optimized weights. Gradient-weighted Class Activation Mapping (Grad-CAM) provides an explainability layer, producing human-interpretable evidence of forgery for end users and forensic investigators, delivered through a live Gradio web interface showing per-model confidence and composite Grad-CAM overlays. On the Face Forensics++ (FF++), Celeb-DF v2, and Deepfake Detection Challenge (DFDC) benchmarks, the ensemble achieves 99.10% accuracy and 98.8% precision, with strong cross-dataset generalization.