Detection of AI Generated Faces and Real Human Faces Using Deep Learning-Based Transfer Learning

Abstract

Recent advances in generative models like Style-GAN2 and diffusion models produce highly realistic AI-generated faces, necessitating reliable detection for identity verification and security applications. This study presents a systematic empirical comparison of Vision Transformer ViT-B/16 and EfficientNetV2-B3 via transfer learning for binary classification of real versus AI-generated faces. A combined dataset of 200,000 images was constructed, integrating real faces from CelebA and FFHQ with synthetic faces from StyleGAN2 and modern text-to-image models including Flux, DALL-E 3, and Stable Diffusion XL. Both models were initialized with ImageNet pretrained weights and optimized using a two-stage fine-tuning scheme that progressively unfreezes backbone layers. Performance was evaluated on accuracy, precision, recall, F1-score, ROC-AUC, PR-AUC, and computational efficiency metrics including latency and throughput. We further analyzed per-generator performance, model interpretability via Grad-CAM and Attention Rollout, robustness against image degradations, and cross-dataset generalization. EfficientNetV2-B3 achieved top performance with 0.9986 accuracy and F1-score, slightly outperforming ViT-B/16 (0.9979 accuracy), with both achieving near-perfect ROC-AUC. Both models maintained above 0.99 accuracy under JPEG compression and resizing. However, ViT-B/16 demonstrated superior cross-dataset generalization (0.9474 accuracy) on an external StyleGAN test set, compared to EfficientNetV2-B3 (0.8901). Since EfficientNetV2-B3 uses approximately 6.7 times fewer parameters and achieves 2.6 times higher throughput, it is recommended for in-distribution high-throughput detection, while ViT-B/16 is preferred for cross-generator scenarios.