A Fusion Framework for Deepfake Video Detection Using Image-Level and Facial Feature Analysis

Abstract

The rapid rise of deepfake technology has raised serious concerns regarding the authenticity of digital media content. This research introduces a hybrid deepfake detection framework that collaboratively combines deep learning and traditional machine learning techniques to improve detection accuracy and robustness. The framework comprises two complementary branches. The first branch employs a CNN-LSTM model to learn spatial-temporal dependencies from video frames, capturing both visual features and temporal patterns associated with deepfakes. The second branch focuses on localized analysis by extracting key facial landmarks such as the eyes, nose, and lips then classifying them using a Random Forest classifier to detect real or fake. The outputs from both branches are integrated using a soft voting mechanism, enabling the system to make a final decision based on the weighted probabilities. This fusion enhances the system's ability to detect a wide range of deepfake manipulations with higher reliability. Additionally, for comparative analysis, the performance of a CNN-Gated Recurrent Unit (CNN-GRU) model, RF and a Vision Transformer (ViT-LSTM) is evaluated independently. Experimental results show that the CNN-LSTM + Random Forest fusion model achieves the highest accuracy of 97.5%, followed by CNN-LSTM (95.0%), RF(93.0%) CNN-GRU (92.5%), and ViT-LSTM (89.82%). The results demonstrate the superior effectiveness of the proposed fusion approach, where CNN-LSTM effectively models temporal dynamics and Random Forest enhances classification through localized facial feature analysis.