Development of Hybrid Distilled Bidirectional Encoder Representations from Transformers and Support Vector Machine Model for Phishing Email Detection

Abstract

Phishing email attacks are popular and common cyber security threats because they become more sophisticated and rely on social engineering techniques. The contextual and semantic features present in phishing emails are often missed by standalone machine-learning methods and transformer-based models like Bidirectional Encoder Representations from Transformers (BERT) have high computational complexity. A hybrid phishing email detection framework was created by using Fine-Tuned DistilBERT and Support Vector Machine (SVM) to boost detection accuracy while keeping low computational costs. The phishing email dataset from Elhanas et al. (2024) that includes 18,650 labeled samples of phishing emails was used. Preprocessing of the data has been done before the model building stage, such as text cleaning, removal of unwanted duplicate records, and encoding of labels. The dataset was randomly split into an 80:20 split between train and test. Three models: Standalone SVM model, Hybrid BERT-SVM model, and the Fine-Tuned DistilBERT-SVM hybrid model. The accuracy, precision, recall, F1 score, confusion matrix, ROC analysis, and computational cost were performed to evaluate the model's performance. Experimental results revealed that the Fine-Tuned DistilBERT-SVM had an accuracy of 97.49%, precision of 97.19%, recall of 97.89%, and an F1 score of 97.54%. The model achieved high accuracy compared with Random Forest and XGBoost classifiers and excelled in phishing detection ability by its high recall performance. Furthermore, the DistilBERT achieved about 40% less parameters than the original BERT, which enhanced the computational efficiency and accelerated inference of the model. Because the results of the study suggest that the Fine-Tuned DistilBERT-SVM framework is effective, robust, and computationally efficient, it appears to be a suitable solution for the phishing email detection system.