A Performance Comparison of Convolutional Neural Networks and Vision Transformers for Malware Detection
Abstract
The detection of malware is a great challenge in cybersecurity because the threat environment keeps on changing. Convolutional Neural Networks (CNNs) are frequently applied to conduct image-based malware detection. On the other hand, Vision Transformers (ViTs) that leverage self-attention mechanisms have emerged as a new deep learning paradigm for image-based malware detection. Thus, it raises the question of which architectural paradigm would perform this task more effectively. The research presented in this paper provides an empirical comparison of ViTs and CNNs for malware family classification. In this paper, six CNNs (namely VGG-16, ResNet-50, DenseNet-121, EfficientNet-B0, RegNetY, ConvNeXt) and six ViTs (namely ViT, DeiT, Swin Transformer V1, Swin Transformer V2, PVT-V2, MaxViT) are adopted for empirical evaluation across two malware datasets, which are the Malware Images (MalImg) and the dataset of Virus-Modified National Institute of Standards and Technology (VirusMNIST). The model performances are assessed using the Macro F1-Score and Accuracy metrics. All twelve models are trained in very strict and fair conditions of the experiment. A statistical test is conducted to compare the classification performances of the ViT and CNN groups, and results are analyzed and discussed. The statistical studies show that ViTs achieve a significantly higher Macro F1-Score than the CNNs on the larger dataset (i.e., Virus-MNIST) while requiring a comparable training duration.