Skin Disease Detection using a Hybrid CNN and Vision Transformer Model
Abstract
Skin disease classification through deep learning has emerged as an important research field because of its ability to facilitate the accurate and timely medical diagnosis. A hybrid deep learning model combining Convolutional Neural Networks (CNN) and a Vision Transformer (ViT) is introduced in the present study to classify skin diseases automatically from dermoscopic images. Images selected are dermoscopic for their ability to provide detailed information about the boundary, distribution of pigmentation, and texture characteristics of lesions, key elements for accurate diagnosis. The experimental dataset is made up of 2000 dermoscopic images divided into 1400 images for model training and 600 images for testing purposes. The proposed framework is compared with the standalone CNN model, with the same experimental settings. Balanced sampling is combined with data augmentation techniques in order to reduce the impact of class imbalance within different categories of skin diseases during the training procedure. Common metrics used to evaluate the effectiveness of models are accuracy, precision, recall and F1-score. The experimental results have shown that the proposed Hybrid CNN–ViT model achieves 91.7% accuracy, which is higher than 84.1% accuracy obtained by the standalone CNN model. CNN-based local feature extraction is combined with the global contextual representation ability of the Vision Transformer, achieving the improved performance is due to the feature fusion between the two, which enables the model to have a better discrimination of visually similar skin lesions. The integration constitutes the main novelty of the proposed work. Moreover, the model is tested with the full test data to obtain the constant and reliable evaluation of the model. The hybrid architecture consumes more computational resources than a conventional CNN model, but the results show that it can achieve better accuracy in skin disease classification, strength, and reliability when detecting skin diseases automatically.