Performance Comparison of Support Vector Machine and Decision Trees for the Classification of Handwritten Digits

Abstract

Handwritten digit recognition remains one of the fundamental applications of machine learning and pattern recognition due to its widespread use in banking, postal services, document processing and intelligent information systems. Although deep learning approaches have demonstrated remarkable performance, classical machine learning algorithms such as Support Vector Machine (SVM) and Decision Tree (DT) remain attractive because of their simplicity, computational efficiency and interpretability. This study evaluates and compares the performance of Support Vector Machine and Decision Tree classifiers for handwritten digit recognition using the Modified National Institute of Standards and Technology (MNIST) dataset. The dataset comprises 70,000 grayscale images of handwritten digits (0–9), each with a resolution of 28 × 28 pixels. Image preprocessing involved normalization of pixel values and standard data preparation before model implementation. Both classifiers were implemented using Python and the Scikit-learn library under identical experimental conditions, and model performance was evaluated using accuracy, precision, recall and F1-score. Experimental results showed that the Support Vector Machine outperformed the Decision Tree across all evaluation metrics. The SVM achieved an accuracy of 94.32%, precision of 94.41%, recall of 94.32%, and an F1-score of 94.35%, whereas the Decision Tree recorded 87.36% accuracy, 87.42% precision, 87.36% recall and 87.35% F1-score. The findings indicate that SVM provides superior classification performance for handwritten digit recognition, while Decision Tree offers faster implementation and greater interpretability. The study concludes that SVM is more suitable for applications requiring high recognition accuracy, whereas Decision Tree remains appropriate for applications where computational simplicity and model transparency are prioritized