Performance Comparison of Support Vector Machine and Naïve Bayes Classifiers on a Synthetic Dataset: A Comprehensive Statistical Analysis
TL;DR
Under resource limits, NB is the preferred choice even though predictive accuracy does not differ significantly from SVM, and the practical distinction is computational: NB trained 22.5× faster, predicted 12× faster, used 2.9× less memory, and produced a 12× smaller model file.
Abstract
We compare a radial-basis-function support vector machine (SVM) with a Gaussian naive Bayes (NB) classifier on a reproducible synthetic binary task designed for controlled evaluation. The data matrix contains 1,000 samples and 20 features, of which 15 are informative and 5 are redundant linear combinations; class labels include a controlled flip rate of 0.1. Models were fitted in scikit-learn 1.3.0 (SVM: RBF kernel, C = 1.0, gamma = 'scale'; NB: GaussianNB with default variance smoothing) under an 80/20 stratified hold-out and 5-fold stratified cross-validation. On the test partition, NB attained accuracy 89.95%, precision 91.35%, and F1-score 90.48%, while SVM attained 89.45%, 90.48%, and 90.05%, respectively; recall was identical at 89.62%. McNemar's test (p = 0.8231) and Cohen's kappa (NB 0.7984; SVM 0.7882; Δκ = 0.0102) indicate that the two predictors are statistically equivalent in discriminative accuracy. The practical distinction is computational: NB trained 22.5× faster, predicted 12× faster, used 2.9× less memory, and produced a 12× smaller model file. Under resource limits, therefore, NB is the preferred choice even though predictive accuracy does not differ significantly from SVM.