Comparative Evaluation of Machine Learning Classifiers for SMS Spam Detection with Optimized Feature Extraction

Abstract

The proliferation of mobile communication has led to a significant rise in unsolicited SMS spam messages, posing threats to user privacy, security, and overall mobile experience. Existing rule-based spam filters are increasingly inadequate against evolving spammer tactics, necessitating more adaptive and accurate detection approaches. This study investigates the application of machine learning techniques to text-based SMS spam detection using the UCI SMS Spam Collection Dataset, which comprises 5,572 messages with a pronounced class imbalance of 86.6% ham to 13.4% spam. Three classifiers were evaluated: Multinomial Naive Bayes, Support Vector Machines, and Random Forest, combined with TF-IDF feature extraction and bigram analysis. Hyperparameter optimization was performed via grid search, and model robustness was assessed through 5-fold cross-validation. The Multinomial Naive Bayes classifier with optimized TF-IDF parameters achieved 99% accuracy on the held-out test set, with a spam-class precision of 99.28%, recall of 92.61%, and an F1-score of 95.83%. Cross-validation yielded a mean F1-score of 0.9556, confirming consistent generalization. Feature importance analysis identified promotional terms such as "claim," "prize," and "have won" as the strongest spam indicators, while informal conversational terms were strongly predictive of legitimate messages. These findings demonstrate that lightweight probabilistic classifiers, when paired with effective feature engineering, can achieve near-perfect spam detection performance and offer a practical, interpretable foundation for real-world SMS filtering systems.