Performance and Explainability of Open-Weight Large Language Models for Spam Email Detection

Abstract

Despite the advancements made by researchers, spam emails remain one of the biggest challenges in the field of cybersecurity. Spam emails can serve as phishing emails or carry viruses that compromise the security of an organization's system. Current detection techniques depend on supervised learning or rely on cloud-based services, which can compromise user data privacy and affect implementation flexibility. This paper evaluates the capability of five large language models (LLMs) in zero-shot spam email classification. The models used in this study include llama3.1:8b, deepseek-r1:8b, gemma3:4b, falcon3:7b, and mistral:7b. In addition to predicting whether the email is spam or not, the LLM was also asked to generate an explanation of its prediction in natural language form. The experiments were conducted on two benchmark datasets: the Ling and TREC2007 datasets. In terms of performance, llama3.1:8b outperformed other LLMs when evaluated on the TREC2007 dataset (98.78% accuracy) and deepseek-r1:8b had the best performance on the Ling dataset (98.79%). The results show that open-weight LLMs can achieve competitive spam detection performance in a local, privacy-preserving environment without any fine-tuning.