Few-shot email spam detection for low-resource languages: A setFit-based approach on Turkish and English datasets

Abstract

Email spam detection in English has been studied extensively under full supervision, but for low-resource languages such as Turkish the largest public dataset is more than an order of magnitude smaller, and every existing Turkish spam detector assumes full supervision. This study asks whether spam can be detected from only a handful of labeled examples per class, and how the leading few-shot approaches compare, on the Enron-Spam and Turkish corpora across three encoder backbones and shot sizes from 2 to 64. SetFit, a contrastive few-shot method, attains a macro-F1 of 0.911 on Turkish and 0.947 on English at 64-shot, within 3.8 and 4.7 F1 points of the three-seed full fine-tuning reference. Against additional baselines (TF-IDF, FastFit, LoRA, and in-context prompting of Mistral-7B and Llama-3.1-8B), in-context LLM prompting is strongest at the lowest shot sizes but gains little from more demonstrations, so the fine-tuned methods match it on English by 64-shot and overtake it on Turkish from 32-shot. SetFit is therefore not uniformly best, but it reaches competitive accuracy with a far smaller model and no generative inference. Purely cross-lingual transfer fails, confounding language and domain shift, whereas bilingual training recovers most of the monolingual performance.