Synthetic Versus Real-World Traffic: An Empirical Evaluation of Machine Learning–Based DDoS Detection

Abstract

Machine learning-based Distributed Denial-of-Service (DDoS) detection has been widely studied, with many approaches reporting very high detection performance on public benchmark datasets. However, benchmark performance does not necessarily demonstrate operational readiness because synthetic and laboratory-generated traffic may not reflect the noise, overlap, temporal variation, and class imbalance of real cloud environments. This paper evaluates the generalization gap between controlled benchmark traffic and long-term real-world unsolicited traffic. A Random Forest classifier is evaluated using a common experimental protocol on CIC-DDoS2019, BoT-IoT, and a real-world dataset collected for 28 months from a private cloud server. The benchmark datasets maintain approximately 99% accuracy, precision, recall, and F1-score across evaluated class distributions. In contrast, the real-world dataset shows lower balanced performance and a clear recall decline under severe imbalance: accuracy reaches 98.6%, but recall decreases to 84.8% and F1-score to 90.4%. The results show that accuracy alone can overstate practical DDoS detection reliability and that recall and F1-score are essential when evaluating detectors for deployment in cloud and IoT environments.