Statistical Drift detection with parametric Gamma–Weibull and adaptive deep learning for malware detection in IoT data streams

Abstract

Machine learning (ML) and deep learning (DL) models for Internet of Things (IoT) malware detection may experience performance degradation in non-stationary data streams due to evolving malware behavior, system updates, and dynamic network conditions. In this study, we present a statistically based drift-adaptive framework that integrates a Gamma–Weibull log-likelihood-ratio CUSUM detector with lightweight, fully automated fine-tuning of LSTM and CNN classifiers. The Gamma–Weibull pairing is adopted not because it is claimed to provide the closest statistical fit under every condition, but because it offers a favorable balance between distributional modeling capability and the analytical tractability required for a closed-form, ARL0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_0$$\end{document}-calibrated decision rule, providing an interpretable alternative to generic distribution-drift detectors; adaptation is performed via pseudo-labeling and clustering only after a statistically significant drift alarm. We evaluate the proposed framework on simulated data streams exhibiting abrupt, gradual, distribution-mismatched, mixed, noisy, and recurring drift, as well as on real-world IoT-23 malware traffic, and compare it with four established drift detectors using repeated Monte Carlo experiments and hyperparameter ablation studies. Furthermore, we performed an extended goodness-of-fit analysis against Lognormal and Pareto alternatives, which shows that although the Lognormal distribution provides the closest unconditional fit to the IoT-23 duration streams, the Weibull distribution yields a better-fitting member of the analytically tractable Gamma–Weibull model underlying the proposed detector. The results demonstrate accurate drift detection with zero false alarms in noise-free simulations, a favorable sensitivity-false-alarm trade-off relative to established detectors, and consistent improvements in classification accuracy and F1-score through drift-aware adaptation that one-sided Wilcoxon signed-rank tests, applied across repeated Monte Carlo and repeated-seed trials, confirm are statistically significant. These findings show that the proposed Gamma–Weibull CUSUM detector captures heavy-tailed distributional shifts in malware-driven IoT traffic, while selective, statistically gated adaptation improves classification robustness under non-stationary conditions, particularly for underrepresented attack classes, albeit with dataset-dependent trade-offs between accuracy and F1-score for the more class-imbalance-sensitive CNN architecture.