Robustness exceeds LOAO detection for every model for every model, and the gap recurs for every model on CSE-CIC-IDS2018, so a robustness score should not be read as a zero-day score, and the two should be reported separately.
Abstract
Synthetic variants of known attacks, including feature-space adversarial perturbations, are sometimes used as stand-ins for unseen ("zero-day") attacks when evaluating machine-learning network intrusion detection systems (NIDS). We test whether this proxy holds. On the corrected CIC-IDS2017 release we train Random Forest, XGBoost, a 1D-CNN and an FT-Transformer and compare, at a fixed 1% false-positive rate over five seeds, detection under constrained perturbations of known attacks (gradient-based and black-box attacks on 42 attacker-controllable features) with leave-one-attack-family-out (LOAO) detection, where an entire family is absent from training. Detection under the strongest perturbation we tested, with budgets of at most 0.2, stays between 90.9% and 99.7%, yet mean detection of held-out families falls to 76.2% (RF), 40.2% (XGBoost), 25.8% (1D-CNN) and 22.8% (FT-Transformer). Robustness exceeds LOAO detection for every model (gaps of 24 to 72 percentage points, 23 to 66 when robustness is also averaged over families; Holm-adjusted p $\le$ 0.00096), and the two measures agree at most on a coarse ranking of the models. An exact mixed-integer evasion search confirms the tree ensembles'scores. Benign-only anomaly detectors and a simple multi-class open-set baseline do not close the gap. A temporal split on 5 seeds (3 of the 4 models) leaves the gap in place. On CSE-CIC-IDS2018 the gap recurs for every model (16 to 68 points). On these benchmarks and under this threat model, a robustness score should not be read as a zero-day score, and the two should be reported separately. Code and per-seed results are publicly available.
Machine learning-based Network Intrusion Detection Systems (NIDS) remain highly vulnerable to test-time evasion (TTE) attacks, yet existing research relies heavily on legacy datasets and ignores the impact of evaluation scale. This study presents a systematic evaluation of TTE attacks using the Adversarial Robustness T...
Rifa Tasfia, Anitha Chennamaneni, Abhijit Kumar Nag· International Journal of Inf...· 0 citations
Intrusion Detection Systems (IDS) play a vital role in modern networks by monitoring network activities, identifying malicious behavior and mitigating potential cyber threats. Traditional Intrusion Detection Systems are plagued with novel hacking techniques, zero-day attacks and infiltrations. These threats strategical...
Smitha Poyyara Rajan, S. V. Kumar· 2026 International Conferenc...· 0 citations
Machine learning-based Network Intrusion Detection Systems (NIDS) have become critical security infrastructure, but the vast majority of published research reports accuracy only on clean, unperturbed traffic, leaving their robustness to adversarial evasion unexplored. Existing research into adversarial robustness, when...
K. Akash, Abuzar A. Dr. Mohammed· Indian Journal of Computer S...· 0 citations
This work characterizes the complementary relationship between reconstruction-based and robustness-based paradigms in the accuracy–efficiency design space under the evaluated conditions: the former suits compute-unconstrained scenarios while the latter serves latency-constrained deployments.
This study investigates the transferability of adversarial attacks across XGBoost and LightGBM models in IDS and uses Explainable AI (XAI) techniques to identify features that influence model decisions and shows that feature-importance patterns can be used to examine model vulnerability under adversarial perturbations.
Abdlelah Abdlatef, Marwa D. Abdulkareem, Duygu Çakır· International Journal of Inf...· 2 citations
Machine learning-based intrusion detection systems (IDS) are increasingly used in resource-constrained Internet of Things (IoT) environments, yet their robustness is often evaluated against static attacks rather than adversaries that adapt to detection feedback. This paper investigates adaptive port-scan evasion agains...
Logan Andrew North, Priya Sanjay Kaluskar, Shasi Kumar Ramachandran Prabhu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.