Skip to content
Preprint

Adversarial Robustness Is Not Zero-Day Generalization: A Controlled Evaluation of Flow-Based Intrusion Detectors on Corrected CIC-IDS2017

Oct 2026 · 0 citations · 26 references
Computer Science

TL;DR

Robustness exceeds LOAO detection for every model for every model, and the gap recurs for every model on CSE-CIC-IDS2018, so a robustness score should not be read as a zero-day score, and the two should be reported separately.

Abstract

Synthetic variants of known attacks, including feature-space adversarial perturbations, are sometimes used as stand-ins for unseen ("zero-day") attacks when evaluating machine-learning network intrusion detection systems (NIDS). We test whether this proxy holds. On the corrected CIC-IDS2017 release we train Random Forest, XGBoost, a 1D-CNN and an FT-Transformer and compare, at a fixed 1% false-positive rate over five seeds, detection under constrained perturbations of known attacks (gradient-based and black-box attacks on 42 attacker-controllable features) with leave-one-attack-family-out (LOAO) detection, where an entire family is absent from training. Detection under the strongest perturbation we tested, with budgets of at most 0.2, stays between 90.9% and 99.7%, yet mean detection of held-out families falls to 76.2% (RF), 40.2% (XGBoost), 25.8% (1D-CNN) and 22.8% (FT-Transformer). Robustness exceeds LOAO detection for every model (gaps of 24 to 72 percentage points, 23 to 66 when robustness is also averaged over families; Holm-adjusted p $\le$ 0.00096), and the two measures agree at most on a coarse ranking of the models. An exact mixed-integer evasion search confirms the tree ensembles'scores. Benign-only anomaly detectors and a simple multi-class open-set baseline do not close the gap. A temporal split on 5 seeds (3 of the 4 models) leaves the gap in place. On CSE-CIC-IDS2018 the gap recurs for every model (16 to 68 points). On these benchmarks and under this threat model, a robustness score should not be read as a zero-day score, and the two should be reported separately. Code and per-seed results are publicly available.

View source

Similar papers

Open access Nov 2026

Revisiting adversarial robustness in NIDS: a multi-dataset evaluation of test-time evasion attacks

Machine learning-based Network Intrusion Detection Systems (NIDS) remain highly vulnerable to test-time evasion (TTE) attacks, yet existing research relies heavily on legacy datasets and ignores the impact of evaluation scale. This study presents a systematic evaluation of TTE attacks using the Adversarial Robustness T...

Rifa Tasfia, Anitha Chennamaneni, Abhijit Kumar Nag · 0 citations
Conference Open access Sep 2026

A Systematic Survey of Generative Adversarial Networks (GAN) and Transformer Models for Zero-Day Intrusion Detection

Intrusion Detection Systems (IDS) play a vital role in modern networks by monitoring network activities, identifying malicious behavior and mitigating potential cyber threats. Traditional Intrusion Detection Systems are plagued with novel hacking techniques, zero-day attacks and infiltrations. These threats strategical...

Smitha Poyyara Rajan, S. V. Kumar · 0 citations
Open access Oct 2026

AdvRoNIDS: A Constraint-Aware Framework for Adversarially Robust Network Intrusion Detection via Min-Max Training and Attribution Analysis

Machine learning-based Network Intrusion Detection Systems (NIDS) have become critical security infrastructure, but the vast majority of published research reports accuracy only on clean, unperturbed traffic, leaving their robustness to adversarial evasion unexplored. Existing research into adversarial robustness, when...

K. Akash, Abuzar A. Dr. Mohammed · 0 citations
Open access Sep 2026

PatchGuard-Freq: Zero-Overhead Adversarial Patch Defense via Frequency Detection and Data-Driven Robustness

This work characterizes the complementary relationship between reconstruction-based and robustness-based paradigms in the accuracy–efficiency design space under the evaluated conditions: the former suits compute-unconstrained scenarios while the latter serves latency-constrained deployments.

De-Jie Luan, Cheng-Hua Li, Chun-Jie Zhang et al. · 0 citations
#explainable ai Open access Sep 2026

Estimating adversarial transferability from feature-importance similarity in explainable intrusion detection systems

This study investigates the transferability of adversarial attacks across XGBoost and LightGBM models in IDS and uses Explainable AI (XAI) techniques to identify features that influence model decisions and shows that feature-importance patterns can be used to examine model vulnerability under adversarial perturbations.

Abdlelah Abdlatef, Marwa D. Abdulkareem, Duygu Çakır · 2 citations
#machine learning Preprint Oct 2026

Adversarial RL for Port-Scan Evasion: Attacker Feature Visibility in Edge-Deployed IDS

Machine learning-based intrusion detection systems (IDS) are increasingly used in resource-constrained Internet of Things (IoT) environments, yet their robustness is often evaluated against static attacks rather than adversaries that adapt to detection feedback. This paper investigates adaptive port-scan evasion agains...

Logan Andrew North, Priya Sanjay Kaluskar, Shasi Kumar Ramachandran Prabhu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.