Skip to content
Open access

Revisiting adversarial robustness in NIDS: a multi-dataset evaluation of test-time evasion attacks

Nov 2026 · International Journal of Information Security · Vol 25 · 0 citations · 29 references

Abstract

Machine learning-based Network Intrusion Detection Systems (NIDS) remain highly vulnerable to test-time evasion (TTE) attacks, yet existing research relies heavily on legacy datasets and ignores the impact of evaluation scale. This study presents a systematic evaluation of TTE attacks using the Adversarial Robustness Toolbox (ART) across three modern, domain-diverse datasets: UNR-IDD, CICIOMT-2024, and UAV-IDS-2025. We evaluate a PyTorch-based fully connected neural network classifier and a 1D Convolutional Neural Network (CNN)-based classifier against white-box (AutoAttack, Auto-PGD, Carlini & Wagner) and black-box (HopSkipJump) threat models. Furthermore, we present a scalability analysis by evaluating performance across incremental test-set proportions from 10 to 100%. Results demonstrate severe susceptibility across models: despite achieving clean baseline accuracies up to 100%, Carlini& Wagner and HopSkipJump attacks reduce classification accuracy by nearly 80%. While 1D CNNs exhibit greater structural resilience than the PyTorch classifier, their adversarial robustness consistently deteriorates as the test set volume increases. These findings confirm that the evaluation scale and feature structure strongly influence adversarial performance, underscoring the urgent need for scalable evaluation standards and adaptive defenses in modern NIDS.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.