Detecting Adversarial Evasion Attacks Against Autoencoder-Based Network Intrusion Detection Systems

Adversarial Evasion in Machine-Learning-Based Network Intrusion Detection: A Systematic Review, Threat Modeling, and Research Roadmap

Machine learning-based Network Intrusion Detection Systems (ML-based NIDS) have become central to modern cyber defense because they can learn complex traffic patterns and detect evolving malicious behavior beyond predefined signatures. However, their reliance on statistical feature representations also exposes them to adversarial evasion, where malicious traffic is modified at inference time to induce misclassification while preserving its operational intent. In network environments, such attacks are shaped by protocol semantics, feature dependencies, traffic timing, and functionality-preservation constraints, making adversarial evasion in ML-based NIDS distinct from adversarial examples in more continuous domains. This review synthesizes 186 studies published between 2018 and 2026—a fast-growing but fragmented body of work—using a Kitchenham-informed systematic literature review methodology. It introduces a lifecycle-aware, NIDS-specific threat model and organizes the literature into three domains—adversarial generation, robustness evaluation, and defense—developing a hierarchical taxonomy within each and conducting a two-tier critical analysis that separates domain-specific limitations from cross-cutting methodological, empirical, and operational constraints. The synthesis shows that much reported evasion success reflects diagnostic feature-space sensitivity rather than executable, deployable adversarial risk, and that many defenses demonstrate component-level robustness rather than system-level resilience. The review also develops a perturbation-realism taxonomy, ranging from feature-level manipulation to executable packet-level attacks, that clarifies when reported success corresponds to deployable risk. Finally, it derives evidence-grounded lessons learned and a future research roadmap for improving realistic evaluation, reproducible reporting, and deployable robustness in adversarial ML-based NIDS.

Detecting adversarial evasion in deep learning intrusion detection systems using explainable AI

Deep learning based network intrusion detection systems (IDS) can achieve strong traffic classification performance, but their resilience to adversarial manipulation remains a critical concern. This study evaluates the adversarial robustness of Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models in a multiclass intrusion detection setting using the Train_Test_Network dataset with ten traffic classes. The models were trained on true sliding flow-window sequences under a unified preprocessing pipeline to support fair comparison. Adversarial robustness was first assessed under a white-box Fast Gradient Sign Method (FGSM) setting and then broadened through additional FGSM and Projected Gradient Descent (PGD) stress testing. SHapley Additive exPlanations (SHAP) were further used to analyse explanation instability under clean and adversarial conditions, and explanation-drift features were evaluated as a secondary adversarial detection signal. Under clean evaluation, both models achieved strong and nearly identical performance, with accuracies of 0.9614 for LSTM and 0.9615 for GRU and weighted F1-scores of 0.9597 and 0.9598, respectively. Under the main FGSM condition, performance declined substantially: the LSTM achieved adversarial accuracy of 0.6094 and weighted F1-score of 0.6290 with an evasion rate of 37.38%, while the GRU achieved adversarial accuracy of 0.5130 and weighted F1-score of 0.5690 with an evasion rate of 47.02%. The broader robustness sweep showed that iterative PGD exposed stronger fragility than FGSM alone. SHAP analysis indicated that adversarial perturbation altered both prediction outcomes and local explanation structure. A learned explanation-driven detector improved over the rule-based baseline, while larger-scale validation confirmed that explanation drift remained informative, though not perfectly separable, at broader scale. Overall, the results show that strong clean performance does not imply adversarial robustness, and that explanation drift provides a useful auxiliary signal for adversarial monitoring in recurrent IDS models.

Adversarial attacks detection for network intrusion detection systems using outlier-filtered principal component analysis

Cybersecurity frameworks are increasingly incorporating machine learning-based Intrusion Detection Systems (IDS) into their security measures. Despite the effectiveness of these systems, they remain susceptible to different forms of attacks that take advantage of their operation; specifically, those that are designed to circumvent their protective mechanisms. For example, modifications made to network traffic can produce “adversarial samples,” which are designed to go undetected. To tackle this issue, two systems based on Principal Component Analysis (PCA) have been proposed for spotting adversarial samples: Standard Principal Component Analysis (SPCA) and Outlier Filtered Principal Component Analysis (OFPCA). SPCA identifies the basic structure of normal network traffic through principal components and detects adversarial attacks by looking at reconstruction errors. A sample is projected onto the principal components and then reconstructed in the original space. The difference between the original and reconstructed features is the reconstruction error. Larger errors can indicate manipulation. OFPCA, on the other hand, is trained only on normal samples after removing outlier data points from the training set. When testing SPCA method using the NSL-KDD dataset, it achieved an AUC-ROC score of 0.97 in detecting FGSM adversarial samples. OFPCA had a higher AUC-ROC score of 0.99 in identifying FGSM adversarial samples. OFPCA performed better than SPCA and other techniques, when tested under different adversarial attacks.

AdvAttack: Adversarial attack against machine learning-based Android malware detection models

Android malware detection has become an essential area of cybersecurity, but the rise of adversarial attacks poses significant risks to the effectiveness of current detection systems. Existing adversarial attack methods face great limitations, especially the imprecision of perturbations generated by random combinations or opaque processes, lack of interpretability. This results in an inability to balance high query efficiency with a high success rate in bypassing detection systems. To address these issues, we propose AdvAttack, a novel black-box adversarial attack framework, which specifically targets static feature-based Android malware detection models. AdvAttack leverages a combination of sensitive function call graph (SFCG) analysis and perturbation insertion techniques to generate highly effective adversarial samples. Our framework introduces a precise perturbation set construction process based on the relationships between sensitive API calls, which ensures high success rates in evading detection while maintaining malware functionality. We evaluated AdvAttack against four advanced detection methods: APIGraph, MaMaDroid, CNN-LSTM and DC-GAN. The results show that AdvAttack can achieve an average attack success rate of over 95% with a query budget of 40. Compared to existing attack methods, AdvAttack strikes a better balance between attack success rate and query efficiency. Additionally, we demonstrate the threat posed by adversarial samples through poisoning attacks on classification models. Finally, the experimental results demonstrate that AdvAttack exhibits superior performance in attacking commercial antivirus (AV) engines.

Defending and mitigating the impact of targeted white-box adversarial attacks in computer network security

Ensuring the robustness of Network Intrusion Detection Systems (NIDS) against adversarial manipulation is an increasingly important problem in the field of computer network security today. Recent breakthroughs in adversarial machine learning have brought to light the vulnerability of NIDS that rely on deep learning models to carefully crafted perturbations, which could significantly degrade the detection and classification performance. In the present investigation, we carry out a systematic empirical investigation of targeted white box adversarial attacks upon a deep learning based multi-class NIDS. A DNN-based intrusion detection model is first evaluated in benign operational condition and then, is subjected to targeted adversarial perturbations introduced using the Fast Gradient Sign Method. To address the observed vulnerabilities, we follow Adversarial Training as a heuristic defence mechanism. All the experiments made use of a subset of the Canadian Institutes of Cybersecurity Intrusion Detection System benchmark data. The proposed NIDS achieves an accuracy of 97.11% in pre-attack conditions. When the model is fed with targeted adversarial perturbations, the classification accuracy decreases to a large degree to 33.27%, 52.15%, 37.89%, and 29.45% when the Benign, DDoS, DoS-Hulk, and PortScan classes are taken as the target, respectively. Incorporating Adversarial Training improves the robustness and increases the accuracy to 73.83%, 76.82%, 75.71% and 71.96% in respective targeted scenarios, although the recovery still depends on the class. Rather than present new attack or defence, this study presents a detailed empirical assessment of targeted adversarial vulnerability and defensive behaviour in a multi-class NIDS setting. The results highlight long-standing, class-specific, robustness gaps and provide insights that could be used to design more robust intrusion detection systems.

Enhancing robustness of deep learning-based malware detection against adversarial attacks