Cross-Domain Generalization Failure in Lightweight Intrusion Detection Models for IIoT Networks

Generalizing across Networks: Evaluating Model Transferability for Intrusion Detection

: As cyber threats continue to evolve, attackers improve their techniques to optimize the effectiveness of their malicious activities. In response, there has been a significant increase in the implementation of Machine Learning (ML) and Deep Learning (DL) in Network Intrusion Detection Systems (NIDS). Choosing high-quality, representative datasets is a critical step in improving detection accuracy; however, many datasets have been available for a long time, some for over two decades, raising questions about the ability of ML and DL models to generalize across different and possibly outdated network scenarios. This study evaluates the generalization capability of models such as LGBM, RF, XGB, and LSTM, particularly in identifying previously unseen attacks. A significant challenge addressed is the inconsistency introduced by different flow exporters, which compute features differently, affecting model reliability. To mitigate this, HERA is used to rebuild datasets directly from original PCAP files. Additionally, we investigated the impact of feature selection on generalization and examined how performance changes when combining different datasets. When evaluated on alternative datasets, however, their behavior diverges, and in some cases, similar traffic patterns allow for reasonable generalization, while in others, the uniqueness of the dataset leads to significant performance drops.

A domain-agnostic explainable framework for network attack detection across diverse traffic datasets

An explainable deep learning framework evaluated across multiple heterogeneous cyber attack datasets, including Kitsune, Server-Based network data,enterprise logs, and Malware Traffic datasets, demonstrating the effectiveness of the proposed framework in handling heterogeneous network traffic while providing interpretable insights into model predictions.

Detecting adversarial evasion in deep learning intrusion detection systems using explainable AI

Deep learning based network intrusion detection systems (IDS) can achieve strong traffic classification performance, but their resilience to adversarial manipulation remains a critical concern. This study evaluates the adversarial robustness of Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models in a multiclass intrusion detection setting using the Train_Test_Network dataset with ten traffic classes. The models were trained on true sliding flow-window sequences under a unified preprocessing pipeline to support fair comparison. Adversarial robustness was first assessed under a white-box Fast Gradient Sign Method (FGSM) setting and then broadened through additional FGSM and Projected Gradient Descent (PGD) stress testing. SHapley Additive exPlanations (SHAP) were further used to analyse explanation instability under clean and adversarial conditions, and explanation-drift features were evaluated as a secondary adversarial detection signal. Under clean evaluation, both models achieved strong and nearly identical performance, with accuracies of 0.9614 for LSTM and 0.9615 for GRU and weighted F1-scores of 0.9597 and 0.9598, respectively. Under the main FGSM condition, performance declined substantially: the LSTM achieved adversarial accuracy of 0.6094 and weighted F1-score of 0.6290 with an evasion rate of 37.38%, while the GRU achieved adversarial accuracy of 0.5130 and weighted F1-score of 0.5690 with an evasion rate of 47.02%. The broader robustness sweep showed that iterative PGD exposed stronger fragility than FGSM alone. SHAP analysis indicated that adversarial perturbation altered both prediction outcomes and local explanation structure. A learned explanation-driven detector improved over the rule-based baseline, while larger-scale validation confirmed that explanation drift remained informative, though not perfectly separable, at broader scale. Overall, the results show that strong clean performance does not imply adversarial robustness, and that explanation drift provides a useful auxiliary signal for adversarial monitoring in recurrent IDS models.

Multi-Class Network Intrusion Detection: Machine and Deep Learning Benchmark with Live Stream Deployment

BHM-IDS: Behavior-Driven Hierarchy and Multi-Dataset Training for Cross-Dataset Generalization

Digital infrastructures are increasingly exposed to diverse and evolving cyber threats, highlighting the need for robust intrusion detection systems (IDSs). Although machine learning (ML)-based IDSs have achieved strong performance, most existing frameworks are still developed and evaluated mainly under intra-dataset settings, providing limited evidence of their ability to generalize across unseen environments. Moreover, few studies go beyond simply reporting cross-dataset performance to propose dedicated mechanisms for improving generalization. To address this limitation, we propose BHM-IDS, a three-stage hierarchical intrusion detection framework that combines behavior-driven hierarchy with multi-dataset training to improve generalization. The first stage performs binary detection of benign versus malicious traffic, while the second stage classifies malicious traffic into two behaviorally distinct groups: the first corresponding to flood and exhaustion attacks and the second to infiltration and exploitation attacks. The final stage performs fine-grained attack classification through two specialized multi-class classifiers. To expose the framework to more diverse attacks, CIC-IDS2017 is enriched with CIC-DDoS2019 during training, while CSE-CIC-IDS2018 is used as an external test dataset to evaluate generalization. The cross-dataset validation results yielded stage-wise accuracies of 0.93, 0.96, and 0.99, respectively, while the complete end-to-end framework achieved a weighted recall of 0.93. Recall values ranging from 0.76 to 1.00 were obtained for several major classes, including benign traffic, Patator, DoS, and DDoS, although limitations remained for certain attack categories, particularly Web Attack. Overall, the proposed framework demonstrated promising and competitive performance compared with simpler frameworks and existing state-of-the-art approaches. These findings highlight the potential of combining behavior-driven hierarchical classification with multi-dataset training to improve cross-dataset generalization in IDSs.

Bridging Training–Deployment Gap in Intrusion Detection With Source-Free Domain Adaptation

Machine learning (ML)–based intrusion detection systems (IDS) frequently degrade when deployed across heterogeneous networks due to domain shifts in traffic composition and monitoring configurations. Conventional domain adaptation (DA) methods mitigate this issue by aligning source and target distributions, but they often rely on retaining source-domain data at deployment—an impractical requirement that undermines operational scalability and reusability. To address this gap, we propose TRANSFA-IDS (Transformer Source-Free Adaptation for IDS), a lightweight source-free adaptation framework that recalibrates a source-trained IDS using only target traffic data. TRANSFA-IDS converts tabular flow records into structured RGB image embeddings and employs a compact Vision Transformer with a Deep Support Vector Data Description (Deep-SVDD) head to learn transferable normal representations. At deployment, adaptation is performed by fine-tuning only the last transformer block on a small target buffer, realigning target representations without retraining or access to source data. Experiments on cross-dataset transfer between CIC-IDS-2018 and UNSW-NB15 show that TRANSFA-IDS achieves AUROC of 0.9177 and 0.9071 in the two transfer directions, reduces target-domain benign false positives by over 60% relative to the same source-pretrained model deployed without source-free adaptation, and adapts substantially faster than supervised and unsupervised DA baselines while using at most 20% of the target-domain data. These results indicate that source-free adaptation can achieve both strong detection performance and a practical deployment-oriented design, with cross-benchmark evidence of scalable adaptation across heterogeneous network environments.