LSTM-PPO: A Hybrid Deep Reinforcement Learning Framework with Asymmetric Reward Engineering for Real-Time Intrusion Detection in IIoT and SCADA Networks

Abstract

The integration of SCADA systems with the Industrial Internet of Things (IIoT) has dramatically expanded the attack surface of critical infrastructure. Traditional intrusion detection systems (IDS) struggle with evolving threats, class imbalance (attack samples often <5%), and real-time constraints (<100 ms). Deep reinforcement learning (DRL) offers a sequential decision-making paradigm that adapts over time. This paper presents a hybrid LSTM-PPO framework that unifies temporal feature extraction (LSTM), synthetic minority oversampling (SMOTE), asymmetric reward engineering, and Proximal Policy Optimization (PPO). The LSTM captures multi-stage attack patterns, SMOTE addresses class imbalance exclusively on training data to prevent leakage, and the asymmetric reward heavily penalizes false negatives (-50) compared to false positives (-10), aligning with industrial safety priorities. PPO ensures stable and efficient policy learning. Extensive experiments on three benchmark datasets (WUSTL-IIoT-2021, NF-UNSW-NB15-v2, WUSTL-SCADA-2018) demonstrate near-perfect detection (up to 100% F1 on WUSTL-IIoT-2021, 99.99% accuracy on NF-UNSW-NB15-v2, 99.96% on WUSTL-SCADA-2018) with sub-microsecond inference latency (≈1 μs per sample on GPU batching, <25 μs for single sample). Cross-validation and ablation studies confirm robustness against overfitting and the contribution of each component. The framework meets real-time industrial requirements and outperforms state-of-the-art supervised and DRL-based IDS. Limitations include binary classification and adversarial robustness, which are left for future work.