Multi-agent reinforcement learning for DoS mitigation in industrial control systems

Abstract

The increasing frequency of cyber-attacks targeting industrial control systems (ICS) demonstrates the importance of digital resilience for Critical National Infrastructures. Denial of service (DoS) attacks have been shown to be highly effective against ICS environments due to the resource-constrained nature of industrial components and networks, such as programmable logic controllers (PLCs). Reinforcement learning (RL) can be used to address this challenge by dynamically managing and mitigating attack effects through statistical monitoring of underlying system behaviours, rather than using pre-trained data models commonly used in supervised and unsupervised machine learning approaches. In this paper, we propose a multi-agent reinforcement learning (MARL) framework that utilises software-defined networking (SDN) to achieve near real-time mitigation of DoS attacks targeting ICS networks. Specifically, we formulate network states using measurements extracted from industrial network communication protocols such as EtherNet/IP. Moreover, we exploit the scalability benefits of hierarchical learning to provide more granular agent decision making through a composite action space. We integrate an SDN controller with a virtualised OpenFlow switch to monitor network performance measurements such as average packet inter-arrival time to establish state behaviours. Through an emulated ICS network representing a live system, we mitigate the effects of different variants of DoS attacks with differing intensities, and highlight the performance improvements of using MARL compared to a single-agent framework. Our results show that MARL provides DoS recovery rates up to 83% faster than standard single-agent implementations, and improves network recovery rates compared with existing research.