Dynamic defense strategies for cyber-physical systems using Stackelberg games and deep reinforcement learning in discrete and continuous time

Abstract

As cyber threats to power grid infrastructures escalate, the urgency of understanding how to protect cyber-physical systems (CPS) has never been greater. These systems, which integrate physical processes with digital control, are increasingly susceptible to sophisticated cyberattacks that can lead to widespread disruption. While most existing defense models function within either discrete or continuous-time frameworks, this research addresses an important limitation in the literature: the limited comparative treatment of both temporal domains within a common strategic framework. This study presents a dual-domain defense framework that combines Stackelberg game theory with Deep Reinforcement Learning (DRL). Rather than merging discrete-time and continuous-time dynamics into a single hybrid model, the two temporal formulations are evaluated as alternative representations of the same CPS security interaction under identical attacker–defender scenarios and metrics, enabling a systematic comparison of strategic decision-making and physical system behavior. The objective of this framework is to facilitate proactive defense decisions that can anticipate and respond to attacks with strategic precision. We conducted extensive simulations using Python to assess the proposed model in both discrete and continuous-time scenarios. Our approach was evaluated through extensive simulations under realistic adversarial conditions to confirm its resilience and cost-effectiveness. Key findings indicate that defender-first strategies in discrete time effectively minimize system damage and alleviate computational burdens, while continuous-time responses, although immediate, demand significantly higher resource investment. This dual-domain solution offers a robust, adaptable toolset for CPS defense by clarifying the trade-offs between temporal abstractions in nonlinear and dynamic environments.