Enhancing Virtual Inertia Control in Microgrid Clusters: A Novel Frequency Response Model Based on Deep Reinforcement Learning
TL;DR
The proposed Agent SAC-Reinforcement Learning control was tested on a two-area microgrid cluster, which demonstrated greater frequency stability and a frequency nadir deviation improvement and ROCOF reduction compared to PID-based strategies.
Abstract
Traditional control strategies, such as droop-frequency and PI controllers, often show limited adaptability when the system operates under highly variable renewable generation and load conditions. In order to overcome this limitation, this study proposes the design and simulation of a clustered microgrid supported by reinforcement learning (RL)-based virtual inertia control. Three continuous-control RL algorithms were evaluated: Deep Deterministic Policy Gradient (DDPG), Twin-Delayed Deep Deterministic Policy Gradient (TD3), and Soft Actor-Critic (SAC). The SAC agent provided the most robust training performance, reaching stable convergence after approximately 400 episodes and a final reward close to 86 units after 600 episodes. DDPG presented the second-best behavior, whereas TD3 achieved the lowest final reward, approximately 43 units. The proposed Agent SAC-Reinforcement Learning control was tested on a two-area microgrid cluster, which demonstrated greater frequency stability. Results indicate a frequency nadir of 59.82 Hz and ROCOF of 0.1484 Hz/s, with 6.78% nadir deviation improvement and 37.23% ROCOF reduction compared to PID-based strategies.