Bi-Timescale Hierarchical Safe Reinforcement Learning for Coordinated Virtual Inertia and Damping Control in Grid-Forming Converters
Abstract
The increasing penetration of inverter-based renewable generation has reduced system inertia and posed new challenges to frequency stability in modern power systems. Virtual synchronous generator (VSG) control can provide virtual inertia and damping support, but its performance strongly depends on the proper coordination of these parameters under varying operating conditions. Existing adaptive and reinforcement-learning-based methods usually regulate virtual inertia and damping at the same timescale, which may ignore their distinct physical roles and lead to coupled parameter variations. To address this issue, this paper proposes a bi-timescale hierarchical safe reinforcement learning framework, termed BiTS-HSRL-JD, for coordinated virtual inertia and damping control in grid-forming converters. In the proposed framework, a slow-timescale policy schedules virtual inertia according to operating conditions, while a fast-timescale policy adjusts the damping coefficient to suppress transient oscillations. A stage-aware state representation and a safety projection layer are further introduced to improve transient adaptability and enforce practical constraints on parameter bounds and variation rates. The proposed method is validated using a MATLAB/Simulink-based VSG system under strong-grid and weak-grid conditions, power-step disturbances, and load-switching events. Comparative results show that BiTS-HSRL-JD reduces RoCoF, improves frequency recovery, and suppresses oscillations more effectively than fixed-parameter, rule-based adaptive, and single-policy reinforcement learning methods.