SCALABLRESOURCE SCHEDULING FOR VIRTUAL POWER PLANTS USING MULTI-AGENT DEEP Q-NETWORKS FOR OPTIMIZED DISTRIBUTED ENERGY RESOURCE MANAGEMENT

Abstract

The increasing integration of Distributed Energy Resources (DERs) within modern power systems requires Virtual Power Plants (VPPs) to adopt intelligent, adaptive, and scalable scheduling mechanisms to ensure efficient, reliable, and economically optimal operation. This research proposes a hybrid reinforcement learning–based framework that combines decentralized Deep Q-Networks (DQN) in the first layer with a Graph-Constrained Multi-Agent Proximal Policy Optimization method enhanced by a Safety Layer (GC-MAPPO-SL) in the second layer. In the first layer, decentralized DQN agents independently regulate local DER behavior such as generation, storage, and consumption using only local observations, including demand forecasts, renewable generation estimates, and state-of-charge values. The decentralized organization provides the ability of the system to be scalable to heterogeneous DERs and optimally achieve adaptability to varying operating conditions. The second layer, GC-MAPPO-SL, optimizes the global operational decisions with graph-based embeddings in the second layer, but a lightweight centralized critic is used, which does not require communication delays or privacy invasion. One layer provides operational restrictions dynamically, such as power balance, line-flow limit, state-of-charge limit, and ramping limit, which are necessary to guarantee feasibility and operational safety in any circumstance. The hybrid framework was tested on a 24-hour simulation horizon with 15-minute intervals with real historical data on the load and renewable generation. Findings indicate good performance, an overall operating cost of $812, a renewable energy use of 77, zero constraint violations, convergence in 120 episodes, as well as scalability of three heterogeneous DERs. Besides, the technique had a load balance variation of 7.25 kW and a quick reaction of 0.3 s/step. These results underscore the efficacy, dependability, and applicability of the framework to real-life VPP scheduling.