Real-Time Dispatching of a Strongly Constrained Virtual Power Plant Based on an Improved Deep Q-Network Algorithm

Abstract

To address the strongly constrained real-time dispatch problem of virtual power plants (VPPs), this paper proposes an improved deep Q-network (IDQN)-based scheduling method for hourly online decision-making. The studied VPP includes wind power, photovoltaic generation, gas turbines, energy storage, and demand response, where start–stop constraints, ramping constraints, state-of-charge evolution, continuous demand response calls, supply–demand balance, and planning deviations create strong temporal coupling across decision periods. To enhance decision-making performance under such coupled constraints, the proposed method integrates a dueling network, Double deep Q-network (Double DQN), prioritized experience replay, NoisyNet exploration, and an N-step return mechanism, and further introduces a constraint-aware action correction strategy to improve action feasibility during execution. Simulation results show that the proposed IDQN achieves a daily average net profit of 3,894.2 yuan, which is higher than those of the deep Q-network (DQN), deep deterministic policy gradient (DDPG), and twin delayed deep deterministic policy gradient (TD3) methods. Meanwhile, the proposed method maintains zero average daily carbon emissions and the lowest deviation penalty cost among the compared algorithms. The results indicate that the proposed approach can improve the economic performance, low-carbon operation capability, and online dispatch feasibility of strongly constrained VPPs, showing practical value for real-time intelligent energy management.