Distribution Network Optimization with Aggregation and Reinforcement Learning Under Massive Distributed Resources Integration

Abstract

The integration of large-scale distributed energy resources (DERs) into distribution networks (DNs) brings challenges to the effective control of DNs. In traditional approaches, mathematical or reinforcement learning (RL)-based solution algorithms are commonly used. However, the exponential increase in the number of DERs reduces the effectiveness of these strategies. Mathematical methods struggle to cope with the dynamic uncertainty caused by the high penetration of renewable energy, while RL algorithms relying on global data training may violate multi-agent privacy protocols. This paper proposes a DNs cooperative optimization method based on resource aggregation and RL. To reduce optimization dimensionality and ensure the privacy of resource data, a dynamic aggregation strategy is employed to aggregate a large number of distributed energy resources into aggregated entities, and the adjustable active–reactive power boundaries of each aggregated entity are derived. To fully exploit the regulation capability of DNs, data centers (DCs), as novel devices, are considered as flexible loads. To improve the convergence speed of model training and decision-making accuracy, evolution strategies (ES) and prioritized experience replay (PER) are integrated into the Soft Actor-Critic (SAC) algorithm, respectively. The proposed method is validated on the IEEE 33-bus and IEEE 123-bus systems. The results demonstrate the effectiveness and superiority of the proposed method in ensuring the secure operation of DNs.