Real-Time Energy Management of PV-ESS Integrated Active Distribution Networks Using Digital Twin-Enabled Deep Reinforcement Learning
TL;DR
The proposed DT-DRL framework establishes a closed-loop cyber–physical architecture in which continuously synchronized Digital Twin states are directly incorporated into a Proximal Policy Optimization (PPO)-based decision-making process to jointly minimize operating cost, voltage deviation, and battery degradation while satisfying network operational constraints.
Abstract
The increasing penetration of photovoltaic (PV) generation and battery energy storage systems (BESSs) has significantly increased the operational complexity of active distribution networks, where real-time energy management must simultaneously address renewable uncertainty, voltage regulation, and battery lifetime preservation. Existing Digital Twin-based energy management approaches primarily support monitoring and visualization, whereas deep reinforcement learning (DRL) controllers are commonly developed independently of real-time system synchronization, limiting their adaptability under rapidly changing operating conditions. To overcome these limitations, this paper proposes a Digital Twin-enabled Deep Reinforcement Learning (DT-DRL) framework for coordinated PV–BESS energy management in active distribution networks. The proposed framework establishes a closed-loop cyber–physical architecture in which continuously synchronized Digital Twin states are directly incorporated into a Proximal Policy Optimization (PPO)-based decision-making process. A multi-objective formulation is developed to jointly minimize operating cost, voltage deviation, and battery degradation while satisfying network operational constraints. Renewable generation and load uncertainties are represented using Monte Carlo-based stochastic scenarios to improve policy robustness under practical operating conditions. The proposed framework is validated on the IEEE 33-bus distribution system and compared with rule-based control (RBC), optimal power flow (OPF), and conventional DRL approaches. Simulation results demonstrate that the proposed method reduces the daily operating cost by 22.2%, decreases the maximum voltage deviation to 0.039 p.u., and achieves more stable BESS operation with lower operational variability under uncertain conditions. Furthermore, the complete Digital Twin synchronization and PPO decision-making process requires only 0.41 s per control interval, satisfying the timing requirements of distribution-level energy management systems. These results demonstrate that the proposed DT-DRL framework provides an accurate, computationally efficient, and practically deployable solution for real-time energy management in renewable-rich active distribution networks.