Deep Reinforcement Learning-Based Intelligent Energy Management of Bidirectional Power Converters for V2g Systems

Abstract

Modern developments in electrification have rendered bidirectional Electric Vehicle (EV) charging a challenge due to the need for transactions in Vehicle-to-Grid (V2G) systems, which must address issues such as renewable generation, tariff fluctuations, and distribution grid support while also aiming to prolong device life by minimizing battery degradation. The specific research gap relates to traditional uncontrolled and rule-based Energy Management Systems (EMSs), which are not able to adapt well to nonlinear converter dynamics, uncertain operating conditions, and competing economic and technical targets. This study proposes a Deep Reinforcement Learning (DRL) framework for a three-phase bidirectional Alternating Current-Direct Current (AC-DC) converter, which captures grid-current dynamics, Direct Current (DC)-link regulation, battery State of Charge (SoC), power limits, ramp constraints and degradation-aware rewards. The performance of Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Soft Actor-Critic (SAC) controllers is evaluated in a dynamic simulation environment. SAC converges the quickest and lands at the highest reward, nearly 70 after 600 training episodes, where PPO only manages about 120 and DDPG 150. The DRL-based EMS achieves around 76% normalized energy cost, 63% peak demand, 30% Root Mean Square (RMS) ramp rate, and 57% grid-power deviation reduction compared to the base case and successfully enables a constant DC-link voltage as close as 800 V under bidirectional disturbances.