Prioritized Adaptive Dual-Critic Deep Reinforcement Learning Approach for Electric Vehicle Charging Optimization
Abstract
The large-scale integration of electric vehicles (EVs) poses significant challenges for the energy management of charging infrastructure. In particular, electricity price fluctuations and diverse user demands require control strategies to effectively operate in complex environments. To tackle these issues, this article proposes a priority-adaptive dual-critic deep deterministic policy-gradient (DDPG) method to dynamically optimize charging strategies, aiming to minimize operational costs while maximizing user satisfaction. To improve sample utilization efficiency and accelerate convergence, this article designs a prioritized experience replay mechanism that dynamically adjusts the sampling probability through temporal difference errors. In addition, a hybrid prediction module integrating convolutional neural networks, long short-term memory networks, and self-attention mechanisms is constructed to accurately capture the temporal features of electricity prices and PV data, providing reliable state information for agent decision-making. To mitigate the Q-value overestimation problem, an adaptive weighting dual-critic network architecture is proposed to enhance algorithm stability by dynamically adjusting the weights of dual Q-value estimates. Experimental results show that compared with DDPG, TD3, and soft actor-critic algorithms, the proposed algorithm exhibits significant advantages in cost reduction and convergence speed, with an average reward increase of at least 8%.