A deep reinforcement learning framework for hybrid electric vehicle energy management: Integrating real-world data augmentation and multi-scale perception

Abstract

To solve the generalization bottleneck and environmental perception limitation of deep reinforcement learning (DRL) in the charge-sustaining (CS) stage of hybrid electric vehicles, this paper proposes a novel adaptive hierarchical energy management strategy, which combines real-world data enhancement and multi-scale perception. Aiming at the problems of over-fitting and short-sighted decision-making commonly existing in traditional strategies, this study constructed a fresh enhanced training set covering all-round driving cycles based on real driving data, thus breaking through the training restrictions brought by standard driving cycles. In addition, the historical average speed window is introduced as the enhanced state, which enables the strategy to capture the macro traffic flow trend. In terms of control architecture, a bi-level coupled mechanism based on soft actor-critic (SAC) and equivalent consumption minimization strategy (ECMS) is designed. Experimental results indicate that across multi-type driving cycle tests, the final SOC deviation of the agent based on the augmented training set is maintained within ±2%. Under identical operating conditions, the augmented agent incorporating a 30 s average velocity observation achieves a 43.55% reduction in the standard deviation of SOC fluctuations compared to its counterpart without such observation. While ensuring battery SOC robustness, the proposed strategy improves fuel economy by 5.2% on average across various driving cycles compared to adaptive ECMS (AECMS).