Three-dimensional path planning algorithm for UAV based on CL-PER-ID3QN

Abstract

To address the challenges of high-dimensional state exploration, random wind field disturbances, near-obstacle collision risks, and unstable value estimation in UAV three-dimensional path planning under urban low-altitude complex environments, a path planning algorithm named CL-PER-ID3QN is proposed, which integrates Curriculum Learning (CL), Prioritized Experience Replay (PER), and the Improved Dueling Double Deep Q-Network (ID3QN). Firstly, the ID3QN architecture is proposed, incorporating PReLU and the dynamic learning rate to enhance the model’s feature extraction capability and training stability; the Adaptive-Gated Value-Advantage Aggregation (AG-VAA) mechanism is proposed to improve the ability to discriminate action values. Secondly, a three-dimensional grid environment for random wind-field perception is constructed, and the wind-obstacle coupled risk proxy model is proposed, enabling the UAV to approach the target while accounting for energy consumption and safety. Finally, the CL and PER mechanisms are introduced to alleviate the initial exploration difficulties and increase the utilization rate of key experience samples. Ablation experiments verify the effectiveness of various improvement strategies. Comparative experimental results show that, compared with the baseline D3QN algorithm, the CL-PER-ID3QN algorithm achieves a 10.46 percentage-point increase in Success Rate (SR), reaching 83.59%, and a 4.27 percentage-point decrease in Collision Rate (CR).