A 3D path planning method for safe UAV escape in mountainous environments based on a Q-learning parameter-adaptive RRT* algorithm

Abstract

In response to the challenges of high-risk close-proximity flight and 3D path planning for unmanned aerial vehicles in mountainous power line inspection scenarios characterized by complex terrain and densely distributed power lines and trees, this paper proposes a risk-aware rapidly-exploring random tree (RRT)* algorithm with reinforcement learning-based adaptive parameter scheduling, termed Q-RARRT*. The proposed method first constructs a three-dimensional environment model that incorporates constraints on mountainous terrain and power-line and tree obstacles, using capsule envelopes as collision representations for these obstacles. On this basis, a risk density field is established, and the cumulative exposure to hazardous obstacles is incorporated into the path planning cost. Subsequently, within the RRT* framework, goal-biased sampling, a risk-guided artificial potential field, and a dynamic step-size strategy are introduced. At the same time, Q-learning is used to adaptively adjust key parameters online based on current states, thereby achieving a better balance between exploration and exploitation. In the path post-processing stage, quadratic Bézier curves are adopted to smooth the generated path and improve trajectory executability. Finally, the proposed algorithm is compared with RRT* and its variants, including Informed-RRT*, Bi-RRT*, and APF-RRT*. Simulation results demonstrate that Q-RARRT* exhibits superior safety, planning efficiency, and environmental adaptability in complex mountainous scenarios.