Aug 2026· Engineering Research Express· Vol 8, pp. 175214· 0 citations· 31 references
Physics
TL;DR
This paper develops a hybrid path-planning method, rapidly-exploring random tree star (RRT*)-deep Q-network (DQN), which combines the fast global search capability of RRT* with the deep learning-based heuristic prediction of a DQN.
Abstract
Path-planning for autonomous unmanned aerial vehicles (UAVs) must be highly efficient, safe, and dynamically feasible for manoeuvring through 3D environments densely packed with obstacles. Traditional heuristic and sampling methods for path-planning often struggle to ensure spatial safety while simultaneously optimizing path quality. They often produce rough and inefficient paths, which lead to instability problems when the vehicle is airborne and higher battery consumption due to longer flight times. We tackle the challenge with this paper by developing a hybrid path-planning method, rapidly-exploring random tree star (RRT*)-deep Q-network (DQN), which combines the fast global search capability of RRT* with the deep learning-based heuristic prediction of a DQN. To demonstrate the merits of our approach, the new system has been tested in four very different 3D space scenarios and its performance compared to seven other leading methods (RRT*, informed RRT*, potential field-based RRT*, heuristic potential optimization RRT*, adaptive grey wolf optimizer, A*, and improved Dijkstra), each method being run 30 times independently with random seeds to allow for a meaningful statistical analysis. The numerical data clearly show that RRT*-DQN outperforms other methods in terms of path optimality almost all the time, with path efficiency reaching up to 97.7% and a trajectory smoothness exceeding 99.86%, while remaining only slightly less competitive in terms of obstacle clearance. Deep reinforcement learning-based inference does bring about a modest computational overhead during the first offline planning stage; however, this is a modest computational overhead, given that it directly produces paths suitable for subsequent flight execution that avoid the kinematically challenging manoeuvres typical of traditional planners. Thus, the RRT*-DQN algorithm offers a very strong candidate for UAVs operating in real-world environments by trading increased offline computational time for improved path quality and potentially more efficient physical execution.
To address the challenges of three-dimensional (3D) flight path planning for Unmanned Aerial Vehicles (UAVs) in complex urban environments, this paper proposes a reinforcement learning approach based on the Deep Q-Network (DQN) algorithm. The method enables intelligent flight path planning within a discretized 3D urban space, dynamically avoiding obstacles in real-time through the UAV's sensory perception. The UAV agent is trained in a simulated 100×100×20 virtual urban environment, with training scenarios categorized into high, medium, and low difficulty levels to progressively enhance the agent's decision-making capabilities. Throughout the training process, a greedy strategy is adopted to balance the exploration of new potential paths and the exploitation of known optimal routes. Once over 80% of the UAV agents successfully reach their designated target points, the training program automatically advances to the next difficulty level. Experimental results validate the effectiveness of the proposed method, demonstrating its superior obstacle avoidance capabilities and exceptional energy optimization performance in complex urban settings.
Yang Li, Xinjie Qian, Yanxiu Wang et al.· International Conference on...· 0 citations
Low-altitude unmanned aerial vehicles operating in mountainous environments require continuous spatial reference paths that satisfy terrain-clearance, altitude, pitch, and curvature constraints. Conventional sampling-based planners can efficiently discover collision-free paths but often produce geometrically irregular solutions, whereas swarm-based optimizers are sensitive to the quality and feasibility of their initial populations. This study proposes a feasibility-first hybrid reference-path planner, termed RRT*-DLPSO, which combines Rapidly Exploring Random Tree Star initialization with a particle swarm optimizer incorporating Differential Evolution and Lévy-flight mechanisms. RRT* first supplies a terrain-aware initial polyline, which is compressed into curvature-selected control points and represented by a cubic spline. Because compression and interpolation can invalidate a collision-free polyline, every reconstructed path is checked using the full set of hard constraints. DLPSO then refines the path using time-varying learning factors, stage-adaptive differential trial generation, and stagnation-triggered Lévy perturbations. Feasible paths are ranked using a weighted objective that considers path length, altitude, pitch demand, and horizontal curvature. Experiments were conducted in three DEM-based mountainous scenarios using 50 independent runs per method. The comparison includes a complete RRT*-PSO ablation chain and an RRT*-STOMP baseline. RRT*-DLPSO achieved a 100% continuously certified feasibility rate and the lowest mean final objective value in all three scenarios with respect to the adopted spline and interpolated DEM model. Relative to the strongest competing method in each scenario, it reduced the mean objective by 6.4%, 5.1%, and 3.9%, respectively. Compared with DLPSO without RRT* initialization, the corresponding reductions were 13.0%, 19.6%, and 32.9%. Paired Wilcoxon tests with Holm correction confirmed statistically significant improvements over all major competing methods. The method also reached the scenario-specific objective thresholds in 86%, 100%, and 96% of the runs. Sensitivity analyses repeated in two DEM scenarios supported nine control points and an RRT* step-size and rewiring-radius coefficient pair of (0.020, 0.040) among the tested settings. These results support RRT*-DLPSO as an effective offline spatial reference-path planner for known static mountainous terrain represented by DEM data.
Hu Liu, Zi-Han Wang, Yongliang Tian et al.· Drones· 0 citations
Urban fire rescue poses severe challenges to the real-time performance and obstacle avoidance capabilities of unmanned aerial vehicle (UAV) path planning. Existing methods (such as A*, RRT, and standard DQN) have problems such as low search efficiency, insufficient obstacle avoidance ability, or slow convergence in complex environments. This paper proposes an improved deep Q-network (DQN) algorithm, introducing a priority experience replay mechanism to improve sample utilization, and designing a composite reward function including arrival reward, step penalty, direction guidance, and safety penalty to guide the UAV to plan safe and efficient flight paths in complex urban environments. A threedimensional grid simulation environment was constructed based on the real fire incident at Chongqing California Garden. Experimental results show that the improved DQN algorithm outperforms the traditional DQN and RRT algorithms. This method provides a feasible technical solution for multi-UAV collaborative rescue in urban fire scenarios.
Rui Qin, Han-Jing Zhou· International Conference on...· 0 citations
In response to the challenges of high-risk close-proximity flight and 3D path planning for unmanned aerial vehicles in mountainous power line inspection scenarios characterized by complex terrain and densely distributed power lines and trees, this paper proposes a risk-aware rapidly-exploring random tree (RRT)* algorithm with reinforcement learning-based adaptive parameter scheduling, termed Q-RARRT*. The proposed method first constructs a three-dimensional environment model that incorporates constraints on mountainous terrain and power-line and tree obstacles, using capsule envelopes as collision representations for these obstacles. On this basis, a risk density field is established, and the cumulative exposure to hazardous obstacles is incorporated into the path planning cost. Subsequently, within the RRT* framework, goal-biased sampling, a risk-guided artificial potential field, and a dynamic step-size strategy are introduced. At the same time, Q-learning is used to adaptively adjust key parameters online based on current states, thereby achieving a better balance between exploration and exploitation. In the path post-processing stage, quadratic Bézier curves are adopted to smooth the generated path and improve trajectory executability. Finally, the proposed algorithm is compared with RRT* and its variants, including Informed-RRT*, Bi-RRT*, and APF-RRT*. Simulation results demonstrate that Q-RARRT* exhibits superior safety, planning efficiency, and environmental adaptability in complex mountainous scenarios.
Jian Cao, Shun-Jie Han, Xing-Guang Cheng et al.· Engineering Research Express· 0 citations
In complex three-dimensional airspace, UAV trajectory planning is subject to stringent real-time constraints and must rapidly generate collision-free, near-optimal and curvature-continuous feasible flight paths within a limited computational window. Although mainstream bidirectional RRT-based algorithms improve the basic search speed through parallel dual-tree expansion, they still suffer from inherent limitations, including blind sampling, fixed expansion strategies and poor environmental adaptability. To address these limitations, this study proposes a Dynamic Hybrid Multi-strategy RRT (DHM-RRT*) algorithm. In the sampling stage, a hybrid strategy combining frontier-density adaptive sampling, Halton low-discrepancy sampling and uniform random sampling is adopted. In the expansion stage, a four-level progressive expansion mechanism is designed, comprising goal-directed expansion, dual-distance scoring tangent-cone obstacle avoidance, improved artificial-potential-field guidance and random fallback expansion. The failure rate of each strategy is estimated online using an exponential moving average, and the expansion probabilities are dynamically and adaptively assigned. After path generation, path quality is further improved through greedy direct connection near the stitching seam and B-spline smoothing. The algorithm was independently evaluated in three MATLAB three-dimensional obstacle environments and compared with the best-performing baseline algorithm in each environment. The proposed algorithm reduced the average path length by 1.78%, 0.42% and 5.49%, respectively, and reduced the planning time by 38.89%, 34.78% and 29.03%, respectively. The simulation results demonstrate that the proposed algorithm provides clear advantages in convergence speed, path length, smoothness and environmental robustness under complex obstacle constraints.
The Improved Dhole Optimization Algorithm is proposed, which enhances the original DOA framework by integrating a logistic-map-based chaotic mapping, a dynamic chaotic perturbation mechanism, and an adaptive stage-division strategy, and significantly outperforms the original DOA in terms of convergence speed and final path optimality.
Wei-Qi Feng, Hongyu Chen, Yu-Jie Fu et al.· Aerospace· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
