A Multi-UAV Cooperative Path-Planning Method for Complex Obstacle Environments

Abstract

Reinforcement learning techniques have been widely applied to multi-UAV cooperative path-planning tasks. However, existing multi-agent reinforcement learning methods are still affected by environmental non-stationarity, cooperation difficulties among agents, and low utilization efficiency of experience samples in complex obstacle environments. These issues often lead to slow convergence and unstable training performance. To address these problems, an Improved Experience Replay Multi-Agent Deep Deterministic Policy Gradient (IER-MADDPG) algorithm is proposed for multi-UAV cooperative path planning. First, a cooperative path-planning model is established under the Centralized Training Distributed Execution framework. Second, a dual-layer replay buffer structure consisting of a global replay buffer and a local replay buffer is designed to preserve both global cooperative information and individual experience. Third, a fusion experience sampling mechanism is introduced by combining prioritized experience replay and random uniform sampling to improve sample utilization efficiency and training stability. Finally, training experiments were conducted in environments with different obstacle configurations to evaluate the proposed method. Experimental results demonstrate that IER-MADDPG outperforms other comparison algorithms in terms of convergence speed, training stability, and path-planning performance.