Reinforcement learning-based strip allocation optimization for multi-UAV cooperative coverage

Abstract

Efficient allocation of strips and flight paths for multiple unmanned aerial vehicles (UAVs) carrying out cooperative area coverage is a basic issue in aerial robotics, especially when fixed-wing kinematic constraints and multi-pass reentry coordination need to be considered together. This paper introduces a reinforcement learning (RL)-based hierarchical optimization model which simultaneously optimizes strip division, flight route design and multi-UAV reentry coordination for area coverage tasks. The proposed method combines block and overlapping allocation techniques with Dubins path planning to specifically consider the minimum turning radius restrictions of fixed-wing unmanned aerial vehicles. An automatic selection mechanism for an adaptive mode, based on a spacing-ratio criterion which is obtained from the kinematic-geometric connection between the turn radius and area dimensions, determines the best allocation strategy according to the mission parameters. A Soft Actor-Critic (SAC) agent adjusts the strip count. It looks for the best configuration in various tasks. We also have a reentry optimization algorithm. This algorithm focuses on load balancing. With this new objective, it coordinates drones well. The drones stay well-synchronized during multi-pass handovers. Experimental results show that the cooperation of multiple UAVs results in a 79.4% decrease in the total objective time when the number of UAVs changes from 2 to 12, and balanced reentry optimization leads to an up to 22.7% improvement for configurations with poor exit-side distributions. The framework can generalize to different mission parameters, such as large turn radius scenarios, where the adaptive selector automatically implements an interleaved allocation.