FALCON-MASAC is presented, a safety-integrated multi-agent reinforcement learning framework that decomposes this task into four complementary layers: a hierarchical leader-follower paradigm that pairs a pre-trained virtual leader with followers learning a distributed cooperative policy, and a bypass-side commitment coordination layer that suppresses trajectory chattering and mitigates crossing conflicts among neighboring UAVs.
Abstract
Safe cooperative navigation of unmanned aerial vehicle (UAV) formations through three-dimensional environments with dense obstacles, dynamic threats, and unstructured terrain requires jointly addressing goal-directed navigation, formation keeping, hard safety constraints, and formation-topology consistency. We present FALCON-MASAC, a safety-integrated multi-agent reinforcement learning framework that decomposes this task into four complementary layers: (1) a hierarchical leader-follower paradigm that pairs a pre-trained virtual leader with followers learning a distributed cooperative policy; (2) a dual-scale entity-risk attention encoder (DSER-AE) that structures heterogeneous observations into entity-scale and risk-scale semantic tokens and fuses them through intra-scale self-attention and inter-scale full-sequence self-attention over the concatenated tokens; (3) a safety shield built on signed distance functions and high-order control barrier functions (SDF-HOCBF) that maintains the conditional safety certificate while the robust quadratic program remains feasible and the high-order admissibility conditions hold, and switches to a bounded best-effort fallback otherwise; and (4) a bypass-side commitment coordination layer that suppresses trajectory chattering and mitigates crossing conflicts among neighboring UAVs. The safety analysis explicitly quantifies the SDF linearization error and dynamic-obstacle prediction uncertainty and describes the bounded fallback used when certified execution is unavailable. Under the centralized-training-with-decentralized-execution (CTDE) paradigm, simulation experiments show that FALCON-MASAC substantially outperforms representative baselines: it attains a 97.3% success rate (21.7 percentage points above the strongest pure-MARL baseline and 13.0 percentage points above a post-hoc CBF-RL variant), a success-conditional steady-tracking formation error of 1.524 m, and a minimum clearance of 2.732 m. The framework also generalizes well along two out-of-distribution dimensions: obstacle density and dynamic-obstacle speed.
Cooperative navigation of multiple unmanned aerial vehicles (UAVs) in disaster search-and-rescue scenarios is challenging due to dense obstacles, partial observability, and strong inter-agent coupling, which often result in path conflicts, collision risks, and limited policy generalization. To address these challenges, this paper proposes a Multi-Agent Deep Deterministic Policy Gradient framework with a Graph-Attention-based Staged Actor (GS-MADDPG). Under a centralized training and decentralized execution paradigm, GNNs are employed to model local interaction relationships among UAVs, enabling effective information aggregation and cooperative decision-making under partial observability. Furthermore, the Actor network is decomposed into perception, goal-guidance, and feature fusion subnetworks, allowing hierarchical decoupling and coordinated integration of local obstacle avoidance behaviors and global navigation objectives. Simulation results conducted in a complex three-dimensional urban environment demonstrate that, compared to traditional methods, GS-MADDPG improves the navigation success rate, robustness, and generalization performance. When the obstacle density reaches 50% and the number of UAVs increases from 2 to 10, the navigation success rate of GS-MADDPG is approximately 40% higher than that of the benchmark algorithm; even in cases with higher obstacle density, GS-MADDPG still achieves a relatively high success rate. This verifies its effectiveness in multi-UAV cooperative navigation for search and rescue tasks.
Li Tan, Hai-Xia Zhao, Jia-Qin Chai et al.· Unmanned Systems· 0 citations
A multi-agent deep reinforcement learning framework that addresses issues through coordinated exploration, demonstration exploitation, safe curriculum scheduling, and structure-aware generalisation is proposed, demonstrating strong performance in collaboration success rate, navigation robustness, zero-shot cross-scenario generalisation, and dynamic environment adaptability.
SLM-A* is designed as a language-native planning agent that can interoperate with multi-agent LLM frameworks as a callable planning sub-module, thus providing an architectural capability not available to conventional planners that operate on numerical graph representations.
This work provides a feasible technical pathway and reproducible evaluation benchmark for the collaborative deployment of lightweight LLM planner, sub-goal guidance, sensor observations, cooperative reward, and reward shaping components and quantifies the indispensability of the LLM planner.
Yuting Cao, Zheng Zhao, Jiekai Wu et al.· Journal of King Saud Univers...· 0 citations
Coordinating heterogeneous aerial and ground vehicles with fundamentally different dynamic time-scales is a longstanding challenge in multi-agent robotics. This paper proposes a distributed reinforcement learning (DRL) framework for cooperative Unmanned Aerial Vehicle (UAV) and Unmanned Ground Vehicle (UGV) formation control that explicitly accounts for multirate dynamics: UAVs execute high-frequency control at 10 Hz while UGVs operate at 2 Hz. Each agent maintains a local actor-critic network trained via a variant of MultiAgent Proximal Policy Optimisation (MAPPO) augmented with a consensus-based communication graph and a multirate synchronisation module that bridges the temporal mismatch between the two vehicle classes. A shaped reward formulation penalises formation deviation, inter-agent collisions and communication dropout simultaneously. Simulation experiments In a physicsaccurate Gazebo/ROS2 environment with two UAV leaders, two UAV followers, and two UGVs demonstrate a task success rate of 96.4%, a formation accuracy of 97.1%, and a collision rate of only 0.8%, outperforming MADDPG, MAPPO (singlerate), centralised DDPG and PID baselines by margins of up to 24.9 percentage points. The framework is further validated in a disaster-response scenario with a dynamic obstacle fields confirming its suitability for real-world heterogeneous multirobot missions.
Sankara Rao Allada, P. Palpandi, J. G. B. Patturose et al.· International Conference on...· 0 citations
Coordinating multiple unmanned aerial vehicles (UAVs) for inspection, delivery, and search-and-rescue requires routing strategies that are both efficient and safety-aware under partial observability. Conventional optimisation methods and single-level reinforcement-learning policies often struggle when fleet size and obstacle density increase, since local collision avoidance and long-horizon route coordination must be learned within the same primitive-action policy. We reformulate multi-UAV path planning as a hierarchical reinforcement-learning problem and propose a two-tier controller in which a recurrent PPO+LSTM Manager selects coarse waypoint-level decisions, while a shared recurrent PPO+LSTM Worker executes local collision-aware movements. The Worker is pre-trained and then frozen, allowing the recurrent Manager to learn macro-level coordination over a reduced decision horizon. Under idealised deterministic grid assumptions, we show that enlarging the waypoint dictionary weakly expands the feasible macro-action set and can preserve safe alternatives when such alternatives exist. Experiments on controlled \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$10\times 10$$\end{document} grid scenarios with 2–6 UAVs and 20–40 % obstacle coverage show that hierarchical PPO+LSTM achieves 90–93 % mission success and collision rates of at most 10 %, outperforming flat PPO and PPO+LSTM baselines. The evaluation further includes reward-coefficient sensitivity analysis, high-resolution path-planning visualisations, a hard bottleneck feasibility illustration, MAPPO-style diagnostic convergence curves, and rollout-loop runtime diagnostics. Statistical validation using Welch’s t-tests supports the significance of the main performance improvements. The results indicate that separating waypoint-level strategy from recurrent local execution improves mission reliability and collision avoidance in the tested grid environments, while larger random-map benchmarks, fully controlled MAPPO/QMIX comparisons, and continuous 3-D simulation remain important future work.
