REMP: A transformer with role-specific experts and multi-scale positional encoding for autonomous multi-UAV air combat

Abstract

Autonomous decision-making in multi-UAV air combat involves high-dimensional state spaces, distinct tactical roles, and variable numbers of agents. While multi-agent reinforcement learning frameworks have shown promise, they still face three primary challenges in complex aerial engagements: (1) feature entanglement caused by shared encoders that cannot separate cooperative and adversarial representations; (2) spectral bias, which limits the extraction of high-frequency spatial features required for precise maneuvering; and (3) scalability constraints, as fixed-input architectures struggle to accommodate variable agent populations. To address these limitations, we propose REMP, a Transformer-based framework that integrates Role-Specific Experts with Multi-Scale Positional Encoding. The Role-Specific Expert module uses separate encoders for the ego agent, allies, and enemies to disentangle latent representations, thereby mitigating gradient interference and feature homogenization. The Multi-Scale Positional Encoding module uses Fourier feature mapping to alleviate spectral bias and capture fine-grained 3D spatial features that are critical for tactical maneuvering. These components are embedded in a permutation-equivariant Transformer backbone to support relational reasoning across variable numbers of agents. Experiments in 5-vs-5 engagements against a rule-based expert demonstrate that REMP significantly outperforms existing baselines across all key metrics, achieving a 90.7% win rate, a 62.3% wipe-out rate, and a loss-exchange ratio of 0.21, while also exhibiting promising scalability and generalization capabilities.