Secrecy Energy Efficiency Maximization for UAV Swarm-Assisted Secure Communication Against an Aerial Eavesdropper via Deep Reinforcement Learning

Abstract

Unmanned aerial vehicle (UAV) swarms have become a promising solution to enhance wireless communication in complicated environments. In this paper, we study a UAV swarmassisted secure communication system, where multiple UAVs cooperatively construct an aerial virtual antenna array (AVAA) to deliver confidential information to ground users under the threat of an aerial eavesdropper. we seek to maximize the secrecy energy efficiency (SEE) through the joint optimization of UAV positions and excitation current weights. To handle this highly non-convex problem, we cast it as a Markov decision process (MDP) and propose an energy-penalty based TD3 (EP-TD3) algorithm. In particular, the sum secrecy rate is adopted as the main reward term, while energy consumption and constraint violations are incorporated as penalty terms to guide the learning process toward a better balance of secrecy enhancement and propulsion energy expenditure. Numerical simulation results reveal that the proposed EP-TD3 achieves better overall performance compared with benchmark schemes regarding average sum secrecy rate, total energy consumption, and average SEE.