Dynamic Spectrum Aggregation and Resource Allocation for Embodied-Enhanced Cognitive UAV Networks Using Hybrid Action Space DRL

Abstract

Uncrewed aerial vehicles (UAVs) identified as agents are promising for future communications and networking due to their flexibility and intelligence. However, UAVs are subjected to the severe spectrum scarcity problem. To tackle this challenge, an embodied-enhanced cognitive UAV network is investigated, where an embodied UAV agent is deployed to autonomously perceive the environment, make adaptive decisions, and execute actions. Then, a dynamic spectrum aggregation and resource allocation problem is formulated to maximize the average sum throughput of the secondary network. Moreover, a hybrid action space deep reinforcement learning (DRL) framework is proposed to enable the embodied UAV agent to make joint discrete spectrum allocation and continuous trajectory decisions. Specifically, the proposed framework decomposes the hybrid policy into parallel continuous and discrete components with a shared state encoder, thereby optimizing the continuous and discrete actions simultaneously. By exploiting the proposed framework, an intelligent joint spectrum allocation and UAV trajectory optimization scheme is developed for the embodied-enhanced cognitive UAV network. Finally, simulation results validate the effectiveness of our proposed scheme, and demonstrate the superior performance in convergence and resource utilization relative to the traditional DRL-based schemes. Moreover, our proposed scheme maintains lower computational complexity and shorter inference time compared to the benchmark schemes.