Joint Trajectory Design and Resource Allocation for QoS-Aware Emergency Data Collection in UAV-Assisted WPCNs: A Hierarchical DRL Approach

Abstract

Unmanned aerial vehicle (UAV)-assisted wireless-powered communication networks (WPCNs) have emerged as a promising solution for energy-constrained Industrial Internet of Things systems, where ground sensor nodes are often deployed in harsh and hard-to-reach environments. However, efficient UAV-assisted data collection remains challenging due to limited UAV onboard energy, realistic propulsion consumption, and varying quality-of-service requirements of industrial nodes. This paper investigates an energy efficiency maximization problem in a UAV-assisted WPCN by jointly optimizing the UAV trajectory, hovering altitude, and hybrid TDMA/NOMA resource allocation. To solve the resulting high-dimensional and highly coupled problem, we propose a deep reinforcement learning-driven Hierarchical Energy-Efficient Data Collection scheme, which we name DRL-HEEC. Specifically, a Double Deep Q-Network is employed at the upper level to optimize the UAV trajectory with altitude state inheritance, while a lower-level optimization engine based on the Dinkelbach method, block coordinate descent, and successive convex approximation is developed to handle heterogeneous resource allocation. Simulation results show that the proposed DRL-HEEC scheme outperforms other baselines. In particular, DRL-HEEC improves the system energy efficiency by approximately 9% to 12% compared to other reinforcement learning-based algorithms while ensuring QoS satisfaction on the part of emergency nodes.