Reinforcement Learning Method for Collaborative Multi-Objective Path Planning of IIoT Drones and Unmanned Vehicles in Random Demand and Dynamic Delivery Environments
TL;DR
A guidance-factor-enhanced MADDPG framework for collaborative multi-objective path planning of UAVs and UGVs in random-demand and dynamic delivery environments provides an adaptive reinforcement-learning solution for UAV–UGV cooperative logistics in dynamic IIoT environments.
Abstract
INTRODUCTION: Dynamic Industrial Internet of Things (IIoT) delivery is characterized by stochastic order arrivals, heterogeneous UAV–UGV capabilities, and real-time demand updates, making conventional static routing methods insufficient for collaborative delivery planning. Existing UAV trajectory-planning and truck–drone routing studies usually focus on either single-platform trajectory control or offline vehicle routing, and they rarely integrate stochastic IIoT demand updating, UAV–UGV rendezvous coordination, operational constraints, and multi-objective decision-making into a unified reinforcement-learning framework.
Objectives
To address this limitation, this study proposes a guidance-factor-enhanced MADDPG framework for collaborative multi-objective path planning of UAVs and UGVs in random-demand and dynamic delivery environments.
Methods
Order arrivals are modeled by a Poisson process, while order location, payload, priority, and time-window attributes are generated through multivariate random distributions and updated by IIoT terminals every 5 s. Payload, endurance, speed, rendezvous, and task-sequencing constraints are incorporated into a normalized multi-objective function considering delivery time, delivery cost, demand satisfaction, and UAV endurance loss. A distance–demand–priority guidance factor and a reconstructed reward function are further introduced to alleviate sparse-reward effects and guide agents toward high-value demand regions.
Results
Simulation and ablation results show that the reconstructed reward is essential for stable learning, while the guidance factor significantly accelerates convergence. Under 40 IIoT nodes, the proposed method converges after approximately 2,000 episodes, whereas the model without guidance information requires about 5,000 episodes. Comparative experiments further show that the proposed method improves system throughput and collaborative delivery efficiency while maintaining feasible UAV trajectories.
Conclusion
The proposed framework provides an adaptive reinforcement-learning solution for UAV–UGV cooperative logistics in dynamic IIoT environments.