Traffic-Aware Mobility Management Based on Decentralized Federated Deep Reinforcement Learning in Satellite-Terrestrial Vehicular Networks

Abstract

Excessive traffic generated by in-vehicle applications can cause congestion or overflow in the transmit buffer queue of connected autonomous vehicles (CAVs), leading to high queuing delays and even service outage in satellite-terrestrial vehicular networks (STVN). Therefore, a decentralized federated deep reinforcement learning based intelligent handover (DF-DRL-IHO) algorithm is proposed. Specifically, a dueling double deep Q-network (D3QN) based local intelligent handover method (LIHO) is introduced, incorporating a priority experience replay strategy to dynamically adjust the experience priority, enabling each CAV to independently optimize handover decision and power control. Subsequently, a decentralized federated learning based aggregation (DFLA) framework is proposed to aggregate the LIHO model updates across different CAVs in the form of clustering. The DFLA employs an efficient aggregation method based on soft clustering to manage non-IID data generated by high mobility of CAVs, improving the stability and efficiency of cluster partitioning, as well as the model training efficiency and generalization capability. Furthermore, to mitigate high communication overhead caused by repeatedly transmitting parameters, an adaptive threshold based compressive sensing method is introduced to compress and transmit differential parameters, thereby accelerating the exchange of aggregation framework parameters. Simulation results demonstrate that proposed DF-DRL-IHO effectively reduces transmission overhead while maintaining highly reliable data transmission.