Distributed Routing for LEO Satellite Networks: A Multi-Agent Deep Reinforcement Learning Approach With State Information Lag

Abstract

Multi-agent deep reinforcement learning (MADRL) offers a promising solution for routing in low Earth orbit (LEO) satellite networks. However, large inter-satellite propagation delays lead to severe state information lag in agent interactions, giving rise to decision biases and degraded routing timeliness. To this end, this paper proposes a distributed routing algorithm named time-aware prediction and dynamic attention routing (TAP-DAR). Specifically, it constructs a delay compensation model that incorporates ephemeris data and queue prediction to generate near real-time neighbor state estimates. In addition, a multi-head attention fusion mechanism considering temporal reliability is designed to achieve adaptive aggregation of asynchronous neighbor states. Simulation results demonstrate that across various constellation configurations and network load conditions, the proposed algorithm achieves a maximum reduction of 16.16% in end-to-end (E2E) latency, an average decrease of nearly 30% in packet loss rate, and a maximum improvement of 19.41% in throughput compared to the baseline. Moreover, it substantially curtails communication overhead by more than 90% relative to the global state flooding mechanism.