Deep reinforcement learning-based Dynamic routing for ultra-reliable low-latency communication in 6G networks

Abstract

Ultra-reliable low-latency communication (URLLC) is one of the hardest service classes that 6G has to carry, because a packet that misses its deadline is useless even if it eventually arrives. Routing based on fixed shortest paths ignores queue build-up, and link-state protocols that rebuild their tables every few milliseconds react too slowly to bursty traffic. This paper formulates hop-by-hop routing of deadline-constrained packets as a Markov decision process and solves it with a shared Double Deep Q-Network (DQN) agent that sees only local information: the destination, the remaining time budget and the queue state of its own outgoing links. The agent was trained and tested in a packet-level simulator of a 14-node, 21-link transport mesh with bursty background traffic. In five test runs per load level, it reduced the probability of exceeding a 1 ms deadline by a factor of 42 compared with static shortest-path routing and by 3.6 compared with load-aware routing refreshed every 2.5 ms (load factor 1.4). Removing queue information from the state erased most of the gain, which shows that the benefit comes from reacting to congestion and not only from learning the topology. A link-state scheme refreshed every 0.5 ms still gave a lower tail at the highest loads, but only by paying for network-wide signalling that the learned policy does not need.