Adaptive Routing Optimization With Cost and Deadline Awareness Using Hierarchical Deep Reinforcement Learning

Abstract

Timely and cost-efficient data transfers in large-scale networks remain challenging due to diverse topologies, non-uniform pricing models, and variable traffic demands. Existing literature often relies on multi-objective optimization, employing heuristic methods to reduce computational complexity; however, these approaches typically assume stable or predictable demand and struggle to scale effectively. Reinforcement Learning (RL) has been explored for its adaptability, yet many RL-based methods remain single-objective or topology-agnostic. This paper introduces CD-DRL, a hierarchical Deep RL framework that jointly optimizes transmission cost and deadline satisfaction, two objectives that often conflict in large-scale networks, through two cooperative agents. A routing agent, built on a Graph Neural Network, selects paths over a structured, multi-binary action space, enabling topology-aware routing across varying network scales and demand patterns. An adaptive tuning agent observes network state and recent performance to dynamically adjust the cost-deadline tradeoff to best fit current conditions. This hierarchical design allows CD-DRL to respond to dynamic network events such as congestion and bandwidth fluctuations, where no single fixed tradeoff remains optimal. We validate CD-DRL through extensive experiments on diverse backbone topologies and request distributions under static and time-varying network conditions. Compared with a state-of-the-art GNN-based RL method and traditional heuristics, CD-DRL improves the deadline-met ratio by up to 25% while maintaining competitive total cost and demonstrating strong scalability. Additionally, CD-DRL achieves faster execution time than mathematical optimization baselines, enabling high-throughput, latency-sensitive routing in dynamic environments.