Deep Reinforcement Learning-Based QoS-Aware Routing Protocol for Space–Air–Ground Integrated Networks

Energy-aware deep reinforcement learning routing algorithm for space–air–ground–sea integrated networks

As a core component of future 6G architectures, the Space-Air-Ground-Sea Integrated Network (SAGS) is essential for marine environmental monitoring and emergency communications. However, constrained by scarce energy replenishment and the heterogeneous distribution of marine relay nodes, traditional shortest-path protocols often induce load imbalance and central node congestion, leading to premature failure and network connectivity loss. To address this "energy hole" problem, an Energy-Aware routing framework based on Proximal Policy Optimization (PPO) is proposed. Specifically, a One-Hot encoding mechanism is introduced to reconstruct the network state space, enabling the accurate capture of topological structural features. Furthermore, a composite reward function incorporating an energy penalty term is designed to guide routing decisions toward an optimal balance between path length and residual node energy. Experimental results in a high-fidelity simulation environment characterized by severe energy constraints demonstrate that the proposed algorithm effectively bypasses low-battery nodes while maintaining a 100% Packet Delivery Ratio. Notably, compared to Dijkstra’s algorithm, the proposed method significantly increases the average residual energy of network bottleneck nodes from 45.40% to 66.80%.

Robust Offline Multi-Agent Reinforcement Learning for Latency-Aware SDN Path Control in 6G-Oriented Network Softwarization

Future sixth-generation (6G)-oriented networks require programmable control that can adapt routing to latency and congestion without unsafe online exploration. This study evaluates offline multi-agent deep deterministic policy gradient (MADDPG) with behavior-adjusted training rewards for latency-aware path control in software-defined networking (SDN). Each traffic pair is modeled as an agent selecting one of three retained candidate paths, while centralized critics learn coordinated decisions from topology-specific Ryu–Mininet transition datasets. Nine policies are compared using ten paired seeds on fat-tree, mesh-grid, and WAN-corridors topologies under a deployed utilization–latency weighting of 0.60/0.40, together with flow-completion, latency, congestion, architectural-comparison, sensitivity, robustness, statistical, and controller-overhead analyses. The utilization-aware path heuristic achieves the strongest overall reward ranking. MADDPG is the strongest learned policy on fat-tree, is not significantly outperformed by any evaluated policy on mesh-grid, and remains statistically tied with completion-matched policies on WAN-corridors. Behavior adjustment is topology-dependent rather than uniformly beneficial. The exported policy requires approximately 52μs per joint decision, whereas complete control-loop timing is dominated by network-statistics polling. These results support offline multi-agent SDN control as a competitive, low-overhead option when interpreted jointly with topology structure, flow completion, and strong heuristic baselines.

Distributed Routing for LEO Satellite Networks: A Multi-Agent Deep Reinforcement Learning Approach With State Information Lag

Multi-agent deep reinforcement learning (MADRL) offers a promising solution for routing in low Earth orbit (LEO) satellite networks. However, large inter-satellite propagation delays lead to severe state information lag in agent interactions, giving rise to decision biases and degraded routing timeliness. To this end, this paper proposes a distributed routing algorithm named time-aware prediction and dynamic attention routing (TAP-DAR). Specifically, it constructs a delay compensation model that incorporates ephemeris data and queue prediction to generate near real-time neighbor state estimates. In addition, a multi-head attention fusion mechanism considering temporal reliability is designed to achieve adaptive aggregation of asynchronous neighbor states. Simulation results demonstrate that across various constellation configurations and network load conditions, the proposed algorithm achieves a maximum reduction of 16.16% in end-to-end (E2E) latency, an average decrease of nearly 30% in packet loss rate, and a maximum improvement of 19.41% in throughput compared to the baseline. Moreover, it substantially curtails communication overhead by more than 90% relative to the global state flooding mechanism.

Feasibility-Aware Reinforcement Learning for Reliable Hop-Constrained Routing in Wireless Sensor Networks

: Hop-constrained packet routing is a fundamental problem in wireless sensor networks (WSNs), where latency constraints, energy limitations, and practical feasibility requirements greatly restrict routing choices. Traditional methods based on shortest path and greedy routing have low complexity but cannot adapt to dynamic network changes well, while reinforcement learning for routing has the potential to adapt to network variations but has not been well explored in the hard hop-constrained setting. The current study attempts to fill the gap by modeling hop-constrained routing as the decision-making problem in a finite-horizon setting. An integrated simulation environment is proposed that unifies the concept of feasibility-aware action masking, energy-and trust-aware routing mechanisms, and simulation-related evaluation criteria. In this unified environment, four representative reinforcement learning methods, REINFORCE, Advantage Actor–Critic (A2C), Proximal Policy Optimization (PPO), and Deep Q-Network (DQN), are applied and validated against greedy forwarding, shortest-path routing, and Dijkstra routing under strict ( H = 5) and relaxed ( H = 15) hop limits using multi-seed testing. Under tight hop constraints, Dijkstra achieves a delivery success rate of 1.000, while greedy routing reaches 0.950 ± 0.014. Among the learning algorithms, PPO, A2C, and DQN reach approximately 0.945 ± 0.014 at their best checkpoints with near-baseline hop efficiency, indicated by an average hop count of about 4.34 ± 0.04. Under relaxed hop constraints, Dijkstra remains at 1.000, greedy forwarding reaches 0.984 ± 0.008, and PPO, A2C, and DQN achieve high best-checkpoint success rates of approximately 0.991– 0.992. REINFORCE improves under the relaxed setting but remains less stable than the stronger learned policies. The experiments show that feasibility-aware learning can approach deterministic baseline reliability while retaining learned forwarding capability under hop constraints. The ablation results further confirm that action masking is the dominant mechanism for maintaining feasible routing decisions, whereas trust mainly provides reliability-aware regularization. These observations emphasize the need to evaluate RL-based routing using deployment-level metrics, including success probability, hop-count distribution, invalid-action rate, route-risk rate, and return, rather than relying only on training reward.

A Cooperative MADRL Approach for Joint Computing Offloading and Route Optimization in Cloud-Edge Networks

Hybrid Deep Reinforcement Learning with Kookaburra Optimization for QOS-Aware Bandwidth Allocation in GMPLS Optical Networks

Objectives: To address dynamic bandwidth allocation with strict Quality of Service (QoS) requirements in Generalized Multi-Protocol Label Switching (GMPLS) optical networks under strain from internet services, real-time multimedia, and cloud infrastructure. Method: A Hybrid Deep Reinforcement Learning (Hyb-DRL) framework combined with the Kookaburra Optimization Algorithm (KkOA) for adaptive weight adjustment is proposed. Dynamically generated input data, including user request rates, queue lengths, server availability, and link stability metrics, were used to simulate real-world traffic. The Hyb-DRL agent learned optimal routing and bandwidth provisioning policies while KkOA optimized model weights for faster convergence and stability. Findings: The simulation results show that the suggested Hyb-DRL-KkOA algorithm performs better than the conventional bandwidth allocation algorithms. In contrast to conventional algorithms, it reduces the blocking probability, makespan, cost, and energy utilization while improving the throughput; therefore, providing enhanced quality of service (QoS). The proposed framework achieves a lower blocking probability by 78%, makespan by 64%, energy consumption by 51%, and operational cost by 47%. In addition to this, it provides better throughput performance by 69% than other conventional techniques. Moreover, it provided a delay of 0.0189 s, minimal energy consumption of 33 mJ, and maximal throughput of 950 Mbps. Novelty: A combination of reinforcement learning and meta-heuristic optimization leads to adaptive decision-making regarding routing and bandwidth allocation in the face of different traffic demands. The performance gain in terms of QoS is due to optimal utilization of network resources with low blocking probability, energy and operational cost. It provides a scalable and adaptive solution for high-speed, reliable data transmission in modern communication networks. Keywords: GMPLS Optical Networks, Kookaburra Optimization Algorithm, Bandwidth Allocation, Quality of Service (QoS), Blocking Probability