Energy-aware deep reinforcement learning routing algorithm for space–air–ground–sea integrated networks

Abstract

As a core component of future 6G architectures, the Space-Air-Ground-Sea Integrated Network (SAGS) is essential for marine environmental monitoring and emergency communications. However, constrained by scarce energy replenishment and the heterogeneous distribution of marine relay nodes, traditional shortest-path protocols often induce load imbalance and central node congestion, leading to premature failure and network connectivity loss. To address this "energy hole" problem, an Energy-Aware routing framework based on Proximal Policy Optimization (PPO) is proposed. Specifically, a One-Hot encoding mechanism is introduced to reconstruct the network state space, enabling the accurate capture of topological structural features. Furthermore, a composite reward function incorporating an energy penalty term is designed to guide routing decisions toward an optimal balance between path length and residual node energy. Experimental results in a high-fidelity simulation environment characterized by severe energy constraints demonstrate that the proposed algorithm effectively bypasses low-battery nodes while maintaining a 100% Packet Delivery Ratio. Notably, compared to Dijkstra’s algorithm, the proposed method significantly increases the average residual energy of network bottleneck nodes from 45.40% to 66.80%.