Intelligent Communication Channel Selection for Heterogeneous Maritime Networks: A Q-Learning and Hysteresis-Based Approach

Abstract

The rapid digitization of the maritime industry has exponentially increased the demand for reliable, high-speed, and cost-effective communication. Modern autonomous and semiautonomous vessels rely on a heterogeneous aggregation of network interfaces, including Geostationary Earth Orbit (GEO) satellites, Low Earth Orbit (LEO) satellites, 4G/5G cellular networks, Wi-Fi, and short-range radio frequencies. However, the maritime environment is inherently volatile, leading to unpredictable fluctuations in network latency, bandwidth availability, and signal stability. Traditional static routing and multi-criteria decision-making algorithms often fail to adapt to these rapid environmental changes, resulting in suboptimal data transmission, exorbitant satellite communication costs, and potentially dangerous delays in critical telemetry delivery. To address these critical challenges, this paper proposes an intelligent, autonomous communication channel selection system powered by Reinforcement Learning (RL). Specifically, a Q-Learning agent is designed to dynamically evaluate real-time state parameters (latency, bandwidth) and classify incoming data packets into critical, normal, and background priorities. The agent continuously learns to map specific data profiles to the most optimal network interface, penalizing high costs and extreme delays while rewarding connection stability. Furthermore, a Hysteresis Decision Engine is integrated as a stabilizing threshold layer. Results demonstrate that the proposed dual-layer architecture not only minimizes transmission delays for high-priority navigation data.