A Multi-Agent Reinforcement Learning Congestion Control Protocol for Wireless Networks

Abstract

Multi-hop wireless ad-hoc networks (WANETs) are expected to expand significantly over the next years. The limited resources characterizing many WANET deployments, make necessary the efficient use of resources. One common practice to improve efficiency is congestion control. The highly dynamic settings and the need for decentralized control found in WANETs mean that a congestion control protocol applied to such networks should be able to adapt to changes in the environment and allow each node to make its own decisions based on their own particular context. Reinforcement Learning (RL) allows each node (agent) to learn a congestion control model while operating in the network, adapting to changes in the environment. Although RL-based congestion control has been widely studied in TCP environments, where connection-based communications and packet acknowledgment messages make the feedback mechanism explicit, limited research is available for UDP-based, connectionless communications. In this paper, we propose a multi-agent RL congestion control mechanism for WANETs, introduce a traffic prioritization mechanism and, through simulations in diverse conditions, compare our solution with existing ones, both in terms of performance and energy awareness. The proposed method increases delivered throughput by 29–80% compared to the best baseline across varying congestion and physical-layer conditions, while maintaining low energy consumption.