TLQ-Geo: a two-level q-learning-based geographic routing protocol for flying ad hoc networks

Abstract

In flying ad hoc networks (FANETs), high node mobility, dynamic topology, and limited resources, such as energy and bandwidth, lead to unstable links and short-lived routes. In such an environment, although Q-learning-based routing methods are adaptable, they face serious challenges in practice due to large state space, high computational load, and slow convergence. To address these issues, this paper proposes a two-level Q-learning-based geographic routing protocol called TLQ-Geo for FANETs. This protocol integrates hierarchical decision-making with adaptive reinforcement learning. TLQ-Geo divides the routing process into two layers: the guided region selection (GRS) layer and the Q-learning-based routing (QRL) layer. The GRS layer determines a bounded search corridor between the source and the destination using a chain of intelligent decision points (IDPs), while the QRL layer performs distributed path optimization within this virtual corridor via Q-learning. By restricting the state space to the region guided by IDPs, TLQ-Geo significantly reduces convergence time and computational overhead. In addition, a dynamic inter-layer feedback mechanism periodically evaluates the performance of each IDP chain and adaptively reconfigures it under topology variations. Extensive simulations demonstrate that when the node density varies, TLQ-Geo achieves higher network lifespan (approximately 4.51%), improved packet delivery ratio (about 1.25%), lower routing overhead (around 3.38%), and better energy efficiency (about 20.79%), while the delay increases by about 13.84%, compared to three basic routing methods, namely QRCF, QRF, and QFAN. Also, when the node speed changes, TLQ-Geo yields better network lifespan (approximately 5.46%), higher packet delivery ratio (about 1.69%), lower overhead (around 2.80%), and better energy efficiency (about 6.42%), while the delay increases by about 9.09%.