DQN-Based Autonomous UAV Target Navigation in a GridWorld Environment with a Q-Learning Baseline
Abstract
This study presents a Deep Reinforcement Learning (DRL)-based autonomous target navigation approach for an unmanned aerial vehicle (UAV) in a simulated 2D GridWorld environment. The UAV is modeled as an agent that learns to reach a target by selecting one of four discrete actions: up, down, left, and right. A custom Gymnasium-based environment with fixed static obstacles was developed, and a Deep Q-Network (DQN) model was trained using Stable-Baselines3. A tabular Q-learning algorithm was also implemented as a baseline. To provide a consistent comparison, both agents were trained using the same Manhattan-distance-based reward-shaping mechanism. The models were evaluated using an all-goals test in which all valid target positions were tested individually, and breadth-first search (BFS) was used only during evaluation to compute the shortest collision-free paths. The Q-learning baseline achieved a 100.00% success rate by reaching all 91 valid target positions, while the DQN agent achieved a 98.90% success rate by reaching 90 out of 91 targets. However, the DQN agent achieved a higher optimal path success rate of 98.90%, compared with 81.32% for Q-learning. These results show that tabular Q-learning is highly effective for target reaching in this small, discrete, and fixed-layout environment, whereas DQN follows shortest collision-free paths more consistently and provides a more scalable framework for future extensions involving larger or more complex state spaces.