MAQDRL: QuadTree-Based Multi-Agent Federated Deep Q-Learning for Collision Avoidance and Routing Optimization with MEC-Aware Communication Analysis
TL;DR
Experimental results demonstrate that the proposed reinforcement learning-based Multi-agent Federated Deep Q-Network model, which integrates federated learning with QuadTree spatial partitioning and indexing framework for collision detection, is a promising solution for advancing autonomous vehicle routing systems.
Abstract
Autonomous vehicle routing in complex urban environments suffers from high computational overheads, energy inefficiency, and increased collision risks using traditional routing algorithms. These limitations necessitate innovative approaches to address scalable and real-time decision-making requirements for multi-agent systems. This research proposes a reinforcement learning-based Multi-agent Federated Deep Q-Network (MAQDRL) model, which integrates federated learning with QuadTree spatial partitioning and indexing framework for collision detection. The model employs a decentralized training approach to optimize routing efficiency and reduce energy consumption. Each agent of MAQDRL trains the Deep Q-Network (DQN) independently using local state information. Model parameters are periodically synchronized through federated learning to align policies while preserving data privacy. The QuadTree partitions the environment dynamically based on agent density, focusing computational resources on high-interaction areas to ensure efficient collision detection and proximity querying. Advanced preprocessing techniques like state normalization, action encoding, and reward scaling stabilize training and enhance policy convergence. The framework further employs an adaptive exploration-exploitation strategy and decentralized decision-making, allowing agents to collaboratively achieve optimized routing while mitigating collisions and reducing energy consumption in complex, high-density scenarios. Experimental results demonstrate that the proposed MAQDRL is a promising solution for advancing autonomous vehicle routing systems. MAQDRL achieves a 97.8% success rate, outperforming existing research - MARC (92.2%), MAPPO (94.8%), and MADDPG (95.2%).MAQDRL reduces collision frequency by 73.08% over MAPPO and 66.13% over MADDPG, while energy consumption is lowered by 56.52% ov er MAPPO and 55.98% over MADDPG on a CVRPTW-derived spatial navigation benchmark.
