Traffic Signal Dynamic Optimization Algorithm Based on Deep Reinforcement Learning
TL;DR
A traffic signal dynamic optimization algorithm, the Cross-Attention Mechanism and Dueling Double DQN (CAM-D3QN), which utilizes a novel crisscross attention module to dynamically model spatial dependencies between intersections and incorporates the Dueling Double DQN architecture for robust Q-value estimation.
Abstract
With the acceleration of urbanization, traffic congestion at multiple intersections has become a core bottleneck restricting urban operational efficiency. To address this, this paper proposes a traffic signal dynamic optimization algorithm, the Cross-Attention Mechanism and Dueling Double DQN (CAM-D3QN). This method utilizes a novel crisscross attention module to dynamically model spatial dependencies between intersections and incorporates the Dueling Double DQN architecture for robust Q-value estimation. Validated on CityFlow using Grid-4×4 and Hangzhou-real networks, CAM-D3QN significantly outperforms the state-of-the-art baseline, GPLight, achieving relative improvements of approximately 10.5% in average vehicle delay, 11.3% in average queue length, 3.3% in throughput, alongside notable reductions in stops (12.1%) and fuel consumption (7.2%). Ablation experiments further demonstrate that removing the cross-attention module increases queue length by 30.6% in sudden congestion scenarios. The proposed method achieves superior performance to the baseline across four typical traffic scenarios on the Hangzhou-real road network, demonstrating its generalization capabilities. By leveraging the coordinated optimization of dynamic spatial perception and robust value assessment, this paper provides an effective solution for efficient and robust coordinated traffic signal control. The framework can be combined with traffic states acquired from radar, roadside sensors or wireless communication units in intelligent transportation systems.