Deep reinforcement learning-based traffic signal control in multi-intersection environments: a comparative study of DQN variants

Abstract

Traffic congestion at urban intersections is commonly associated with non-adaptive Fixed Time Signal Control (FTSC), which cannot respond effectively to variations in vehicle types, traffic demand, and intersection policies. Although deep reinforcement learning (DRL) has been increasingly applied to traffic signal control, comprehensive evaluations under multi-intersection environments with heterogeneous vehicles and different turning policies remain limited. This study evaluates four value-based DRL algorithms, namely DQN, DDQN, Dueling DQN, and Dueling DDQN, for optimizing traffic signal control in a simulated four-intersection network. The simulation incorporates heterogeneous vehicle types, priority-weighted vehicles, and two turning policy scenarios, and the revised evaluation also includes Fixed Time Signal Control, Longest Queue First, and Max Pressure as baseline controllers. Results from repeated training and testing evaluations show that the DRL-based controllers generally outperform FTSC and remain competitive against adaptive baselines across waiting time and speed metrics. Case 2, which allows direct left turns, consistently performs better than Case 1; however, this improvement is interpreted as the combined effect of DRL-based control and a less restrictive traffic policy. In offline testing for Case 2, Dueling DQN reduces ambulance waiting time from 167.6 to 42.2 s, corresponding to a 74.83% reduction. Overall, the findings demonstrate the potential of DRL-based traffic signal control in controlled simulation conditions and highlight that algorithm performance is strongly influenced by traffic policy design and environmental complexity.