Beyond static snapshots: gated recurrent units and graph attention networks for smarter multi-agent traffic control

Abstract

Existing multi-agent reinforcement learning (MARL) approaches for traffic signal control often fail to capture two critical characteristics of real-world urban traffic: the dynamic spatio-temporal propagation of congestion across intersections and the frequent imperfection of sensor data. To address these fundamental gaps, this paper introduces hybrid coordinated spatio-temporal graph reinforcement learning (HC-STGRL), a novel framework that integrates three complementary mechanisms-gated recurrent unit (GRU), graph attention network (GAT), dueling double deep Q-Network (3DQN). Each intersection is equipped with a GRU to retain and update short-term memory of evolving traffic patterns, while a GAT enables selective and adaptive information sharing with neighboring intersections, allowing agents to weigh the relevance of spatio-temporal dependencies from different sources. The final phase decisions of the signal are derived using 3DQN. The proposed system is extensively evaluated using SUMO simulations on two synthetic grid networks and two real-world traffic networks. Experimental results demonstrate that HC-STGRL consistently reduces average trip times and delays compared to state-of-the-art MARL baselines, achieving up to a 11.3% reduction in travel time and a 14.9% reduction in waiting time relative to the strongest baseline, CoLight, on real-world road networks. Furthermore, the framework exhibits notable robustness, maintaining efficient operation even when 20% of sensors malfunction, a condition that significantly degrades the performance of existing methods: HC-STGRL incurs only a 25.4% performance degradation under 20% sensor failure, compared with 44.7% for CoLight. Importantly, HC-STGRL incorporates built-in mechanisms to prevent major arterial roads from starving adjacent intersections, thereby overcoming a well-documented limitation of conventional throughput-maximizing control strategies.