Coordinated ramp metering under asymmetric traffic demand: a continuous deep reinforcement learning approach

Abstract

Asymmetric ramp demand is common in multi-ramp freeway corridors and can complicate coordinated ramp metering. Existing approaches may not explicitly represent differences in short-term queue growth across ramps, while discrete metering actions can limit control resolution. This study proposes a soft actor–critic-based coordinated ramp-metering framework in which the control problem is formulated as a continuous-action Markov decision process. A queue-growth asymmetry index is incorporated into the state representation to characterise uneven queue development, and continuous green times are assigned jointly to the controlled ramps. The framework is evaluated under symmetric, static asymmetric and dynamic asymmetric demand scenarios. Comparisons with no control, heuristic ramp-metering coordination and model predictive control are conducted using five independent random seeds. Relative to no control, the maximum travel-time reduction and speed improvement are 31.36% and 64.91%, respectively, while the maximum ramp queues remain below the overflow threshold. Ablation results show that removing the queue-growth asymmetry index or replacing continuous actions with discrete green-time levels reduces traffic performance. Reward-component ablations further indicate that the overflow penalty mainly limits ramp queue accumulation, whereas the occupancy-gradient penalty contributes more to mainline speed. These results indicate that the proposed framework provides effective coordinated control under the tested demand conditions.