Adaptive Dynamic Model-Free Neuro-Fuzzy System for Traffic Signal Control

Abstract

With the development of the economy, urban traffic congestion has become increasingly serious, causing a series of problems such as safety and environmental pollution. To address the challenge of urban traffic congestion problems, traffic signal configuration optimization based on time-varying traffic states and real-time performance is an important direction with practical engineering significance. In this study, we design a traffic signal controller that learns online without requiring a traffic model. A model-free action-dependent adaptive dynamic programming (ADP) that employs two neural networks (an action network and a critic network) to approximate the Hamilton–Jacobi–Bellman equation is employed to provide self-learning optimization ability with varying traffic states. The proposed controller employs a neuro-fuzzy system that functions as an action network to generate control decisions utilizing expertise and mitigates the stochastic exploration inefficiency inherent in conventional ADP. ADP provides reinforcement signals that indicate a reward or punishment for the neuro-fuzzy system to guide it in adjusting its parameters. Subsequently, actions associated with lower cumulative delay are reinforced. The proposed traffic signal controller can reduce the blindness of learning by using experience and the inaccuracy of the traffic model. An artificial bee colony algorithm was employed to train the ADP to meet the real-time requirement for traffic signal control. The controller can adapt to fluctuating traffic states by training continuously, and achieves a smaller average delay in the long run. The simulation results demonstrate that proposed controller achieves a reduced delay through supervised learning with an accelerated training speed.