Smart Traffic Signal Optimization Using Real-Time Data: A Review

Machine Learning and Autonomous Systems

In this paper, methods have been explored to effectively optimise traffic signal control to minimise waiting times and queue lengths, thereby increasing traffic flow. The traffic intersection was first defined as a Markov Decision Process, and a state representation, actions and rewards were chosen. Simulation of Urban MObility (SUMO) was used to simulate an intersection and then compare a Round Robin Scheduler, a Feedback Control mechanism and two Reinforcement Learning techniques-Deep Q Network (DQN) and Advantage Actor-Critic (A2C), as the policy for the traffic signal in the simulation under different scenarios. Finally, the methods were tested on a simulation of a real-world intersection in Bengaluru, India.

Urban traffic congestion has become a serious issue, and improving the flow of traffic through cities is critical for environmental, social and economic reasons. Improvements in Adaptive Traffic Signal Control (ATSC) have a pivotal role to play in the future development of Smart Cities, and in the alleviation of traffic congestion. Here we describe an autonomic method for ATSC; namely Reinforcement Learning (RL). This chapter presents a comprehensive review of the applications of RL to the traffic control problem to date, along with a case study that showcases our developing Multi Agent Traffic Control architecture. Three different RL algorithms are presented and evaluated experimentally. We also look towards the future and discuss some important challenges that still need to be addressed in this field.

IEEE Transactions on Intelligent Transportation Systems, 2006

Real-time traffic signal control is an integral part of the urban traffic control system, and providing effective real-time traffic signal control for a large complex traffic network is an extremely challenging distributed control problem. This paper adopts the multiagent system approach to develop distributed unsupervised traffic responsive signal control models, where each agent in the system is a local traffic signal controller for one intersection in the traffic network. The first multiagent system is developed using hybrid computational intelligent techniques. Each agent employs a multistage online learning process to update and adapt its knowledge base and decision-making mechanism. The second multiagent system is developed by integrating the simultaneous perturbation stochastic approximation theorem in fuzzy neural networks (NN). The problem of real-time traffic signal control is especially challenging if the agents are used for an infinite horizon problem, where online learning has to take place continuously once the agent-based traffic signal controllers are implemented into the traffic network. A comprehensive simulation model of a section of the Central Business District of Singapore has been developed using PARAMICS microscopic simulation program. Simulation results show that the hybrid multiagent system provides significant improvement in traffic conditions when evaluated against an existing traffic signal control algorithm as well as the SPSA-NN-based multiagent system as the complexity of the simulation scenario increases. Using the hybrid NN-based multiagent system, the mean delay of each vehicle was reduced by 78% and the mean stoppage time, by 85% compared to the existing traffic signal control algorithm. The promising results demonstrate the efficacy of the hybrid NN-based multiagent system in solving large-scale traffic signal control problems in a distributed manner.

IET Intelligent Transport Systems, 2010

This study presents a distributed multi-agent-based traffic signal control for optimising green timing in an urban arterial road network to reduce the total travel time and delay experienced by vehicles. The proposed multi-agent architecture uses traffic data collected by sensors at each intersection, stored historical traffic patterns and data communicated from agents in adjacent intersections to compute green time for a phase. The parameters like weights, threshold values used in computing the green time is fine tuned by online reinforcement learning with an objective to reduce overall delay. PARAMICS software was used as a platform to simulate 29 signalised intersection at Central Business District of Singapore and test the performance of proposed multi-agent traffic signal control for different traffic scenarios. The proposed multi-agent reinforcement learning (RLA) signal control showed significant improvement in mean time delay and speed in comparison to other traffic control system like hierarchical multi-agent system (HMS), cooperative ensemble (CE) and actuated control.

IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2003

This paper presents a new hybrid, synergistic approach in applying computational intelligence concepts to implement a cooperative, hierarchical, multiagent system for real-time traffic signal control of a complex traffic network. The large-scale traffic signal control problem is divided into various subproblems, and each subproblem is handled by an intelligent agent with fuzzy neural decision-making module. The decisions made by lower-level agents are mediated by their respective higher-level agents. Through adopting a cooperative distributed problem solving approach, coordinated control by the agents is achieved. In order for the multiagent architecture to adapt itself continuously to the dynamically changing problem domain, a multistage online learning process for each agent is implemented involving reinforcement learning, learning rate and weight adjustment as well as dynamic update of fuzzy relations using evolutionary algorithm. The test bed used for this research is a section of the Central Business District of Singapore. The performance of the proposed multiagent architecture is evaluated against the set of signal plans used by the current real-time adaptive traffic control system. The multiagent architecture produces significant improvements in the conditions of the traffic network, reducing the total mean delay by 40% and total vehicle stoppage time by 50%.

2004

Abstract Vehicular travel is increasing throughout the world, particularly in large urban areas. Therefore the need arises for simulating and optimizing traffic control algorithms to better accommodate this increasing demand. In this paper we study the simulation and optimization of traffic light controllers in a city and present an adaptive optimization algorithm based on reinforcement learning.

SSRN Electronic Journal

In the urban environment, traffic congestion has become a significant concern. Congestion negatively influences the economy, the environment, and the quality of life in general. Unfortunately, traditional traffic control systems fail to control traffic discipline due to inadequate human resource management and limited extension of current infrastructure, resulting in increased traffic congestion and road infractions. This paper aims to create an intelligent system dashboard to make judgments on its own, detect congested areas and actual congestion locations, and plan alternative routes. The system should collect all available data from different cities and create forecasts based on the previous year's data. The designing Artificial Intelligence traffic controllers in our proposal can adapt to current data from sensors to perform constant optimizations on the signal timing plan for intersections in a network to minimize traffic congestions by using real-time traffic data, which is the main issue in traffic flow control today. A new technology known as Radio Frequency Identification (RFID) has been introduced, which can be used in conjunction with the existing signalling system to provide real-time smart traffic control. Traffic congestion will be decreased as a result of the use of this innovative technology. In addition, bottlenecks and traffic violations will be spotted early, allowing for early preventative actions to be implemented, saving the motorist time and money. Long-term decision-making is aided by traffic monitoring, mainly when designing transportation plans and budgets. It also helps law enforcement agencies identify the different types of traffic and take appropriate precautions, such as installing security cameras and other control mechanisms.

2019

The Study group participants for the Intelligent Traffic Light Control assignment were tasked with investigating the use of autonomous traffic light controls for a network of road intersections such as the road map of a city. Smart Traffic, the software used and designed by our client Sweco, is designed to control a single intersection. Our task involved optimising the traffic flow without the need for global (e.g. city wide) governance of all traffic lights. This would both fit with the existing approach in Smart Traffic, avoids bad experiences in the past, and mathematically avoids a huge explosion of the state space. Unfortunately, the Smart Traffic software was not available for use and study, and this somewhat limited the approaches we considered. Instead, the software was treated as a black-box, for which we only knew an idealised version of the cost function that the software tried to optimise. We therefore focused on finding a method to have multiple instances of this contro...

International Journal of Engineering and Technology, 2012

For a long time it was believed that the systems responding to real time traffic would enable significant benefits. However, numerous limitations have appeared such as the existence of the models with high level of detail precision, the uncertainty in predicting future traffic flows, the difficulty in arrival time estimation, the lack of self-adjusting mechanism. The difficulties in optimising the signal control strategy have initiated new researches. The results highlight the artificial intelligence methods as a possible solution. These systems are characterized with the ability to accumulate and use knowledge, set a problem, learn, process, conclude, solve the problem and exchange knowledge. The research presented in this paper proposes an adaptive signal control performed by a control agent able to adapt to an optimal policy by learning from the environment. The goal to be achieved is minimization of the delays in the system. First, the problem of reinforcement learning has been set. The first computation results of the Q-learning application for adaptive traffic signal control are presented. It is concluded that the results obtained are in favor of the adaptive signal control strategy compared to the fixed and actuated signal control.

Journal of Transportation Engineering, 2003

The ability to exert real-time, adaptive control of transportation processes is the core of many intelligent transportation systems decision support tools. Reinforcement learning, an artificial intelligence approach undergoing development in the machinelearning community, offers key advantages in this regard. The ability of a control agent to learn relationships between control actions and their effect on the environment while pursuing a goal is a distinct improvement over prespecified models of the environment. Prespecified models are a prerequisite of conventional control methods and their accuracy limits the performance of control agents. This paper contains an introduction to Q-learning, a simple yet powerful reinforcement learning algorithm, and presents a case study involving application to traffic signal control. Encouraging results of the application to an isolated traffic signal, particularly under variable traffic conditions, are presented. A broader research effort is outlined, including extension to linear and networked signal systems and integration with dynamic route guidance. The research objective involves optimal control of heavily congested traffic across a two-dimensional road network-a challenging task for conventional traffic signal control methodologies.