Search

Recent advancements in connected automated vehicles (CAVs) and reinforcement learning (RL) hold significant promise for enhancing intelligent traffic control systems. This paper conducts a systematic review of studies on RL-based urban traffic control at signalised intersections, highlighting the significant impact of CAVs on traffic control performance improvement. We first review the fundamental concepts of RL algorithms, establishing a foundational understanding for subsequent RL-based traffic control methods. We then review recent progress in RL-based traffic signal control using CV/CAV trajectory data, RL-based CAV trajectory planning, and the cooperative control of both traffic signals and CAVs at signalised intersections. Our aim is to provide researchers with a comprehensive roadmap for future research in RL-based traffic control at signalised intersections. [ABSTRACT FROM AUTHOR]

As the global economy develops rapidly, traffic congestion has become a major problem for first-tier cities in various countries. In order to address the problem of failed real-time control of the traffic flow data by the traditional traffic light control as well as malicious attack and other security problems faced by the intelligent traffic light (ITL) control system, a multi-agent distributed ITL control method was proposed based on the fog computing platform and the Q learning algorithm used for the reinforcement learning in this study, and the simulation comparison was conducted by using the simulation platform jointly constructed based on the VISSIM-Excel VBA-MATLAB software. Subsequently, on the basis of puzzle difficulty of the computational Diffie–Helleman (CDH) and Hash Collision, the applicable security control scheme of ITL under the fog computing was proposed. The results reveal that the proposed intelligent control system prolongs the time of green light properly when the number of vehicles increases, thereby reducing the delay time and retention rate of vehicles; the security control scheme of ITL based on the puzzle of CDH is less efficient when the vehicle density increases, while that based on the puzzle of Hash collision is very friendly to the fog equipment. In conclusion, the proposed control method of ITL based on the fog computing and Q learning algorithm can alleviate the traffic congestion effectively, so the proposed method has high security. [ABSTRACT FROM AUTHOR]

For intersection traffic congestion phenomenon, combined fog computing and reinforcement learning theory, this paper proposed a traffic light control model FRTL. The model performed intelligent coordinated control of traffic lights based on real-time traffic flow information. The fog node uploaded the collected real-time traffic flow information to the fog server, and the fog server realized information sharing on the fog platform, and the fog platform combined the processed shared data and Q learning to formulate a traffic light control algorithm. The algorithm used the detected real-time traffic data to calculate a suitable traffic light timing scheme, which was finally applied to the traffic light. The simulation results show that compared with the traditional time-phase control method and the main road control method (ATL), the FRTL control method improves the throughput of intersections, reduces the average waiting time of vehicles, and achieves the goal of properly regulating traffic lights and alleviating traffic congestion. [ABSTRACT FROM AUTHOR]

Group-based control is an advanced traffic signal strategy capable of dynamically generating phase sequences at intersections. Combined with the phasing scheme, vehicle actuated timing is often adopted to respond to the detected traffic. However, the parameters of a signal controller are often predetermined in practice, and the control performance may suffer from deterioration when dealing with highly fluctuating traffic demand. This study proposes a group-based signal control approach capable of making decisions based on its understanding of traffic conditions at the intersection level. In particular, the control problem is formulated using a framework of stochastic optimal control for multi-agent system in which each signal group is modeled as an intelligent agent. The agents learn how to react to traffic environment and make optimal timing decisions according to the perceived system states. Reinforcement learning, enhanced by multiple-step backups, is employed as the kernel of the intelligent control algorithm, where each agent updates its knowledge on-line based on a sequence of states during the process. In addition, the proposed system is designated to be compatible with the prevailing signal system. A case study was carried out in a simulation environment to compare the proposed control approach with a benchmark controller used in practice, group-based vehicle actuated (GBVA) controller, whose parameters were off-line optimized using a genetic algorithm. Simulation results show that the proposed adaptive group-based control system outperforms the optimized GBVA control system mainly because of its real-time adaptive learning capacity in response to the changes in traffic demand. [ABSTRACT FROM AUTHOR]

Abstract: This research presents the application of intelligent computing models in Air Traffic Flow Management (ATFM). Firstly, multi-agent system in grid computing environment is applied to deal with the problem of ATFM synchronization. The developed system consists of software agents, which are implemented in a Computational Grid platform for congestions identification, conflicts resolution and agreements negotiation among the participating airports. A metric criterion, called Agent’s Balancing Standard (ABS), is used as a basic index to measure the effectiveness of reducing both the amount of communication among agents and the delay of flights. Secondly, a brief discussion about the Meta-Level Control model is introduced in ATFM issues to improve the efficiency of communication among the agents. The system is developed to analyze the traffic flow information received, identify its importance and process it in the most adequate order. [Copyright &y& Elsevier]

The intelligent control of the traffic signal is critical to the optimization of transportation systems. To achieve global optimal traffic efficiency in large-scale road networks, recent works have focused on coordination among intersections, which have shown promising results. However, existing studies paid more attention to observations sharing among intersections (both explicit and implicit) and did not care about the consequences after decisions. In this paper, we design a multi-agent coordination framework based on Deep Reinforcement Learning method for traffic signal control, defined as γ - Reward that includes both original γ - Reward and γ - Attention-Reward. Specifically, we propose the Spatial Differentiation method for coordination which uses the temporal–spatial information in the replay buffer to amend the reward of each action. A concise theoretical analysis that proves the proposed model can converge to Nash equilibrium is given. By extending the idea of Markov Chain to the dimension of space–time, this truly decentralized coordination mechanism replaces the graph attention method and realizes the decoupling of the road network, which is more scalable and more in line with practice. The simulation results show that the proposed model remains a state-of-the-art performance even not use a centralized setting. Code is available in https://github.com/Skylark0924/Gamma_Reward. [ABSTRACT FROM AUTHOR]