Search

As urban areas continue to expand, traffic congestion has emerged as a significant challenge impacting urban governance and economic development. Frequent regional traffic congestion has become a primary factor hindering urban economic growth and social activities, necessitating improved regional traffic management. Addressing regional traffic optimization and control methods based on the characteristics of regional congestion has become a crucial and complex issue in the field of traffic management and control research. This paper focuses on the macroscopic fundamental diagram (MFD) and aims to tackle the control problem without relying on traffic determination information. To address this, we introduce the Q-learning (QL) algorithm in reinforcement learning and the Deep Deterministic Policy Gradient (DDPG) algorithm in deep reinforcement learning. Subsequently, we propose the MFD-QL perimeter control model and the MFD-DDPG perimeter control model. We conduct numerical analysis and simulation experiments to verify the effectiveness of the MFD-QL and MFD-DDPG algorithms. The experimental results show that the algorithms converge rapidly to a stable state and achieve superior control effects in optimizing regional perimeter control. [ABSTRACT FROM AUTHOR]

To contend traffic congestion on urban networks, existing studies have made great efforts to develop traffic-responsive signal timing algorithms in the last decade. More recently, as an alternative to conventional model-based algorithms, machine learning-based methods have been tested on traffic light timing problems and show promising potentials. However, many researchers and practitioners still questioned the feasibility and applicability of adopting machine learning techniques in the ATSC domain. One of the reasons is that these methods assumed flawless detectors and heavily relied on simulators for training and evaluations. To address such a critical concern, this article customizes a Deep Q-learning Learning (DQL) method to optimize traffic signal timings at urban intersections, where the partial observations from identity-based detectors are inputs, and the green splits are outputs. A simulation-free data-driven prediction model is also developed to train the DQL with reduced computational time. Then the machine learning-based methods are evaluated on a real-world case with Automatic Number-Plate Recognition (ANPR) data. Experiments show the proposed data-driven model can predict the traffic state in limited computational time, and the DQL algorithm is 3.9% better than the field experiment performance from the adaptive control system, SCOOT, and 22% better than the time-of-day plan by SYNCHRO. The results indicate the DQL methods can only yield marginal improvement with restrictive input and output settings in congested traffic flow in comparison to the conventional adaptive method. [ABSTRACT FROM AUTHOR]

The Internet of Things has a large number of terminal devices and a wide range of deployment. This feature has higher requirements for reliable transmission and parallel connection, and is more prone to network congestion resulting in the loss of power information flow and excessive delay. The traditional congestion control algorithm is not suitable for the high concurrent mass heterogeneous iot terminal access control and mass information flow storage control. In this paper, the Transformer BBR algorithm is proposed, which is a congestion control algorithm based on BBR deep reinforcement learning combined with Transformer's excellent long-term prediction ability. In the BBR algorithm bandwidth detection stage, transformer model is used as an agent to detect the network state and map it to the congestion window, so as to find the network changing state in time and make the corresponding decision action. Firstly, the congestion control strategy is learned in the simulation network environment, and then the efficiency is verified in the simulation environment. Finally, the experiment shows that the algorithm can reduce the delay while ensuring good bandwidth performance, and is superior to the main application algorithms in network throughput. [ABSTRACT FROM AUTHOR]

Traffic congestion alleviation around intersections has been a growing challenge, and a competent traffic signal control scheme plays a pivotal role in intelligent transportation systems. Recent studies using deep reinforcement learning techniques have shown promising results for traffic signal control, but they only focus on extracting features from traffic conditions of isolated or adjacent intersections. In this work, we employed navigation information for traffic signal control, greatly enriching the features for traffic signal control with deep reinforcement learning. In addition, we are the first to propose a novel scheme DeepNavi to exploit the temporal-spatial relations from numerous navigation routes and extract dynamic real-time and future traffic features. We tested our scheme on a challenging real-world traffic dataset with 16 intersections in a residential district of Hangzhou, China. Extensive experiments were conducted and the results demonstrated that our DeepNavi scheme achieves superior performance over five popular and state-of-the-art baseline methods on different metrics, including queue length, speed, travel time and accumulative waiting time. In addition, with our method, vehicles suffer the least red lights and enjoy the most green waves, which further validates that our scheme greatly relieves the congestion and provides excellent experience for drivers. Simulations with different penetration levels of navigation routes showed that even with only part of navigation routes available in the traffic network, our scheme can obtain superior performance, further demonstrating the effectiveness and feasibility of DeepNavi. [ABSTRACT FROM AUTHOR]

• Develop a novel two-layer coordinated multi-agent reinforcement learning algorithm. • Synergize traffic signals through local cooperation and global coordination. • Optimization of vehicle emissions and traffic congestion as simultaneous goals. • Proposed method is performed in synthetic network and Cologne urban network on SUMO. Intersection traffic signal control considering vehicle emissions has become an important topic, however, the decision complexity of traffic signal control increases dramatically in a dynamic traffic environment with multi-intersections. It is a severe challenge to coordinate traffic signals at multi-intersections based on Internet of Things information to improve the traffic condition of the road network. This paper proposes a two-layer coordination algorithm based on multi-agent reinforcement learning—Multi-agent Coordinated Policy Optimization (MACoPO), for solving traffic signal control at multi-intersections. MACoPO consists of local cooperation, which adjusts the weights of individual rewards and neighborhood agents' rewards by using local cooperation factors (LCF), and global coordination, which updates the LCF to maximize global rewards. The state and reward functions are designed in terms of the current state of the signal, waiting queue length, vehicle density and emission concentration in the lane, vehicle delay, and vehicle emissions, thus making full use of the intersection state information. The proposed method is extensively assessed through simulation experiments using artificial and real road networks and the numerical results confirm its effectiveness in complex and dynamic real-time traffic environments with multi-intersections. [ABSTRACT FROM AUTHOR]

• A decentralized DRL signal control for multi-modal traffic networks is introduced based on a microscopic environment. • Car traffic and bus performances are improved simultaneously and traded off. • The proposed method is applicable to different bus lane layouts and multiple bus lines. • The scalability and transferability of the model are verified. Managing traffic flow at intersections in a large-scale network remains challenging. Multi-modal signalized intersections integrate various objectives, including minimizing the queue length and maintaining constant bus headway. Inefficient traffic signals and bus headway control strategies may cause severe traffic jams, high delays for bus passengers, and bus bunching that harms bus line operations. To simultaneously improve the level of service for car traffic and the bus system in a multi-modal network, this paper integrates bus priority and holding with traffic signal control via decentralized controllers based on Reinforcement Learning (RL). The controller agents act and learn from a synthetic traffic environment built with the microscopic traffic simulator SUMO. Action information is shared among agents to achieve cooperation, forming a Multi-Agent Reinforcement Learning (MARL) framework. The agents simultaneously aim to minimize vehicles' total stopping time and homogenize the forward and backward space headways for buses approaching intersections at each decision step. The Deep Q-Network (DQN) algorithm is applied to manage the continuity of the state space. The tradeoff between the bus transit and car traffic objectives is discussed using various numerical experiments. The introduced method is tested in scenarios with distinct bus lane layouts and bus line deployments. The proposed controller outperforms model-based adaptive control methods and the centralized RL method regarding global traffic efficiency and bus transit stability. Furthermore, the remarkable scalability and transferability of trained models are demonstrated by applying them to several different test networks without retraining. [ABSTRACT FROM AUTHOR]

• A lane-level VSL (LVSL) approach with actor-critic framework is proposed to provide differential VSL control. • A hybrid reward function is presented to adjust relationship between the reducing crash risk and improving traffic efficiency. • A twin delayed deep deterministic policy gradient (TD3)-based LVSL approach is established for the first time. • The TD3LVSL shows better performance than baseline models in terms of improving traffic safety and traffic efficiency. Variable speed limit (VSL) control plays a vital role in the emerging connected automated vehicle highway (CAVH) system, which can alleviate recurrent traffic congestion caused by capacity drop and accidents. However, the effect of the VSL on improving traffic efficiency is still controversial. It is necessary to study how to explore the potential benefits of the VSL by balancing its influence on reducing crash risk and enhancing traffic efficiency. To fill the technological gap above, this paper proposes a reinforcement learning-based lane-level VSL (LVSL) control approach to conduct refined traffic control on the mainlines. Firstly, an actor-critic framework is developed to generate and evaluate the discrete speed limits of each lane in continuous action space. To optimize traffic control performance, a hybrid reward function is developed by synchronously considering traffic safety and traffic efficiency of the bottleneck area. Then, to solve the overestimation bias problem of the actor-critic methods caused by function approximation error, a twin delayed deep deterministic policy gradient (TD3) method is introduced to train the framework of the LVSL method. Finally, a real-world recurrent bottleneck of the State Route 91 highway in California is simulated with consideration of the connected automated vehicles to examine the performance of the TD3-based LVSL (TD3LVSL) controller. The simulation results reveal that the proposed method is capable of reducing crash risk and improving traffic efficiency synchronously. Compared with the LVSL controller based on the deep deterministic policy gradient approach, the TD3LVSL controller shows better performance in terms of traffic safety and efficiency. These findings indicate that the proposed controller could contribute to future traffic control in a CAVH environment. [ABSTRACT FROM AUTHOR]

Recently, Adaptive Traffic Signal Control (ATSC) in the multi-intersection system is considered as one of the most critical issues in Intelligent Transportation Systems (ITS). Among the proposed AI -based approaches, Deep Reinforcement Learning (DRL) has been largely applied while showing better performances. This paper proposes a new DRL -based cooperative approach for controlling multiple intersections. The problem is modelled as a Multi-Agent Reinforcement Learning (MARL) system, while each agent is trained to select the best action to control an intersection by obtaining information about its local lanes state. The cooperation aspect is manifested in this approach by considering the effect of the state, action and reward of neighbour agents in the process of policy learning. An intersection controller applies a Deep Q-Network (DQN) method, while transferring state, action and reward received from their neighbour agents to its own loss function during the learning process. The experimental results under different scenarios shows that the proposed approach outperforms many state-of-the-art approaches in terms of three metrics: Average Waiting Time (AWT), Average Queue Length (AQL) and Average Emission CO 2 (AEC). In addition, the cooperation between the different trained DRL -based controllers allows the system to continuously learn and improve its performance by interacting with the environment, particularly when the traffic is congested. [ABSTRACT FROM AUTHOR]