Search

Efficient traffic signal management is crucial for regulating traffic flow and fostering sustainable development within road transportation systems. To address the challenges in traffic management, numerous studies have applied the Adaptive Traffic Signal Control (ATSC) technology, using Deep Reinforcement Learning (DRL) to decrease vehicles' average waiting times. Nonetheless, the intricate nature of DRL, characterized by its extensive parameter connections, often complicates the assurance of real-time responsiveness. Additionally, by prioritizing reduced waiting times, these methods may overlook potential rises in queue lengths, risking congestion. In this paper, we propose an Efficient Double Broad Reinforcement Learning (E-DBRL) algorithm based on a Double Broad Q-Network (Double BQN) to alleviate the overestimation of action values common in Broad Reinforcement Learning (BRL). To enhance the Quality of Experience (QoE) of drivers, we develop a new reward function that optimizes the average waiting time and the range between the longest and shortest waiting times, thus avoiding the need for dimension normalization. Moreover, we conduct simulation experiments using actual traffic data collected from Hangzhou, China. The experimental results indicate that, compared to the traditional Double DQN, the proposed E-DBRL algorithm achieves a 45.78% reduction in the average training time per round and a 5.57% increase in the average rewards. [ABSTRACT FROM AUTHOR]

Compared to traditional traffic signal control methods, the method driven by Deep Reinforcement Learning (DRL) has shown better performance. But the problem of low sample utilization in reinforcement learning also arises. To deal with the problem, this paper presents a novel Twin Delayed Deep Deterministic Policy Gradient with Dual Buffer (TD3_DB) for traffic signal control. In the proposed framework, two experience buffers are used to store important samples and normal samples, separately, and the proportion of the two buffers is adjusted adaptively. In addition, lane pressure, describing the dynamic feature of lane traffic flow, is used for the state design of the TD3 agent, which enhances the perception of the agent toward intersections. Comprehensive experiments on different traffic flow modes has shown, the dual experience replay scheme can improve the sample utilization, and the proposed TD3_DB performs better than other methods such as original TD3, Proximal Policy Optimization (PPO), etc., effectively reducing vehicle queue length and waiting time. [ABSTRACT FROM AUTHOR]

One of the focal points in the field of intelligent transportation is the intelligent control of traffic signals (TS), aimed at enhancing the efficiency of urban road networks through specific algorithms. Deep Reinforcement Learning (DRL) algorithms have become mainstream, yet they suffer from inefficient training sample selection, leading to slow convergence. Additionally, enhancing model robustness is crucial for adapting to diverse traffic conditions. Hence, this paper proposes an enhanced method for traffic signal control (TSC) based on DRL. This approach utilizes dueling network and double q-learning to alleviate the overestimation issue of DRL. Additionally, it introduces a priority sampling mechanism to enhance the utilization efficiency of samples in memory. Moreover, noise parameters are integrated into the neural network model during training to bolster its robustness. By representing high-dimensional real-time traffic information as matrices, and employing a phase-cycled action space to guide the decision-making of intelligent agents. Additionally, utilizing a reward function that closely mirrors real-world scenarios to guide model training. Experimental results demonstrate faster convergence and optimal performance in metrics such as queue length and waiting time. Testing experiments further validate the method's robustness across different traffic flow scenarios. [ABSTRACT FROM AUTHOR]

In many real-life scenarios, multiple agents necessitate cooperation to accomplish tasks. Benefiting from the significant success of deep learning, many single-agent deep reinforcement learning algorithms have been extended to multi-agent scenarios. Overestimation in value estimation of Q-learning is a significant issue that has been studied comprehensively in the single-agent domains, but rarely in multi-agent reinforcement learning. In this paper, we first demonstrate that Q-learning-based multi-agent reinforcement learning (MARL) methods generally have notably serious overestimation issues, which cannot be alleviated by current methods. To tackle this problem, we introduce the double critic networks structure and the delayed policy update to Q-learning-based multi-agent MARL methods, which reduce the overestimation and enhance the quality of policy updating. To demonstrate the versatility of our proposed method, we select several Q-learning based MARL methods and evaluate them on several multi-agent tasks on the multi-agent particle environment and SMAC. Experimental results demonstrate that the proposed method can avoid the overestimation problem and significantly improve performance. Besides, application in the Traffic Signal Control verifies the feasibility of applying the proposed method in real-world scenarios. [ABSTRACT FROM AUTHOR]

Traffic lights are crucial for urban traffic management, as they significantly impact congestion reduction and travel safety. Traditional methods relying on hand-crafted rules and operator experience are limited in their ability to adapt to changing traffic environments. To address this challenge, we have been exploring intelligent traffic light control using deep reinforcement learning techniques. However, current approaches often suffer from inadequate training data and unstable training processes, leading to suboptimal performance and real-world consequences. In this study, we propose RELight, a novel random ensemble reinforcement learning-based traffic light control framework. RELight effectively utilizes collected empirical data, ensuring a stable and efficient training process. To evaluate the performance of our proposed framework, we conducted a comprehensive set of experiments on a variety of datasets, including four synthetic datasets and a real traffic dataset collected from surveillance cameras at an intersection in Hangzhou, China. The results show that RELight outperforms existing approaches, demonstrating its superiority and potential for practical traffic light control applications. [ABSTRACT FROM AUTHOR]