Search

Traffic congestion alleviation around intersections has been a growing challenge, and a competent traffic signal control scheme plays a pivotal role in intelligent transportation systems. Recent studies using deep reinforcement learning techniques have shown promising results for traffic signal control, but they only focus on extracting features from traffic conditions of isolated or adjacent intersections. In this work, we employed navigation information for traffic signal control, greatly enriching the features for traffic signal control with deep reinforcement learning. In addition, we are the first to propose a novel scheme DeepNavi to exploit the temporal-spatial relations from numerous navigation routes and extract dynamic real-time and future traffic features. We tested our scheme on a challenging real-world traffic dataset with 16 intersections in a residential district of Hangzhou, China. Extensive experiments were conducted and the results demonstrated that our DeepNavi scheme achieves superior performance over five popular and state-of-the-art baseline methods on different metrics, including queue length, speed, travel time and accumulative waiting time. In addition, with our method, vehicles suffer the least red lights and enjoy the most green waves, which further validates that our scheme greatly relieves the congestion and provides excellent experience for drivers. Simulations with different penetration levels of navigation routes showed that even with only part of navigation routes available in the traffic network, our scheme can obtain superior performance, further demonstrating the effectiveness and feasibility of DeepNavi. [ABSTRACT FROM AUTHOR]

In recent years, the growing development of Connected Autonomous Vehicles (CAV), Intelligent Transport Systems (ITS), and 5G communication networks have led to the advent of Autonomous Intersection Management (AIM) systems. AIMs present a new paradigm for CAV control in future cities, taking control of CAVs in scenarios where cooperation is necessary and allowing safe and efficient traffic flows, eliminating traffic signals. So far, the development of AIM algorithms has been based on basic control algorithms, without the ability to adapt or keep learning new situations. To solve this, in this paper we present a new advanced AIM approach based on end-to-end Multi-Agent Deep Reinforcement Learning (MADRL) and trained using Curriculum through Self-Play, called advanced Reinforced AIM (adv.RAIM). adv.RAIM enables the control of CAVs at intersections in a collaborative way, autonomously learning complex real-life traffic dynamics. In addition, adv.RAIM provides a new way to build smarter AIMs capable of proactively controlling CAVs in other highly complex scenarios. Results show remarkable improvements when compared to traffic light control techniques (reducing travel time by 59% or reducing time lost due to congestion by 95%), as well as outperforming other recently proposed AIMs (reducing waiting time by 56%), highlighting the advantages of using MADRL. [ABSTRACT FROM AUTHOR]

Intelligent traffic light control is one of the modern approaches to solve traffic congestion, where reinforcement learning is a widely used method. Conventionally, reinforcement learning is used to determine whether to change the current phase (or choose a traffic phase) after each small interval. One major drawback of these approaches is that it makes the current traffic light phase duration uncertain before the current phase terminates. Directly determining the duration of the traffic light phase can effectively avoid this shortcoming. An adaptive traffic light timing system is proposed in this paper which can directly control the phase duration. In the proposed system, the Q-learning algorithm is employed and the action space is defined as all possible phase durations. In addition, the reward function is redefined to guide the agent to balance more traffic metrics, and the state is redefined to reduce the state space. Finally, the proposed system is evaluated by equal, unequal, and complex traffic scenarios. Results show that the proposed system has a better performance compared with other methods in controlling traffic lights, even on complex traffic scenarios. [ABSTRACT FROM AUTHOR]

The development of intelligent traffic light control systems is essential for smart transportation management. While some efforts have been made to optimize the use of individual traffic lights in an isolated way, related studies have largely ignored the fact that the use of multi-intersection traffic lights is spatially influenced, as well as the temporal dependency of historical traffic status for current traffic light control. To that end, in this article, we propose a novel Spatio-Temporal Multi-Agent Reinforcement Learning (STMARL) framework for effectively capturing the spatio-temporal dependency of multiple related traffic lights and control these traffic lights in a coordinating way. Specifically, we first construct the traffic light adjacency graph based on the spatial structure among traffic lights. Then, historical traffic records will be integrated with current traffic status via Recurrent Neural Network structure. Moreover, based on the temporally-dependent traffic information, we design a Graph Neural Network based model to represent relationships among multiple traffic lights, and the decision for each traffic light will be made in a distributed way by the deep Q-learning method. Finally, the experimental results on both synthetic and real-world data have demonstrated the effectiveness of our STMARL framework, which also provides an insightful understanding of the influence mechanism among multi-intersection traffic lights. [ABSTRACT FROM AUTHOR]

Improving the efficiency of traffic signal control is an effective way to alleviate traffic congestion at signalized intersections. To achieve effective management of the system-wide traffic flows, current research tends to focus on applying reinforcement learning (RL) techniques for collaborative traffic signal control in a traffic road network. However, the existing collaboration-based methods often ignore the impact of transmission delay for exchanging traffic flow information on the system. Most of the studies assume that the signal controllers can collect all instantaneous vehicular features without delay. To fill the gap, we propose an RL-based cooperative traffic signal control scheme considering the data transmission delay issue in a traffic road network. In this paper, we (1) design our new RL agents to cooperatively control the traffic signals by improving the reward and state representation based on the state-of-the-art max-pressure control theory; (2) propose a traffic state prediction method to address the data transmission delay issue by decreasing the discrepancy between the real-time and delayed traffic conditions; (3) evaluated the performance of our proposed work on both synthetic and real-world scenarios with a different range of data transmission delays. The results demonstrate that our method surpassed the performance of the previous max-pressure-based traffic signal control methods and addressed the data transmission delay issue. [ABSTRACT FROM AUTHOR]

In the era of internet of vehicles, both the traffic signal control and the rerouting of connected and autonomous vehicles (CAVs) are essential techniques to alleviate traffic congestion and improve traffic efficiency. However, these two schemes have been treated separately without a unified framework allowing for joint optimization in the previous studies. In the case of traffic congestion at the bottleneck intersections, either of the two schemes alone is not enough. To this end, we propose to jointly control the traffic lights and the CAVs under the reinforcement learning (RL) framework. To lower the dimensionality of this learning problem, we propose to control the route choices of the CAVs through a few common scalar parameters instead of controlling the CAVs individually. In particular, the rerouting ratios are dynamically adjusted. A model-based method is further proposed to estimate the expected travel time with reduced estimation error. We then exploit the tools from deep RL and put forth an efficient algorithm that is able to control the green time allocation of traffic signals and the rerouting of CAVs jointly. Comprehensive numerical experiments demonstrate the validity of our proposed models and show the traffic efficiency can be significantly improved with our proposed algorithm. [ABSTRACT FROM AUTHOR]

The adaptive traffic signal control (ATSC) problem can be modeled as a multiagent cooperative game among urban intersections, where intersections cooperate to counter the city's traffic conditions. Recently, reinforcement learning (RL) has achieved marked successes in managing sequential decision making problems, which motivates us to apply RL in the ATSC problem. One of the largest challenges of this problem is that the observation of intersection is typically partially observable, which limits the learning performance of RL algorithms. Considering the large scale of intersections in an urban traffic environment, we use independent RL to solve ATSC problem in this study. We model ATSC problem as a partially observable weak cooperative traffic model (PO-WCTM). Different from a traditional IRL task that averages the returns of all agents in fully cooperative games, the learning goal of each intersection in PO-WCTM is to reduce the cooperative difficulty of learning, which is also consistent with the traffic environment hypothesis. To achieve the optimal cooperative strategy of PO-WCTM, we propose an IRL algorithm called Cooperative Important Lenient Double DQN (CIL-DDQN), which extends Double DQN (DDQN) algorithm using two mechanisms: the forgetful experience mechanism and the lenient weight training mechanism. The former mechanism decreases the importance of experiences stored in the experience reply buffers, while the latter mechanism increases the weight experiences with high estimation and ‘leniently’ trains the DDQN neural network. Experiments in two real traffic scenarios and one simulated traffic scenarios show that, CIL-DDQN outperforms other methods in almost all performance indicators of ATSC. [ABSTRACT FROM AUTHOR]

Multiagent systems provide proper modeling in real-world applications such as intelligent transportation systems. The interaction between the agents can be represented by the graph theory. In this article, a fuzzy graph is used for urban traffic network modeling. A network composed of several intersections is considered as a multiagent system composed of multiple interacting agents. The interaction between the agents can be represented by a fuzzy graph in which each vertex shows an agent in the network. The network is divided into correlated agent's sets. In each set, collective learning composed of Q-learning and function approximation method is used to learn the optimal control policy. The total average energy of the sets of correlated agents as fuzzy subgraphs is computed and the relationship between these values and the effectiveness of the collective learning is studied. Experimental results show that the proposed collective learning method leads to better results compared to the independent mode in which each agent controls the intersection individually. In addition, the energy of fuzzy subgraphs related to the set of correlated agents are computed and the dependence of the energy and effectiveness of the collective learning method is studied in the results. [ABSTRACT FROM AUTHOR]

Reinforcement learning (RL) approaches have recently been spotlighted for use in adaptive traffic-signal control on an area-wide level. Most researchers have employed multi-agent reinforcement learning (MARL) algorithms wherein each agent shares a holistic traffic state and cooperates with other agents to reach a common goal. However, MARL algorithms cannot guarantee a global optimal solution unless the actions of all agents are fully coordinated. The present study employs a RL algorithm that recognizes an entire traffic state and jointly controls all the traffic signals of multiple intersections. With this approach, a deep Q-network (DQN) that depends solely on traffic images is extended to overcome the curse of dimensionality that is associated with a large state and action space. Several front layers in a deep convolutional neural network (CNN) to approximate the true Q-function are shared by each intersection approach. Weight parameters connecting the last hidden layer to the output layer are fixed. The proposed methodology outperforms a fixed-signal operation, a fully actuated signal operation, a multi-agent RL control without coordination, and a multi-agent RL control with partial coordination. [ABSTRACT FROM AUTHOR]

During rescue missions, transferring injured people from accident scenes to rescue sites is considered crucial and time-sensitive. In particular, a one-second delay could put more lives in danger. Though ambulances are commonly equipped with standard siren devices, such siren signals are not recognized by traffic light controllers. Therefore, rescue missions could be delayed at intersections due to an urgency-unaware traffic light control system. In the worst case, pile-up accidents could also happen when the ambulances lawfully ignores the traffic lights. This paper proposes A2T to enhance the efficiency of rescue missions, by establishing a communication mechanism among ambulances and infrastructures (e.g., traffic light controllers). Thailand, which had the highest road traffic death rate in the world in 2015, is selected as a use case in this paper. A2T promotes information sharing between ambulances and traffic light controllers along the rescue path in advance. Such information, including speeds, locations, and emergency routes, allows the traffic light controllers to launch a prioritized green traffic light accordingly. This allows the ambulances to go through any road intersection efficiently and safely. Our comprehensive performance evaluation shows that A2T achieves 100 percent waiting time reduction for the ambulances, with only 2.48 percent increase in delay of other vehicles at the intersections. [ABSTRACT FROM AUTHOR]

Periodicity correlations between roadside PM $_{2.5}$ concentration and traffic volume are difficult to explore due to the complicated process of production and diffusion of PM $_{2.5}$ : a reaction of vehicle emission with atmosphere, dependence with wind speed, pavement width, vehicle speed, vehicle type, etc. We propose a framework to explore the periodicity correlation between roadside PM $_{2.5}$ concentration and traffic volume using wavelet transform. The framework utilizes wavelet spectrum to obtain the characteristic periods of PM $_{2.5}$ concentration and traffic volume, and utilizes cross-wavelet spectrum to explore the hysteresis response of PM $_{2.5}$ concentration with respect to traffic volume. Different from traditional research work which handle pollutants and traffic data of macro scales, our study examines the micro-scale relationship of PM $_{2.5}$ concentration and traffic volume. We conduct experiments from four urban streets of Beijing, and reveal two micro-scale rules: (a) the characteristic period of PM $_{2.5}$ concentration approximates the characteristic period of traffic volume, and both of them are significantly affected by the traffic signal cycle; (b) PM $_{2.5}$ concentration exhibits a time delay within 0.3–0.9 minutes with respect to traffic volume. The research will benefit the further study on the evolution of PM $_{2.5}$ concentration and traffic volume, and provide a reference for the mitigation of PM $_{2.5}$ pollution by traffic control as well as practical traffic pollutant regulations in metropolises. [ABSTRACT FROM AUTHOR]

In this paper, we propose a game-based policy for the signal control of an isolated intersection together with the rerouting of the vehicles in this intersection using V2I technology. The game starts at the beginning of each signal cycle on the condition that there are vehicles delayed in the intersection at the end of current cycle. Then the delayed vehicles negotiate with the signal controller to decide the signal setting for next cycle while rerouting their next movements. During the game, each delayed vehicle selects one route from their own acceptable route space to reduce the intersection delay time, while the signal controller adjusts the signal setting to meet the traffic demand of the delayed vehicles as well as maximize the throughput. The game can be terminated with end conditions in finite rounds. The vehicles coming after the end of game reroute in a similar way with the delayed vehicles but based on the game results. Numerical experiments on a calibrated network of Caohejing District in Shanghai indicate that our proposed method can effectively mitigate road congestions and improve the network capacity, especially at high traffic loads. [ABSTRACT FROM AUTHOR]

Traffic control at intersections is a key issue and hot research topic in intelligent transportation systems. Existing approaches, including traffic light scheduling and trajectory maneuver, are either inaccurate and inflexible or complicated and costly. More importantly, due to the dynamics of traffic, it is really difficult to obtain the optimal solution in a real-time way. Inspired by the emergence of vehicular ad hoc network, we propose a novel approach to traffic control at intersections. Via vehicle to vehicle or vehicle to infrastructure communications, vehicles can compete for the privilege of passing the intersection, i.e., traffic is controlled via coordination among vehicles. Such an approach is flexible and efficient. To realize the coordination among vehicles, we first model the problem as a new variant of the classic mutual exclusion problem, and then design algorithms to solve new problem. Both centralized and distributed algorithms are. We conduct extensive simulations to evaluate the performance of our proposed algorithms. The results show that, our approach is efficient and outperforms a reference algorithm based on optimal traffic light scheduling. Moreover, our approach does not rely on traffic light or intersection controller facilities, which makes it flexible and applicable to various kinds of intersections. [ABSTRACT FROM PUBLISHER]