Search

Traffic congestion alleviation around intersections has been a growing challenge, and a competent traffic signal control scheme plays a pivotal role in intelligent transportation systems. Recent studies using deep reinforcement learning techniques have shown promising results for traffic signal control, but they only focus on extracting features from traffic conditions of isolated or adjacent intersections. In this work, we employed navigation information for traffic signal control, greatly enriching the features for traffic signal control with deep reinforcement learning. In addition, we are the first to propose a novel scheme DeepNavi to exploit the temporal-spatial relations from numerous navigation routes and extract dynamic real-time and future traffic features. We tested our scheme on a challenging real-world traffic dataset with 16 intersections in a residential district of Hangzhou, China. Extensive experiments were conducted and the results demonstrated that our DeepNavi scheme achieves superior performance over five popular and state-of-the-art baseline methods on different metrics, including queue length, speed, travel time and accumulative waiting time. In addition, with our method, vehicles suffer the least red lights and enjoy the most green waves, which further validates that our scheme greatly relieves the congestion and provides excellent experience for drivers. Simulations with different penetration levels of navigation routes showed that even with only part of navigation routes available in the traffic network, our scheme can obtain superior performance, further demonstrating the effectiveness and feasibility of DeepNavi. [ABSTRACT FROM AUTHOR]

In recent years, the growing development of Connected Autonomous Vehicles (CAV), Intelligent Transport Systems (ITS), and 5G communication networks have led to the advent of Autonomous Intersection Management (AIM) systems. AIMs present a new paradigm for CAV control in future cities, taking control of CAVs in scenarios where cooperation is necessary and allowing safe and efficient traffic flows, eliminating traffic signals. So far, the development of AIM algorithms has been based on basic control algorithms, without the ability to adapt or keep learning new situations. To solve this, in this paper we present a new advanced AIM approach based on end-to-end Multi-Agent Deep Reinforcement Learning (MADRL) and trained using Curriculum through Self-Play, called advanced Reinforced AIM (adv.RAIM). adv.RAIM enables the control of CAVs at intersections in a collaborative way, autonomously learning complex real-life traffic dynamics. In addition, adv.RAIM provides a new way to build smarter AIMs capable of proactively controlling CAVs in other highly complex scenarios. Results show remarkable improvements when compared to traffic light control techniques (reducing travel time by 59% or reducing time lost due to congestion by 95%), as well as outperforming other recently proposed AIMs (reducing waiting time by 56%), highlighting the advantages of using MADRL. [ABSTRACT FROM AUTHOR]

Intelligent traffic light control is one of the modern approaches to solve traffic congestion, where reinforcement learning is a widely used method. Conventionally, reinforcement learning is used to determine whether to change the current phase (or choose a traffic phase) after each small interval. One major drawback of these approaches is that it makes the current traffic light phase duration uncertain before the current phase terminates. Directly determining the duration of the traffic light phase can effectively avoid this shortcoming. An adaptive traffic light timing system is proposed in this paper which can directly control the phase duration. In the proposed system, the Q-learning algorithm is employed and the action space is defined as all possible phase durations. In addition, the reward function is redefined to guide the agent to balance more traffic metrics, and the state is redefined to reduce the state space. Finally, the proposed system is evaluated by equal, unequal, and complex traffic scenarios. Results show that the proposed system has a better performance compared with other methods in controlling traffic lights, even on complex traffic scenarios. [ABSTRACT FROM AUTHOR]

Improving the efficiency of traffic signal control is an effective way to alleviate traffic congestion at signalized intersections. To achieve effective management of the system-wide traffic flows, current research tends to focus on applying reinforcement learning (RL) techniques for collaborative traffic signal control in a traffic road network. However, the existing collaboration-based methods often ignore the impact of transmission delay for exchanging traffic flow information on the system. Most of the studies assume that the signal controllers can collect all instantaneous vehicular features without delay. To fill the gap, we propose an RL-based cooperative traffic signal control scheme considering the data transmission delay issue in a traffic road network. In this paper, we (1) design our new RL agents to cooperatively control the traffic signals by improving the reward and state representation based on the state-of-the-art max-pressure control theory; (2) propose a traffic state prediction method to address the data transmission delay issue by decreasing the discrepancy between the real-time and delayed traffic conditions; (3) evaluated the performance of our proposed work on both synthetic and real-world scenarios with a different range of data transmission delays. The results demonstrate that our method surpassed the performance of the previous max-pressure-based traffic signal control methods and addressed the data transmission delay issue. [ABSTRACT FROM AUTHOR]

In the era of internet of vehicles, both the traffic signal control and the rerouting of connected and autonomous vehicles (CAVs) are essential techniques to alleviate traffic congestion and improve traffic efficiency. However, these two schemes have been treated separately without a unified framework allowing for joint optimization in the previous studies. In the case of traffic congestion at the bottleneck intersections, either of the two schemes alone is not enough. To this end, we propose to jointly control the traffic lights and the CAVs under the reinforcement learning (RL) framework. To lower the dimensionality of this learning problem, we propose to control the route choices of the CAVs through a few common scalar parameters instead of controlling the CAVs individually. In particular, the rerouting ratios are dynamically adjusted. A model-based method is further proposed to estimate the expected travel time with reduced estimation error. We then exploit the tools from deep RL and put forth an efficient algorithm that is able to control the green time allocation of traffic signals and the rerouting of CAVs jointly. Comprehensive numerical experiments demonstrate the validity of our proposed models and show the traffic efficiency can be significantly improved with our proposed algorithm. [ABSTRACT FROM AUTHOR]

The adaptive traffic signal control (ATSC) problem can be modeled as a multiagent cooperative game among urban intersections, where intersections cooperate to counter the city's traffic conditions. Recently, reinforcement learning (RL) has achieved marked successes in managing sequential decision making problems, which motivates us to apply RL in the ATSC problem. One of the largest challenges of this problem is that the observation of intersection is typically partially observable, which limits the learning performance of RL algorithms. Considering the large scale of intersections in an urban traffic environment, we use independent RL to solve ATSC problem in this study. We model ATSC problem as a partially observable weak cooperative traffic model (PO-WCTM). Different from a traditional IRL task that averages the returns of all agents in fully cooperative games, the learning goal of each intersection in PO-WCTM is to reduce the cooperative difficulty of learning, which is also consistent with the traffic environment hypothesis. To achieve the optimal cooperative strategy of PO-WCTM, we propose an IRL algorithm called Cooperative Important Lenient Double DQN (CIL-DDQN), which extends Double DQN (DDQN) algorithm using two mechanisms: the forgetful experience mechanism and the lenient weight training mechanism. The former mechanism decreases the importance of experiences stored in the experience reply buffers, while the latter mechanism increases the weight experiences with high estimation and ‘leniently’ trains the DDQN neural network. Experiments in two real traffic scenarios and one simulated traffic scenarios show that, CIL-DDQN outperforms other methods in almost all performance indicators of ATSC. [ABSTRACT FROM AUTHOR]

Multiagent systems provide proper modeling in real-world applications such as intelligent transportation systems. The interaction between the agents can be represented by the graph theory. In this article, a fuzzy graph is used for urban traffic network modeling. A network composed of several intersections is considered as a multiagent system composed of multiple interacting agents. The interaction between the agents can be represented by a fuzzy graph in which each vertex shows an agent in the network. The network is divided into correlated agent's sets. In each set, collective learning composed of Q-learning and function approximation method is used to learn the optimal control policy. The total average energy of the sets of correlated agents as fuzzy subgraphs is computed and the relationship between these values and the effectiveness of the collective learning is studied. Experimental results show that the proposed collective learning method leads to better results compared to the independent mode in which each agent controls the intersection individually. In addition, the energy of fuzzy subgraphs related to the set of correlated agents are computed and the dependence of the energy and effectiveness of the collective learning method is studied in the results. [ABSTRACT FROM AUTHOR]

Reinforcement learning (RL) approaches have recently been spotlighted for use in adaptive traffic-signal control on an area-wide level. Most researchers have employed multi-agent reinforcement learning (MARL) algorithms wherein each agent shares a holistic traffic state and cooperates with other agents to reach a common goal. However, MARL algorithms cannot guarantee a global optimal solution unless the actions of all agents are fully coordinated. The present study employs a RL algorithm that recognizes an entire traffic state and jointly controls all the traffic signals of multiple intersections. With this approach, a deep Q-network (DQN) that depends solely on traffic images is extended to overcome the curse of dimensionality that is associated with a large state and action space. Several front layers in a deep convolutional neural network (CNN) to approximate the true Q-function are shared by each intersection approach. Weight parameters connecting the last hidden layer to the output layer are fixed. The proposed methodology outperforms a fixed-signal operation, a fully actuated signal operation, a multi-agent RL control without coordination, and a multi-agent RL control with partial coordination. [ABSTRACT FROM AUTHOR]

During rescue missions, transferring injured people from accident scenes to rescue sites is considered crucial and time-sensitive. In particular, a one-second delay could put more lives in danger. Though ambulances are commonly equipped with standard siren devices, such siren signals are not recognized by traffic light controllers. Therefore, rescue missions could be delayed at intersections due to an urgency-unaware traffic light control system. In the worst case, pile-up accidents could also happen when the ambulances lawfully ignores the traffic lights. This paper proposes A2T to enhance the efficiency of rescue missions, by establishing a communication mechanism among ambulances and infrastructures (e.g., traffic light controllers). Thailand, which had the highest road traffic death rate in the world in 2015, is selected as a use case in this paper. A2T promotes information sharing between ambulances and traffic light controllers along the rescue path in advance. Such information, including speeds, locations, and emergency routes, allows the traffic light controllers to launch a prioritized green traffic light accordingly. This allows the ambulances to go through any road intersection efficiently and safely. Our comprehensive performance evaluation shows that A2T achieves 100 percent waiting time reduction for the ambulances, with only 2.48 percent increase in delay of other vehicles at the intersections. [ABSTRACT FROM AUTHOR]

Periodicity correlations between roadside PM $_{2.5}$ concentration and traffic volume are difficult to explore due to the complicated process of production and diffusion of PM $_{2.5}$ : a reaction of vehicle emission with atmosphere, dependence with wind speed, pavement width, vehicle speed, vehicle type, etc. We propose a framework to explore the periodicity correlation between roadside PM $_{2.5}$ concentration and traffic volume using wavelet transform. The framework utilizes wavelet spectrum to obtain the characteristic periods of PM $_{2.5}$ concentration and traffic volume, and utilizes cross-wavelet spectrum to explore the hysteresis response of PM $_{2.5}$ concentration with respect to traffic volume. Different from traditional research work which handle pollutants and traffic data of macro scales, our study examines the micro-scale relationship of PM $_{2.5}$ concentration and traffic volume. We conduct experiments from four urban streets of Beijing, and reveal two micro-scale rules: (a) the characteristic period of PM $_{2.5}$ concentration approximates the characteristic period of traffic volume, and both of them are significantly affected by the traffic signal cycle; (b) PM $_{2.5}$ concentration exhibits a time delay within 0.3–0.9 minutes with respect to traffic volume. The research will benefit the further study on the evolution of PM $_{2.5}$ concentration and traffic volume, and provide a reference for the mitigation of PM $_{2.5}$ pollution by traffic control as well as practical traffic pollutant regulations in metropolises. [ABSTRACT FROM AUTHOR]

In this paper, we propose a game-based policy for the signal control of an isolated intersection together with the rerouting of the vehicles in this intersection using V2I technology. The game starts at the beginning of each signal cycle on the condition that there are vehicles delayed in the intersection at the end of current cycle. Then the delayed vehicles negotiate with the signal controller to decide the signal setting for next cycle while rerouting their next movements. During the game, each delayed vehicle selects one route from their own acceptable route space to reduce the intersection delay time, while the signal controller adjusts the signal setting to meet the traffic demand of the delayed vehicles as well as maximize the throughput. The game can be terminated with end conditions in finite rounds. The vehicles coming after the end of game reroute in a similar way with the delayed vehicles but based on the game results. Numerical experiments on a calibrated network of Caohejing District in Shanghai indicate that our proposed method can effectively mitigate road congestions and improve the network capacity, especially at high traffic loads. [ABSTRACT FROM AUTHOR]

Traffic control at intersections is a key issue and hot research topic in intelligent transportation systems. Existing approaches, including traffic light scheduling and trajectory maneuver, are either inaccurate and inflexible or complicated and costly. More importantly, due to the dynamics of traffic, it is really difficult to obtain the optimal solution in a real-time way. Inspired by the emergence of vehicular ad hoc network, we propose a novel approach to traffic control at intersections. Via vehicle to vehicle or vehicle to infrastructure communications, vehicles can compete for the privilege of passing the intersection, i.e., traffic is controlled via coordination among vehicles. Such an approach is flexible and efficient. To realize the coordination among vehicles, we first model the problem as a new variant of the classic mutual exclusion problem, and then design algorithms to solve new problem. Both centralized and distributed algorithms are. We conduct extensive simulations to evaluate the performance of our proposed algorithms. The results show that, our approach is efficient and outperforms a reference algorithm based on optimal traffic light scheduling. Moreover, our approach does not rely on traffic light or intersection controller facilities, which makes it flexible and applicable to various kinds of intersections. [ABSTRACT FROM PUBLISHER]

Traffic light sensing aims to detect the status of traffic lights which is valuable for many applications such as traffic management, traffic light optimization, and real-time vehicle navigation. In this work, we develop a system called POVA for traffic light sensing in large-scale urban areas. The system employs pervasive probe vehicles that just report real-time states of position and speed from time to time. POVA has advantages of wide coverage and low deployment cost. The important observation motivating the design of POVA is that a traffic light has a considerable impact on mobility of vehicles on the road attached to the traffic light. However, the system design faces three unique challenges: 1) Probe reports are by nature discrete while the goal of traffic light sensing is to determine the state of a traffic light at any time; 2) there may be a very limited number of probe reports in a given duration for traffic light state estimation; and 3) a traffic light may change its state with a variable interval. To tackle the challenges, we develop a new technique that makes the best use of limited probe reports as well as statistical features of light states. It first estimates the state of a traffic light at the time instant of a report by applying maximum a posterior estimation. Then, we formulate the state estimation of a light at any time into a joint optimization problem that is solved by an efficient heuristic algorithm. We have implemented the system and tested it with a fleet of around 4,000 probe taxis and 2,000 buses in Shanghai, China. Trace-driven experimentation and field study show that nearly 60 percent of traffic lights have an estimation error lower than 19 percent if 20,000 probe vehicles would be employed in the urban area of Shanghai. We further demonstrate that the estimation error rate is as low as 18 percent even when the number of available reports is merely 1 per minute. [ABSTRACT FROM AUTHOR]

Adaptive control of traffic lights is a key component of any intelligent transportation system. Many real-time traffic light control (TLC) algorithms are based on graded thresholds, because precise information about the traffic congestion in the road network is hard to obtain in practice. For example, using thresholds L1 and L2, we could mark the congestion level on a particular lane as “low,” “medium,” or “high” based on whether the queue length on the lane is below L1, between L1 and L2, or above L2, respectively. However, the TLC algorithms that were proposed in the literature incorporate fixed values for the thresholds, which, in general, are not optimal for all traffic conditions. In this paper, we present an algorithm based on stochastic optimization to tune the thresholds that are associated with a TLC algorithm for optimal performance. We also propose the following three novel TLC algorithms: 1) a full-state Q-learning algorithm with state aggregation, 2) a Q-learning algorithm with function approximation that involves an enhanced feature selection scheme, and 3) a priority-based TLC scheme. All these algorithms are threshold based. Next, we combine the threshold-tuning algorithm with the three aforementioned algorithms. Such a combination results in several interesting consequences. For example, in the case of Q-learning with full-state representation, our threshold-tuning algorithm suggests an optimal way of clustering states to reduce the cardinality of the state space, and in the case of the Q-learning algorithm with function approximation, our (threshold-tuning) algorithm provides a novel feature adaptation scheme to obtain an “optimal” selection of features. Our tuning algorithm is an incremental-update online scheme with proven convergence to the optimal values of thresholds. Moreover, the additional computational effort that is required because of the integration of the tuning scheme in any of the graded-threshold-based TLC algorithms is minimal. Simulation results show a significant gain in performance when our threshold-tuning algorithm is used in conjunction with various TLC algorithms compared to the original TLC algorithms without tuning and with fixed thresholds. [ABSTRACT FROM PUBLISHER]

Robust road-boundary extraction/tracking is one of the main problems in autonomous roadway navigation. Although the road boundary can be defined by various means including lane markings, curbs, and borders of vegetation, this paper focuses on road-boundary tracking using curbs. A vehicle-mounted (downward tilted) 2-D laser-measurement system is utilized to detect the curbs. The tracking problem is difficult because both the vehicle is moving and the target is disappearing, reappearing, and maneuvering in clutter. The interacting-multiple-model probabilistic-data-association filter (IMMPDAF) is proposed to solve the problems after detailed analysis. Track initiation, confirmation, and deletion are performed using the sequential-probability-ratio test. Extensive simulations followed by experiments in a campus environment show that the road-boundary tracking utilizing curbs is possible and robust through IMMPDAF. [ABSTRACT FROM AUTHOR]

In this paper, an iterative learning approach for the freeway density control under ramp metering and speed regulation is developed in a macroscopic level traffic environment. Rigorous analyses show that the proposed learning control schemes guarantee the asymptotic convergence of the traffic density to the desired one. The two major features of the learning-based density control are 1) less prior modeling knowledge required in the control system design and 2) the ability to reject exogenous traffic perturbations. The control schemes are applied to a freeway model, and simulation results confirm the efficacy of the proposed approach. [ABSTRACT FROM AUTHOR]

The article discusses the development of an agent-based control for networked traffic management systems in the United States. Control algorithms have been developed to run various industrial devices, including intersection traffic controllers. The agent-based control method decomposes a control algorithm into many simple task-oriented control agents distributed over a network, generally a wide-area network. Control agents that can be deployed and replaced over the network as operating conditions vary will run network-enabled devices. In this way, a network-enabled device can operate on a control on demand basis.

Aiming at the problems of known traffic signal control methods, such as heavy system and huge computation, this paper presents a macroscopic two-dimensional cellular automata (CA) model of an urban traffic signal control system, in which each intersection is regarded as a cell and the flowpressure is treated as the state. The hierarchical control architecture of traffic signal is built. Off the support of the upper level (traffic control center), the lower level (intersection controllers) can implement self-organizing control of traffic signal. The signal decision at an intersection is based on the thought of "perception-decision." We establish the simple state perception rule of typical intersection. By means of the computer simulation, the application of CA in signal self-organizing control is proved effective in urban traffic. [ABSTRACT FROM AUTHOR]