Search

Strategic traffic management is crucial for combating traffic congestion at the macroscopic level. However, such a field is still relatively unexplored, particularly for microscopic control objects, such as intersections and coordinated intersection groups. This article proposes a human-in-the-loop recommendation system for strategic urban traffic management, which follows an agent-based structure. A regional agent dispatcher is defined to assign agents for operation whenever “operation on-demand” is required. Such a requirement is identified by a daily-dependent operational mode on strategic traffic operations at a control object level. The strategic management scheme for each control object is guided by a strategic agent (customized), which is essentially a deep recommender model with a specific architecture. By featuring the multiagent design, a customized operational scheme can be generated at the intersection level, which instructs the corresponding controller to take specific operations. The utility of the recommendation system is demonstrated via a case study using real-world traffic data. In both offline and online evaluations, the system performs consistently at traffic operational recommendations in different scenarios and has the potential to provide more reasonable traffic operational strategies than a human-operated system. [ABSTRACT FROM AUTHOR]

Cooperatively optimizing a vast number of agents over a large-scale network faces unprecedented scalability challenges. The scalability of existing optimization algorithms is limited by either the agent population size or the network dimension. As a radical improvement, this article, for the first time, constructs a two-facet scalable distributed optimization framework. This novel framework distributes the computing load among agents (scalability w.r.t. population size) and enables each agent to only consider partial network constraints in its primal variable updates (scalability w.r.t. network dimension). We first develop a systemic network dimension reduction technique to virtually cluster the agents and lower the dimension of network-induced constraints and then constitute a novel shrunken primal-multi-dual subgradient (SPMDS) algorithm based on the reduced-dimension network for strongly coupled convex optimization problems. Optimality and convergence of the proposed distributed optimization framework are rigorously proved. The SPMDS-based optimization framework is free of agent-to-agent or cluster-to-cluster communication. Besides, the proposed method can achieve significant floating-point operations (FLOPs) reduction compared with full-dimension cases. The efficiency and efficacy of the proposed approaches are demonstrated, in comparison with benchmark methods, through simulations of electric vehicle charging control problems and traffic congestion control problems. [ABSTRACT FROM AUTHOR]

Traffic congestion alleviation around intersections has been a growing challenge, and a competent traffic signal control scheme plays a pivotal role in intelligent transportation systems. Recent studies using deep reinforcement learning techniques have shown promising results for traffic signal control, but they only focus on extracting features from traffic conditions of isolated or adjacent intersections. In this work, we employed navigation information for traffic signal control, greatly enriching the features for traffic signal control with deep reinforcement learning. In addition, we are the first to propose a novel scheme DeepNavi to exploit the temporal-spatial relations from numerous navigation routes and extract dynamic real-time and future traffic features. We tested our scheme on a challenging real-world traffic dataset with 16 intersections in a residential district of Hangzhou, China. Extensive experiments were conducted and the results demonstrated that our DeepNavi scheme achieves superior performance over five popular and state-of-the-art baseline methods on different metrics, including queue length, speed, travel time and accumulative waiting time. In addition, with our method, vehicles suffer the least red lights and enjoy the most green waves, which further validates that our scheme greatly relieves the congestion and provides excellent experience for drivers. Simulations with different penetration levels of navigation routes showed that even with only part of navigation routes available in the traffic network, our scheme can obtain superior performance, further demonstrating the effectiveness and feasibility of DeepNavi. [ABSTRACT FROM AUTHOR]

We develop reinforcement learning (RL) boundary controllers to mitigate stop-and-go traffic congestion on a freeway segment. The traffic dynamics of the freeway segment are governed by a macroscopic Aw–Rascle–Zhang (ARZ) model, consisting of 2 $\times $ 2 quasi-linear partial differential equations (PDEs) for traffic density and velocity. The boundary stabilization of the linearized ARZ PDE model has been solved by PDE backstepping, guaranteeing spatial $L^{2}$ norm regulation of the traffic state to uniform density and velocity and ensuring that traffic oscillations are suppressed. Collocated proportional (P) and proportional–integral (PI) controllers also provide stability guarantees for allowable control gains and are always applicable as model-free control options through gain tuning by trial and error, or by model-free optimization. Although these approaches are mathematically elegant, the stabilization result only holds locally and is usually affected by the change of model parameters. Therefore, we reformulate the PDE boundary control problem as an RL problem that pursues stabilization without knowing the system dynamics, simply by observing the state values. The proximal policy optimization (PPO), a neural network-based policy gradient algorithm, is employed to obtain RL controllers by interacting with a numerical simulator of the ARZ PDE. Being stabilization-inspired, the RL state-feedback boundary controllers are compared and evaluated against the rigorously stabilizing controllers in two cases: 1) in a system with perfect knowledge of the traffic flow dynamics and then 2) in one with only partial knowledge. We obtain RL controllers that nearly recover the performance of the backstepping, P, and PI controllers with perfect knowledge and outperform them in some cases with partial knowledge. It must be noted, however, that the RL controllers are obtained by conducting about one thousand episodes of iterative training on a simulation model. This training cannot be performed in a collision-free fashion in real traffic, nor convergence guaranteed when training. Thus, we demonstrate that the RL approach has learning (i.e., adaptation) potential for traffic PDE systems under uncertain and changing conditions, but RL is neither simple nor a fully safe substitute for model-based control in real traffic systems. [ABSTRACT FROM AUTHOR]

Intelligent traffic light control is one of the modern approaches to solve traffic congestion, where reinforcement learning is a widely used method. Conventionally, reinforcement learning is used to determine whether to change the current phase (or choose a traffic phase) after each small interval. One major drawback of these approaches is that it makes the current traffic light phase duration uncertain before the current phase terminates. Directly determining the duration of the traffic light phase can effectively avoid this shortcoming. An adaptive traffic light timing system is proposed in this paper which can directly control the phase duration. In the proposed system, the Q-learning algorithm is employed and the action space is defined as all possible phase durations. In addition, the reward function is redefined to guide the agent to balance more traffic metrics, and the state is redefined to reduce the state space. Finally, the proposed system is evaluated by equal, unequal, and complex traffic scenarios. Results show that the proposed system has a better performance compared with other methods in controlling traffic lights, even on complex traffic scenarios. [ABSTRACT FROM AUTHOR]

Improving the efficiency of traffic signal control is an effective way to alleviate traffic congestion at signalized intersections. To achieve effective management of the system-wide traffic flows, current research tends to focus on applying reinforcement learning (RL) techniques for collaborative traffic signal control in a traffic road network. However, the existing collaboration-based methods often ignore the impact of transmission delay for exchanging traffic flow information on the system. Most of the studies assume that the signal controllers can collect all instantaneous vehicular features without delay. To fill the gap, we propose an RL-based cooperative traffic signal control scheme considering the data transmission delay issue in a traffic road network. In this paper, we (1) design our new RL agents to cooperatively control the traffic signals by improving the reward and state representation based on the state-of-the-art max-pressure control theory; (2) propose a traffic state prediction method to address the data transmission delay issue by decreasing the discrepancy between the real-time and delayed traffic conditions; (3) evaluated the performance of our proposed work on both synthetic and real-world scenarios with a different range of data transmission delays. The results demonstrate that our method surpassed the performance of the previous max-pressure-based traffic signal control methods and addressed the data transmission delay issue. [ABSTRACT FROM AUTHOR]

In the era of internet of vehicles, both the traffic signal control and the rerouting of connected and autonomous vehicles (CAVs) are essential techniques to alleviate traffic congestion and improve traffic efficiency. However, these two schemes have been treated separately without a unified framework allowing for joint optimization in the previous studies. In the case of traffic congestion at the bottleneck intersections, either of the two schemes alone is not enough. To this end, we propose to jointly control the traffic lights and the CAVs under the reinforcement learning (RL) framework. To lower the dimensionality of this learning problem, we propose to control the route choices of the CAVs through a few common scalar parameters instead of controlling the CAVs individually. In particular, the rerouting ratios are dynamically adjusted. A model-based method is further proposed to estimate the expected travel time with reduced estimation error. We then exploit the tools from deep RL and put forth an efficient algorithm that is able to control the green time allocation of traffic signals and the rerouting of CAVs jointly. Comprehensive numerical experiments demonstrate the validity of our proposed models and show the traffic efficiency can be significantly improved with our proposed algorithm. [ABSTRACT FROM AUTHOR]

As urban traffic condition is diverse and complicated, applying reinforcement learning to reduce traffic congestion becomes one of the hot and promising topics. Especially, how to coordinate the traffic light controllers of multiple intersections is a key challenge for multi-agent reinforcement learning (MARL). Most existing MARL studies are based on traditional $Q$ -learning, but unstable environment leads to poor learning in the complicated and dynamic traffic scenarios. In this paper, we propose a novel multi-agent recurrent deep deterministic policy gradient (MARDDPG) algorithm based on deep deterministic policy gradient (DDPG) algorithm for traffic light control (TLC) in vehiclar networks. Specifically, the centralized learning in each critic network enables each agent to estimate the policies of other agents in the decision-making process and each agent can coordinate with each other, alleviating the problem of poor learning performance caused by environmental instability. The decentralized execution enables each agent to make decisions independently. We share parameters in actor networks to speed up the training process and reduce the memory footprint. The addition of LSTM is beneficial to alleviate the instability of the environment caused by partial observable state. We utilize surveillance cameras and vehicular networks to collect status information for each intersection. Unlike previous work, we have not only considered the vehicle but also considered the pedestrians waiting to pass through the intersection. Moreover, we also set different priorities for buses and ordinary vehicles. The experimental results in a vehicular network show that our method can run stably in various scenarios and coordinate multiple intersections, which significantly reduces vehicle congestion and pedestrian congestion. [ABSTRACT FROM AUTHOR]

Potential adversary and maliciously-behaved vehicles may cause traffic chaos or even lead to traffic accidents (particularly on traffic congestion crossroads). To address this issue, a novel security scheme is proposed for facilitating an efficient vehicular communication by employing a quantum secret sharing method. In the proposed scheme, the traffic control information is broadcasted regularly and shared among vehicles to reduce traffic congestion, and thus avoiding traffic accidents. Finally, we also demonstrate the superiority performance of our proposed scheme through theoretical and numerical simulation analysis. [ABSTRACT FROM AUTHOR]

Urban freeway traffic control is of great importance for traffic management and intelligent transportation systems. Various approaches have been proposed to relieve urban freeway traffic jam, among which, ALINEA, a ramp metering strategy, is commonly implemented with fixed triggering threshold and static controller parameter. However, such a strategy may not be able to effectively alleviate the traffic congestion while maintaining certain ramp throughput due to two reasons: i). the congestion threshold can be time-varying due to different factors, such as segment ID, weather condition, time, and etc. ii). The congestion evolution patterns are time-varying even for the same segment. In this paper, based on over 890 million records of vehicles collected on ramps in Hangzhou, China, we established dynamic congestion threshold for each road segment with external factors. Based on such dynamic congestion threshold, we further clustered the congestion evolution patterns, and designed adaptive ramp controller which could switch the controller parameter according to the predicted congestion evolution pattern. Finally, in order to show the performance among different strategies, we introduced three baseline groups, which are ‘without Controller’, ‘ALINEA controller’, and ‘Direct RBF(radial basis function)-neural network controller’, respectively.The evaluation of proposed controller design over real large-scale data indicated that our method achieves 8.4%(7.2%), 4.62%(9.48%) efficiency improvement in terms of average speed in km/h (average traffic flow in veh/h) than the performance with normal ALINEA controller and RBF-neural network controller respectively. [ABSTRACT FROM AUTHOR]

Major international automakers have considered the deployment of the 5.9 GHz dedicated short-range communications (DSRC) on their vehicle fleets for wireless connectivity. DSRC-enabled vehicle-to-vehicle (V2V) communication through broadcast of basic safety messages enables safety applications for crash warning and avoidance. However, in dense traffic conditions as the V2V deployment scales up, the resultant channel load increases and leads to channel congestion and may adversely affect the performance of the safety applications. The Society of Automotive Engineers J2945/1 standard that builds atop Institute of Electrical and Electronics Engineers (IEEE) 802.11p and IEEE 1609 standards provides the minimum performance requirements for V2V safety communications. Specifically, it provides a congestion control protocol for transmission rate and power adaptations to achieve robust performance in dense vehicular networks. The primary contribution of this paper is that using a congestion generation testbed that emulates channel congestion including a large number of remote vehicles, we can validate and test any V2V equipped vehicle for compliance with the J2945/1 standard. Our paper also demonstrates that under heavy congestion, even with 600 ms of inter-transmit time, a moving vehicle can be tracked to a lane-level accuracy. [ABSTRACT FROM AUTHOR]

Macroscopic fundamental diagram (MFD) describes a unimodal, low-scatter relationship between network vehicle density and network space mean flow for homogenous network regions, and provides a possible solution to traffic signal control for urban road networks. Transfer between multiple states has been ignored in a few control methods, despite continuous advances in the areas of congestion perimeter control because different states should have different control objectives. This paper proposes a boundary hybrid control for two-region urban cities based on three network states: free flow, critical flow, and congestion flow as characterized by MFD. The hybrid perimeter control for two-region urban cities with different states can be set up according to different control objectives by analyzing the control adaptation characteristics for the single-control method of Bang–Bang, PI feedback control, and nonlinear PI control based on the MFD curve at a single-boundary entrance under different conditions. Furthermore, the confirmation of rapid reduction in the saturation, traffic distribution, and signal timing of the boundary intersection is derived. An urban center network located approximately 20 km2 in Hefei is taken as an example. Comparison results show that the hybrid control performs significantly better than the single-control scheme in completing vehicle travel, decreasing travel delay, and preventing network oversaturation. [ABSTRACT FROM AUTHOR]

The urban road networks undergo frequent traffic congestions during the peak hours and around the city center. Capturing the spatiotemporal evolution of the congestion scenario in real-time in an urban-scale can aid in developing smart traffic management systems, and guiding commuters in making informed decision about route choice. The congestion scenario is often represented by a set of distinguishable network partitions that have a homogeneous level of congestion inside them but are heterogeneous to others. Due to the dynamic nature of traffic, these partitions evolve with time in terms of their structure and location. In this paper, we propose a comprehensive framework to capture the evolution by incrementally updating the partitions in an efficient manner using a two-layer approach. The physical layer maintains a set of small-sized road network building blocks in a fine granularity, and performs low-level computations to incrementally update them, whereas the logical layer performs high-level computations in order to serve as an interface to query the physical layer about the congested partitions in a coarse granularity. We also propose an in-memory index called Bin that compactly stores the historical sets of building blocks in the main memory with no information loss, and facilitates their efficient retrieval. Our experimental results show that the proposed method is much efficient than the existing re-partitioning methods without significant sacrifice in accuracy. The proposed Bin consumes a minimum space with least redundancy at different time stamps. [ABSTRACT FROM AUTHOR]

This paper proposes a self-adaptive interactive navigation tool (SAINT), which is tailored for cloud-based vehicular traffic optimization in road networks. The legacy navigation systems make vehicles navigate toward their destination less effectively with individually optimal navigation paths rather than network-wide optimal navigation paths, particularly during rush hours. To the best of our knowledge, SAINT is the first attempt to investigate a self-adaptive interactive navigation approach through the interaction between vehicles and vehicular cloud. The vehicles report their navigation experiences and travel paths to the vehicular cloud so that the vehicular cloud can know real-time road traffic conditions and vehicle trajectories for better navigation guidance for other vehicles. With these traffic conditions and vehicle trajectories, the vehicular cloud uses a mathematical model to calculate road segment congestion estimation for global traffic optimization. This model provides each vehicle with a navigation path that has minimum traffic congestion in the target road network. Using the simulation with a realistic road network, it is shown that our SAINT outperforms the legacy navigation scheme, which is based on Dijkstra's algorithm with a real-time road traffic snapshot. On a road map of Manhattan in New York City, our SAINT can significantly reduce the travel delay during rush hours by 19%. [ABSTRACT FROM PUBLISHER]