Search

Recent advancements in connected automated vehicles (CAVs) and reinforcement learning (RL) hold significant promise for enhancing intelligent traffic control systems. This paper conducts a systematic review of studies on RL-based urban traffic control at signalised intersections, highlighting the significant impact of CAVs on traffic control performance improvement. We first review the fundamental concepts of RL algorithms, establishing a foundational understanding for subsequent RL-based traffic control methods. We then review recent progress in RL-based traffic signal control using CV/CAV trajectory data, RL-based CAV trajectory planning, and the cooperative control of both traffic signals and CAVs at signalised intersections. Our aim is to provide researchers with a comprehensive roadmap for future research in RL-based traffic control at signalised intersections. [ABSTRACT FROM AUTHOR]

With the development of autonomous driving technology utilizing machine learning, AI, and sensors, research on autonomous driving control has become more active, and a large number of innovative studies are underway. In the near future, all autonomous vehicle fleets will be able to communicate with each other for sharing information and overall optimal traffic control will be achieved. One of the vehicle control systems that are based on the premise of such a fully automated society is the "signal-less intersection." There is an intersection traffic control method that achieves safe and rational route selection by using virtual walls (VWs), which are virtual obstacles, but there are issues in terms of total route length and reduction of computation time. To address the issues, we propose a method that (1) prunes unneeded paths and (2) arranges VWs in a stepwise manner. The effectiveness of the proposed method was evaluated by simulation, and the results showed that the total route length and execution time were reduced. [ABSTRACT FROM AUTHOR]

Three-colour four-way traffic light completed 100 years in 2020. Even though the traffic light in the form of Semaphore arms has been in use in London since 1868, electric traffic lights came into existence in 1912 and the standard three-colour four-way light in 1920. Research is continuously being carried out to develop better algorithms to improve safety, reduce travel delays, and optimize road capacity. Hence a review of the evolution of traffic lights is warranted. This paper presents an all-round review using a six-prong approach. Timeline of the evolution of the literature in the last 100 years, the evolution of hardware, algorithms, traffic control schemes, standards and the pedestrian lights and count down timer are the six areas in which the review is carried out. A timeline of the different keywords related to the various algorithms in use is presented. This article delves into the thinking and meticulous approach of early researchers and practitioners of the field while dwelling on the past. They laid the rock-solid foundation of today's research. Also, future research areas like connected vehicles and automated vehicles are pointed out, and a summary of the findings is presented at the end. [ABSTRACT FROM AUTHOR]

• A virtual risk computation based on matching degree was introduced. • A universal planning method for pre-merging in mixed traffic and regular car-following has been proposed. • A virtual risk-based longitudinal planning model has been formulated. • Enhanced stability and improvements in safety are characteristics of the proposed planning approach. Ramp-merging zones have perennially served as bottlenecks for both safety and efficiency within road traffic. Addressing the inherent uncertainties and anomalous behaviors potentially exhibited by human-driven vehicles (HDVs) in mixed traffic flow, this study introduced a distributed merging control algorithm based on virtual risk assessment. Initially, we proposed a progressive vehicle projection method based on matching degree, formulated to engender smooth longitudinal accelerations. Building upon this foundation, a virtual risk-based longitudinal planning model named V-RQM was established, furnishing longitudinal accelerations with robust stability and safety to Connected and Autonomous Vehicles (CAVs). Results from four diverse simulation experiments corroborate that V-RQM is aptly versatile, suited for conventional one-dimensional car-following environments as well as pre-merging control in heterogeneous traffic flow. The model not only mitigates oscillatory amplitudes while sustaining high traffic efficiency but also exhibits commendable resilience against uncertainties and anomalies indigenous to mixed traffic flow. This research pioneers a novel model-driven approach to ramp-merging control, setting the stage for the full exploitation of CAV capabilities. [ABSTRACT FROM AUTHOR]

To reduce energy consumption and transportation emissions during car-following behavior on highways, we propose a car-following model for connected automated vehicles (CAVs). This new model considers spacing variation of the surrounding vehicles located immediately upstream and downstream. We optimize the key parameter of forward control weight of the proposed CAV model to stabilize traffic flow and ensure stable conditions. Then simulation experiments are conducted to validate the effectiveness of our proposed CAV model in enhancing traffic flow stability and energy-saving in mixed traffic consisting of both CAVs and regular vehicles (RVs). Results show that traffic flow controlled by the proposed CAV model is less affected by disturbances. The increase of CAV penetration rates can gradually improve stability of the mixed traffic flow. CAVs equipped with our proposed model can effectively reduce energy consumption and transportation emissions, which decrease with an increase of CAV penetration rates under constant speed conditions. When CAV penetration rate reaches 100 %, the average reduction of energy consumption, CO 2 emissions, and NO x emissions at various speeds reach a peak value of 22.15 %, 31.00 %, and 56.41 %, respectively. Speed also has significant influence on energy consumption and emissions, with potential reductions when speed falls within an appropriate range. • Propose a car-following model of connected automated vehicles (CAVs) for energy-saving. • Present an optimization design for our proposed model to stabilize traffic flow with CAVs. • Conduct simulation experiments to validate the effectiveness of the proposed CAV car-following model in energy-saving. [ABSTRACT FROM AUTHOR]

Urban traffic control (UTC) is inherently multiscale in both temporal and spatial domains. With the wide deployment of connected and automated vehicles (CAVs), data is increasingly available to help reveal this multiscale nature of UTC and the inter-dynamics among different scales. This paper applies the multiscale UTC framework we proposed earlier and extends it to UTC on a network of traffic signals with a mixed flow of CAVs and human-driven vehicles (HDVs). We adopt distributed control as the basic scheme for network-wide control and use information sharing to achieve cooperation among different intersections. We use CAV information to estimate HDV state and develop a "safety check" technique to control CAVs in the mixed traffic flow. Together, we propose a network-wide UTC framework with mixed traffic flow. To address the computation issue of the model-based multiscale method, we develop an imitation learning (IL) enhanced data-driven method to improve the computation efficiency. Specifically, we use a convolutional neural network (CNN) to represent the policies of the slower-scale signal control problem and use the data aggregation method as the learning framework to improve the policies. This algorithm's unique feature is that IL policies' training is based on the optimized results from the model-based multiscale control method. We test the model-based and IL-based methods in simulation, under various traffic scenarios and on multiple networks. We also test the transferability property of the IL-based method by training individual intersection control separately in small networks and applying them to larger networks with various types of intersections. [ABSTRACT FROM AUTHOR]

• This paper proposes the Grid-Based Conflict Index (GBCI) to quantify the collision risks and severity among the vehicles. • This paper introduced a calculation method for the grid-based Post Encroachment Time (PET) and the total kinetic energy change. • This paper proposed a traffic-safety-based AIM model aimed at minimizing the weighted sum of total delay and conflict risk at the intersection. Existing studies on autonomous intersection management (AIM) primarily focus on traffic efficiency, often overlooking the overall intersection safety, where conflict separation is simplified and traffic conflicts are inadequately assessed. In this paper, we introduce a calculation method for the grid-based Post Encroachment Time (PET) and the total kinetic energy change before and after collisions. The improved grid-based PET metric provides a more accurate estimation of collision probability, and the total kinetic energy change serves as a precise measure of collision severity. Consequently, we establish the Grid-Based Conflict Index (GBCI) to systematically quantify collision risks between vehicles at an autonomous intersection. Then, we propose a traffic-safety-based AIM model aimed at minimizing the weighted sum of total delay and conflict risk at the intersection. This entails the optimization of entry time and trajectory for each vehicle within the intersection, achieving traffic control that prioritizes overall intersection safety. Our results demonstrate that GBCI effectively assesses conflict risks within the intersection, and the proposed AIM model significantly reduces conflict risks between vehicles and enhances traffic safety while ensuring intersection efficiency. [ABSTRACT FROM AUTHOR]

The shared autonomous vehicle (SAV), combining shared mobility and vehicle automation, will play a pivotal role in urban transportation. Effectively matching SAVs to stochastic customer demand poses a significant challenge, particularly when addressing microscopic multi-AV coordination to prevent conflicts and congestion within a network. To tackle this issue, we propose a novel traffic control scheme called the Integrated Operation Framework for Order Matching and AV Coordination (I-OFOMAC). This scheme introduces a third-party coordinator responsible for integrating online order matching for the SAV platform and real-time AV coordination for the road manager. Our approach utilizes an offline deep reinforcement learning method to train a state value neural network, which facilitates strategic order matching. Additionally, a neural network approximating bisimulation metrics is trained to enhance exploration and accelerate learning. To achieve scalability in network-level AV coordination, we adopt a scheduled transportation scheme. The coordinator solves the I-OFOMAC during each decision epoch, optimizing order matching and AV coordination. The I-OFOMAC aims to balance the goals of minimizing AV travel delay and maximizing SAV service revenue. We conduct extensive simulation studies in the realistic Wangjing network to evaluate the proposed I-OFOMAC scheme. Compared to two-step benchmark frameworks that implement order matching and AV coordination sequentially, the I-OFOMAC achieves an average increase of 13.28% in platform revenue (ranging from 2% to 22.53%) and a 4.48% reduction in average PAV travel delay (ranging from −6.55% to 20.17%). • Mixed AV zone. • Online operations of joint order matching and AV coordination. • Offline deep reinforcement learning framework. • Bisimulation-guided exploration. • A scalable approximate algorithm. • Numerical studies in realistic Wangjing network. [ABSTRACT FROM AUTHOR]

Traditional traffic signal control is one of the most widely used urban traffic control methods, due to its high efficiency, robustness, and simplicity. However, in the era of connected and automated vehicles (CAVs), it is prospective to enhance traffic control precision and achieve vehicle coordination in a more efficient manner. Focusing on an isolated intersection, this study proposes an innovative concept of Fine-grained Phase (FP) for CAV-enabled traffic control, which realizes more refined right-of-way allocation than traditional signal phases while preserving the simplicity and scalability. The development of FP jointly optimizes traffic control strategy and intersection layout design to fully utilize time-space resources, and multiple forms of FP could be generated in response to various traffic demands; meanwhile, vehicle coordination is achieved in an offline–online manner to balance solution quality and efficiency. Under the FP-based control, each lane is assigned with periodical entry time points using FP design results, then vehicles are supposed to catch up with the appropriate time points to fulfill their trips. Theoretical analyses are conducted under stationary vehicle arrival patterns, where the average delay and the intersection capacity associated with different forms of FP can be analytically acquired. For the implementation of FP-based control, we construct a general offline–online framework, consisting of an offline model that optimizes the geometry of the conflict zone at intersections and the control strategy of vehicles simultaneously, which is formulated as a mixed-integer linear programming problem, and an online FP adoption mechanism that can adapt to varying demand patterns. Under the framework, limited online computation is required, facilitating its practical real-time implementation. Simulation studies under a variety of demand settings show that FP-based control is advantageous over benchmark methods in reducing vehicle delay and also improving intersection capacity in most cases; even for cases with extremely imbalanced demands, FP-based control can admit comparable throughput, while the average delay is much smaller. In addition, results of computation time reveal that FP-based control could realize real-time implementation with negligible computation burden. • An innovative concept called FP for intersection traffic control with CAVs. • Joint optimization of traffic control and intersection layout design. • Offline–online implementation framework to balance solution quality and efficiency. • Theoretical analysis for average vehicle delay and admissible traffic capacity. • Improvement over signalized control and the state-of-art reservation strategy. [ABSTRACT FROM AUTHOR]

With the development of autonomous driving and vehicle-to-infrastructure technologies, connected automated vehicles (CAVs) will gradually replace human-driven vehicles (HDVs). However, during the widespread adoption of CAVs, the roads will experience a new type of mixed traffic flow composed of vehicles with different automation levels for an extended period. To analyze the impact of different automation levels on mixed traffic flow, this paper proposes a stability analysis framework for mixed traffic flow that considers different automation levels. The influence of automation level classification on the composition of mixed traffic flow is analyzed first, and the degradation of CAVs in mixed traffic flow is further explored. Then, a stability analysis framework for mixed traffic flow considering different automation levels is proposed, and the impact of key parameters on the stability is analyzed. Finally, simulation experiments are designed to validate the theoretical analysis results. The research findings indicate that (1) the stability of mixed traffic flow improves with the increase in steady-state speed. When vehicle speed exceeds 20 m/s, mixed traffic flow remains stable. (2) When the CAV penetration rate is low, the stability of mixed traffic flow is poor due to the degradation of CAVs. However, with the increase in the penetration rate and automation level of CAVs, mixed traffic flow rapidly tends toward a stable state. (3) Numerous factors influence the stability of mixed traffic flow, with the order of importance being the CAV penetration rate, automation level, headway, and maximum acceleration. Therefore, this study can provide theoretical support for managing and controlling mixed traffic flow with different automation levels in the future. [ABSTRACT FROM AUTHOR]

The emerging technologies in both wireless communication and automation driving that have enabled vehicles to communicate with infrastructure, and with each other, are collectively known as connected vehicle and automated vehicle (CAV) technology. In existing connected automated vehicle-based and traditional traffic control systems, urban arterials with signalized intersections still face issues, including local demand uncertainty and automation level compatibility. Here, a distributed signal control system that integrates automated vehicle path planning information is adapted to address these issues. The proposed method includes planning-based local demand estimation and demand-aware max pressure control in an edge computing environment. A bivariable automation model is developed to represent the various, i.e., L1-L5, automatic driving levels. A real district with typical scenarios is implemented to verify the proposed method. The simulation results show that the proposed demand-aware edge control provides better control results than benchmarks for almost all different performance indices and network demands. The proposed control reduces the waiting time by up to 30%, and the time loss decreased by up to 20%. Moreover, better performances are always obtained at different automation levels, so there is a good compatibility of the automation driving for the adaptation of the CAV flows, especially at the L3-L5 levels. [ABSTRACT FROM AUTHOR]

• Here are the highlights to this paper: • Classify various cyberattaks injection modes. • Propose three car-following models to analyze the cyberattack effects on platoon. • Apply six indicators to comprehensively evaluate driver tolerance, vehicle adaptability, and environmental resistance. The high-level integration and interaction between the information flow at the cyber layer and the physical subjects at the vehicular layer enables the connected automated vehicles (CAVs) to achieve rapid, cooperative and shared travel. However, the cyber layer is challenged by malicious attacks and the shortage of communication resources, which makes the vehicular layer suffer from system nonlinearity, disturbance randomness and behavior uncertainty, thus interfering with the stable operation of the platoon. So far, scholars usually adopt the method of assuming or improving the car-following model to explore the platoon behavior and the defense mechanism in cyberattacks, but they have not considered whether the model itself has disturbance and impact on cyberattack defenses. In other words, it is still being determined whether the car-following model designed can be fully applicable to such cyberattacks. To provide a theoretical basis for vehicular layer modeling, it is necessary to comprehend the self-resistance of different car-following models faced on various cyberattacks. First, we review the car-following models adopted on the vehicular layer in cyberattacks, involving traffic engineering, physical statistics, and platoon dynamics. Based on the review, we divide the malicious attacks faced by the cyber layer into explicit attacks and implicit attacks. Second, we develop a cooperative generalized force model (CGFM), which combines and unifies the r-predecessors following communication topology. The proposed models, labeled the vulnerable cooperative intelligent driver model (VCIDM), the vulnerable cooperative optimal velocity model (VCOVM), and the vulnerable cooperative platoon dynamics model (VCPDM), incorporate the CGFM model and assorted cyberattack injection modes to explain the cyberattack effects on the platoon self-resistance capability. Upon the described models, we provide six indicators in three dimensions from the basic traffic element, including drivers, vehicles, and environment. These indicators illustrate driver tolerance, vehicle adaptability, and environmental resistance when a platoon faces attacks such as bogus information, replay/delay, and communication interruption. We arrange and reorganize the car-following models and the cyberattack injection modes to complete the research on the self-resistance capability of the platoon, which has positive research value and practical significance for enhancing the endogenous security at the vehicular layer and improving the intrusion tolerability at the cyber layer. [ABSTRACT FROM AUTHOR]

• Thirty autonomous vehicle (AV) intersection pre-crash scenarios were identified. • The proportion of AVs being rear-ended by conventional vehicles was 58.38%. • Contributing factors of AV scenarios were identified using association rules. • Key roadway factors were intersection-related area, signal control, and on-street parking. • Insufficient stop, deceleration, and collision avoidance decisions were important causes for AV crashes. Intersections are high-risk locations for autonomous vehicles (AVs). Crash causation analysis based on pre-crash scenarios can provide new insight into these crashes that can lead to effective countermeasures, but there are significant differences in pre-crash scenarios between autonomous and conventional vehicles, and inadequate AV data has put limits on research. The association rule method, however, can yield useful results despite these limits. This study therefore aims to use the method with pre-crash scenarios to understand the characteristics and contributing factors of AV crashes at intersections from the latest 5-year AV crash data. Analysis of 197 AV crashes at intersections revealed 30 types of pre-crash scenarios. The rear-end crash (58.88%) and lane change crash (16.24%) were the most frequently occurring scenarios for AVs. The proportion of AVs being rear-ended by conventional vehicles was 58.38%. The main contributing factors of these two most common AV scenarios were identified by association rules and crash causes were analyzed from the perspective of AV decision-making. The main factors contributing to the AV rear-end scenario were location outside the intersection in the intersection-related area, traffic signal control, autonomous engaged mode, mixed-use or public land, and weekdays, while those for lane change scenarios were on-street parking and the time of 8:00 a.m. Important causes of rear-end crashes attributable to the AV were inadequate stop and deceleration decisions by the AV's automated driving system (ADS) and insufficient collision avoidance decisions in lane change crashes. Identification of the pre-crash characteristics and contributing factors provide new insight into AV crash causation and can be used in the determination of the AV's operational design domain and the development and optimization of the AV's ADS at intersections. These findings can also play a role in guiding traffic safety agencies to discover AV hotspots and propose AV management regulations. [ABSTRACT FROM AUTHOR]

• An internal model-Kalman filtering-based optimal hybrid feedforward/feedback control strategy is developed. • A practically decoupled platoon regulation around an equilibrium state is established. • String stability of the control system thus obtained is analyzed. • Numerical simulations verified the theoretical analysis in decoupling between successive vehicles. This paper presents an internal model-Kalman filtering-based optimal hybrid feedforward/feedback control strategy for traffic platoon control coordination enabled by SAE Automation Level 2 or Level 3 vehicles, i.e., partially automated vehicles (PAVs). Based on the Helly linear car-following model, a PAV platoon is established. Taking each vehicle's characteristic polynomial as the dominant internal model polynomial, an augmented system state model with filtered inputs is formed, and then both the system states and the external disturbance/internal perturbation from previous vehicle can be estimated by a delicately designed Kalman filter. By using a linear quadratic regulation approach, a distributed hybrid optimal feedforward/feedback controller utilizing the local estimated states is constructed to achieve a practically decoupled platoon regulation around an equilibrium state. String stability of the control system thus obtained is further analyzed. Finally, extensive numerical simulations verified the theoretical analysis in decoupling between successive vehicles, in suppressing the influence of various external disturbances, and in maintaining string stability. [ABSTRACT FROM AUTHOR]

• Develop a framework including both stability analysis and implementation constraints for traffic platoon control via sparse autonomous vehicles. • Identify the number of human-driven vehicles that can be stabilized by one single autonomous vehicle. • Establish error dynamics in traffic platoon considering multiple disturbances and error composition. • Propose a novel ring string stability that is flexible and applicable to mixed ring traffic. The emergence of autonomous vehicles (AVs) is expected to revolutionize road transportation and it has been demonstrated that AVs can be employed as mobile actuators to effectively dissipate stop and go waves. This paper proposes a comprehensive theoretical framework including stability and implementation considerations for analyzing traffic platoon control via sparse AVs and discussing the number of HDVs that can be stabilized by one AV. Considering multiple disturbance sources, we first discuss the output error composition and analyze traffic dynamics in a platoon system. Then we discuss the Lyapunov asymptotic stability (LAS) of the system based on eigenvalue analysis and propose a novel ring string stability (RSS), which ensures the boundedness of disturbance error and guarantees that the effect of disturbance will not be amplified over a lap. The proposed RSS is flexible and applicable to mixed traffic, since the string instability for predecessor-follower pairs are allowed, provided they can be compensated by AVs. In addition, the practical constraints including driving safety and vehicle physical limitations are modeled. The worst-case evaluation of constraints is conducted to reproduce the critical scenarios in real life. Finally, we conduct numerical experiments and analyze the traffic platoon control from the aspects of both stability and practical implementations. New findings include: (1) The introduction of AVs is beneficial to dampen the waves and stabilize the HDV flow. (2) Considering the implementation issues, there is an upper bound of HDVs that can be stabilized by one single AV. (3) As the number and amplitudes of exogenous disturbances increase, it will be more challenging for sparse AVs to ensure stability and meanwhile satisfy the practical constraints. [ABSTRACT FROM AUTHOR]