Search

Autonomous vehicles are rapidly advancing and have the potential to revolutionize transportation in the future. This paper primarily focuses on vehicle motion trajectory planning algorithms, examining the methods for estimating collision risks based on sensed environmental information and approaches for achieving user-aligned trajectory planning results. It investigates the different categories of planning algorithms within the scope of local trajectory planning applications for autonomous driving, discussing and differentiating their properties in detail through a review of the recent studies. The risk estimation methods are classified and introduced based on their descriptions of the sensed collision risks in traffic environments and their integration with trajectory planning algorithms. Additionally, various user experience-oriented methods, which utilize human data to enhance the trajectory planning performance and generate human-like trajectories, are explored. The paper provides comparative analyses of these algorithms and methods from different perspectives, revealing the interconnections between these topics. The current challenges and future prospects of the trajectory planning tasks in autonomous vehicles are also discussed. [ABSTRACT FROM AUTHOR]

Object detection is one of the core technologies for autonomous driving. Current road object detection mainly relies on visible light, which is prone to missed detections and false alarms in rainy, night-time, and foggy scenes. Multispectral object detection based on the fusion of RGB and infrared images can effectively address the challenges of complex and changing road scenes, improving the detection performance of current algorithms in complex scenarios. However, previous multispectral detection algorithms suffer from issues such as poor fusion of dual-mode information, poor detection performance for multi-scale objects, and inadequate utilization of semantic information. To address these challenges and enhance the detection performance in complex road scenes, this paper proposes a novel multispectral object detection algorithm called MRD-YOLO. In MRD-YOLO, we utilize interaction-based feature extraction to effectively fuse information and introduce the BIC-Fusion module with attention guidance to fuse different modal information. We also incorporate the SAConv module to improve the model's detection performance for multi-scale objects and utilize the AIFI structure to enhance the utilization of semantic information. Finally, we conduct experiments on two major public datasets, FLIR_Aligned and M3FD. The experimental results demonstrate that compared to other algorithms, the proposed algorithm achieves superior detection performance in complex road scenes. [ABSTRACT FROM AUTHOR]

This paper presents the development of path-tracking control strategies for an over-actuated autonomous electric vehicle. The vehicle platform is equipped with four-wheel steering (4WS) as well as torque vectoring (TV) capabilities, which enable the control of vehicle dynamics to be enhanced. A nonlinear model predictive controller is proposed taking into account the nonlinearities in vehicle dynamics at the limits of handling as well as the crucial actuator constraints. Controllers with different actuation formulations are presented and compared to study the path-tracking performance of the vehicle with different levels of actuation. The controllers are implemented in a high-fidelity simulation environment considering scenarios of vehicle handling limits. According to the simulation results, the vehicle achieves the best overall path-tracking performance with combined 4WS and TV, which illustrates that the over-actuation topology can enhance the path-tracking performance during conditions under the limits of handling. In addition, the performance of the over-actuation controller is further assessed with different sampling times as well as prediction horizons in order to investigate the effect of such parameters on the control performance, and its capability for real-time execution. In the end, the over-actuation control strategy is implemented on a target machine for real-time validation. The control formulation proposed in this paper is proven to be compatible with different levels of actuation, and it is also demonstrated in this work that it is possible to include the particular over-actuation formulation and specific nonlinear vehicle dynamics in real-time operation, with the sampling time and prediction time providing a compromise between path-tracking performance and computational time. [ABSTRACT FROM AUTHOR]

This paper considers the interactive effects between the ego vehicle and other vehicles in a dynamic driving environment and proposes an autonomous vehicle lane-changing behavior decision-making and trajectory planning method based on graph convolutional networks (GCNs) and multi-segment polynomial curve optimization. Firstly, hierarchical modeling is applied to the dynamic driving environment, aggregating the dynamic interaction information of driving scenes in the form of graph-structured data. Graph convolutional neural networks are employed to process interaction information and generate ego vehicle's driving behavior decision commands. Subsequently, collision-free drivable areas are constructed based on the dynamic driving scene information. An optimization-based multi-segment polynomial curve trajectory planning method is employed to solve the optimization model, obtaining collision-free motion trajectories satisfying dynamic constraints and efficiently completing the lane-changing behavior of the vehicle. Finally, simulation and on-road vehicle experiments are conducted for the proposed method. The experimental results demonstrate that the proposed method outperforms traditional decision-making and planning methods, exhibiting good robustness, real-time performance, and strong scenario generalization capabilities. [ABSTRACT FROM AUTHOR]

For the RRT* algorithm, there are problems such as greater randomness, longer time consumption, more redundant nodes, and inability to perform local obstacle avoidance when encountering unknown obstacles in the path planning process of autonomous vehicles. And the artificial potential field method (APF) applied to autonomous vehicles is prone to problems such as local optimality, unreachable targets, and inapplicability to global scenarios. A fusion algorithm combining the improved RRT* algorithm and the improved artificial potential field method is proposed. First of all, for the RRT* algorithm, the concept of the artificial potential field and probability sampling optimization strategy are introduced, and the adaptive step size is designed according to the road curvature. The path post-processing of the planned global path is carried out to reduce the redundant nodes of the generated path, enhance the purpose of sampling, solve the problem where oscillation may occur when expanding near the target point, reduce the randomness of RRT* node sampling, and improve the efficiency of path generation. Secondly, for the artificial potential field method, by designing obstacle avoidance constraints, adding a road boundary repulsion potential field, and optimizing the repulsion function and safety ellipse, the problem of unreachable targets can be solved, unnecessary steering in the path can be reduced, and the safety of the planned path can be improved. In the face of U-shaped obstacles, virtual gravity points are generated to solve the local minimum problem and improve the passing performance of the obstacles. Finally, the fusion algorithm, which combines the improved RRT* algorithm and the improved artificial potential field method, is designed. The former first plans the global path, extracts the path node as the temporary target point of the latter, guides the vehicle to drive, and avoids local obstacles through the improved artificial potential field method when encountered with unknown obstacles, and then smooths the path planned by the fusion algorithm, making the path satisfy the vehicle kinematic constraints. The simulation results in the different road scenes show that the method proposed in this paper can quickly plan a smooth path that is more stable, more accurate, and suitable for vehicle driving. [ABSTRACT FROM AUTHOR]

The paper demonstrates the validity of sensors and the model in the algorithm for a lane change controller. The paper presents the systematic derivation of the chosen model from the ground up and the important role played by the sensors used in this system. The whole concept of the system on which the tests were carried out is presented step by step. Simulations were realised in the Matlab and Simulink environments. Preliminary tests were performed to confirm the need for the controller in a closed-loop system. On the other hand, sensitivity (the influence of noise and offset) studies showed the advantages and disadvantages of the developed algorithm. This allowed us to create a research path for future work with the aim of improving the operation of the proposed system. [ABSTRACT FROM AUTHOR]

Complete autonomous systems such as self-driving cars to ensure the high reliability and safety of humans need the most efficient combination of four-dimensional (4D) detection, exact localization, and artificial intelligent (AI) networking to establish a fully automated smart transportation system. At present, multiple integrated sensors such as light detection and ranging (LiDAR), radio detection and ranging (RADAR), and car cameras are frequently used for object detection and localization in the conventional autonomous transportation system. Moreover, the global positioning system (GPS) is used for the positioning of autonomous vehicles (AV). These individual systems' detection, localization, and positioning efficiency are insufficient for AV systems. In addition, they do not have any reliable networking system for self-driving cars carrying us and goods on the road. Although the sensor fusion technology of car sensors came up with good efficiency for detection and location, the proposed convolutional neural networking approach will assist to achieve a higher accuracy of 4D detection, precise localization, and real-time positioning. Moreover, this work will establish a strong AI network for AV far monitoring and data transmission systems. The proposed networking system efficiency remains the same on under-sky highways as well in various tunnel roads where GPS does not work properly. For the first time, modified traffic surveillance cameras have been exploited in this conceptual paper as an external image source for AV and anchor sensing nodes to complete AI networking transportation systems. This work approaches a model that solves AVs' fundamental detection, localization, positioning, and networking challenges with advanced image processing, sensor fusion, feathers matching, and AI networking technology. This paper also provides an experienced AI driver concept for a smart transportation system with deep learning technology. [ABSTRACT FROM AUTHOR]

In this paper, the low-level velocity controller of an autonomous vehicle is studied. The performance of the traditional controller used in this kind of system, a PID, is analyzed. This kind of controller cannot follow ramp references without error, so when the reference implies a change in the speed, the vehicle cannot follow the proposed reference, and there is a significant difference between the actual and desired vehicle behaviors. A fractional controller is proposed which changes the ordinary dynamics allowing faster responses for small times, at the cost of slower responses for large times. The idea is to take advantage of this fact to follow fast setpoint changes with a smaller error than that obtained with a classic non-fractional PI controller. Using this controller, the vehicle can follow variable speed references with zero stationary error, significantly reducing the difference between reference and actual vehicle behavior. The paper presents the fractional controller, studies its stability in function of the fractional parameters, designs the controller, and tests its stability. The designed controller is tested on a real prototype, and its behavior is compared to a standard PID controller. The designed fractional PID controller overcomes the results of the standard PID controller. [ABSTRACT FROM AUTHOR]

Intersections are at the core of congestion in urban areas. After the end of the Second World War, the problem of intersection management has benefited from a growing body of advances to address the optimization of the traffic lights' phase splits, timing, and offset. These contributions have significantly improved traffic safety and efficiency in urban areas. However, with the growth of transportation demand and motorization, traffic lights show their limits. At the end of the 1990s, the perspective of autonomous and connected driving systems motivated researchers to introduce a paradigm shift for controlling intersections. This new paradigm is well known today as autonomous intersection management (AIM). It harnesses the self-organization ability of future vehicles to provide more accurate control approaches that use the smallest available time window to reach unprecedented traffic performances. This is achieved by optimizing two main points of the interaction of connected and autonomous vehicles at intersections: the motion control of vehicles and the schedule of their accesses. Considering the great potential of AIM and the complexity of the problem, the proposed approaches are very different, starting from various assumptions. With the increasing popularity of AIM, this paper provides readers with a comprehensive vision of noticeable advances toward enhancing traffic efficiency. It shows that it is possible to tailor vehicles' speed and schedule according to the traffic demand by using distributed particle swarm optimization. Moreover, it brings the most relevant contributions in the light of traffic engineering, where flow–speed diagrams are used to measure the impact of the proposed optimizations. Finally, this paper presents the current challenging issues to be addressed. [ABSTRACT FROM AUTHOR]

Autonomous driving technology plays an essential role in reducing road traffic accidents and ensuring more convenience while driving, so it has been widely studied in industrial and academic communities. The lane-changing decision-making process is challenging but critical for ensuring autonomous vehicles' (AVs) safe and smooth maneuvering. This paper presents a closed-loop lane-changing behavioral decision-making framework suitable for AVs in fully autonomous driving environments to achieve both safety and high efficiency. The framework is based on a complete information non-cooperative game theory. Moreover, we attempt to introduce human driver-specific driving styles (reflected by aggressiveness types) and micro-interaction behaviors for both sides of the game in this model, enabling users to understand, adapt, and utilize intelligent lane-changing techniques. Additionally, a model predictive control controller based on the host-vehicle (HV) driving risk field (DRF) is proposed. The controller's optimizer is used to find the optimal path with the lowest driving risk by using its optimizer and simultaneously adjusting its control variables to track the path. The method can synchronize path planning and motion control and provide real-time vehicle state feedback to the decision-making module. Simulations in several typical traffic scenarios demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

With the advent of autonomous vehicles, sensors and algorithm testing have become crucial parts of the autonomous vehicle development cycle. Having access to real-world sensors and vehicles is a dream for researchers and small-scale original equipment manufacturers (OEMs) due to the software and hardware development life-cycle duration and high costs. Therefore, simulator-based virtual testing has gained traction over the years as the preferred testing method due to its low cost, efficiency, and effectiveness in executing a wide range of testing scenarios. Companies like ANSYS and NVIDIA have come up with robust simulators, and open-source simulators such as CARLA have also populated the market. However, there is a lack of lightweight and simple simulators catering to specific test cases. In this paper, we introduce the SLAV-Sim, a lightweight simulator that specifically trains the behaviour of a self-learning autonomous vehicle. This simulator has been created using the Unity engine and provides an end-to-end virtual testing framework for different reinforcement learning (RL) algorithms in a variety of scenarios using camera sensors and raycasts. [ABSTRACT FROM AUTHOR]

Ensuring that intelligent vehicles do not cause fatal collisions remains a persistent challenge due to pedestrians' unpredictable movements and behavior. The potential for risky situations or collisions arising from even minor misunderstandings in vehicle–pedestrian interactions is a cause for great concern. Considerable research has been dedicated to the advancement of predictive models for pedestrian behavior through trajectory prediction, as well as the exploration of the intricate dynamics of vehicle–pedestrian interactions. However, it is important to note that these studies have certain limitations. In this paper, we propose a novel graph-based trajectory prediction model for vehicle–pedestrian interactions called Holistic Spatio-Temporal Graph Attention (HSTGA) to address these limitations. HSTGA first extracts vehicle–pedestrian interaction spatial features using a multi-layer perceptron (MLP) sub-network and max pooling. Then, the vehicle–pedestrian interaction features are aggregated with the spatial features of pedestrians and vehicles to be fed into the LSTM. The LSTM is modified to learn the vehicle–pedestrian interactions adaptively. Moreover, HSTGA models temporal interactions using an additional LSTM. Then, it models the spatial interactions among pedestrians and between pedestrians and vehicles using graph attention networks (GATs) to combine the hidden states of the LSTMs. We evaluate the performance of HSTGA on three different scenario datasets, including complex unsignalized roundabouts with no crosswalks and unsignalized intersections. The results show that HSTGA outperforms several state-of-the-art methods in predicting linear, curvilinear, and piece-wise linear trajectories of vehicles and pedestrians. Our approach provides a more comprehensive understanding of social interactions, enabling more accurate trajectory prediction for safe vehicle navigation. [ABSTRACT FROM AUTHOR]

Methods based on 64-beam LiDAR can provide very precise 3D object detection. However, highly accurate LiDAR sensors are extremely costly: a 64-beam model can cost approximately USD 75,000. We previously proposed SLS–Fusion (sparse LiDAR and stereo fusion) to fuse low-cost four-beam LiDAR with stereo cameras that outperform most advanced stereo–LiDAR fusion methods. In this paper, and according to the number of LiDAR beams used, we analyzed how the stereo and LiDAR sensors contributed to the performance of the SLS–Fusion model for 3D object detection. Data coming from the stereo camera play a significant role in the fusion model. However, it is necessary to quantify this contribution and identify the variations in such a contribution with respect to the number of LiDAR beams used inside the model. Thus, to evaluate the roles of the parts of the SLS–Fusion network that represent LiDAR and stereo camera architectures, we propose dividing the model into two independent decoder networks. The results of this study show that—starting from four beams—increasing the number of LiDAR beams has no significant impact on the SLS–Fusion performance. The presented results can guide the design decisions by practitioners. [ABSTRACT FROM AUTHOR]

It is important to estimate the exact depth from 2D images, and many studies have been conducted for a long period of time to solve depth estimation problems. Recently, as research on estimating depth from monocular camera images based on deep learning is progressing, research for estimating accurate depths using various techniques is being conducted. However, depth estimation from 2D images has been a problem in predicting the boundary between objects. In this paper, we aim to predict sophisticated depths by emphasizing the precise boundaries between objects. We propose a depth estimation network with encoder–decoder structures using the Laplacian pyramid and local planar guidance method. In the process of upsampling the learned features using the encoder, the purpose of this step is to obtain a clearer depth map by guiding a more sophisticated boundary of an object using the Laplacian pyramid and local planar guidance techniques. We train and test our models with KITTI and NYU Depth V2 datasets. The proposed network constructs a DNN using only convolution and uses the ConvNext networks as a backbone. A trained model shows the performance of the absolute relative error (Abs_rel) 0.054 and root mean square error (RMSE) 2.252 based on the KITTI dataset and absolute relative error (Abs_rel) 0.102 and root mean square error 0.355 based on the NYU Depth V2 dataset. On the state-of-the-art monocular depth estimation, our network performance shows the fifth-best performance based on the KITTI Eigen split and the eighth-best performance based on the NYU Depth V2. [ABSTRACT FROM AUTHOR]

The automobile industry has developed dramatically in recent years, the supply of vehicles has also increased, and thus it has become deeply established in everyday life. Recently, as the supply of vehicles with autonomous driving functions increases, the safety of vehicles is also an emerging issue. Various car-following models for the safe driving of vehicles have long been studied by various people, and recently, a Responsibility-Sensitive Safety (RSS) model has been proposed by Mobileye. However, in existing car-following models or the RSS model, the safe distance between vehicles is presented using only vehicle speed and acceleration information, so there is a limitation in that it cannot respond to changes in road conditions due to the weather. In this paper, in order to ensure safety when the RSS model is applied to a variable focus function camera, an improved RSS model is presented in consideration of the changes in road conditions due to changes in weather, and a safety distance is derived based on the proposed model. [ABSTRACT FROM AUTHOR]

This paper presents a sliding mode control (SMC)-based path-tracking algorithm for autonomous vehicles by considering model-free adaptive feedback actions. In autonomous vehicles, safe path tracking requires adaptive and robust control algorithms because driving environment and vehicle conditions vary in real time. In this study, the SMC was adopted as a robust control method to adjust the switching gain, taking into account the sliding surface and unknown uncertainty to make the control error zero. The sliding surface can be designed mathematically, but it is difficult to express the unknown uncertainty mathematically. Information of priori bounded uncertainties is needed to obtain closed-loop stability of the control system, and the unknown uncertainty can vary with changes in internal and external factors. In the literature, ongoing efforts have been made to overcome the limitation of losing control stability due to unknown uncertainty. This study proposes an integrated method of adaptive feedback control (AFC) and SMC that can adjust a bounded uncertainty. Some illustrative and representative examples, such as autonomous driving scenarios, are also provided to show the main properties of the designed integrated controller. The examples show superior control performance, and it is expected that the integrated controller could be widely used for the path-tracking algorithms of autonomous vehicles. [ABSTRACT FROM AUTHOR]

This paper presents an interactive lane keeping model for an advanced driver assistant system and autonomous vehicle. The proposed model considers not only the lane markers but also the interaction with surrounding vehicles in determining steering inputs. The proposed algorithm is designed based on the Recurrent Neural Network (RNN) with long short-term memory cells, which are configured by the collected driving data. A data collection vehicle is equipped with a front camera, LiDAR, and DGPS. The input features of the RNN consist of lane information, surrounding targets, and ego vehicle states. The output feature is the steering wheel angle to keep the lane. The proposed algorithm is evaluated through similarity analysis and a case study with driving data. The proposed algorithm shows accurate results compared to the conventional algorithm, which only considers the lane markers. In addition, the proposed algorithm effectively responds to the surrounding targets by considering the interaction with the ego vehicle. [ABSTRACT FROM AUTHOR]

This paper presents a novel vehicular environment identification approach based on deep learning. It consists of exploiting the vehicular wireless channel characteristics in the form of Channel State Information (CSI) in the receiver side of a connected vehicle in order to identify the environment type in which the vehicle is driving, without any need to implement specific sensors such as cameras or radars. We consider environment identification as a classification problem, and propose a new convolutional neural network (CNN) architecture to deal with it. The estimated CSI is used as the input feature to train the model. To perform the identification process, the model is targeted for implementation in an autonomous vehicle connected to a vehicular network (VN). The proposed model is extensively evaluated, showing that it can reliably recognize the surrounding environment with high accuracy ( 96.48 % ). Our model is compared to related approaches and state-of-the-art classification architectures. The experiments show that our proposed model yields favorable performance compared to all other considered methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new free space detection algorithm for autonomous vehicle driving. Previous free space detection algorithms often use only the location information of every frame, without information on the speed of the obstacle. In this case, there is a possibility of creating an inefficient path because the behavior of the obstacle cannot be predicted. In order to compensate for the shortcomings of the previous algorithm, the proposed algorithm uses the speed information of the obstacle. Through object tracking, the dynamic behavior of obstacles around the vehicle is identified and predicted, and free space is detected based on this. In the free space, it is possible to classify an area in which driving is possible and an area in which it is not possible, and a route is created according to the classification result. By comparing and evaluating the path generated by the previous algorithm and the path generated by the proposed algorithm, it is confirmed that the proposed algorithm is more efficient in generating the vehicle driving path. [ABSTRACT FROM AUTHOR]

The fatigue life of orthotropic steel decks (OSDs) is significantly affected by vehicle loads, and the local stress response of OSDs is sensitive to the transverse position of vehicle loads. However, the presence of autonomous vehicles is likely to change the transverse distribution of vehicles within the lane, thereby affecting vehicle-induced fatigue damage to OSDs. Therefore, it is necessary to evaluate the potential effect of autonomous vehicles on the fatigue life of OSDs so that appropriate strategies can be implemented to control the transverse positions of autonomous vehicles passing the bridge deck. To this end, fatigue damages of several typical fatigue details in a conventional OSD (COSD) and a lightweight composite OSD (LWCD) induced by vehicle loads were calculated based on finite element analysis, and their fatigue lives were evaluated based on Miner's Rule, in which different transverse distribution patterns of autonomous vehicles and their proportions in the mixed traffic flow were considered. The results indicate that fatigue lives of both the COSD and the LWCD can be negatively affected by autonomous vehicles traveling across the bridge without any constraints on the transverse distribution, especially when their proportion in the mixed traffic flow exceeds 30%. Compared to the scenario without autonomous vehicles, the fatigue damage of most fatigue details in OSDs may increase by 51% to 210% in the most unfavorable case due to the presence of autonomous vehicles. Nevertheless, it is feasible to extend the fatigue life of OSDs by optimizing the transverse distribution of autonomous vehicles. Specifically, the fatigue life of most fatigue details in the COSD could be extended by more than 86% in the most favorable case when a bimodal Gaussian distribution is adopted as the transverse distribution pattern of autonomous vehicles. Moreover, both the negative and positive effects of autonomous vehicles on the fatigue life of the COSD are more significant than those of the LWCD in most cases. The results can provide references for the maintenance of OSDs under the action of autonomous vehicles. [ABSTRACT FROM AUTHOR]

This paper describes a real-time motion planner based on the drivers' visual behavior-guided rapidly exploring random tree (RRT) approach, which is applicable to on-road driving of autonomous vehicles. The primary novelty is in the use of the guidance of drivers' visual search behavior in the framework of RRT motion planner. RRT is an incremental sampling-based method that is widely used to solve the robotic motion planning problems. However, RRT is often unreliable in a number of practical applications such as autonomous vehicles used for on-road driving because of the unnatural trajectory, useless sampling, and slow exploration. To address these problems, we present an interesting RRT algorithm that introduces an effective guided sampling strategy based on the drivers' visual search behavior on road and a continuous-curvature smooth method based on B-spline. The proposed algorithm is implemented on a real autonomous vehicle and verified against several different traffic scenarios. A large number of the experimental results demonstrate that our algorithm is feasible and efficient for on-road autonomous driving. Furthermore, the comparative test and statistical analyses illustrate that its excellent performance is superior to other previous algorithms. [ABSTRACT FROM AUTHOR]

Electric-vehicle technology is an emerging area offering several benefits such as economy due to low running costs. Electric vehicles can also help to significantly reduce CO2 emission, which is a vital factor for environmental pollution. Modern vehicles are equipped with driver-assistance systems that facilitate drivers by offloading some of the tasks a driver does while driving. Human beings are prone to errors. Therefore, accidents and fatalities can happen if the driver fails to perform a particular task within the deadline. In electric vehicles, the focus has always been to optimize the power and battery life, and thus, any additional hardware can affect their battery life significantly. In this paper, the design of driver-assistance systems has been introduced to automate and assist in some of the vital tasks, such as a braking system, in an optimized manner. We revamp the idea of the traditional driver-assistance system and propose a generic lightweight system based on the leading factors and their impact on accidents. We model tasks for these factors and simulate a low-cost driver-assistance system in a real-time context, where these scenarios are investigated and tasks schedulability is formally proved before deploying them in electric vehicles. The proposed driver-assistance system offers many advantages. It decreases the risk of accidents and monitors the safety of driving. If, at some point, the risk index is above a certain threshold, an automated control algorithm is triggered to reduce it by activating different actuators. At the same time, it is lightweight and does not require any dedicated hardware, which in turn has a significant advantage in terms of battery life. Results show that the proposed system not only is accurate but also has a very negligible effect on energy consumption and battery life. [ABSTRACT FROM AUTHOR]

LiDAR-based Simultaneous Localization And Mapping (SLAM), which provides environmental information for autonomous vehicles by map building, is a major challenge for autonomous driving. In addition, the semantic information has been used for the LiDAR-based SLAM with the advent of deep neural network-based semantic segmentation algorithms. The semantic segmented point clouds provide a much greater range of functionality for autonomous vehicles than geometry alone, which can play an important role in the mapping step. However, due to the uncertainty of the semantic segmentation algorithms, the semantic segmented point clouds have limitations in being directly used for SLAM. In order to solve the limitations, this paper proposes a semantic segmentation-based LiDAR SLAM system considering the uncertainty of the semantic segmentation algorithms. The uncertainty is explicitly modeled by proposed probability models which are come from the data-driven approaches. Based on the probability models, this paper proposes semantic registration which calculates the transformation relationship of consecutive point clouds using semantic information with proposed probability models. Furthermore, the proposed probability models are used to determine the semantic class of the points when the multiple scans indicate different classes due to the uncertainty. The proposed framework is verified and evaluated by the KITTI dataset and outdoor environments. The experiment results show that the proposed semantic mapping framework reduces the errors of the mapping poses and eliminates the ambiguity of the semantic information of the generated semantic map. [ABSTRACT FROM AUTHOR]

With the continuous development of new sensor features and tracking algorithms for object tracking, researchers have opportunities to experiment using different combinations. However, there is no standard or agreed method for selecting an appropriate architecture for autonomous vehicle (AV) crash reconstruction using multi-sensor-based sensor fusion. This study proposes a novel simulation method for tracking performance evaluation (SMTPE) to solve this problem. The SMTPE helps select the best tracking architecture for AV crash reconstruction. This study reveals that a radar-camera-based centralized tracking architecture of multi-sensor fusion performed the best among three different architectures tested with varying sensor setups, sampling rates, and vehicle crash scenarios. We provide a brief guideline for the best practices in selecting appropriate sensor fusion and tracking architecture arrangements, which can be helpful for future vehicle crash reconstruction and other AV improvement research. [ABSTRACT FROM AUTHOR]

This paper describes the environment perception system designed for intelligent vehicle SmartV-II, which won the 2010 Future Challenge. This system utilizes the cooperation of multiple lasers and cameras to realize several necessary functions of autonomous navigation: road curb detection, lane detection and traffic sign recognition. Multiple single scan lasers are integrated to detect the road curb based on Z-variance method. Vision based lane detection is realized by two scans method combining with image model. Haar-like feature based method is applied for traffic sign detection and SURF matching method is used for sign classification. The results of experiments validate the effectiveness of the proposed algorithms and the whole system. [ABSTRACT FROM AUTHOR]

Road and sidewalk detection in urban scenarios is a challenging task because of the road imperfections and high sensor data bandwidth. Traditional free space and ground filter algorithms are not sensitive enough for small height differences. Camera-based or sensor-fusion solutions are widely used to classify drivable road from sidewalk or pavement. A LIDAR sensor contains all the necessary information from which the feature extraction can be done. Therefore, this paper focuses on LIDAR-based feature extraction. For road and sidewalk detection, the current paper presents a real-time (20 Hz+) solution. This solution can also be used for local path planning. Sidewalk edge detection is the combination of three algorithms working parallelly. To validate the result, the de facto standard benchmark dataset, KITTI, was used alongside our measurements. The data and the source code to reproduce the results are shared publicly on our GitHub repository. [ABSTRACT FROM AUTHOR]

Autonomous Vehicles (AVs) have the potential to solve many traffic problems, such as accidents, congestion and pollution. However, there are still challenges to overcome, for instance, AVs need to accurately perceive their environment to safely navigate in busy urban scenarios. The aim of this paper is to review recent articles on computer vision techniques that can be used to build an AV perception system. AV perception systems need to accurately detect non-static objects and predict their behaviour, as well as to detect static objects and recognise the information they are providing. This paper, in particular, focuses on the computer vision techniques used to detect pedestrians and vehicles. There have been many papers and reviews on pedestrians and vehicles detection so far. However, most of the past papers only reviewed pedestrian or vehicle detection separately. This review aims to present an overview of the AV systems in general, and then review and investigate several detection computer vision techniques for pedestrians and vehicles. The review concludes that both traditional and Deep Learning (DL) techniques have been used for pedestrian and vehicle detection; however, DL techniques have shown the best results. Although good detection results have been achieved for pedestrians and vehicles, the current algorithms still struggle to detect small, occluded, and truncated objects. In addition, there is limited research on how to improve detection performance in difficult light and weather conditions. Most of the algorithms have been tested on well-recognised datasets such as Caltech and KITTI; however, these datasets have their own limitations. Therefore, this paper recommends that future works should be implemented on more new challenging datasets, such as PIE and BDD100K. [ABSTRACT FROM AUTHOR]

The present paper offers the chance to experimentally measure, for the first time, the internal tire strain by optical fiber sensors during the tire rolling in real operating conditions. The phenomena that take place during the tire rolling are in fact far from being completely understood. Despite several models available in the technical literature, there is not a correspondently large set of experimental observations. The paper includes the detailed description of the new multi-sensing technology for an ongoing vehicle measurement, which the research group has developed in the context of the project OPTYRE. The experimental apparatus is mainly based on the use of optical fibers with embedded Fiber Bragg Gratings sensors for the acquisition of the circumferential tire strain. Other sensors are also installed on the tire, such as a phonic wheel, a uniaxial accelerometer, and a dynamic temperature sensor. The acquired information is used as input variables in dedicated algorithms that allow the identification of key parameters, such as the dynamic contact patch, instantaneous dissipation and instantaneous grip. The OPTYRE project brings a contribution into the field of experimental grip monitoring of wheeled vehicles, with implications both on passive and active safety characteristics of cars and motorbikes. [ABSTRACT FROM AUTHOR]

Real-time continuous localization is a key technology in the development of intelligent transportation systems. In these systems, it is very important to have accurate information about the position and heading angle of the vehicle at all times. The most widely implemented methods for positioning are the global positioning system (GPS), vision-based system, and magnetic marker system. Among these methods, the magnetic marker system is less vulnerable to indoor and outdoor environment conditions; moreover, it requires minimal maintenance expenses. In this paper, we present a position estimation scheme based on magnetic markers and odometry sensors for an all-wheel-steering vehicle. The heading angle of the vehicle is determined by using the position coordinates of the last two detected magnetic markers and odometer data. The instant position and heading angle of the vehicle are integrated with an extended Kalman filter to estimate the continuous position. GPS data with the real-time kinematics mode was obtained to evaluate the performance of the proposed position estimation system. The test results show that the performance of the proposed localization algorithm is accurate (mean error: 3 cm; max error: 9 cm) and reliable under unexpected missing markers or incorrect markers. [ABSTRACT FROM AUTHOR]

Nowadays, deep learning methods based on a virtual environment are widely applied to research and technology development for autonomous vehicle's smart sensors and devices. Learning various driving environments in advance is important to handle unexpected situations that can exist in the real world and to continue driving without accident. For training smart sensors and devices of an autonomous vehicle well, a virtual simulator should create scenarios of various possible real-world situations. To create reality-based scenarios, data on the real environment must be collected from a real driving vehicle or a scenario analysis process conducted by experts. However, these two approaches increase the period and the cost of scenario generation as more scenarios are created. This paper proposes a scenario generation method based on deep learning to create scenarios automatically for training autonomous vehicle smart sensors and devices. To generate various scenarios, the proposed method extracts multiple events from a video which is taken on a real road by using deep learning and generates the multiple event in a virtual simulator. First, Faster-region based convolution neural network (Faster-RCNN) extracts bounding boxes of each object in a driving video. Second, the high-level event bounding boxes are calculated. Third, long-term recurrent convolution networks (LRCN) classify each type of extracted event. Finally, all multiple event classification results are combined into one scenario. The generated scenarios can be used in an autonomous driving simulator to teach multiple events that occur during real-world driving. To verify the performance of the proposed scenario generation method, experiments using real driving video data and a virtual simulator were conducted. The results for deep learning model show an accuracy of 95.6%; furthermore, multiple high-level events were extracted, and various scenarios were generated in a virtual simulator for smart sensors and devices of an autonomous vehicle. [ABSTRACT FROM AUTHOR]

With the emergence of autonomous functions in road vehicles, there has been increased use of Advanced Driver Assistance Systems comprising various sensors to perform automated tasks. Light Detection and Ranging (LiDAR) is one of the most important types of optical sensor, detecting the positions of obstacles by representing them as clusters of points in three-dimensional space. LiDAR performance degrades significantly when a vehicle is driving in the rain as raindrops adhere to the outer surface of the sensor assembly. Performance degradation behaviors include missing points and reduced reflectivity of the points. It was found that the extent of degradation is highly dependent on the interface material properties. This subsequently affects the shapes of the adherent droplets, causing different perturbations to the optical rays. A fundamental investigation is performed on the protective polycarbonate cover of a LiDAR assembly coated with four classes of material—hydrophilic, almost-hydrophobic, hydrophobic, and superhydrophobic. Water droplets are controllably dispensed onto the cover to quantify the signal alteration due to the different droplets of various sizes and shapes. To further understand the effects of droplet motion on LiDAR signals, sliding droplet conditions are simulated using numerical analysis. The results are validated with physical optical tests, using a 905 nm laser source and receiver to mimic the LiDAR detection mechanism. Comprehensive explanations of LiDAR performance degradation in rain are presented from both material and optical perspectives. These can aid component selection and the development of signal-enhancing strategies for the integration of LiDARs into vehicle designs to minimize the impact of rain. [ABSTRACT FROM AUTHOR]

Detecting road cracks is essential for inspecting and assessing the integrity of concrete pavement structures. Traditional image-based methods often require complex preprocessing to extract crack features, making them challenging when dealing with noisy concrete surfaces in diverse real-world scenarios, such as autonomous vehicle road detection. This study introduces an image-based crack detection approach that combines a Random Forest machine learning classifier with a deep convolutional neural network (CNN) to address these challenges. Three state-of-the-art models, namely MobileNet, InceptionV3, and Xception, were employed and trained using a dataset of 30,000 images to build an effective CNN. A systematic comparison of validation accuracy across various base learning rates identified a base learning rate of 0.001 as optimal, achieving a maximum validation accuracy of 99.97%. This optimal learning rate was then applied in the subsequent testing phase. The robustness and flexibility of the trained models were evaluated using 6,000 test photos, each with a resolution of 224 × 224 pixels, which were not part of the training or validation sets. The outstanding results, boasting a remarkable 99.95% accuracy, 99.95% precision, 99.94% recall, and a matching 99.94% F1 Score, unequivocally affirm the efficacy of the proposed technique in precisely identifying road fractures in photographs taken on real concrete surfaces. [ABSTRACT FROM AUTHOR]

In transportation at open-pit mines, rocks dropped as a mining truck is driven will wear out the tires of the vehicle, thus increasing the mining cost. In the case of autonomous vehicles, the vehicle must automatically detect rocks on the transportation roads during the driving process. This will be a new challenge: rough road, rocks of small size and irregular shape, long detection distance, etc. This paper presents a detection method based on light detection and ranging (lidar). It includes two stages: (1) using the modified cloth simulation method to filter out the ground points; (2) using the regional growth method based on grid division to cluster non-ground points. Experimental results show that the method can detect rocks with a size of 20–30 cm at a distance of 40 m in front of the vehicle, and it takes only 0.3 s on an ordinary personal computer (PC). This method is easy to understand, and it has fewer parameters to be adjusted. Therefore, it is a better method for detecting small, irregular obstacles on a low-speed, unstructured and rough road. [ABSTRACT FROM AUTHOR]

Visibility is a critical factor for transportation, even if we refer to air, water, or ground transportation. The biggest trend in the automotive industry is autonomous driving, the number of autonomous vehicles will increase exponentially, prompting changes in the industry and user segment. Unfortunately, these vehicles still have some drawbacks and one, always in attention and topical, will be treated in this paper—visibility distance issue in bad weather conditions, particularly in fog. The way and the speed with which vehicles will determine objects, obstacles, pedestrians, or traffic signs, especially in bad visibility, will determine how the vehicle will behave. In this paper, a new experimental set up is featured, for analyzing the effect of the fog when the laser and LIDAR (Light Detection And Ranging) radiation are used in visibility distance estimation on public roads. While using our experimental set up, in the laboratory, the information offered by these measurement systems (laser and LIDAR) are evaluated and compared with results offered by human observers in the same fog conditions. The goal is to validate and unitarily apply the results regarding visibility distance, based on information arrives from different systems that are able to estimate this parameter (in foggy weather conditions). Finally, will be notifying the drivers in case of unexpected situations. It is a combination of stationary and of moving systems. The stationary system will be installed on highways or express roads in areas prone to fog, while the moving systems are, or can be, directly installed on the vehicles (autonomous but also non-autonomous). [ABSTRACT FROM AUTHOR]

The fault detection and isolation are very important for the driving safety of autonomous vehicles. At present, scholars have conducted extensive research on model-based fault detection and isolation algorithms in vehicle systems, but few of them have been applied for path tracking control. This paper determines the conditions for model establishment of a single-track 3-DOF vehicle dynamics model and then performs Taylor expansion for modeling linearization. On the basis of that, a novel fault-tolerant model predictive control algorithm (FTMPC) is proposed for robust path tracking control of autonomous vehicle. First, the linear time-varying model predictive control algorithm for lateral motion control of vehicle is designed by constructing the objective function and considering the front wheel declination and dynamic constraint of tire cornering. Then, the motion state information obtained by multi-sensory perception systems of vision, GPS, and LIDAR is fused by using an improved weighted fusion algorithm based on the output error variance. A novel fault signal detection algorithm based on Kalman filtering and Chi-square detector is also designed in our work. The output of the fault signal detector is a fault detection matrix. Finally, the fault signals are isolated by multiplication of signal matrix, fault detection matrix, and weight matrix in the process of data fusion. The effectiveness of the proposed method is validated with simulation experiment of lane changing path tracking control. The comparative analysis of simulation results shows that the proposed method can achieve the expected fault-tolerant performance and much better path tracking control performance in case of sensor failure. [ABSTRACT FROM AUTHOR]

Pedestrian detection is a critical task for safety-critical systems, but detecting pedestrians is challenging in low-light and adverse weather conditions. Thermal images can be used to improve robustness by providing complementary information to RGB images. Previous studies have shown that multi-modal feature fusion using convolution operation can be effective, but such methods rely solely on local feature correlations, which can degrade the performance capabilities. To address this issue, we propose an attention-based novel fusion network, referred to as INSANet (INtra-INter Spectral Attention Network), that captures global intra- and inter-information. It consists of intra- and inter-spectral attention blocks that allow the model to learn mutual spectral relationships. Additionally, we identified an imbalance in the multispectral dataset caused by several factors and designed an augmentation strategy that mitigates concentrated distributions and enables the model to learn the diverse locations of pedestrians. Extensive experiments demonstrate the effectiveness of the proposed methods, which achieve state-of-the-art performance on the KAIST dataset and LLVIP dataset. Finally, we conduct a regional performance evaluation to demonstrate the effectiveness of our proposed network in various regions. [ABSTRACT FROM AUTHOR]

In the new agricultural scenarios, the interaction between autonomous tractors and a human operator is important when they jointly perform a task. Obtaining and exchanging accurate localization information between autonomous tractors and the human operator, working as a team, is a critical to maintaining safety, synchronization, and efficiency during the execution of a mission. An advanced localization system for both entities involved in the joint work, i.e., the autonomous tractors and the human operator, provides a basis for meeting the task requirements. In this paper, different localization techniques for a human operator and an autonomous tractor in a field environment were tested. First, we compared the localization performances of two global navigation satellite systems' (GNSS) receivers carried by the human operator: (1) an internal GNSS receiver built into a handheld device; and (2) an external DGNSS receiver with centimeter-level accuracy. To investigate autonomous tractor localization, a real-time kinematic (RTK)-based localization system installed on autonomous tractor developed for agricultural applications was evaluated. Finally, a hybrid localization approach, which combines distance estimates obtained using a wireless scheme with the position of an autonomous tractor obtained using an RTK-GNSS system, is proposed. The hybrid solution is intended for user localization in unstructured environments in which the GNSS signal is obstructed. The hybrid localization approach has two components: (1) a localization algorithm based on the received signal strength indication (RSSI) from the wireless environment; and (2) the acquisition of the tractor RTK coordinates when the human operator is near the tractor. In five RSSI tests, the best result achieved was an average localization error of 4 m. In tests of real-time position correction between rows, RMS error of 2.4 cm demonstrated that the passes were straight, as was desired for the autonomous tractor. From these preliminary results, future work will address the use of autonomous tractor localization in the hybrid localization approach. [ABSTRACT FROM AUTHOR]

The autonomous vehicle is an automated system equipped with features like environment perception, decision-making, motion planning, and control and execution technology. Navigating in an unstructured and complex environment is a huge challenge for autonomous vehicles, due to the irregular shape of road, the requirement of real-time planning, and the nonholonomic constraints of vehicle. This paper presents a motion planning method, based on the Radial Basis Function (RBF) neural network, to guide the autonomous vehicle in unstructured environments. The proposed algorithm extracts the drivable region from the perception grid map based on the global path, which is available in the road network. The sample points are randomly selected in the drivable region, and a gradient descent method is used to train the RBF network. The parameters of the motion-planning algorithm are verified through the simulation and experiment. It is observed that the proposed approach produces a flexible, smooth, and safe path that can fit any road shape. The method is implemented on autonomous vehicle and verified against many outdoor scenes; furthermore, a comparison of proposed method with the existing well-known Rapidly-exploring Random Tree (RRT) method is presented. The experimental results show that the proposed method is highly effective in planning the vehicle path and offers better motion quality. [ABSTRACT FROM AUTHOR]

When blind and deaf people are passengers in fully autonomous vehicles, an intuitive and accurate visualization screen should be provided for the deaf, and an audification system with speech-to-text (STT) and text-to-speech (TTS) functions should be provided for the blind. However, these systems cannot know the fault self-diagnosis information and the instrument cluster information that indicates the current state of the vehicle when driving. This paper proposes an audification and visualization system (AVS) of an autonomous vehicle for blind and deaf people based on deep learning to solve this problem. The AVS consists of three modules. The data collection and management module (DCMM) stores and manages the data collected from the vehicle. The audification conversion module (ACM) has a speech-to-text submodule (STS) that recognizes a user's speech and converts it to text data, and a text-to-wave submodule (TWS) that converts text data to voice. The data visualization module (DVM) visualizes the collected sensor data, fault self-diagnosis data, etc., and places the visualized data according to the size of the vehicle's display. The experiment shows that the time taken to adjust visualization graphic components in on-board diagnostics (OBD) was approximately 2.5 times faster than the time taken in a cloud server. In addition, the overall computational time of the AVS system was approximately 2 ms faster than the existing instrument cluster. Therefore, because the AVS proposed in this paper can enable blind and deaf people to select only what they want to hear and see, it reduces the overload of transmission and greatly increases the safety of the vehicle. If the AVS is introduced in a real vehicle, it can prevent accidents for disabled and other passengers in advance. [ABSTRACT FROM AUTHOR]

This paper presents a systematic review of the perception systems and simulators for autonomous vehicles (AV). This work has been divided into three parts. In the first part, perception systems are categorized as environment perception systems and positioning estimation systems. The paper presents the physical fundamentals, principle functioning, and electromagnetic spectrum used to operate the most common sensors used in perception systems (ultrasonic, RADAR, LiDAR, cameras, IMU, GNSS, RTK, etc.). Furthermore, their strengths and weaknesses are shown, and the quantification of their features using spider charts will allow proper selection of different sensors depending on 11 features. In the second part, the main elements to be taken into account in the simulation of a perception system of an AV are presented. For this purpose, the paper describes simulators for model-based development, the main game engines that can be used for simulation, simulators from the robotics field, and lastly simulators used specifically for AV. Finally, the current state of regulations that are being applied in different countries around the world on issues concerning the implementation of autonomous vehicles is presented. [ABSTRACT FROM AUTHOR]

Autonomous parking valet systems improve users' comfort, helping with the task of searching for a parking space and parking maneuvering; and due to the simple infrastructure design and low speeds, this maneuver is quite feasible for automated vehicles. Various demonstrations have been performed in both closed parking and in open air parking; scenarios that allow the use of specific technological tools for navigation and searching for a parking space. However, there are still challenges. The purpose of this paper was the integration of perception, positioning, decision-making, and maneuvering algorithms for the control of an autonomous vehicle in a parking lot with the support of a single LiDAR sensor, and with no additional sensors in the infrastructure. Based on a digital map, which was as simplified as possible, the driver can choose the range of parking spaces in which the vehicle must look for a space. From that moment on, the vehicle moves, looking for free places until an available one in the range selected by the driver is found. Then, the vehicle performs the parking maneuver, choosing between two alternatives to optimize the required space. Tests in a real parking lot, with spaces covered with metallic canopies, showed an accurate behavior. [ABSTRACT FROM AUTHOR]

Electric wheelchairs make it easier for disabled and elderly people to live, move, interact, and participate in society. Moving a wheelchair in open spaces is relatively easy, but in closed and small spaces, maneuvering is difficult. Solutions to such problems for people with disabilities are applicable to a relatively small group of recipients and are mostly custom-made solutions, whose considerable cost is a significant barrier to accessibility. New technologies can provide an opportunity to improve the quality of life of people with disabilities in this aspect. Using selected elements of complex automation and control systems, cost-effective solutions can be created that facilitate the functioning of people with disabilities. This paper presents an analysis of hazards and problems when maneuvering a wheelchair in narrow passageways, as well as the authors' solution to this problem, and the concept and assumptions of a mechatronic anti-collision system based on 2D LiDAR laser scanners. This solution is composed of a proprietary 2D rotating scanner mechanism that ensures the acquisition of 3D images of the environment around the wheelchair. Preliminary tests of this solution yielded promising results. Further research will include miniaturization of the device. [ABSTRACT FROM AUTHOR]

Detecting buildings in the surroundings of an urban vehicle and matching them to building models available on map services is an emerging trend in robotics localization for urban vehicles. In this paper, we present a novel technique, which improves a previous work by detecting building façade, their positions, and finding the correspondences with their 3D models, available in OpenStreetMap. The proposed technique uses segmented point clouds produced using stereo images, processed by a convolutional neural network. The point clouds of the façades are then matched against a reference point cloud, produced extruding the buildings' outlines, which are available on OpenStreetMap (OSM). In order to produce a lane-level localization of the vehicle, the resulting information is then fed into our probabilistic framework, called Road Layout Estimation (RLE). We prove the effectiveness of this proposal, testing it on sequences from the well-known KITTI dataset and comparing the results concerning a basic RLE version without the proposed pipeline. [ABSTRACT FROM AUTHOR]

The strapdown inertial navigation system (SINS) is widely used in autonomous vehicles. However, the random drift error of gyroscope leads to serious accumulated navigation errors during long continuous operation of SINS alone. In this paper, we propose to combine the Inertial Measurement Unit (IMU) data with the line feature parameters from a camera to improve the navigation accuracy. The proposed method can also maintain the autonomy of the navigation system. Experimental results show that the proposed inertial-visual navigation system can mitigate the SINS drift and improve the accuracy, stability, and reliability of the navigation system. [ABSTRACT FROM AUTHOR]

Multi-object tracking is a crucial problem for autonomous vehicle. Most state-of-the-art approaches adopt the tracking-by-detection strategy, which is a two-step procedure consisting of the detection module and the tracking module. In this paper, we improve both steps. We improve the detection module by incorporating the temporal information, which is beneficial for detecting small objects. For the tracking module, we propose a novel compressed deep Convolutional Neural Network (CNN) feature based Correlation Filter tracker. By carefully integrating these two modules, the proposed multi-object tracking approach has the ability of re-identification (ReID) once the tracked object gets lost. Extensive experiments were performed on the KITTI and MOT2015 tracking benchmarks. Results indicate that our approach outperforms most state-of-the-art tracking approaches. [ABSTRACT FROM AUTHOR]

This study presents a novel multimodal heterogeneous perception cross-fusion framework for intelligent vehicles that combines data from millimeter-wave radar and camera to enhance target tracking accuracy and handle system uncertainties. The framework employs a multimodal interaction strategy to predict target motion more accurately and an improved joint probability data association method to match measurement data with targets. An adaptive root-mean-square cubature Kalman filter is used to estimate the statistical characteristics of noise under complex traffic scenarios with varying process and measurement noise. Experiments conducted on a real vehicle platform demonstrate that the proposed framework improves reliability and robustness in challenging environments. It overcomes the challenges of insufficient data fusion utilization, frequent leakage, and misjudgment of dangerous obstructions around vehicles, and inaccurate prediction of collision risks. The proposed framework has the potential to advance the state of the art in target tracking and perception for intelligent vehicles. [ABSTRACT FROM AUTHOR]

Autonomous vehicles are expected to take complete control of the driving process, enabling the former drivers to act as passengers only. This could lead to increased sickness as they can be engaged in tasks other than driving. Adopting different sickness mitigation techniques gives us unique types of motion sickness in autonomous vehicles to be studied. In this paper, we report on a study where we explored the possibilities of assessing motion sickness with electrogastrography (EGG), a non-invasive method used to measure the myoelectric activity of the stomach, and its potential usage in autonomous vehicles (AVs). The study was conducted in a high-fidelity driving simulator with a virtual reality (VR) headset. There separate EGG measurements were performed: before, during and after the driving AV simulation video in VR. During the driving, the participants encountered two driving environments: a straight and less dynamic highway road and a highly dynamic and curvy countryside road. The EGG signal was recorded with a proprietary 3-channel recording device and Ag/AgCl cutaneous electrodes. In addition, participants were asked to signalize whenever they felt uncomfortable and nauseated by pressing a special button. After the drive they completed also the Simulator Sickness Questionnaire (SSQ) and reported on their overall subjective perception of sickness symptoms. The EGG results showed a significant increase of the dominant frequency (DF) and the percentage of the high power spectrum density (FSD) as well as a significant decrease of the power spectrum density Crest factor (CF) during the AV simulation. The vast majority of participants reported nausea during more dynamic conditions, accompanied by an increase in the amplitude and the RMS value of EGG. Reported nausea occurred simultaneously with the increase in EGG amplitude. Based on the results, we conclude that EGG could be used for assessment of motion sickness in autonomous vehicles. DF, CF and FSD can be used as overall sickness indicators, while the relative increase in amplitude of EGG signal and duration of that increase can be used as short-term sickness indicators where the driving environment may affect the driver. [ABSTRACT FROM AUTHOR]

This paper presents the results of the design, simulation, and implementation of a virtual vehicle. Such a process employs the Unity videogame platform and its Machine Learning-Agents library. The virtual vehicle is implemented in Unity considering mechanisms that represent accurately the dynamics of a real automobile, such as motor torque curve, suspension system, differential, and anti-roll bar, among others. Intelligent agents are designed and implemented to drive the virtual automobile, and they are trained using imitation or reinforcement. In the former method, learning by imitation, a human expert interacts with an intelligent agent through a control interface that simulates a real vehicle; in this way, the human expert receives motion signals and has stereoscopic vision, among other capabilities. In learning by reinforcement, a reward function that stimulates the intelligent agent to exert a soft control over the virtual automobile is designed. In the training stage, the intelligent agents are introduced into a scenario that simulates a four-lane highway. In the test stage, instead, they are located in unknown roads created based on random spline curves. Finally, graphs of the telemetric variables are presented, which are obtained from the automobile dynamics when the vehicle is controlled by the intelligent agents and their human counterpart, both in the training and the test track. [ABSTRACT FROM AUTHOR]

Recent decades have witnessed the breakthrough of autonomous vehicles (AVs), and the sensing capabilities of AVs have been dramatically improved. Various sensors installed on AVs will be collecting massive data and perceiving the surrounding traffic continuously. In fact, a fleet of AVs can serve as floating (or probe) sensors, which can be utilized to infer traffic information while cruising around the roadway networks. Unlike conventional traffic sensing methods relying on fixed location sensors or moving sensors that acquire only the information of their carrying vehicle, this paper leverages data from AVs carrying sensors for not only the information of the AVs, but also the characteristics of the surrounding traffic. A high-resolution data-driven traffic sensing framework is proposed, which estimates the fundamental traffic state characteristics, namely, flow, density and speed in high spatio-temporal resolutions and of each lane on a general road, and it is developed under different levels of AV perception capabilities and for any AV market penetration rate. Experimental results show that the proposed method achieves high accuracy even with a low AV market penetration rate. This study would help policymakers and private sectors (e.g., Waymo) to understand the values of massive data collected by AVs in traffic operation and management. [ABSTRACT FROM AUTHOR]

A key research area in autonomous driving is how to model the driver's decision-making behavior, due to the fact it is significant for a self-driving vehicles considering their traffic safety and efficiency. However, the uncertain characteristics of vehicle and pedestrian trajectories affect urban roads, which poses severe challenges to the cognitive understanding and decision-making of autonomous vehicle systems in terms of accuracy and robustness. To overcome the abovementioned problems, this paper proposes a Bayesian driver agent (BDA) model which is a vision-based autonomous vehicle system with learning and inference methods inspired by human driver's cognitive psychology. Different from the end-to-end learning method and traditional rule-based methods, our approach breaks the driving system up into a scene recognition module and a decision inference module. The perception module, which is based on a multi-task learning neural network (CNN), takes a driver's-view image as its input and predicts the traffic scene's feature values. The decision module based on dynamic Bayesian network (DBN) then makes an inferred decision using the traffic scene's feature values. To explore the validity of the Bayesian driver agent model, we performed experiments on a driving simulation platform. The BDA model can extract the scene feature values effectively and predict the probability distribution of the human driver's decision-making process accurately based on inference. We take the lane changing scenario as an example to verify the model, the intraclass correlation coefficient (ICC) correlation between the BDA model and human driver's decision process reached 0.984. This work suggests a research in scene perception and autonomous decision-making that may apply to autonomous vehicle system. [ABSTRACT FROM AUTHOR]