Search

In this paper, the problem of decision making for autonomous vehicles changing lanes is addressed by formulating multiple games in normal form for pairs of agents. This formulation generates the optimal action for the Ego vehicle at a given state and does not consider global optimality for all agents. The payoff matrices of the games are designed based on a user-defined set of rules. The constant parameters of these payoffs are then adjusted using neural learning to generate optimal behavior among the vehicles. An algorithm integrating deep reinforcement learning and game theory, regarded as Nash Q-learning, is included in the decision-making scheme. The applicability of the proposed method in a lane-changing scenario is tested via simulation. [ABSTRACT FROM AUTHOR]

Robust and adaptive vehicle state estimation and tracking algorithms have become a very important part within the autonomous driving field. The family of interacting multiple model (IMM) filters has shown to provide very effective and accurate state estimation in systems whose behavior patterns change significantly over time. The main reason for the improved performance of IMM filters compared to single model approaches is the mode mixing, which constantly aligns the different models. This paper proposes an innovative way for the mode mixing, when the state-vectors of the models are of different size. The proposed mixing approach consists of two parts: firstly mixing the common states and secondly weighting between original and mixed states based on the model probabilities. Results from artificial simulations and real world measurement setups are shown to demonstrate the validity of the approach. Compared to previously suggested solutions, the proposed approach is more general and the overall complexity of the mode mixing step is reduced, which is the main contribution of the presented paper. [ABSTRACT FROM AUTHOR]

Traffic awareness is the prerequisite of autonomous driving. Given the limitation of on-board sensors (e.g., precision and price), remote measurement from either infrastructure or other vehicles can improve traffic safety. However, the wireless communication carrying the measurement result undergoes fading, noise and interference and has a certain probability of outage. When the communication fails, the vehicle state can only be predicted by Bayesian filtering with a low precision. Higher communication resource utilization (e.g., transmission power) reduces the outage probability and hence results in an improved estimation precision. The power control subject to an estimate variance constraint is a difficult problem due to the complicated mapping from transmit power to vehicle-state estimate variance. In this paper, we develop an estimator consisting of several Kalman filters (KFs) or extended Kalman filters (EKFs) and an interacting multiple model (IMM) to estimate and predict the vehicle state. We propose to apply deep reinforcement learning (DRL) for the transmit power optimization. In particular, we consider an intersection and a lane-changing scenario and apply proximal policy optimization (PPO) and soft actor-critic (SAC) to train the DRL model. Testing results show satisfactory power control strategies confining estimate variances below given threshold. SAC achieves higher performance compared to PPO. [ABSTRACT FROM AUTHOR]

Autonomous vehicles will generate data from the variety of sensors they employ to track the surrounding environment. This data is inherently valuable, as it gives algorithm designers the potential to leverage prior experience in order to improve driving performance over time. This paper uses the lens of autonomous racing to provide an example of how data from previous iterations of driving can be used to improve quantitative metrics of performance. Two complementary algorithms are demonstrated in this paper. The first algorithm uses iterative learning control (ILC) to simultaneously improve lateral and longitudinal tracking of the desired racing trajectory over multiple laps, while the second algorithm is focused on altering the trajectory itself using a search method. When driven experimentally at the limits of handling, the result is a reduction in lap time of nearly 1.4 seconds, a major improvement. [ABSTRACT FROM AUTHOR]

GNSS/INS integrated solution has been extensively studied over the past decades. However, its performance relies heavily on environmental conditions and sensor costs. The GNSS positioning can obtain satisfactory performance in the open area. Unfortunately, its accuracy can be severely degraded in a highly urbanized area, due to the notorious multipath effects and none-line-of-sight (NLOS) receptions. As a result, excessive GNSS outliers occur, which causes a huge error in GNSS/INS integration. This paper proposes to apply a fish-eye camera to capture the sky view image to further classify the NLOS and line-of-sight (LOS) measurements. In addition, the raw INS and GNSS measurements are tightly integrated using a state-of-the-art probabilistic factor graph model. Instead of excluding the NLOS receptions, this paper makes use of both the NLOS and LOS measurements by treating them with different weightings. Experiments conducted in typical urban canyons of Hong Kong showed that the proposed method could effectively mitigate the effects of GNSS outliers, and an improved accuracy of GNSS/INS integration was obtained, when compared with the conventional GNSS/INS integration. [ABSTRACT FROM AUTHOR]

The localization system is one of the key components of autonomous vehicles. Widely-used GNSS (low-cost) alone can not meet the centimeter accuracy and is unstable in urban environment. Typically LiDAR-based localization system is very computationally demanding due to the usage of the dense point cloud map. In this paper, we propose a novel lightweight LiDAR-based localization system for autonomous vehicle in this paper. The proposed system only relies on lightweight poles and curbs landmark map, which is highly robust and efficient compared to other localization systems. Poles and curbs are selected as landmarks because of their commonality and stability. We novelly propose a Branch-and-Bound (BnB)-based global optimization method to tackle the data association problem of poles. Motion decoupling is adopted to decouple translation and rotation to improve the efficiency of the BnB-based algorithm. Besides, we propose a new local grid map-based representation for curbs to make better use of curb information. Cost functions for pole and curb are defined respectively and then fused for the subsequent non-linear optimization method to obtain the vehicle location. Experiments on KITTI dataset and our self-collected dataset demonstrate the efficiency and accuracy of our system. [ABSTRACT FROM AUTHOR]

3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications. In this paper, we extend our preliminary work PointRCNN to a novel and strong point-cloud-based 3D object detection framework, the part-aware and aggregation neural network (Part- $A^2$ A 2 net). The whole framework consists of the part-aware stage and the part-aggregation stage. First, the part-aware stage for the first time fully utilizes free-of-charge part supervisions derived from 3D ground-truth boxes to simultaneously predict high quality 3D proposals and accurate intra-object part locations. The predicted intra-object part locations within the same proposal are grouped by our new-designed RoI-aware point cloud pooling module, which results in an effective representation to encode the geometry-specific features of each 3D proposal. Then the part-aggregation stage learns to re-score the box and refine the box location by exploring the spatial relationship of the pooled intra-object part locations. Extensive experiments are conducted to demonstrate the performance improvements from each component of our proposed framework. Our Part- $A^2$ A 2 net outperforms all existing 3D detection methods and achieves new state-of-the-art on KITTI 3D object detection dataset by utilizing only the LiDAR point cloud data. [ABSTRACT FROM AUTHOR]

Lane detection on road segments with complex topologies such as lane merge/split and highway ramps is not yet a solved problem. This paper presents a novel graph-embedded solution. It consists of two key parts, a learning-based low-level lane feature extraction algorithm, and a graph-embedded lane inference algorithm. The former reduces the over-reliance on customized annotated/labeled lane data. We leveraged several open-source semantic segmentation datasets (e.g., Cityscape, Vistas, and Apollo) and designed a dedicated network that can be trained across these heterogeneous datasets to extract lane attributes. The latter algorithm constructs a graph to represent the lane geometry and topology. It does not rely on strong geometric assumptions such as lane lines are a set of parallel polynomials. Instead, it constructs a graph based on detected lane nodes. The lane parameters in the world coordinate are inferred by efficient graph-based searching and calculation. The performance of the proposed method is verified on both open source and our own collected data. On-vehicle experiments were also conducted and the comparison with Mobileye EyeQ2 shows favorable results. [ABSTRACT FROM AUTHOR]

3D neural networks are widely used in real-world applications (e.g., AR/VR headsets, self-driving cars). They are required to be fast and accurate; however, limited hardware resources on edge devices make these requirements rather challenging. Previous work processes 3D data using either voxel-based or point-based neural networks, but both types of 3D models are not hardware-efficient due to the large memory footprint and random memory access. In this paper, we study 3D deep learning from the efficiency perspective. We first systematically analyze the bottlenecks of previous 3D methods. We then combine the best from point-based and voxel-based models together and propose a novel hardware-efficient 3D primitive, Point-Voxel Convolution (PVConv). We further enhance this primitive with the sparse convolution to make it more effective in processing large (outdoor) scenes. Based on our designed 3D primitive, we introduce 3D Neural Architecture Search (3D-NAS) to explore the best 3D network architecture given a resource constraint. We evaluate our proposed method on six representative benchmark datasets, achieving state-of-the-art performance with 1.8-23.7× measured speedup. Furthermore, our method has been deployed to the autonomous racing vehicle of MIT Driverless, achieving larger detection range, higher accuracy and lower latency. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel system named Disp R-CNN for 3D object detection from stereo images. Many recent works solve this problem by first recovering point clouds with disparity estimation and then apply a 3D detector. The disparity map is computed for the entire image, which is costly and fails to leverage category-specific prior. In contrast, we design an instance disparity estimation network (iDispNet) that predicts disparity only for pixels on objects of interest and learns a category-specific shape prior for more accurate disparity estimation. To address the challenge from scarcity of disparity annotation in training, we propose to use a statistical shape model to generate dense disparity pseudo-ground-truth without the need of LiDAR point clouds, which makes our system more widely applicable. Experiments on the KITTI dataset show that, when LiDAR ground-truth is not used at training time, Disp R-CNN outperforms previous state-of-the-art methods based on stereo input by 20 percent in terms of average precision for all categories. The code and pseudo-ground-truth data are available at the project page: https://github.com/zju3dv/disprcnn. [ABSTRACT FROM AUTHOR]

The Internet of Vehicles (IoV) as a promising application of Internet of Things (IoT) has played a significant role in autonomous driving, by connecting intelligent vehicles. Autonomous driving needs to process the mass environmental sensing data in coordination with surrounding vehicles, and makes an accurate driving judgment accordingly. Since the vehicles always have limited computing resources, processing these data in parallel with efficient task scheduling is one of the most important topics. Most current work focuses on formulating special scenarios and service requirements as optimization problems. However, the complicated and dynamic environment of vehicular computing is hard to model, predict and control, making those previous methods unscalable and unable to reflect the real scenario. In this paper, a Multi-task Deep reinforcement learning approach for scalable parallel Task Scheduling (MDTS) is firstly devised. For avoiding the curse of dimensionality when coping with complex parallel computing environments and jobs with diverse properties, we extend the action selection in Deep Reinforcement Learning (DRL) to a multi-task decision, where the output branches of multi- task learning are fine-matched to parallel scheduling tasks. Child tasks of a job are accordingly assigned to distributed nodes without any human knowledge while the resource competition among parallel tasks is leveraged through shared neural network layers. Moreover, we design an appropriate reward function to optimize multiple metrics simultaneously, with emphasis on specific scenarios. Extensive experiments show that the MDTS significantly increases the overall reward compared with least- connection scheduling and particle swarm optimization algorithm from −16.71, −0.67 to 2.93, respectively. [ABSTRACT FROM AUTHOR]

Rear-end collision prediction has gained an increasing attention for safety improvement in smart cities. It is urgent to design efficient warning strategies for rear-end collisions which is one of the main causes of traffic accidents. The existing researches have been conducted to predict the collisions. Learning-based methods are proposed to solve this complicated issue, which the traditional methods are difficult to solve. However, due to some limitations in terms of the feature extraction and prediction performance, back-propagation learning methods are facing challenge. In this paper, we proposed a novel Rear-end Collision Prediction Mechanism with deep learning method (RCPM), in which a convolutional neural network model is established. In RCPM, the dataset is smoothed and expanded based on genetic theory to alleviate the class imbalance problem. The preprocessed dataset is divided into training and testing sets as the input to train our convolutional neural network model. The experimental results show that compared with the Honda, Berkeley and multi-layer perception neural-network-based algorithms, RCPM effectively improves performance to predict rear-end collisions. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI)-powered autonomous vehicles (AVs) can integrate different machine learning (ML) techniques to build up a complex autonomous driving system. However, single AV intelligence is not enough to cope with ever-changing driving environments. The underlying reason is that, with current neural network design and training algorithms, it is challenging for the driving model to generalize to diverse driving environments all at once due to sample inefficiency and the curse of dimensionality. Powerful computing resources and massive amount of data can be used to train a good driving model offline. However, the driving model obtained offline might fail in corner case scenarios. In this paper, we propose an intelligence networking framework among AVs assisted by multi-access edge computing (MEC) with end-to-end learning for demonstration. In this framework, driving road is divided into segments and data is collected for each road segment separately. Assisted by MEC networks, a continuously updated driving model is produced in near real-time for each road segment when the environment changes. By dividing the road into segments, we aim to reduce the burden of generalization since a single model only needs to adapt to a specific road segment. Simulation results show that our solutions can produce updated driving model for each road segment to adapt to environmental changes better than the existing scheme. Upcoming AVs can then adapt to changing environments by downloading the updated driving model. [ABSTRACT FROM AUTHOR]

Monocular 3D object detection has the merit of low cost and can be served as an auxiliary module for autonomous driving system, becoming a growing concern in recent years. In this paper, we present a monocular 3D object detection method with feature enhancement networks, which we call MonoFENet. Specifically, with the estimated disparity from the input monocular image, the features of both the 2D and 3D streams can be enhanced and utilized for accurate 3D localization. For the 2D stream, the input image is used to generate 2D region proposals as well as to extract appearance features. For the 3D stream, the estimated disparity is transformed into 3D dense point cloud, which is then enhanced by the associated front view maps. With the RoI Mean Pooling layer, 3D geometric features of RoI point clouds are further enhanced by the proposed point feature enhancement (PointFE) network. The region-wise features of image and point cloud are fused for the final 2D and 3D bounding boxes regression. The experimental results on the KITTI benchmark reveal that our method can achieve state-of-the-art performance for monocular 3D object detection. [ABSTRACT FROM AUTHOR]

The three main modules of autonomous vehicles, i.e., sensing, decision making, and motion controlling, have been studied separately in most existing works on autonomous driving, which overlook the correlations among these modules, leading to a result of unsatisfactory performance. In this paper, we propose a novel scheme that first tactfully processes the sensing data, then jointly learns and optimizes the decision-making and motion-controlling using reinforcement learning (RL). Specifically, the proposed scheme designs a novel state representation mechanism, where the sensing data goes through the attention layer and the convolutional neural network (CNN) layer sequentially. The attention layer focuses on extracting the most important local information and then CNN layer takes a broad view to comprehensively consider the global information for a better representation. Furthermore, the proposed scheme jointly learns decision-making and motion-controlling, therefore, the relevance of these two modules is implicitly considered, which helps to achieve a better autonomous driving policy. Extensive simulation results show that the proposed scheme is better than classic control methods and some RL methods in terms of safety, velocity, etc. We also demonstrate the respective functions of the attention layer and the CNN layer through ablation studies. Finally, we construct a traffic scene with a real autonomous vehicle, and verified the feasibility of the proposed scheme. [ABSTRACT FROM AUTHOR]

Edge federated learning (FL) is an emerging paradigm that trains a global parametric model from distributed datasets based on wireless communications. This paper proposes a unit-modulus over-the-air computation (UMAirComp) framework to facilitate efficient edge federated learning, which simultaneously uploads local model parameters and updates global model parameters via analog beamforming. The proposed framework avoids sophisticated baseband signal processing, leading to low communication delays and implementation costs. Training loss bounds of UMAirComp FL systems are derived and two low-complexity large-scale optimization algorithms, termed penalty alternating minimization (PAM) and accelerated gradient projection (AGP), are proposed to minimize the nonconvex nonsmooth loss bound. Simulation results show that the proposed UMAirComp framework with PAM algorithm achieves a smaller mean square error of model parameters’ estimation, training loss, and test error compared with other benchmark schemes. Moreover, the proposed UMAirComp framework with AGP algorithm achieves satisfactory performance while reduces the computational complexity by orders of magnitude compared with existing optimization algorithms. Finally, we demonstrate the implementation of UMAirComp in a vehicle-to-everything autonomous driving simulation platform. It is found that autonomous driving tasks are more sensitive to model parameter errors than other tasks since the neural networks for autonomous driving contain sparser model parameters. [ABSTRACT FROM AUTHOR]

Unlike high-end three-dimensional (3-D) scanners with more than 16 layers which are mainly used in academia, low-end 3-D scanners with a few layers are being developed by sensor makers for installation in commercial advanced driver assistance system. The output of a low-end 3-D scanner is completely different from that of a full 3-D scanner and it is rather similar to the output of a 2-D scanner with a single layer. In this paper, a new framework for moving object detection and subsequent tracking using a low-end 3-D scanner with four layers is proposed. The proposed method uses the contours of the objects to obtain a robust association between a detection and a tracking. The proposed method comprises five steps: preprocessing, contour extraction, hypothesis generation, pruning, and moving object detection. In the preprocessing step, outliers, such as the ground or backlights from preceding vehicles, are removed and the scanned points are decomposed into segments, each of which corresponds to a single object. In the track hypothesis generation step, each segment is associated with an existing track maintained over multiple scans. The association method developed here uses the contour shape of the segments and is motivated by the linear programming and dynamic time warping. In the track hypothesis pruning step, unlikely tracks are removed from the hypothesis trees based on the proposed hypothesis scores. In the last step, moving objects are detected based on the track velocity. The proposed method is applied to four challenging real-world scenarios, and its validity is demonstrated via experimentation. [ABSTRACT FROM AUTHOR]

The knowledge about the placement and appearance of lane markings is a prerequisite for the creation of maps with high precision, necessary for autonomous driving, infrastructure monitoring, lanewise traffic management, and urban planning. Lane markings are one of the important components of such maps. Lane markings convey the rules of roads to drivers. While these rules are learned by humans, an autonomous driving vehicle should be taught to learn them to localize itself. Therefore, accurate and reliable lane-marking semantic segmentation in the imagery of roads and highways is needed to achieve such goals. We use airborne imagery that can capture a large area in a short period of time by introducing an aerial lane marking data set. In this paper, we propose a symmetric fully convolutional neural network enhanced by wavelet transform in order to automatically carry out lane-marking segmentation in aerial imagery. Due to a heavily unbalanced problem in terms of a number of lane-marking pixels compared with background pixels, we use a customized loss function as well as a new type of data augmentation step. We achieve a high accuracy in pixelwise localization of lane markings compared with the state-of-the-art methods without using the third-party information. In this paper, we introduce the first high-quality data set used within our experiments, which contains a broad range of situations and classes of lane markings representative of today’s transportation systems. This data set will be publicly available, and hence, it can be used as the benchmark data set for future algorithms within this domain. [ABSTRACT FROM AUTHOR]

Predicting the state of dynamic objects in a real traffic environment is a key issue in autonomous driving vehicles. Various approaches have been proposed to learn the dynamics from visual observations with static background. However, minimal research has been conducted in a real traffic environment due to the complicated and changeable scenes. This paper proposes an adaptive multi-target future state prediction (position/velocity) method under autonomous driving conditions. In particular, an adaptive visual interaction method and control mechanism are introduced to overcome the change in the number of objects in continuous driving frames. In addition, a two-stream architecture with stage-wise learning is utilized for accurate object state prediction by simultaneously complementing spatial and temporal information. Experiments on two public challenging datasets, namely Udacity (CrowdAI) and Udacity (Autti), demonstrate the effectiveness of the proposed method on multi-target dynamic state prediction in a real traffic environment. [ABSTRACT FROM AUTHOR]

Predicting the future trajectories of surrounding agents has become an crucial problem to be solved for the safety of autonomous vehicles. Recent studies based on Long Short Term Memory (LSTM) networks have shown powerful abilities to model social interactions. However, many of these approaches focus on spatial interactions of the neighborhood agents but ignore temporal interactions that accompany spatial interactions. In this paper, we propose a Hierarchical Spatio-Temporal Attention architecture (HSTA), which activates the utilization of spatial interactions with different weights, and jointly considers the temporal interactions across time steps of all agents. More specially, the graph attention mechanism (GAT) is presented to capture spatial interactions, the multi-head attention mechanism (MHA) is conducted to encode temporal correlations of interactions and a state gated fusion (SGF) layer is used to integrate spatial and temporal interactions. We evaluate our proposed method against baselines on both pedestrian and vehicle datasets. The results show that our model is effective and achieves state-of-the-art achievements. [ABSTRACT FROM AUTHOR]

This paper presents a latent space reinforcement learning method for interpretable decision-making of autonomous vehicles at highway on-ramps. This method is based on the latent model and the combination model of the hidden Markov model and Gaussian mixture regression (HMM-GMR). It is difficult for the traditional decision-making method to understand the environment because its input is high-dimensional and lacks an understanding of the task. By utilizing the HMM-GMR model, we can obtain the interpretable state providing semantic information and environment understanding. A framework is proposed to unify representation learning with the deep reinforcement learning (DRL) approach, in which the latent model is used to reduce the dimension of interpretable state by extracting underlying task-relevant information. Experimental results are presented and the results show the right balance between driving safety and efficiency in the challenging scenarios of highway on-ramps merging. [ABSTRACT FROM AUTHOR]

Driving a car in an urban setting is an extremely difficult problem, incorporating a large number of complex visual tasks; however, this problem is solved daily by most adults with little apparent effort. This paper proposes a novel vision-based approach to autonomous driving that can predict and even anticipate a driver's behavior in real time, using preattentive vision only. Experiments on three large datasets totaling over 200 000 frames show that our preattentive model can 1) detect a wide range of driving-critical context such as crossroads, city center, and road type; however, more surprisingly, it can 2) detect the driver's actions (over 80% of braking and turning actions) and 3) estimate the driver's steering angle accurately. Additionally, our model is consistent with human data: First, the best steering prediction is obtained for a perception to action delay consistent with psychological experiments. Importantly, this prediction can be made before the driver's action. Second, the regions of the visual field used by the computational model strongly correlate with the driver's gaze locations, significantly outperforming many saliency measures and comparable to state-of-the-art approaches. [ABSTRACT FROM PUBLISHER]

Millimeter-wave radars are being increasingly integrated into commercial vehicles to support advanced driver-assistance system features. A key shortcoming for present-day vehicular radar imaging is poor azimuth resolution (for side-looking operation) due to the form factor limits on antenna size and placement. In this paper, we propose a solution via a new multiple-input and multiple-output synthetic aperture radar (MIMO-SAR) imaging technique, that applies coherent SAR principles to vehicular MIMO radar to improve the side-view (angular) resolution. The proposed 2-stage hierarchical MIMO-SAR processing workflow drastically reduces the computation load while preserving image resolution. To enable coherent processing over the synthetic aperture, we integrate a radar odometry algorithm that estimates the trajectory of ego-radar. The MIMO-SAR algorithm is validated by both simulations and real experiment data collected by a vehicle-mounted radar platform. [ABSTRACT FROM AUTHOR]

Modern computer vision requires processing large amounts of data, both while training the model and/or during inference, once the model is deployed. Scenarios where images are captured and processed in physically separated locations are increasingly common (e.g. autonomous vehicles, cloud computing, smartphones). In addition, many devices suffer from limited resources to store or transmit data (e.g. storage space, channel capacity). In these scenarios, lossy image compression plays a crucial role to effectively increase the number of images collected under such constraints. However, lossy compression entails some undesired degradation of the data that may harm the performance of the downstream analysis task at hand, since important semantic information may be lost in the process. Moreover, we may only have compressed images at training time but are able to use original images at inference time (i.e. test), or vice versa, and in such a case, the downstream model suffers from covariate shift. In this paper, we analyze this phenomenon, with a special focus on vision-based perception for autonomous driving as a paradigmatic scenario. We see that loss of semantic information and covariate shift do indeed exist, resulting in a drop in performance that depends on the compression rate. In order to address the problem, we propose dataset restoration, based on image restoration with generative adversarial networks (GANs). Our method is agnostic to both the particular image compression method and the downstream task; and has the advantage of not adding additional cost to the deployed models, which is particularly important in resource-limited devices. The presented experiments focus on semantic segmentation as a challenging use case, cover a broad range of compression rates and diverse datasets, and show how our method is able to significantly alleviate the negative effects of compression on the downstream visual task. [ABSTRACT FROM AUTHOR]

Semantic segmentation, unifying most navigational perception tasks at the pixel level has catalyzed striking progress in the field of autonomous transportation. Modern Convolution Neural Networks (CNNs) are able to perform semantic segmentation both efficiently and accurately, particularly owing to their exploitation of wide context information. However, most segmentation CNNs are benchmarked against pinhole images with limited Field of View (FoV). Despite the growing popularity of panoramic cameras to sense the surroundings, semantic segmenters have not been comprehensively evaluated on omnidirectional wide-FoV data, which features rich and distinct contextual information. In this paper, we propose a concurrent horizontal and vertical attention module to leverage width-wise and height-wise contextual priors markedly available in the panoramas. To yield semantic segmenters suitable for wide-FoV images, we present a multi-source omni-supervised learning scheme with panoramic domain covered in the training via data distillation. To facilitate the evaluation of contemporary CNNs in panoramic imagery, we put forward the Wild PAnoramic Semantic Segmentation (WildPASS) dataset, comprising images from all around the globe, as well as adverse and unconstrained scenes, which further reflects perception challenges of navigation applications in the real world. A comprehensive variety of experiments demonstrates that the proposed methods enable our high-efficiency architecture to attain significant accuracy gains, outperforming the state of the art in panoramic imagery domains. [ABSTRACT FROM AUTHOR]

With the aims of safe, smart and sustainable future mobility, a personalized approach of trajectory planning and control based on user preferences is developed for lane-change of autonomous vehicles in this paper. First, a safe area during the lane change process is identified by using constraint Delaunay triangulation. Then, an improved rapidly-exploring Random Trees (i-RRT) is developed with B-spline to generate the feasible trajectory cluster, which is subject to the safe area boundaries and the vehicle dynamics. To extract a personalized trajectory from this cluster, we firstly adopt the fuzzy linguistic preference relation (FLPR) method to identify users’ preferences on driving, which can be reflected by their subjective objectives including driving safety, ride comfort and vehicle stability. Then, the technique for order preference by similarity to ideal situation (TOPSIS) is utilized to solve the multi-objective optimisation problem formulated by considering the user preferences. The algorithms proposed above are integrated, and both simulation and experimental validation are conducted under lane-change scenarios of autonomous driving. Simulation and experiment results show that proposed approach is able to successfully realize personalized trajectory planning and lane-change control, satisfying users’ various preferences and simultaneously ensure vehicle safety, demonstrating its feasibility and effectiveness. [ABSTRACT FROM AUTHOR]

Autonomous vehicle trajectory planning method based on the driving risk field is one of the current research hotspots. Currently, mainstream driving risk field models (DRFM) rarely consider basic factors such as vehicle geometry and heading angle, and ignore the impact of driving behavior characteristics on the driving risk field, the accuracy of safety assessment needs to be improved. Therefore, this paper proposes an extended driving risk field model (EDRFM) that considers geometric dimensions, heading angle, and driving behavior characteristics, and provides the basis for determining geometric dimensions and shape parameters, expanding the applicability of the DRFM. On this basis, EDRFM is applied to vehicle trajectory planning, and it is found that the target trajectory (denoted as P1) obtained using EDRFM is not achievable and the vehicle is unstable. This paper further proposes an arc correction method considering vehicle dynamics constraints to correct P1 to get P2. To ensure that the trajectory planning results meet the driving stability and motion of the vehicle. The results show that the EDRFM model considering arc correction can effectively complete vehicle trajectory planning tasks in complex scenes, and ensure the feasibility of trajectory planning and vehicle driving stability.

Predicting the future trajectories of surrounding pedestrians is undoubtedly one of the most essential but challenging tasks for safe urban autonomous driving. Despite this importance, there has been limited research conducted on the egocentric view from easy-to-access vehicle-mounted cameras for autonomous driving applications. This paper presents a non-autoregressive transformer based trajectory prediction methodology for pedestrian on egocentric view. Furthermore, our proposed model predicts ego-motion independent future trajectories for utilization in downstream tasks such as motion planning in autonomous vehicles. This approach differs from previous researches as it focuses on predicting the future position of pedestrians based on the current observed image context, rather than their future positions in future observed images. The proposed model, referred to as the TransPred network in this paper, is composed of three main modules: vehicle motion compensation, non-autoregressive transformer, and conditional variational autoencoder(CVAE). The transformer structure is employed to effectively handle raw images and the historical trajectory of the target pedestrian, enabling the generation of advanced future predictions. Additionally, the CVAE module is utilized in the final part of the overall model to predict plausible multiple future trajectories. It contributes to generating diverse and realistic future trajectory predictions. The performance of our model has been evaluated on Nuscenes and In-house dataset obtained from our vehicle equipped with sensors. We achieves the state-of-the-art performance for prioritized trajectories on both datasets. Moreover, the usability of the proposed ego-motion independent trajectories for autonomous driving is demonstrated through risk assessment experiments.

In recent years, vehicle detectors built on deep convolutional neural network (DCNN) have been widely used in autonomous driving. Under the complex conditions of road traffic, the detector is expected to run in high speed and high accuracy. However, due to the limited computing power and storage space on the autonomous vehicle, the deployment of advanced DCNN detectors is often restricted. The design of lightweight and powerful detectors is in a great desire. Recently, group convolution, as a novel convolution algorithm, has been proposed to reduce the floating-point operations and make the detection network lighter and faster. However, in practice, it is found that the increase of group number does not always boost the detection speed, but sometimes leads to the performance degradation. In addition, the existing guidelines for network design do not indicate how to choose the group number in the group convolution in order to maximize the overall detection speed. To this end, in this paper, we propose three new guidelines to determine the valid range of the group number, and design a lightweight detection network—DenseLightNet—based on these new design criteria. The proposed detector runs at a speed of three times faster than the current real-time detector YoloV3, while holding a much smaller model size. [ABSTRACT FROM AUTHOR]

Vehicle localization and mobility tracking are important tasks in autonomous driving. Traditional methods either have insufficient accuracy or rely on additional facilities to reach the desired accuracy for autonomous driving. In this paper, a multi-sensor multi-vehicle localization and mobility tracking framework is developed for autonomous vehicles equipped with GPS, inertial measurement unit (IMU), and an integrated sensing system. Our algorithm fuse the information from local onboard sensors as well as the observations of other vehicles or existing intelligent transportation system infrastructure such as road side units (RSU) to improve the precision and stability of localization and mobility tracking. Specifically, this framework incorporates the dynamic model of vehicles to achieve better localization and tracking performance. The communication delays during the information sharing process are explicitly taken into account in our algorithm development. Simulations manifest that not only the accuracy of localization and mobility tracking could be greatly enhanced in general, but also the robustness can be guaranteed under circumstances where traditional localization and tracking devices fail. [ABSTRACT FROM AUTHOR]

Active safety systems are of significant importance for autonomous vehicles operating in safety-critical situations like an obstacle-avoidance manoeuvre with high vehicle speed or poor road condition. However, a conventional electronic stability control system, may not always yield desired path following and yaw stability performance in such circumstances merely through brake intervention. This paper pursues a detailed investigation on utilising model predictive control (MPC) and torque vectoring for path following and yaw stability control of over-actuated autonomous electric vehicles (AEVs) in dangerous double lane change manoeuvre. The control problem of the AEV is formulated based on MPC by utilising active front steering and torque vectoring, and constraints are imposed explicitly on yaw rate and sideslip angle to ensure yaw stability. Four MPC-based controllers are designed based on double-track vehicle models. Specifically, they include two one-level controllers, i.e. one with torque vectoring and one with equal torque allocation, and two two-level controllers, i.e. one with optimisation-based torque allocation and one with rule-based allocation. These controllers are assessed extensively, with respect to passing velocity, tracking accuracy, tyre utilisation and robustness. The effect of horizon length on the control performance and computational efficiency is also investigated. [ABSTRACT FROM AUTHOR]

This paper introduces a method to extract driving behaviors from a human expert driver which are applied to an autonomous agent to reproduce proactive driving behaviors. Deep learning techniques were used to extract latent features from the collected data. Extracted features were clustered into behaviors and used to create velocity profiles allowing an autonomous driving agent could drive in a human-like manner. By using proactive driving behaviors, the agent could limit potential sources of discomfort such as jerk and uncomfortable velocities. Additionally, we proposed a method to compare trajectories where not only the geometric similarity is considered, but also velocity, acceleration and jerk. Experimental results in a simulator implemented in ROS show that the autonomous agent built with the driving behaviors was capable of driving similarly to expert human drivers. [ABSTRACT FROM AUTHOR]

With the rapid development of autonomous driving, collision avoidance has attracted attention from both academia and industry. Many collision avoidance strategies have emerged in recent years, but the dynamic and complex nature of driving environment poses a challenge to develop robust collision avoidance algorithms. Therefore, in this paper, we propose a decentralized framework named RACE: Reinforced Cooperative Autonomous Vehicle Collision AvoidancE. Leveraging a hierarchical architecture we develop an algorithm named Co-DDPG to efficiently train autonomous vehicles. Through a security abiding channel, the autonomous vehicles distribute their driving policies. We use the relative distances obtained by the opponent sensors to build the VANET instead of locations, which ensures the vehicle's location privacy. With a leader-follower architecture and parameter distribution, RACE accelerates the learning of optimal policies and efficiently utilizes the remaining resources. We implement the RACE framework in the widely used TORCS simulator and conduct various experiments to measure the performance of RACE. Evaluations show that RACE quickly learns optimal driving policies and effectively avoids collisions. Moreover, RACE also scales smoothly with varying number of participating vehicles. We further compared RACE with existing autonomous driving systems and show that RACE outperforms them by experiencing 65% less collisions in the training process and exhibits improved performance under varying vehicle density. [ABSTRACT FROM AUTHOR]

Semantic segmentation is a key step in scene understanding for autonomous driving. Although deep learning has significantly improved the segmentation accuracy, current high-quality models such as PSPNet and DeepLabV3 are inefficient given their complex architectures and reliance on multi-scale inputs. Thus, it is difficult to apply them to real-time or practical applications. On the other hand, existing real-time methods cannot yet produce satisfactory results on small objects such as traffic lights, which are imperative to safe autonomous driving. In this paper, we improve the performance of real-time semantic segmentation from two perspectives, methodology and data. Specifically, we propose a real-time segmentation model coined Narrow Deep Network (NDNet) and build a synthetic dataset by inserting additional small objects into the training images. The proposed method achieves 65.7% mean intersection over union (mIoU) on the Cityscapes test set with only 8.4G floating-point operations (FLOPs) on $1024\times 2048$ inputs. Furthermore, by re-training the existing PSPNet and DeepLabV3 models on our synthetic dataset, we obtained an average 2% mIoU improvement on small objects. [ABSTRACT FROM AUTHOR]

Deep reinforcement learning (DRL) has been successfully applied to end-to-end autonomous driving, especially in simulation environments. However, common DRL approaches used in complex autonomous driving scenarios sometimes are unstable or difficult to converge. This paper proposes two approaches to improve the stability of the policy model training with as few manual data as possible. For the first approach, reinforcement learning is combined with imitation learning to train a feature network with a small amount of manual data for parameters initialization. For the second approach, an auxiliary network is added to the reinforcement learning framework, which can leverage the real-time measurement information to deepen the understanding of environment, without any guide of demonstrators. To verify the effectiveness of these two approaches, simulations in image information-based and lidar information-based end-to-end autonomous driving systems are conducted, respectively. These approaches are not only tested in the virtual game world, but also applied in Gazebo, in which we build a 3D world based on the real vehicle model of Ranger XP900 platform, the real 3D obstacle model, and the real motion constraints with inertial characteristics, so as to ensure that the trained end-to-end autonomous driving model is more suitable for the real world. Experimental results show that the performance is increased by over 45% in the virtual game world, and can converge quickly and stably in Gazebo in which previous methods can hardly converge. [ABSTRACT FROM AUTHOR]

This paper presents an emergency pullover algorithm for fail-safe systems designed for level-4 autonomous vehicles. The proposed algorithm utilizes feedback gain adaptation, based on sensitivity estimation, and cost-based learning. Vehicle failure within this paper does not encompass every type of failure and refers only to any situation where the upper controller or communications from the upper controller shuts down. When this type of failure occurs, the algorithm performs an emergency pullover maneuver. This maneuver does not require any form of independent control from the driver to be performed successfully. However, the highest control priority is still given to the driver if the driver intervenes during the maneuver. The feedback gain adaptation is comprised of two sections: Sensitivity Estimation and Gradient Descent (GD) based Adaptation. For Sensitivity Estimation, a relationship function has been designed with feedback gain, from the feedback gain adaptation, and changes in state error. The sensitivity of state error with respect to feedback gain can then be estimated. This estimation is done through the Recursive Least Squares (RLS) method with multiple forgetting factors through the directional forgetting method. For GD based Adaptation, state errors are applied with parameters for the cost-based learning to give Adaptation Gains. These Adaptation Gains are used in tandem with the estimated sensitivity to update the feedback gain. To reduce the number of tuning parameters required in the GD method, an additional distance condition has been proposed. This condition utilizes feedback change rates and state errors, obtained from the multi-dimensional plane of the feedback gain’s change rates. A proportional coefficient is also required as a tuning parameter for this condition. This parameter is tuned by a cost-based learning algorithm, also designed in this study. Resultantly, these methods allow the adaptive feedback controller to forgo any system information such as mathematical models and system parameters. This indicates that the vehicle model is not expected to hinder performance. Hence, controllers that do not require system information are indeed a preferable algorithm for fail-safe modules. Performance evaluations for the controller has also been conducted with actual vehicle tests, under longitudinal and lateral autonomous driving scenarios.

This paper presents an adaptive and weighted model predictive control (MPC) algorithm for autonomous driving with disturbance estimation and prediction. Unexpected and unpredictable disturbances in the real world limit the performance of MPC. To overcome this limitation, this paper proposes adaptive and weighted prediction methods with a sliding mode observer and a weighting function with the grey prediction model. The sliding mode observer is designed for disturbance estimation with finite stability conditions, and the estimated disturbance is predicted using the grey prediction model. Based on the adaptive and weighted prediction method, the length of prediction horizon and cost value of each predicted state are adjusted in real time to eliminate any negative impact on future predicted states. Meanwhile, a variation in the cost value, which is caused by prediction horizon adaptation and weighted prediction, may harm the control performance as it can excessively increase or decrease the model uncertainty. Therefore, an input weighting factor is adapted in the MPC cost function based on an exponential weighting function. The performance of the proposed adaptive control algorithm is evaluated using CarMaker software under longitudinal and lateral autonomous driving scenarios.

In this paper, we propose a decision-making framework for autonomous driving at road intersections that determines appropriate maneuvers for an autonomous vehicle to navigate an intersection safely and efficiently (regardingmaking progress), even in the face of violation vehicles—one of the most challenging tasks in the domain of autonomous vehicles. The proposed framework uses a digital map to predict future paths of observed vehicles and then uses the predicted future paths to identify potential threats (vehicles) and collision areas, regardless of whether observed vehicles are obeying traffic rules at the intersection. Next, under an independent and distributed reasoning structure, it systematically, reliably, and robustly assesses the potential threats, even under incomplete and uncertain noise data, by way of a threat measure, Bayesian networks, and time window filtering. It then uses this information to determine appropriate maneuvers for the autonomous vehicle to navigate the intersection safely and efficiently. We have tested and evaluated the proposed framework through in-vehicle testing on a closed urban test road under traffic conditions inclusive of nonviolation and violation vehicles. In-vehicle testing results show the performance of the proposed framework to be sufficiently reliable for autonomous driving at intersections regarding reliability, robustness, safety, and efficiency. [ABSTRACT FROM AUTHOR]

Verification and validation of automated driving functions impose large challenges. Currently, scenario-based approaches are investigated in research and industry, aiming at a reduction of testing efforts by specifying safety relevant scenarios. To define those scenarios and operate in a complex real-world design domain, a structured description of the environment is needed. Within the PEGASUS research project, the 6-Layer Model (6LM) was introduced for the description of highway scenarios. This paper refines the 6LM and extends it to urban traffic and environment. As defined in PEGASUS, the 6LM provides the possibility to categorize the environment and, therefore, functions as a structured basis for subsequent scenario description. The model enables a structured description and categorization of the general environment, without incorporating any knowledge or anticipating any functions of actors. Beyond that, there is a variety of other applications of the 6LM, which are elaborated in this paper. The 6LM includes a description of the road network and traffic guidance objects, roadside structures, temporary modifications of the former, dynamic objects, environmental conditions and digital information. The work at hand specifies each layer by categorizing its items. Guidelines are formulated and explanatory examples are given to standardize the application of the model for an objective environment description. In contrast to previous publications, the model and its design are described in far more detail. Finally, the holistic description of the 6LM presented includes remarks on possible future work when expanding the concept to machine perception aspects.

The goal of this paper is to perform 3D object detection in the context of autonomous driving. Our method aims at generating a set of high-quality 3D object proposals by exploiting stereo imagery. We formulate the problem as minimizing an energy function that encodes object size priors, placement of objects on the ground plane as well as several depth informed features that reason about free space, point cloud densities and distance to the ground. We then exploit a CNN on top of these proposals to perform object detection. In particular, we employ a convolutional neural net (CNN) that exploits context and depth information to jointly regress to 3D bounding box coordinates and object pose. Our experiments show significant performance gains over existing RGB and RGB-D object proposal methods on the challenging KITTI benchmark. When combined with the CNN, our approach outperforms all existing results in object detection and orientation estimation tasks for all three KITTI object classes. Furthermore, we experiment also with the setting where LIDAR information is available, and show that using both LIDAR and stereo leads to the best result. [ABSTRACT FROM PUBLISHER]

This paper describes the design, implementation, and evaluation of a virtual target-based overtaking decision, motion planning, and control algorithm for autonomous vehicles. Both driver acceptance and safety, when surrounded by other vehicles, must be considered during autonomous overtaking. These are considered through safe distance based on human driving behavior. Since all vehicles cannot be equipped with a vehicle to vehicle communications at present, autonomous vehicles should perceive the surrounding environment based on local sensors. In this paper, virtual targets are devised to cope with the limitation of cognitive range. A probabilistic prediction is adopted to enhance safety, given the potential behavior of surrounding vehicles. Then, decision-making and motion planning has been designed based on the probabilistic prediction-based safe distance, which could achieve safety performance without a heavy computational burden. The algorithm has considered the decision rules that drivers use when overtaking. For this purpose, concepts of target space, demand, and possibility for lane change are devised. In this paper, three driving modes are developed for active overtaking. The desired driving mode is decided for safe and efficient overtaking. To obtain desired states and constraints, intuitive motion planning is conducted. A stochastic model predictive control has been adopted to determine vehicle control inputs. The proposed autonomous overtaking algorithm has been evaluated through simulation, which reveals the effectiveness of virtual targets. Also, the proposed algorithm has been successfully implemented on an autonomous vehicle and evaluated via real-world driving tests. Safe and comfortable overtaking driving has been demonstrated using a test vehicle.

Road traffic signals and traffic infrastructure have a significant impact on the behaviour of highly automated or autonomous vehicles. However, the increase in automation does not always mean an advantage. Generally, highly automated vehicles strictly follow the traffic rules resulting in near-accident situations, although their goal is to avoid and reduce them. Malfunctions of the automated functions might cause surprising interventions while the vehicle is in motion, drivers cannot react in time and well. This paper highlights the potential danger and uncertainty of highly automated or autonomous vehicles in the context of the current conventional traffic infrastructure system. In the future, special consideration shall be given to the vehicle industry and traffic regulation makers on how the infrastructure should be adapted to automated vehicle functions to have a seamless shift towards automated driving. The paper sums up many problematic situations with two critical problems with sensitivity analysis. The first situation: the speed assist system (based on speed limit sign recognition) conflicts with the traffic infrastructure. The second situation is shown: the ACC (Adaptive Cruise Control) and LKA (Lane Assist) contradicts with the traffic infrastructure. These critical situations were investigated by using an high-fidelity automotive simulation software as proof of concept and were examined by accident reconstruction analysis software.

This paper proposes a strategy for visual perception in the context of autonomous driving. Humans, when not distracted or drunk, are still the best drivers you can currently find. For this reason, we take inspiration from two theoretical ideas about the human mind and its neural organization. The first idea concerns how the brain uses structures of neuron ensembles that expand and compress information to extract abstract concepts from visual experience and code them into compact representations. The second idea suggests that these neural perceptual representations are not neutral but functional to predicting the future state of affairs in the environment. Similarly, the prediction mechanism is not neutral but oriented to the planning of future action. We identify within the deep learning framework two artificial counterparts of the aforementioned neurocognitive theories. We find a correspondence between the first theoretical idea and the architecture of convolutional autoencoders, while we translate the second theory into a training procedure that learns compact representations which are not neutral but oriented to driving tasks, from two distinct perspectives. From a static perspective, we force separate groups of neural units in the compact representations to represent specific concepts crucial to the driving task distinctly. From a dynamic perspective, we bias the compact representations to predict how the current road scenario will change in the future. We successfully learn compact representations that use as few as 16 neural units for each of the two basic driving concepts we consider: cars and lanes. We maintain the two concepts separated in the latent space to facilitate the interpretation and manipulation of the perceptual representations. The source code for this paper is available at https://github.com/3lis/rnn_vae.

The progress in object detection for autonomous driving using LiDAR point cloud data has been remarkable. However, current voxel-based two-stage detectors have not fully capitalized on the wealth of contextual information present in the point cloud data. Typically, Voxel Feature Encoding (VFE) layers tend to focus exclusively on internal voxel information, neglecting the broader context. Additionally, the process of extracting 3D proposal features through Region of Interest (ROI) spatial quantization and pooling downsampling results in a loss of spatial detail within the proposed regions. This limitation in capturing contextual details presents challenges for accurate object detection and positioning, particularly over long distances. In this paper, we propose ContextNet, which leverages comprehensive contextual information for enhanced 3D object detection. Specifically, it comprises two modules: the Voxel Self-Attention Encoding module (VSAE) and the Joint Channel Self-Attention Re-weight module (JCSR). VSAE establishes dependencies between voxels through self-attention, expanding the receptive field and introducing substantial contextual information. JCSR employs joint attention to extract both local channel information and global context information from the raw point cloud within the RoI region. By integrating these two sets of information and re-weighting the point features, the 3D proposal is refined, enabling a more accurate estimation of the object’s position and confidence. Extensive experiments conducted on the KITTI dataset demonstrate that our approach outperforms voxel-based two-stage methods, particularly with a 9.5% improvement in the mAP compared to the baseline on the nuScenes test dataset, and an improved 1.61% hard AP compared to the baseline on the KITTI benchmark.

In the domain of intelligent transportation systems, the advent of autonomous driving technology represents a critical milestone, profoundly shaping the automotive industry’s evolutionary path. This technology’s core, particularly the algorithms facilitating driverless path planning, has attracted significant scholarly interest. This paper presents an advanced Deep Reinforcement Learning algorithm for Path Planning (DRL-PP), designed to rectify the shortcomings inherent in existing path planning techniques. Considering the complex nature of the environment, the DRL-PP algorithm is meticulously crafted to ascertain optimal actions, thereby effectively reducing the propensity for overfitting. The algorithm harnesses the capabilities of deep reinforcement learning, utilizing neural networks to identify the most advantageous action corresponding to a specific state. It then constructs an optimal action sequence, extending from the vehicle’s initial position to its designated target. Additionally, the algorithm enhances the reward function by incorporating data pertinent to the objective. This refinement enables the nuanced differentiation of action values based on dynamically adjusted reward metrics, thereby augmenting the efficiency of the action selection process and yielding improved results in path planning. Empirical results validate the algorithm’s proficiency in stabilizing the reward metric while minimizing exploratory steps, consistently surpassing comparative models in path-finding effectiveness.

In the modern field of multi-view 3D detection, interest has seen a notable rise. Existing methods are predominantly focused on constructing dense BEV (Bird’s Eye View) features or utilizing sparse queries for detection. In this paper, we synergize the methodologies of sparse queries with dense BEV feature representation and integrate robust 2D detection capabilities to propose an innovative structure. Specifically, we leverage the results of 2D detection and their predicted depth to generate a series of sparse queries endowed with strong semantic and positional priors. These queries demonstrate enhanced capability in capturing challenging cases in 3D detection. Additionally, we employ 2D detection and depth information to discern critical foreground BEV queries. This strategy enables a efficient and noiseless feature aggregation, leveraging multi-view image features. Our methodology extends to a comprehensive global modeling of all BEV queries, ensuring the derivation of premium quality BEV features. Furthermore, we enhance BEV features through temporal modeling that differentiates between static and dynamic objects. Our proposed approach achieves state-of-the-art results on the nuScenes test set with 60.4%mAP and 51.7%NDS. Code will be available.

As the issues of social sustainability and the aging of population becomes increasingly severe, Autonomous Driving technology is increasingly being seen as an important issue for future travel. At present, the Human Machine Interface (HMI) of Autonomous Driving system has certain difficulties for middle-aged and elderly users, which further affects their perception of cars status and operation experience. Therefore, in order to design an elderly-oriented HMI design of Autonomous Driving cars that meets the Kansei needs of middle-aged and elderly users, a design flow based on Kansei Engineering/ Rough Set Theory/ Backpropagation Neural Network is proposed. The HMI of Autonomous Driving cars is taken as an example in this paper. Under the framework of Kansei Engineering research, Kansei intention analysis is carried out. Factor Analysis is used to reduce dimension and cluster the collected Kansei words. By using the morphological analysis method, the HMI samples are deconstructed into 14 different design features. The attribute reduction algorithm in Rough Set Theory is used to identify the key design features of HMI that have important influence on the elderly-oriented level. Backpropagation Neural Network is used to establish the mapping model between the Kansei intention of middle-aged and elderly users and the key design features of HMI. The mapping model demonstrates good fit as the errors between the predicted and actual values in the 4 types of kansei semantic evaluation tests are all less than 5%. So that it could meet the needs of middle-aged and elderly users and obtain the design combination with the highest Kansei average value. Based on the experimental results, it is found that the optimal emotion of middle-aged and elderly users could be obtained by overall color type three, mode of speed display type three, font display of rotate speed type three, reminding color of turning type four and mode of fuel indicator display one in the morphological deconstruction table. The research results show that the elderly-oriented HMI design constructed by Kansei Engineering/ Rough Set Theory/ Backpropagation Neural Network can meet the sentimental needs of middle-aged and elderly users and can provide a reference example for relevant elderly-oriented design.

The paper introduces a quadratic programming algorithm for real-time local path planning of autonomous vehicles. The algorithm relies on discretized sampling points and an enhanced cost function. Initially, we formulate the cost function to optimize the reference trajectory and establish the Frenet coordinate system. The drivable region undergoes discretization to generate sampling points in the Frenet coordinate system. We apply the principles of convex spatial obstacle avoidance to define the vehicle’s drivable area, taking into account the vehicle’s kinematics and establishing barrier boundary conditions. Subsequently, quadratic programming is employed to determine an optimal path within the vehicle’s drivable area. Concurrently, two cost functions are devised, the first evaluates the distance between the vehicle and obstacles, while the second assesses ride comfort, these cost functions are employed to evaluate sampling points and speed profiles, facilitating the planning of an optimal speed profile on the selected path. Finally, the algorithm undergoes validation through co-simulation using Matlab/Simulink, PreScan, and CarSim software. Various road scenarios, including straight and S-curve roads with both dynamic and static obstacles, are created to assess the method’s feasibility. The test results demonstrate the algorithm’s efficacy in avoiding moving and stationary obstacles and generating an ideal path compliant with driving conditions.

Lane-level self-localization is a critical task in the field of autonomous driving. Map-based self-localization is commonly employed to achieve lane-level accuracy in urban settings. However, it is known that in certain locations, such as narrow roads and intersections, map-based self-localization poses challenges, leading to failures of autonomous driving tasks. Potential errors in specific locations can be estimated before driving by leveraging the geometrical structures of the environment and can then be utilized to enhance the safety of autonomous driving operations. In contrast to traditional self-localization error estimation methods, which often rely on statistical analyses or regression specific to a particular map format, this research focuses on identifying fundamental errors in self-localization within various map formats. The proposed method employs a general formulation of environmental representation, specifically normal distributions, and a closed-form uncertainty approximation of optimal solutions, enabling the identification of essential self-localization errors in an environment. The results obtained through this method are not only valuable for autonomous driving tasks but also contribute to discussions on the quality of specific map formats. Experiments highlight locations within an urban environment that are susceptible to significant errors in self-localization based on the proposed metric. Furthermore, a comparison of self-localization errors associated with specific map formats using the proposed metric reveals that essential errors in algorithms can be estimated. The discussion presented in this paper reveals the patterns of geometrical features in the urban environment that are likely to result in self-localization errors.

Recently, with the development of semiconductors and VLSI (Very Large Scale Integrated Circuit), the technology required for autonomous driving is rapidly developing. One of the technologies that cannot be left out is the lane detection function. Lane recognition requires a lot of data from the camera sensor. As a result, the data size increases, making it difficult to process on a lightweight embedded board. This paper proposes a sliding-based parallel segment image processing method to solve this problem. Most boards in autonomous vehicles are lightweight, so the technique has been designed to reduce computation and power consumption. After fetching the image’s pixel data, grayscale conversion, Gaussian smoothing, Sobel operator, non-maximum suppression, and hysteresis are performed in parallel. Lanes were detected by performing a Hough transform operation on an image for which edge detection was completed in parallel. Due to the nature of parallel processing, it is more effective when image input is continuous and numerous than single image processing. This algorithm is written in C language and VHDL (VHSIC Hardware Description Language) for two parts in the board, DE1-SoC, FPGA (Field Programmable Gate Array) and HPS (Hard Processor System. Due to the use of the C language and VHDL, parallel programming uses 3.1 times less time, twice as much memory and slightly more power than sequential programming. For hardware languages such as Verilog, the computation algorithms have been converted to a fixed point. When comparing HPS and FPGA, the FPGA consumed significantly fewer resources, with 18 times shorter run time, 50 times fewer clock cycles, 3 times less power, and 183 times less energy. This provides a substantial benefit.