Search

Vehicle localization in highway tunnels is challenging for autonomous vehicle navigation. The drift error increases because GPS signals from satellites cannot be received inside a highway tunnel. Therefore, map-aided localization is essential for correcting the drift error. However, the environment around the tunnel is composed mainly of elliptical walls, and there are few unique feature points for map matching, unlike the case outdoors. In this paper, we propose a tunnel facility–based precise vehicle localization method for highway tunnels using 3D LIDAR. For vehicle localization, a point landmark map storing the center points of the tunnel facilities and a probability distribution map storing the probability distributions of the lane markings is used. In the proposed method, the tunnel facility points are extracted using 3D LIDAR, and position estimation is conducted using an extended Kalman filter-based navigation filter. The proposed localization algorithm was verified through experiments using actual highway driving data. The experimental results verify that tunnel facility–based vehicle localization yields precise results in various situations. [ABSTRACT FROM AUTHOR]

Pedestrian Detection is the most critical module of an Autonomous Driving system. Although a camera is commonly used for this purpose, its quality degrades severely in low-light night time driving scenarios. On the other hand, the quality of a thermal camera image remains unaffected in similar conditions. This paper proposes an end-to-end multimodal fusion model for pedestrian detection using RGB and thermal images. Its novel spatio-contextual deep network architecture is capable of exploiting the multimodal input efficiently. It consists of two distinct deformable ResNeXt-50 encoders for feature extraction from the two modalities. Fusion of these two encoded features takes place inside a multimodal feature embedding module (MuFEm) consisting of several groups of a pair of Graph Attention Network and a feature fusion unit. The output of the last feature fusion unit of MuFEm is subsequently passed to two CRFs for their spatial refinement. Further enhancement of the features is achieved by applying channel-wise attention and extraction of contextual information with the help of four RNNs traversing in four different directions. Finally, these feature maps are used by a single-stage decoder to generate the bounding box of each pedestrian and the score map. We have performed extensive experiments of the proposed framework on three publicly available multimodal pedestrian detection benchmark datasets, namely KAIST, CVC-14, and UTokyo. The results on each of them improved the respective state-of-the-art performance. A short video giving an overview of this work along with its qualitative results can be seen at https://youtu.be/FDJdSifuuCs. Our source code will be released upon publication of the paper. [ABSTRACT FROM AUTHOR]

The rapid advancement of sensor technologies and artificial intelligence are creating new opportunities for traffic safety enhancement. Dashboard cameras (dashcams) have been widely deployed on both human driving vehicles and automated driving vehicles. A computational intelligence model that can accurately and promptly predict accidents from the dashcam video will enhance the preparedness for accident prevention. The spatial-temporal interaction of traffic agents is complex. Visual cues for predicting a future accident are embedded deeply in dashcam video data. Therefore, the early anticipation of traffic accidents remains a challenge. Inspired by the attention behavior of humans in visually perceiving accident risks, this paper proposes a Dynamic Spatial-Temporal Attention (DSTA) network for the early accident anticipation from dashcam videos. The DSTA-network learns to select discriminative temporal segments of a video sequence with a Dynamic Temporal Attention (DTA) module. It also learns to focus on the informative spatial regions of frames with a Dynamic Spatial Attention (DSA) module. A Gated Recurrent Unit (GRU) is trained jointly with the attention modules to predict the probability of a future accident. The evaluation of the DSTA-network on two benchmark datasets confirms that it has exceeded the state-of-the-art performance. A thorough ablation study that assesses the DSTA-network at the component level reveals how the network achieves such performance. Furthermore, this paper proposes a method to fuse the prediction scores from two complementary models and verifies its effectiveness in further boosting the performance of early accident anticipation. [ABSTRACT FROM AUTHOR]

Recent technological advancements in computer vision algorithms and data acquisition devices have greatly facilitated the research and applications of deep learning-based traffic object recognition from Light Detection and Ranging (LiDAR) data. The majority of existing methodologies applied deep learning (DL)-based techniques, especially Convolutional Neural Networks (CNNs), for vehicle detection and tracking on autonomous driving datasets. Nevertheless, fewer studies were focused on DL-based vehicle detection using roadside LiDAR data, partially due to the lack of publicly available roadside LiDAR datasets for network training and testing. This paper develops a novel framework based on CNNs and LiDAR data for automated vehicle detection. It leverages the domain knowledge of CNNs trained on large-scale autonomous driving datasets for vehicle detection from roadside LiDAR data. In the experimental study, roadside LiDAR data were collected at a road intersection in Reno, Nevada, U.S. Meanwhile, a CNN architecture was proposed to detect vehicles from LiDAR data through 3D bounding boxes. The proposed CNN was modified from the established PointPillars network by adding dense connections to the convolutional layers to achieve more comprehensive feature extraction. Three CNNs, including the proposed CNN, PointPillars, and YOLOv4, were trained and tested on PandaSet, a publicly available large-scale autonomous driving LiDAR dataset. Subsequently, the trained CNNs were reused for vehicle detection from the captured roadside LiDAR data. The experimental results demonstrated that the proposed CNN outperformed the others in the testing metrics. All three networks showed good performance on vehicle detection from the captured roadside LiDAR data. [ABSTRACT FROM AUTHOR]

Lane detection plays a vital part in autonomous driving. Conventional studies rely on less robust hand-craft features, while deep learning has improved the performance of lane detection to a great extent. Different from dominant methods based on semantic segmentation, this paper proposes an end-to-end framework named DevNet, which combines deviation awareness with semantic features based on point estimation. It consists of two modules to capture more representative features by integrating information of distance deviation and angle which helps to tackle diverse driving conditions in real environments, such as dim or shiny light conditions, crowdedness, and vanishing lanes. Experiments on public datasets indicate that the proposed method achieves favorable performance when compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

For safe and efficient navigation in driving environment, autonomous vehicles are supposed to predict future trajectories of surrounding vehicles dynamically and tackle the uncertainty of environment. Early works view uncertainty as distributions in different modals, and perform this task by enumerating a mass of trajectories, but can hardly cover all the possible modal distributions. Recently, generative model has risen to hold promise for modelling such distributions, something that we will build upon here. Target at predefined modals of vehicle agents, this paper proposes Spatial-Temporal Generative Model (STGM), a new learning algorithm that leverages a stochastic generative model to mitigate modal distributions problem. To be more specific, a Conditional Variational Auto-Encoder (CVAE) is leveraged to construct the framework of STGM, in which the input is augmented into modal distributions to gain modality-wise trajectories. Besides, for the stability of prediction, we also come up with a new modal-wise sampling trick as an alternative of traditional random sampling in CVAE. Considering that multiple layer perception (MLP) based CVAE is incapable of extracting spatial and temporal features effectively, we use two respective encoders before CVAE: Spatial Encoder and Temporal Encoder, and align them with a closed modal set. Additionally, inspired by the Total Probability Formula, we adopt a Modal Prediction Model to refine the confidences of modal-wise trajectories. After empirical evaluation on two public datasets, we find that STGM almost outperforms the baselines, such as Covernet, MTP, and so on. [ABSTRACT FROM AUTHOR]

Forecasting the trajectories of neighbor vehicles is a crucial step for decision making and motion planning of autonomous vehicles. This paper proposes a graph-based spatial-temporal convolutional network (GSTCN) to predict future trajectory distributions of all neighbor vehicles using past trajectories. This network tackles spatial interactions using a graph convolutional network (GCN), and captures temporal features with a convolutional neural network (CNN). The spatial-temporal features are encoded and decoded by a gated recurrent unit (GRU) network to generate future trajectory distributions. Besides, we propose a weighted adjacency matrix to describe the intensities of mutual influence between vehicles, and the ablation study demonstrates the effectiveness of our scheme. Our network is evaluated on two real-world freeway trajectory datasets: I-80 and US-101 in the Next Generation Simulation (NGSIM). Comparisons in three aspects, including prediction errors, model sizes, and inference speeds, show that our network can achieve state-of-the-art performance. [ABSTRACT FROM AUTHOR]

This paper proposes an algorithm to model and process streams of LiDAR data under an autonomous vehicle framework. LiDAR is assumed to be an exteroceptive sensor that allows the vehicle to have dynamic 3D scene perception of its surroundings. We employ an encoder-decoder architecture based on 3D-Convolutional layers called 3D Convolution Encoder-Decoder (3D-CED), together with a transfer learning strategy to extract a set of features from point clouds, which are relevant in the context of autonomous driving. The resulting features allow to make inferences of the future point cloud data and detect multiple abstraction level anomalies in controlled scenarios by utilizing a probabilistic switching dynamic model called High Dimensional Markov Jump Particle Filter (HD-MJPF). Moreover, a comparison is provided between piecewise linear, piecewise nonlinear, and nonlinear predictive models for anomaly detection at multiple abstraction levels. Our approach is evaluated with data collected from the LiDAR sensors of the autonomous vehicle while performing certain tasks in a controlled environment. [ABSTRACT FROM AUTHOR]

Accurate and fast 3D object detection from point clouds is a key task in autonomous driving. Existing one-stage 3D object detection methods can achieve real-time performance, however, they are dominated by anchor-based detectors which are inefficient and require additional post-processing. In this paper, we eliminate anchors and model an object as a single point—the center point of its bounding box. Based on the center point, we propose an anchor-free CenterNet3D network that performs 3D object detection without anchors. Our CenterNet3D uses keypoint estimation to find center points and directly regresses 3D bounding boxes. However, because inherent sparsity of point clouds, 3D object center points are likely to be in empty space which makes it difficult to estimate accurate boundaries. To solve this issue, we propose an extra corner attention module to enforce the CNN backbone to pay more attention to object boundaries. Besides, considering that one-stage detectors suffer from the discordance between the predicted bounding boxes and corresponding classification confidences, we develop an efficient keypoint-sensitive warping operation to align the confidences to the predicted bounding boxes. Our proposed CenterNet3D is non-maximum suppression free which makes it more efficient and simpler. We evaluate CenterNet3D on the widely used KITTI dataset and more challenging nuScenes dataset. Our method outperforms all state-of-the-art anchor-based one-stage methods and has comparable performance to two-stage methods as well. It has an inference speed of 20 FPS and achieves the best speed and accuracy trade-off. Our source code will be released at https://github.com/wangguojun2018/CenterNet3d. [ABSTRACT FROM AUTHOR]

For autonomous vehicles to operate persistently in a typical urban environment, it is essential to have high accuracy position information. This requires a mapping and localisation system that can adapt to changes over time. A localisation approach based on a single-survey map will not be suitable for long-term operation as it does not incorporate variations in the environment. In this paper, we present new algorithms to maintain a featured-based map. A map maintenance pipeline is proposed that can continuously update a map with the most relevant features taking advantage of the changes in the surroundings. Our pipeline detects and removes transient features based on their geometrical relationships with the vehicle’s pose. Newly identified features became part of a new feature map and are assessed by the pipeline as candidates for the localisation map. By purging out-of-date features and adding newly detected features, we continually update the prior map to more accurately represent the most recent environment. We have validated our approach using the USyd Campus Dataset, which includes more than 18 months of data. The results presented demonstrate that our maintenance pipeline produces a resilient map which can provide sustained localisation performance over time. [ABSTRACT FROM AUTHOR]

Modern vehicles are equipped with various driver-assistance systems, including automatic lane keeping, which prevents unintended lane departures. Traditional lane detection methods incorporate handcrafted or deep learning-based features followed by postprocessing techniques for lane extraction using frame-based RGB cameras. The utilization of frame-based RGB cameras for lane detection tasks is prone to illumination variations, sun glare, and motion blur, which limits the performance of lane detection methods. Incorporating an event camera for lane detection tasks in the perception stack of autonomous driving is one of the most promising solutions for mitigating challenges encountered by frame-based RGB cameras. The main contribution of this work is the design of the lane marking detection model, which employs the dynamic vision sensor. This paper explores the novel application of lane marking detection using an event camera by designing a convolutional encoder followed by the attention-guided decoder. The spatial resolution of the encoded features is retained by a dense atrous spatial pyramid pooling (ASPP) block. The additive attention mechanism in the decoder improves performance for high dimensional input encoded features that promote lane localization and relieve postprocessing computation. The efficacy of the proposed work is evaluated using the DVS dataset for lane extraction (DET). The experimental results show a significant improvement of 5.54% and 5.03% in $F1$ scores in multiclass and binary-class lane marking detection tasks. Additionally, the intersection over union ($IoU$) scores of the proposed method surpass those of the best-performing state-of-the-art method by 6.50% and 9.37% in multiclass and binary-class tasks, respectively. [ABSTRACT FROM AUTHOR]

Model-checking and verification using Kripke structures and computational tree logic* (CTL*) use abstractions from the model/process/application to create the state-transition graphs that verify the model behavior. This scheme of profiling the performance of a process imports that the depth of the process operation correlates with the level abstraction. However, because of state explosion problems, these abstractions tend to restrict the scope to create manageable execution states. Therefore, for context modeling, this procedure does not generate a fine-grained behavioral model as generated states limit the ability of the abstraction to capture the execution time interactions amongst the processes, the hardware, and the kernel. Hence, in this paper, we present an end-to-end framework that comprises auto-encoders and probabilistic models to understand the behavior of system processes and detect deviant behaviors. We test this framework with a publicly available dataset generated from an autonomous aerial vehicle (UAV) application and the results show that by creating a fine-grained model that exploits previously unharnessed properties of the system calls, we can create a dynamic anomaly detection framework that evolves as the threats change. [ABSTRACT FROM AUTHOR]

Object detection plays an important role in autonomous driving systems. LiDAR is widely used in autonomous driving vehicles and robots as a sensor for environmental perception. Recently, the development of computational power and deep learning technology makes it possible to classify and locate objects from LiDAR point cloud in a single end-to-end learnable network. However, objects are sparsely distributed in large point cloud field, and are always been partly scanned by LiDAR, which pose a challenge for accurate and rapid object positioning and classification from the raw point cloud. In this paper, we introduce a new single-shot refinement neural network for fast and accurate 3D object detection from the raw LiDAR point cloud. Firstly, we exploit self-attention mechanism in main object detection branch to enhance object feature representation. Secondly, we apply deformable convolution for learning adaptive receptive fields to fully capture the features of rotating and partially visible objects. Thirdly, an object refinement branch is introduced to produce a finer regression of objects upon the primary estimation from the main detection branch. All proposed modules have been proven to effectively improve the accuracy of object detection. Our method is evaluated on KITTI 3D detection benchmark and achieves state-of-the-art results while maintains real-time efficiency. Furthermore, real-time test in autonomous driving vehicle demonstrates that our method is robust to 16 channels LiDAR and can meet the demands of accuracy, efficiency, and visibility of object detection in various urban scenarios. [ABSTRACT FROM AUTHOR]

Current 3D target detection methods used in the field of autonomous driving generally have low real-time performance and insufficient target context feature to detect dynamic multi-target accurately. In order to solve these problems, a dynamic multi-target detection algorithm of voxel point cloud fusion based on PointRCNN is proposed, which adopts a two-stage detection structure. The first stage directly processes the point cloud to extract key point features and divides voxel space. A novel submanifold sparse convolution is used to extract voxel features. Then key point features and voxel features of the point cloud are merged to generate pre-selection boxes. In the second stage, reference points are set based on the voxel features. The features of key points around reference points are merged for the second time to achieve optimized detection boxes. Finally, for the problem of inconsistent confidence, a mandatory consistency loss function is proposed to improve the accuracy of the detection box. The proposed algorithm was compared with other algorithms in three different datasets, and further tested on a self-made dataset from an actual vehicle platform. Results showed that the proposed algorithm had higher accuracy, better robustness, stronger generalization ability for dynamic multi-target detection. [ABSTRACT FROM AUTHOR]

Object detection for autonomous driving in smart transportation systems requires comprehensive consideration of accuracy, speed and sensitivity for detecting multi-objects. The one-stage algorithm, Single Shot MultiBox Detector (SSD), can basically satisfy the above requirements. However, there are still rooms for improvement of the overall precision due to its dissatisfactory detection rate of small objects, which are abundant in autonomous driving scenarios. In order to solve the inherent defect problem of one-stage algorithm in processing the extreme foreground-background class imbalance encountered during training of dense detectors, we propose a Balanced Atrous Net (BANet) to significantly improve the performance of the SSD for autonomous driving. The BANet combines the atrous convolution and the feature fusion to improve the network structure of the SSD, thereby expanding receptive field and enriching semantic information in shallow layers. Meanwhile, a new loss that alleviates the class imbalance is designed to replace the standard cross entropy loss in the original algorithm. The experimental results show that the BANet outperforms the original SSD in mAP on KITTI with a relatively high speed retained. [ABSTRACT FROM AUTHOR]

For autonomous vehicles driving on roads, future trajectories of surrounding traffic agents (e.g., vehicles, bicycles, pedestrians) are essential information. The prediction of future trajectories is challenging as the motion of traffic agents is constantly affected by spatial-temporal interactions from agents and road infrastructure. To take those interactions into account, this study proposes a Graph Attention Transformer (Gatformer) in which a traffic scene is represented by a sparse graph. To maintain the spatial and temporal information of traffic agents in a traffic scene, Convolutional Neural Networks (CNNs) are utilized to extract spatial features and a position encoder is proposed to encode the spatial features and the corresponding temporal features. Based on the encoded features, a Graph Attention Network (GAT) block is employed to model the agent-agent and agent-infrastructure interactions with the help of attention mechanisms. Finally, a Transformer network is introduced to predict trajectories for multiple agents simultaneously. Experiments are conducted over the Lyft dataset and state-of-the-art methods are introduced for comparison. The results show that the proposed Gatformer could make more accurate predictions while requiring less inference time than its counterparts. [ABSTRACT FROM AUTHOR]

We present a novel compact deep multi-task learning model to handle various autonomous driving perception tasks in one forward pass. The model performs multiple views of semantic segmentation, depth estimation, light detection and ranging (LiDAR) segmentation, and bird’s eye view projection simultaneously without being supported by other models. We also provide an adaptive loss weighting algorithm to tackle the imbalanced learning issue that occurred due to plenty of given tasks. Through data pre-processing and intermediate sensor fusion techniques, the model can process and combine multiple input modalities retrieved from RGB cameras, dynamic vision sensors (DVS), and LiDAR placed at several positions on the ego vehicle. Therefore, a better understanding of a dynamically changing environment can be achieved. Based on the ablation study, the model variant trained with our proposed method achieves a better performance. Furthermore, a comparative study is also conducted to clarify its performance and effectiveness against the combination of some recent models. As a result, our model maintains better performance even with much fewer parameters. Hence, the model can inference faster with less GPU memory utilization. Moreover, the result tends to be consistent in 3 different CARLA simulation datasets and 1 real-world nuScenes-lidarseg dataset. To support future research, we share codes and other files publicly at https://github.com/oskarnatan/compact-perception. [ABSTRACT FROM AUTHOR]

We present a new framework for creating lane-level detailed HD-maps at scale for autonomous vehicles (AVs). In order to overcome scaling challenges of ground survey-based HD-map creation, we propose a number of innovations by leveraging three data sources: high-resolution aerial imagery, aggregated vehicle telemetry, and a navigation map. We first divide map creation problem into several categories based on lane configurations. The road category is predicted in a supervised setting using aerial imagery, which is pre-processed by using aggregated vehicle telemetry without supervision. We utilized the navigation map to guide the process along the road network and used aerial imagery and aggregated vehicle telemetry to extract lane level features at each step. We propose a multi-task convolutional neural network (CNN) to predict road polygons, road-way centerline, number of lanes, lane and road edges using both the aerial imagery and the corresponding aggregated vehicle telemetry. The predicted road features for each image are then stitched along a road segment to construct the road and lane edge polylines, which are then used to predict lane marking and road edge types in a sliding window fashion along the road segment. The extracted features are finally utilized to calculate higher level features for each point in the HD-map. Our experimental results show that the proposed framework works well, offering a flexible solution for creating HD-maps for AVs at scale. [ABSTRACT FROM AUTHOR]

With the development of deep learning, semantic segmentation has received considerable attention within the robotics community. For semantic segmentation to be applied to mobile robots or autonomous vehicles, real-time processing is essential. In this article, a new real-time semantic segmentation network, called the adjacent feature propagation network (AFPNet), is proposed to achieve high performance and fast inference. AFPNet executes in real time on a commercial embedded GPU. The network includes two new modules. The local memory module (LMM) is the first; it improves the upsampling accuracy by propagating the high-level features to the adjacent grids. The cascaded pyramid pooling module (CPPM) is the second; it reduces computational time by changing the structure of the pyramid pooling module. Using these two modules, the proposed AFPNet achieved 76.4% mean intersection-over-union on the Cityscapes test dataset, outperforming other real-time semantic segmentation networks. Furthermore, AFPNet was successfully deployed on an embedded board Jetson AGX Xavier and applied to the real-world navigation of a mobile robot, proving that AFPNet can be effectively used in a variety of real-time applications. [ABSTRACT FROM AUTHOR]

With the gradual popularization of self-driving, it is becoming increasingly important for vehicles to smartly make the right driving decisions and autonomously obey traffic rules by correctly recognizing traffic signs. However, for machine learning-based traffic sign recognition on the Internet of Vehicles (IoV), a large amount of traffic sign data from distributed vehicles is needed to be gathered in a centralized server for model training, which brings serious privacy leakage risk because of traffic sign data containing lots of location privacy information. To address this issue, we first exploit privacy-preserving federated learning to perform collaborative training for accurate recognition models without sharing raw traffic sign data. Nevertheless, due to the limited computing and energy resources of most devices, it is hard for vehicles to continuously undertake complex artificial intelligence tasks. Therefore, we introduce powerful Spike Neural Networks (SNNs) into traffic sign recognition for energy-efficient and fast model training, which is the next generation of neural networks and is practical and well-fitted to IoV scenarios. Furthermore, we design a novel encoding scheme for SNNs based on neuron receptive fields to extract information from the pixel and spatial dimensions of traffic signs to achieve high-accuracy training. Numerical results indicate that the proposed federated SNN outperforms traditional federated convolutional neural networks in terms of accuracy, noise immunity, and energy efficiency as well. [ABSTRACT FROM AUTHOR]

Point cloud completion aims to infer the complete point clouds from incomplete ones, which is used in remote sensing applications such as reconstructing and autonomous driving. However, most existing methods cannot recover accurate structure details of the object. In this article, we propose point shift network (PS-Net). Our main contributions lie in the following three-folds. First, we propose a multiresolution encoder, which extracts and fuses multiresolution point cloud features hierarchically, thus avoiding information loss caused by a single global feature. Second, we design a multiresolution point cloud generation structure, which can be combined with the multiresolution encoder to generate gradually dense point clouds, avoiding the problem of nonuniformly density of the single-layer decoder. Third, we design the shift network (SN), which is used to generate shift vectors to shift the coordinates of each point cloud, so as to further fine-tune the coordinate positions of point clouds, achieving more accurate prediction. We conduct comprehensive experiments on the ShapeNet, KITTI, ScanObjectNN, and ModelNet40 datasets, which demonstrate that the proposed PS-Net achieves better performance than the existing methods and verify the robustness of the proposed method. This article contributes a new method to point cloud completion, realizes fine point cloud shape completion, and brings new possibilities to the research of autonomous driving, registration, and reconstruction. [ABSTRACT FROM AUTHOR]

Object detection is becoming increasingly significant for autonomous-driving system. However, poor accuracy or low inference performance limits current object detectors in applying to autonomous driving. In this work, a fast and accurate object detector termed as SA-YOLOv3, is proposed by introducing dilated convolution and self-attention module (SAM) into the architecture of YOLOv3. Furthermore, loss function based on GIoU and focal loss is reconstructed to further optimize detection performance. With an input size of $512\times 512$ , our proposed SA-YOLOv3 improves YOLOv3 by 2.58 mAP and 2.63 mAP on KITTI and BDD100K benchmarks, with real-time inference (more than 40 FPS). When compared with other state-of-the-art detectors, it reports better trade-off in terms of detection accuracy and speed, indicating the suitability for autonomous-driving application. To our best knowledge, it is the first method that incorporates YOLOv3 with attention mechanism, and we expect this work would guide for autonomous-driving research in the future. [ABSTRACT FROM AUTHOR]

The strong demand of autonomous driving in the industry has led to vigorous interest in 3D object detection and resulted in many excellent 3D object detection algorithms. However, the vast majority of algorithms only model single-frame data, ignoring the temporal clue in video sequence. In this work, we propose a new transformer, called Temporal-Channel Transformer (TCTR), to model the temporal-channel domain and spatial-wise relationships for video object detecting from Lidar data. As the special design of this transformer, the information encoded in the encoder is different from that in the decoder. The encoder encodes temporal-channel information of multiple frames while the decoder decodes the spatial-wise information for the current frame in a voxel-wise manner. Specifically, the temporal-channel encoder of the transformer is designed to encode the information of different channels and frames by utilizing the correlation among features from different channels and frames. On the other hand, the spatial decoder of the transformer decodes the information for each location of the current frame. Before conducting the object detection with detection head, a gate mechanism is further deployed for re-calibrating the features of current frame, which filters out the object-irrelevant information by repetitively refining the representation of target frame along with the up-sampling process. Experimental results reveal that TCTR achieves the state-of-the-art performance in grid voxel-based 3D object detection on the nuScenes benchmark. [ABSTRACT FROM AUTHOR]

Localization in the complex urban environment is an open problem for current methods. The occlusion from dynamic objects, such as vehicles and pedestrians, degenerates the precision of the localization result. This article proposes a pole-like feature-based localization framework to solve this problem. Pole-like objects, such as posts of lamps or traffic sign and tree trunks, widely exist in the urban environment and are robust to occlusion, as they are usually higher than the objects on the road. First, this type of feature is extracted from the point cloud by a robust clustering algorithm. Then, the features from different frames of data are stitched to generate a feature map. For online localization, a Monte Carlo localization (MCL) framework is used to fuse the vehicle motion data and the map-matching result. An improved version of iterative closest point (ICP) that is specifically designed for the pole-like feature association is used for map matching based on the state of every particle. With the MCL scheme, localization is robust to the local minimum or robot kidnapping problem. Experimental results in the real urban environment demonstrate the precision and robustness of the proposed method, with mean absolute errors less than 0.20 m and 0.5°. The results also show that the proposed method outperforms some state-of-the-art localization methods in the complex urban environment. Note to Practitioners—There are some works using features from the 3-D point cloud, e.g., corners, planes, and reflectance, for robot localization. Instead of the abstract features, this article presents an object-feature-based localization scheme. We propose a novel pole-like object extraction algorithm based on the spatial distribution of the 3-D points. This algorithm can extract most types of pole-like objects in the urban environment. As these objects are highly distinct from other types of objects in their surroundings, localization is achieved by associating the pole-like features in the map and the features detected in real time through maximizing the likelihood. The whole system is verified with data collected in the real world, which indicates that its accuracy can fulfill the requirements of autonomous driving. The limitation of the proposed method is that it highly depends on one specific type of feature, which may not work well in the rural environment. In future research, we will address this problem by incorporating more types of semantic features for localization. [ABSTRACT FROM AUTHOR]

The controller area network (CAN) is the most widely used intra-vehicular communication network in the automotive industry. Because of its simplicity in design, it lacks most of the requirements needed for a security-proven communication protocol. However, a safe and secured environment is imperative for autonomous as well as connected vehicles. Therefore CAN security is considered one of the important topics in the automotive research community. In this article, we propose a four-stage intrusion detection system that uses the chi-squared method and can detect any kind of strong and weak cyber attacks in a CAN. This work is the first-ever graph-based defense system proposed for the CAN. Our experimental results show that we have a very low 5.26% misclassification for denial of service (DoS) attack, 10% misclassification for fuzzy attack, 4.76% misclassification for replay attack, and no misclassification for spoofing attack. In addition, the proposed methodology exhibits up to 13.73% better accuracy compared to existing ID sequence-based methods. [ABSTRACT FROM AUTHOR]

This article presents a semi-automated method to extract the lane features along the curved roads from mobile laser scanning (MLS) point clouds. The proposed method consists of four steps. After data pre-processing, a road edge detection algorithm is performed to distinguish road curbs and extract road surfaces. Then, textual and directional road markings such as arrows, symbols, and words, to inform drivers in necessary cases, are detected by intensity thresholding and conditional Euclidean clustering algorithms. Furthermore, lane markings are extracted by local intensity analysis and distance thresholding methods according to road design standards, because they are more regular along the road. Finally, centerline points on lanes are estimated based on the coordinates of extracted lane markings. Our method shows strong feasibility and robustness when creating high-definition (HD) maps from MLS data, by increasing the number of blocks in the curve and the distance threshold control in curved lane centerline extraction. Quantitative evaluations show that the average recall, precision, and F1-score obtained from four datasets for road marking extraction are 93.87%, 93.76%, and 93.73%, respectively. The generated lane centerlines are evaluated by overlaying them on manually labeled reference buffers from 4 cm resolution orthoimagery. The comparative study indicates that the proposed methods can achieve higher accuracy and robustness than most state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Understanding driver interaction behavioral semantics has potential benefits to autonomous car’s decision-making design. This article presents a framework of analyzing various encountering behaviors through decomposing driving encounter sequential data into small building blocks, called traffic primitives, using a Bayesian nonparametric learning (BNPL) approach. This framework offers a flexible way to gain semantic insights into complex driving encounters without any prerequisite knowledge of interaction behavior categories. Its effectiveness is then validated using 976 naturalistic driving encounters from which more than 4000 traffic primitives were learned with the BNPL approach. After that, a dynamic time warping method integrated with $k$ -means clustering is then developed to cluster all these extracted traffic primitives into groups. Experimental results identify 20 kinds of traffic primitives capable of representing the essential components of driving encounters in our database. Based on the results, we conclude that the proposed primitive-based analysis could prove useful for autonomous vehicle applications. [ABSTRACT FROM AUTHOR]

This article proposes the idea of fault classification-based control for the steady-state operation of unmanned surface vehicles (USVs). The idea of fault classification is achieved with the help of wavelet transforms and support vector machines, and the control is performed using a wavelet fuzzy controller. Initially, a brief idea of faults that affect the stable operation of USVs is identified. Furthermore, the surge and sway translational motion of USVs are realized with the help of a ball balancer setup. The fault data are measured in terms of plate angle, ball position, and motor operating voltage for developing the fault classifier. The proposed algorithm depicted improved classification accuracy when compared with conventional methods. To accommodate the operation of the system as per the operating state, a wavelet-based fuzzy controller is proposed. The proposed controller solves the problem of position tracking and balancing for ball and plate system with high precision, hence achieving the stable operation. [ABSTRACT FROM AUTHOR]

This article develops a deep learning object detector to provide accurate six-degree-of-freedom (DoF) information of the drogue relative to a monocular camera onboard a flying unmanned aerial vehicle. An object detector helps to provide the needed information for an autonomous vehicle to dock and refuel without the need for human intervention. This object detector can detect eight different beacons by training on 8746 images of a mock drogue. Once these beacons were detected, a nonlinear least squares algorithm that uses the collinearity equations as a system model takes the beacon's location on the captured image to provide an accurate six-DoF navigation solution. These navigation solutions from the object detector were evaluated on multiple metrics and then compared to navigation solutions provided by a VICON motion tracking system. Finally, Monte Carlo analysis, using the collinearity equations as a system model, evaluates an object detector's performance with various noise levels. [ABSTRACT FROM AUTHOR]

We present a new simulation framework for learning control policies for autonomous vehicles (AVs) based on real-world vehicle data and maps. The framework we propose consists of three major components: a) creating a detailed lane-level map (i.e., a high definition (HD) map) for a region of interest, b) generating an environment on the HD-map using the sensor outputs (e.g., GPS, radar) from vehicles driving in the same region, and c) learning a control policy based on the realistic environment constructed. We created the lane-level HD-maps using open street maps (OSM) and aerial imagery, from which we extracted the lane-level marking and edge features. The extracted image features are then utilized to calculate higher level attributes (e.g., curvature, heading, cross-sections etc.) for each point in the HD-map. The data acquired from vehicle sensors is combined with the constructed map to create a realistic environment. Based on the constructed environment, we learned a policy to control the vehicle laterally using a reinforcement learning algorithm and longitudinally using proportional-integral-derivative (PID) controller. Our experimental results show that the proposed framework works well, offering a flexible and scalable solution for learning control policies for AVs in realistic environments. [ABSTRACT FROM AUTHOR]

Semantic segmentation is a fundamental task in computer vision, and it has various applications in fields such as robotic sensing, video surveillance, and autonomous driving. A major research topic in urban road semantic segmentation is the proper integration and use of cross-modal information for fusion. Here, we attempt to leverage inherent multimodal information and acquire graded features to develop a novel multilabel-learning network for RGB-thermal urban scene semantic segmentation. Specifically, we propose a strategy for graded-feature extraction to split multilevel features into junior, intermediate, and senior levels. Then, we integrate RGB and thermal modalities with two distinct fusion modules, namely a shallow feature fusion module and deep feature fusion module for junior and senior features. Finally, we use multilabel supervision to optimize the network in terms of semantic, binary, and boundary characteristics. Experimental results confirm that the proposed architecture, the graded-feature multilabel-learning network, outperforms state-of-the-art methods for urban scene semantic segmentation, and it can be generalized to depth data. [ABSTRACT FROM AUTHOR]

Three-dimensional object detection plays an important role in autonomous driving because it provides the 3-D locations of objects for subsequent use in decision-making modules. A novel method is proposed using a monocular image and cascade geometric constraints to achieve robust 3-D vehicle detection. The framework is divided into two stages. In the first stage, the monocular image input is processed using a heatmap-based detection network with five branches to regress the orientation, dimension, center projection of the bottom face, viewpoint classification, and 2-D bounding box. In the second stage, the intersection-over-union threshold is increased to filter out imprecise 2-D bounding boxes. Thereafter, cascade geometric constraints are used to obtain the final 3-D box output, which improves the detection performance under truncation and occlusion conditions. The proposed method is tested on the KITTI-3-D benchmark and is shown to be effective and efficient. The proposed framework does not depend on external sources or subnetworks and can be trained in an end-to-end manner. [ABSTRACT FROM AUTHOR]