Search

With the rapid development of the autonomous driving industry, there is increasing research on related perception tasks. However, research on road surface traffic sign detection tasks is still limited. There are two main challenges to this task. First, when the target object's pixel ratio is small, the detection accuracy often decreases. Second, the existing publicly available road surface traffic sign datasets have limited image data. To address these issues, this paper proposes a new instance segmentation network, RTS R-CNN, for road surface traffic sign detection tasks based on Mask R-CNN. The network can accurately perceive road surface traffic signs and provide important information for the autonomous driving decision-making system. Specifically, CSPDarkNet53_ECA is proposed in the feature extraction stage to enhance the performance of deep convolutional networks by increasing inter-channel interactions. Second, to improve the network's detection accuracy for small target objects, GR-PAFPN is proposed in the feature fusion part, which uses a residual feature enhancement module (RFA) and atrous spatial pyramid pooling (ASPP) to optimize PAFPN and introduces a balanced feature pyramid module (BFP) to handle the imbalanced feature information at different resolutions. Finally, data augmentation is used to generate more data and prevent overfitting in specific scenarios. The proposed method has been tested on the open-source dataset Ceymo, achieving a Macro F1-score of 87.56%, which is 2.3% higher than the baseline method, while the inference speed reaches 23.5 FPS. [ABSTRACT FROM AUTHOR]

Autonomous vehicles have become a topic of interest in recent times due to the rapid advancement of automobile and computer vision technology. The ability of autonomous vehicles to drive safely and efficiently relies heavily on their ability to accurately recognize traffic signs. This makes traffic sign recognition a critical component of autonomous driving systems. To address this challenge, researchers have been exploring various approaches to traffic sign recognition, including machine learning and deep learning. Despite these efforts, the variability of traffic signs across different geographical regions, complex background scenes, and changes in illumination still poses significant challenges to the development of reliable traffic sign recognition systems. This paper provides a comprehensive overview of the latest advancements in the field of traffic sign recognition, covering various key areas, including preprocessing techniques, feature extraction methods, classification techniques, datasets, and performance evaluation. The paper also delves into the commonly used traffic sign recognition datasets and their associated challenges. Additionally, this paper sheds light on the limitations and future research prospects of traffic sign recognition. [ABSTRACT FROM AUTHOR]

LiDAR has become a vital sensor for autonomous driving scene understanding. To meet the accuracy and speed of LiDAR point clouds semantic segmentation, an efficient model ACPNet is proposed in this paper. In the feature extraction stage, the backbone is constructed with asymmetric convolutions, so the skeleton of the square convolution kernel is enhanced, which leads to greater robustness to target rotation. Moreover, a contextual feature enhancement module is designed to extract richer contextual features. During training, global scaling and global translation are performed to enrich the diversity of datasets. Compared with the baseline network PolarNet, the mIoU of ACPNet on the SemanticKITTI, SemanticPOSS and nuScenes datasets are improved by 5.1%, 1.6% and 2.9%, respectively. Meanwhile, the speed of ACPNet is 14 FPS, which basically meets the real-time requirements in autonomous driving scenarios. The experimental results show that ACPNet significantly improves the performance of LiDAR point cloud semantic segmentation. [ABSTRACT FROM AUTHOR]

Naturalistic driving data (NDD) are valuable for testing autonomous driving systems under various driving conditions. Automatically identifying scenes from high-dimensional and unlabeled NDD remains a challenging task. This paper presents a novel approach for automatically identifying test scenarios for autonomous driving through deep unsupervised learning. Firstly, US DAS2 NDD are leveraged, and the selection of data variables representing the vehicle state and surrounding environment is conducted to formulate the segmentation criterion. The isolation forest (IF) algorithm is then employed to segment the data, yielding two distinct types of datasets: typical scenarios and extreme scenarios. Secondly, a one-dimensional residual convolutional autoencoder (1D-RCAE) is developed to extract scenario features from the two datasets. Compared to four other autoencoders, the 1D-RCAE can effectively extract crucial information from high-dimensional data with optimal feature extraction capability. Next, considering the varying importance of different features, an information entropy (IE)-optimized K-means algorithm is employed to cluster the features extracted using 1D-RCAE. Finally, statistical analysis is performed on the parameters of each cluster of scenarios to explore their distribution characteristics within each class, and four typical scenarios are identified along with five extreme scenarios. The proposed unsupervised framework, combining IF, 1D-RCAE, and IE-improved K-means algorithms, can automatically identify typical and extreme scenarios from NDD. These identified scenarios can then be applied to test the performance of autonomous driving systems, enriching the library of automated driving test scenarios. [ABSTRACT FROM AUTHOR]

Pedestrian Detection is the most critical module of an Autonomous Driving system. Although a camera is commonly used for this purpose, its quality degrades severely in low-light night time driving scenarios. On the other hand, the quality of a thermal camera image remains unaffected in similar conditions. This paper proposes an end-to-end multimodal fusion model for pedestrian detection using RGB and thermal images. Its novel spatio-contextual deep network architecture is capable of exploiting the multimodal input efficiently. It consists of two distinct deformable ResNeXt-50 encoders for feature extraction from the two modalities. Fusion of these two encoded features takes place inside a multimodal feature embedding module (MuFEm) consisting of several groups of a pair of Graph Attention Network and a feature fusion unit. The output of the last feature fusion unit of MuFEm is subsequently passed to two CRFs for their spatial refinement. Further enhancement of the features is achieved by applying channel-wise attention and extraction of contextual information with the help of four RNNs traversing in four different directions. Finally, these feature maps are used by a single-stage decoder to generate the bounding box of each pedestrian and the score map. We have performed extensive experiments of the proposed framework on three publicly available multimodal pedestrian detection benchmark datasets, namely KAIST, CVC-14, and UTokyo. The results on each of them improved the respective state-of-the-art performance. A short video giving an overview of this work along with its qualitative results can be seen at https://youtu.be/FDJdSifuuCs. Our source code will be released upon publication of the paper. [ABSTRACT FROM AUTHOR]

In recent years, the development of smart transportation has accelerated research on semantic segmentation as it is one of the most important problems in this area. A large receptive field has always been the center of focus when designing convolutional neural networks for semantic segmentation. A majority of recent techniques have used maxpooling to increase the receptive field of a network at an expense of decreasing its spatial resolution. Although this idea has shown improved results in object detection applications, however, when it comes to semantic segmentation, a high spatial resolution also needs to be considered. To address this issue, a new deep learning model, the M-Net is proposed in this paper which satisfies both high spatial resolution and a large enough receptive field while keeping the size of the model to a minimum. The proposed network is based on an encoder-decoder architecture. The encoder uses atrous convolution to encode the features at full resolution, and instead of using heavy transposed convolution, the decoder consists of a multipath feature extraction module that can extract multiscale context information from the encoded features. The experimental results reported in the paper demonstrate the viability of the proposed scheme. [ABSTRACT FROM AUTHOR]

The recent boom of autonomous driving nowadays has made object detection in traffic scenes a hot topic of research. Designed to classify and locate instances in the image, this is a basic but challenging task in the computer vision field. With its powerful feature extraction abilities, which are vital for object detection, deep learning has expanded its application areas to this field during the past several years and thus achieved breakthroughs. However, even with such powerful approaches, traffic scenarios have their own specific challenges, such as real-time detection, changeable weather, and complex lighting conditions. This survey is dedicated to summarizing research and papers on applying deep learning to the transportation environment in recent years. More than 100 research papers are covered, and different aspects such as key generic object detection frameworks, categorized object detection applications in traffic scenario, evaluation metrics, and classified datasets are included. Some open research fields are also provided. We believe that it is the first survey focusing on deep learning-based object detection in traffic scenario. [ABSTRACT FROM AUTHOR]

In recent years, 3D object detection based on LiDAR point clouds is a key component of autonomous driving. In pursuit of enhancing the accuracy of 3D point cloud feature extraction and point cloud detection, this paper introduces a novel 3D object detection model, termed as Graph Self-Attention-RCNN (GA-RCNN). This model is designed to integrate voxel information and point location information, enhancing the quality of 3D object proposals while maintaining contextual accuracy. The first stage rectifies the previous approach that relied on local features for preselected boxes, overlooking crucial global contextual information. An improved method is suggested in this work, utilizing BEV to capture long-range dependencies via a cross-attention mechanism. The second stage addresses the overreliance on local neighborhood point feature extraction. The Graph Self-Attention Pooling method is proposed, characterized by its dynamic computation of contribution weights for inputs. This enhances the model's flexibility and generalization performance. Extensive evaluations on KITTI and Waymo datasets demonstrate GA-RCNN's superior accuracy compared to other methods, affirming its efficacy in 3D object detection. [ABSTRACT FROM AUTHOR]

The upsurge of autonomous vehicles in the automobile industry will lead to better driving experiences while also enabling the users to solve challenging navigation problems. Reaching such capabilities will require significant technological attention and the flawless execution of various complex tasks, one of which is ensuring robust localization and mapping. Recent surveys have not provided a meaningful and comprehensive description of the current approaches in this field. Accordingly, this review is intended to provide adequate coverage of the problems affecting autonomous vehicles in this area, by examining the most recent methods for mapping and localization as well as related feature extraction and data security problems. First, a discussion of the contemporary methods of extracting relevant features from equipped sensors and their categorization as semantic, non-semantic, and deep learning methods is presented. We conclude that representativeness, low cost, and accessibility are crucial constraints in the choice of the methods to be adopted for localization and mapping tasks. Second, the survey focuses on methods to build a vehicle's environment map, considering both the commercial and the academic solutions available. The analysis proposes a difference between two types of environment, known and unknown, and develops solutions in each case. Third, the survey explores different approaches to vehicle localization and also classifies them according to their mathematical characteristics and priorities. Each section concludes by presenting the related challenges and some future directions. The article also highlights the security problems likely to be encountered in self-driving vehicles, with an assessment of possible defense mechanisms that could prevent security attacks in vehicles. Finally, the article ends with a debate on the potential impacts of autonomous driving, spanning energy consumption and emission reduction, sound and light pollution, integration into smart cities, infrastructure optimization, and software refinement. This thorough investigation aims to foster a comprehensive understanding of the diverse implications of autonomous driving across various domains. [ABSTRACT FROM AUTHOR]

Recent technological advancements in computer vision algorithms and data acquisition devices have greatly facilitated the research and applications of deep learning-based traffic object recognition from Light Detection and Ranging (LiDAR) data. The majority of existing methodologies applied deep learning (DL)-based techniques, especially Convolutional Neural Networks (CNNs), for vehicle detection and tracking on autonomous driving datasets. Nevertheless, fewer studies were focused on DL-based vehicle detection using roadside LiDAR data, partially due to the lack of publicly available roadside LiDAR datasets for network training and testing. This paper develops a novel framework based on CNNs and LiDAR data for automated vehicle detection. It leverages the domain knowledge of CNNs trained on large-scale autonomous driving datasets for vehicle detection from roadside LiDAR data. In the experimental study, roadside LiDAR data were collected at a road intersection in Reno, Nevada, U.S. Meanwhile, a CNN architecture was proposed to detect vehicles from LiDAR data through 3D bounding boxes. The proposed CNN was modified from the established PointPillars network by adding dense connections to the convolutional layers to achieve more comprehensive feature extraction. Three CNNs, including the proposed CNN, PointPillars, and YOLOv4, were trained and tested on PandaSet, a publicly available large-scale autonomous driving LiDAR dataset. Subsequently, the trained CNNs were reused for vehicle detection from the captured roadside LiDAR data. The experimental results demonstrated that the proposed CNN outperformed the others in the testing metrics. All three networks showed good performance on vehicle detection from the captured roadside LiDAR data. [ABSTRACT FROM AUTHOR]

Accurate and fast 3D object detection from point clouds is a key task in autonomous driving. Existing one-stage 3D object detection methods can achieve real-time performance, however, they are dominated by anchor-based detectors which are inefficient and require additional post-processing. In this paper, we eliminate anchors and model an object as a single point—the center point of its bounding box. Based on the center point, we propose an anchor-free CenterNet3D network that performs 3D object detection without anchors. Our CenterNet3D uses keypoint estimation to find center points and directly regresses 3D bounding boxes. However, because inherent sparsity of point clouds, 3D object center points are likely to be in empty space which makes it difficult to estimate accurate boundaries. To solve this issue, we propose an extra corner attention module to enforce the CNN backbone to pay more attention to object boundaries. Besides, considering that one-stage detectors suffer from the discordance between the predicted bounding boxes and corresponding classification confidences, we develop an efficient keypoint-sensitive warping operation to align the confidences to the predicted bounding boxes. Our proposed CenterNet3D is non-maximum suppression free which makes it more efficient and simpler. We evaluate CenterNet3D on the widely used KITTI dataset and more challenging nuScenes dataset. Our method outperforms all state-of-the-art anchor-based one-stage methods and has comparable performance to two-stage methods as well. It has an inference speed of 20 FPS and achieves the best speed and accuracy trade-off. Our source code will be released at https://github.com/wangguojun2018/CenterNet3d. [ABSTRACT FROM AUTHOR]

Automated Vehicles (AVs) reform the automotive industry by enabling real-time and efficient data exchange between the vehicles. While connectivity and automation of the vehicles deliver a slew of benefits, they may also introduce new safety, security, and privacy risks. Further, AVs rely entirely on the sensor data and the data from other vehicles too. On the other hand, the sensor data is susceptible to anomalies caused by cyber-attacks, errors, and faults, resulting in accidents and fatalities. Hence, it is essential to create techniques for detecting anomalies and identifying their sources before the wide adoption of AVs. This paper proposes an anomaly detection model using a Modified-Convolutional Neural Network (M-CNN) with Safety Pilot Model Deployment (SPMD) dataset. The M-CNN model comprises specifically trained layers involving the ReLU activation function for feature extraction and detection of AV anomalies. Furthermore, the Adam is used as the optimization algorithm to train the model. The detection accuracy of the proposed model is compared with Isolation Forest (IF) and Support Vector Machine (SVM). The experimental result reveals that the proposed model outperforms the other models with an accuracy of 99.40% in AV anomaly detection. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Beijing University of Technology is the property of Journal of Beijing University of Technology, Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Detecting object locations and semantic classes in an image, such as traffic signs, traffic lights, and guide signs, is the crucial problem for autonomous driving, known as object detection. However, stable object detection in complex real-world environments, such as urban environments, is still challenging because of clutter, time of day, blur etc., even with modern deep convolutional neural networks (DCNNs). On the other hand, a high definition (HD) map is a pre-built information resource for autonomous driving tasks, especially for controls. Besides controls, HD map utilization for detection tasks has been gaining attention in recent years, enabling us to stabilize detection even in complex real-world environments. However, it is challenging to use object information from an HD map as detection directly because the self-localization error affects the transformed object locations on the image coordinate system from the HD map's coordinate system. This paper explores incorporating HD map information into deep feature maps of a DCNN-based model. Two proposed modules implicitly make the feature extraction efficient and stable by utilizing HD map information. As a result of the experiments, the proposed module improved a modern model for challenging images of the urban area Shinjuku by 37 % in mAP, even in self-localization errors. [ABSTRACT FROM AUTHOR]

Effective lane detection technology plays an important role in the current autonomous driving system. Although deep learning models, with their intricate network designs, have proven highly capable of detecting lanes, there persist key areas requiring attention. Firstly, the symmetry inherent in visuals captured by forward-facing automotive cameras is an underexploited resource. Secondly, the vast potential of position information remains untapped, which can undermine detection precision. In response to these challenges, we propose FF-HPINet, a novel approach for lane detection. We introduce the Flipped Feature Extraction module, which models pixel pairwise relationships between the flipped feature and the original feature. This module allows us to capture symmetrical features and obtain high-level semantic feature maps from different receptive fields. Additionally, we design the Hierarchical Position Information Extraction module to meticulously mine the position information of the lanes, vastly improving target identification accuracy. Furthermore, the Deformable Context Extraction module is proposed to distill vital foreground elements and contextual nuances from the surrounding environment, yielding focused and contextually apt feature representations. Our approach achieves excellent performance with the F1 score of 97.00% on the TuSimple dataset and 76.84% on the CULane dataset. [ABSTRACT FROM AUTHOR]

Limitations are identified in the expressive capabilities of the deep feature extraction network employed in deep reinforcement learning (DRL), particularly in complex scenarios. Additionally, learning performance is negatively impacted by compound errors, with consideration given to the potential critical role of sample quality in learning outcomes. Therefore, how to improve the collision avoidance decision-making adaptive and effectiveness for autonomous ships navigating in various scenarios is crucial. To address these issues and overcome data acquisition difficulties, a method called rule-guided vision supervised learning (RGVSL) is proposed in this paper. Through a static collision avoidance decision-making task, a comparison is drawn between a deep feature extraction network and the Nature CNN in DQN, revealing shortcomings in the feature extraction of DRL. Additionally, a collision avoidance decision-making environment with the capability of generalization for maritime encounters is proposed, and the advantages of the method in terms of adaptability and learning cost are validated. Finally, it is demonstrated that the RGVSL method is equivalent to a DQN with γ set to 0, indicating a significant performance improvement without compound errors. Achieving an adaptive decision accuracy of over 90% in various encounter scenarios without retraining, this research substantially reduces learning costs. It can provide innovative and practical solutions for the technological development in the field of autonomous ship collision avoidance decision-making. • Revolutionizes DRL data by using explicit rules for diverse scenarios, boosting model performance. • Exposes Nature CNN's DQN inadequacy, emphasizing the need for effective feature expression in DRL. • RGVSL achieves 90% accuracy in diverse maritime scenarios, cutting retraining costs, enhancing performance. [ABSTRACT FROM AUTHOR]

6DoF camera localization is an important component of autonomous driving and navigation. Deep learning has achieved impressive results in localization, but its robustness in dynamic environments has not been adequately addressed. In this paper, we propose a framework based on hybrid attention mechanism which can be generally applied to existing CNN-based pose regressors to improve their robustness in dynamic environments. Specifically, we propose a novel Transformer Bottleneck (TBo) block including convolution, channel attention, and a position-aware self-attention mechanism, which extracts more geometrically robust features by capturing the corresponding long-term dependencies between pixels. Furthermore, we introduce shuffle attention (SA) before the pose regressor, which integrates feature information in both spatial and channel dimensions, forcing the network to learn geometrically robust features, reducing the effects of dynamic objects and illumination conditions to improve camera localization accuracy. We evaluate our method on commonly benchmarked indoor and outdoor datasets and the experimental results show that our proposed method can significantly improve localization performance compared compare favorably to contemporary pose regressors schemes. In addition, extensive ablation evaluations are conducted to prove the effectiveness of our proposed hybrid attention bottleneck block for pose regression networks. • We propose a novel Transformer Bottleneck block with self-attention and channel attention to overcome the limitations of convolution. This coupling allows them to be optimized in a mutually reinforcing manner, significantly improving fine-grained feature extraction for accurate localization. • We propose a novel end-to-end hybrid attention network for single image localization, which improves the accuracy and robustness of camera localization, especially in dynamic scenes. • We conduct extensive experiments in both indoor and outdoor dataset, which shows that our model performs better than the existing competitive methods. [ABSTRACT FROM AUTHOR]

In the field of intelligent connected vehicles, the precise and real-time identification of speed bumps is critically important for the safety of autonomous driving. To address the issue that existing visual perception algorithms struggle to simultaneously maintain identification accuracy and real-time performance amidst image distortion and complex environmental conditions, this study proposes an enhanced lightweight neural network framework, YOLOv5-FPNet. This framework strengthens perception capabilities in two key phases: feature extraction and loss constraint. Firstly, FPNet, based on FasterNet and Dynamic Snake Convolution, is developed to adaptively extract structural features of distorted speed bumps with accuracy. Subsequently, the C3-SFC module is proposed to augment the adaptability of the neck and head components to distorted features. Furthermore, the SimAM attention mechanism is embedded within the backbone to enhance the ability of key feature extraction. Finally, an adaptive loss function, Inner–WiseIoU, based on a dynamic non-monotonic focusing mechanism, is designed to improve the generalization and fitting ability of bounding boxes. Experimental evaluations on a custom speed bumps dataset demonstrate the superior performance of FPNet, with significant improvements in key metrics such as the mAP, mAP50_95, and FPS by 38.76%, 143.15%, and 51.23%, respectively, compared to conventional lightweight neural networks. Ablation studies confirm the effectiveness of the proposed improvements. This research provides a fast and accurate speed bump detection solution for autonomous vehicles, offering theoretical insights for obstacle recognition in intelligent vehicle systems. [ABSTRACT FROM AUTHOR]

Most existing traffic sign detection models suffer from high computational complexity and superior performance but cannot be deployed on edge devices with limited computational capacity, which cannot meet the direct needs of autonomous vehicles for detection model performance and efficiency. To address the above concerns, this paper proposes an improved SEDG-Yolov5 traffic sign detection method based on knowledge distillation. Firstly, the Slicing Aided Hyper Inference method is used as a local offline data augmentation method for the model training. Secondly, to solve the problems of high-dimensional feature information loss and high model complexity, the inverted residual structure ESGBlock with a fused attention mechanism is proposed, and a lightweight feature extraction backbone network is constructed based on it, while we introduce the GSConv in the feature fusion layer to reduce the computational complexity of the model further. Eventually, an improved response-based objectness scaled knowledge distillation method is proposed to retrain the traffic sign detection model to compensate for the degradation of detection accuracy due to light-weighting. Extensive experiments on two challenging traffic sign datasets show that our proposed method has a good balance on detection precision and detection speed with 2.77M parametric quantities. Furthermore, the inference speed of our method achieves 370 FPS with TensorRT and 35.6 FPS with ONNX at FP16-precision, which satisfies the requirements for real-time sign detection and edge deployment. [ABSTRACT FROM AUTHOR]

In autonomous vehicles, the LiDAR and radar sensors are indispensable components for measuring distances to objects. While deep-learning-based algorithms for LiDAR sensors have been extensively proposed, the same cannot be said for radar sensors. LiDAR and radar share the commonality of measuring distances, but they are used in different environments. LiDAR tends to produce less noisy data and provides precise distance measurements, but it is highly affected by environmental factors like rain and fog. In contrast, radar is less impacted by environmental conditions but tends to generate noisier data. To reduce noise in radar data and enhance radar data augmentation, we propose a LiDAR-to-Radar translation method with a voxel feature extraction module, leveraging the fact that both sensors acquire data in a point-based manner. Because of the translation of high-quality LiDAR data into radar data, this becomes achievable. We demonstrate the superiority of our proposed method by acquiring and using data from both LiDAR and radar sensors in the same environment for validation. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) object detection is crucial for accurate recognition of autonomous driving roads, and the distribution of point clouds in 3D scenes becomes sparse with increasing distance, thus seriously affecting the sensor's perception precision. To address this problem, we propose a two-stage 3D object detection network based on point and voxel feature fusion. In the first stage, a spatial semantic feature fusion module is designed to effectively fuse low-level spatial features and high-level semantic features to generate high-quality proposals. Then, an attention mechanism-based residual module is constructed to expand the receptive field and adaptively aggregate the voxel features in the 3D scene. At the same time, the sampled key points and voxel features are fused to extract the key information in the 3D scene. In the second stage, the graph network pooling module is introduced to construct local graphs on 3D proposals using key point features as nodes to estimate the confidence and location of objects more accurately. Experimental results on the KITTI dataset show that the detection precision is improved significantly in easy, moderate, and hard tasks. [ABSTRACT FROM AUTHOR]

In foggy weather scenarios, the scattering and absorption of light by water droplets and particulate matter cause object features in images to become blurred or lost, presenting a significant challenge for target detection in autonomous driving vehicles. To address this issue, this study proposes a foggy weather detection method based on the YOLOv5s framework, named YOLOv5s-Fog. The model enhances the feature extraction and expression capabilities of YOLOv5s by introducing a novel target detection layer called SwinFocus. Additionally, the decoupled head is incorporated into the model, and the conventional non-maximum suppression method is replaced with Soft-NMS. The experimental results demonstrate that these improvements effectively enhance the detection performance for blurry objects and small targets in foggy weather conditions. Compared to the baseline model, YOLOv5s, YOLOv5s-Fog achieves a 5.4% increase in mAP on the RTTS dataset, reaching 73.4%. This method provides technical support for rapid and accurate target detection in adverse weather conditions, such as foggy weather, for autonomous driving vehicles. [ABSTRACT FROM AUTHOR]

Accurately detect vehicles or pedestrians from 3D point clouds (3D object detection) is a fast developing research topic in autonomous driving and other domains. The fundamental component for feature extraction in 3D object detection is Set Abstraction (SA), which can downsample points while aggregating points to extract features. However, the current SA ignores the geometric and semantic properties of point clouds and may miss to detect remote small objects. In this paper, FocusSA is proposed, which consists two modules for enhancing useful feature extraction in the SA layer to improve 3D object detection accuracy. At first, Focused FPS (FocFPS) is proposed to evaluate the foreground and boundary scores of the points and reweighs the Furthest Point Sampling (FPS) using the evaluated scores to retain more contextual points in downsampling. Then a Geometry-aware Feature Extraction (GeoFE) module is proposed to add geometric information to enrich the awareness of geometric structure in feature aggregation. To evaluate the performances of the proposed methods, we conduct extensive experiments on three difficulty levels of Car class in KITTI dataset. The experimental results show that on "moderate" instances, our results outperform the state-of-the-art method by 1.08%. Moreover, FocusSA is easy to be plugged in popular architectures. [ABSTRACT FROM AUTHOR]

Dense panoptic prediction is a key ingredient in many existing applications such as autonomous driving, automated warehouses, or remote sensing. Many of these applications require fast inference over large input resolutions on affordable or even embedded hardware. We proposed to achieve this goal by trading off backbone capacity for multi-scale feature extraction. In comparison with contemporaneous approaches to panoptic segmentation, the main novelties of our method are efficient scale-equivariant feature extraction, cross-scale upsampling through pyramidal fusion and boundary-aware learning of pixel-to-instance assignment. The proposed method is very well suited for remote sensing imagery due to the huge number of pixels in typical city-wide and region-wide datasets. We present panoptic experiments on Cityscapes, Vistas, COCO, and the BSB-Aerial dataset. Our models outperformed the state-of-the-art on the BSB-Aerial dataset while being able to process more than a hundred 1MPx images per second on an RTX3090 GPU with FP16 precision and TensorRT optimization. [ABSTRACT FROM AUTHOR]

General target detection with deep learning has made tremendous strides in the past few years. However, small target detection sometimes is associated with insufficient sample size and difficulty in extracting complete feature information. For safety during autonomous driving, remote signs and pedestrians need to be detected from driving scenes photographed by car cameras. In the early period of a medical lesion, because of the small area of the lesion, target detection is of great significance to detect masses and tumors for accurate diagnosis and treatment. To deal with these problems, we propose a novel deep learning model, named CenterNet for small targets (ST-CenterNet). First of all, due to the lack of visual information on small targets in the dataset, we extracted less discriminative features. To overcome this shortcoming, the proposed selective small target replication algorithm (SSTRA) was used to realize increasing numbers of small targets by selectively oversampling them. In addition, the difficulty of extracting shallow semantic information for small targets results in incomplete target feature information. Consequently, we developed a target adaptation feature extraction module (TAFEM), which was used to conduct bottom-up and top-down bidirectional feature extraction by combining ResNet with the adaptive feature pyramid network (AFPN). The improved new network model, AFPN, was added to solve the problem of the original feature extraction module, which can only extract the last layer of the feature information. The experimental results demonstrate that the proposed method can accurately detect the small-scale image of distributed targets and simultaneously, at the pixel level, classify whether a subject is wearing a safety helmet. Compared with the detection effect of the original algorithm on the safety helmet wearing dataset (SHWD), we achieved mean average precision (mAP) of 89.06% and frames per second (FPS) of 28.96, an improvement of 18.08% mAP over the previous method. [ABSTRACT FROM AUTHOR]

For autonomous driving without high‐definition maps, we present a model capable of generating multiple plausible paths from egocentric images for autonomous vehicles. Our generative model comprises two neural networks: feature extraction network (FEN) and path generation network (PGN). The FEN extracts meaningful features from an egocentric image, whereas the PGN generates multiple paths from the features, given a driving intention and speed. To ensure that the paths generated are plausible and consistent with the intention, we introduce an attentive discriminator and train it with the PGN under a generative adversarial network framework. Furthermore, we devise an interaction model between the positions in the paths and the intentions hidden in the positions and design a novel PGN architecture that reflects the interaction model for improving the accuracy and diversity of the generated paths. Finally, we introduce ETRIDriving, a dataset for autonomous driving, in which the recorded sensor data are labeled with discrete high‐level driving actions, and demonstrate the state‐of‐the‐art performance of the proposed model on ETRIDriving in terms of accuracy and diversity. [ABSTRACT FROM AUTHOR]

With the development of deep learning, semantic segmentation has received considerable attention within the robotics community. For semantic segmentation to be applied to mobile robots or autonomous vehicles, real-time processing is essential. In this article, a new real-time semantic segmentation network, called the adjacent feature propagation network (AFPNet), is proposed to achieve high performance and fast inference. AFPNet executes in real time on a commercial embedded GPU. The network includes two new modules. The local memory module (LMM) is the first; it improves the upsampling accuracy by propagating the high-level features to the adjacent grids. The cascaded pyramid pooling module (CPPM) is the second; it reduces computational time by changing the structure of the pyramid pooling module. Using these two modules, the proposed AFPNet achieved 76.4% mean intersection-over-union on the Cityscapes test dataset, outperforming other real-time semantic segmentation networks. Furthermore, AFPNet was successfully deployed on an embedded board Jetson AGX Xavier and applied to the real-world navigation of a mobile robot, proving that AFPNet can be effectively used in a variety of real-time applications. [ABSTRACT FROM AUTHOR]

Point cloud completion aims to infer the complete point clouds from incomplete ones, which is used in remote sensing applications such as reconstructing and autonomous driving. However, most existing methods cannot recover accurate structure details of the object. In this article, we propose point shift network (PS-Net). Our main contributions lie in the following three-folds. First, we propose a multiresolution encoder, which extracts and fuses multiresolution point cloud features hierarchically, thus avoiding information loss caused by a single global feature. Second, we design a multiresolution point cloud generation structure, which can be combined with the multiresolution encoder to generate gradually dense point clouds, avoiding the problem of nonuniformly density of the single-layer decoder. Third, we design the shift network (SN), which is used to generate shift vectors to shift the coordinates of each point cloud, so as to further fine-tune the coordinate positions of point clouds, achieving more accurate prediction. We conduct comprehensive experiments on the ShapeNet, KITTI, ScanObjectNN, and ModelNet40 datasets, which demonstrate that the proposed PS-Net achieves better performance than the existing methods and verify the robustness of the proposed method. This article contributes a new method to point cloud completion, realizes fine point cloud shape completion, and brings new possibilities to the research of autonomous driving, registration, and reconstruction. [ABSTRACT FROM AUTHOR]

Object detection is becoming increasingly significant for autonomous-driving system. However, poor accuracy or low inference performance limits current object detectors in applying to autonomous driving. In this work, a fast and accurate object detector termed as SA-YOLOv3, is proposed by introducing dilated convolution and self-attention module (SAM) into the architecture of YOLOv3. Furthermore, loss function based on GIoU and focal loss is reconstructed to further optimize detection performance. With an input size of $512\times 512$ , our proposed SA-YOLOv3 improves YOLOv3 by 2.58 mAP and 2.63 mAP on KITTI and BDD100K benchmarks, with real-time inference (more than 40 FPS). When compared with other state-of-the-art detectors, it reports better trade-off in terms of detection accuracy and speed, indicating the suitability for autonomous-driving application. To our best knowledge, it is the first method that incorporates YOLOv3 with attention mechanism, and we expect this work would guide for autonomous-driving research in the future. [ABSTRACT FROM AUTHOR]

Vision-based Automated Vehicle Recognition (VAVR) has attracted considerable attention recently. Particularly given the reliance on emerging deep learning methods, which have powerful feature extraction and pattern learning abilities, vehicle recognition has made significant progress. VAVR is an essential part of Intelligent Transportation Systems. The VAVR system can fast and accurately locate a target vehicle, which significantly helps improve regional security. A comprehensive VAVR system contains three components: Vehicle Detection (VD), Vehicle Make and Model Recognition (VMMR), and Vehicle Re-identification (VRe-ID). These components perform coarse-to-fine recognition tasks in three steps. In this article, we conduct a thorough review and comparison of the state-of-the-art deep learning-based models proposed for VAVR. We present a detailed introduction to different vehicle recognition datasets used for a comprehensive evaluation of the proposed models. We also critically discuss the major challenges and future research trends involved in each task. Finally, we summarize the characteristics of the methods for each task. Our comprehensive model analysis will help researchers that are interested in VD, VMMR, and VRe-ID and provide them with possible directions to solve current challenges and further improve the performance and robustness of models. [ABSTRACT FROM AUTHOR]

This article develops a deep learning object detector to provide accurate six-degree-of-freedom (DoF) information of the drogue relative to a monocular camera onboard a flying unmanned aerial vehicle. An object detector helps to provide the needed information for an autonomous vehicle to dock and refuel without the need for human intervention. This object detector can detect eight different beacons by training on 8746 images of a mock drogue. Once these beacons were detected, a nonlinear least squares algorithm that uses the collinearity equations as a system model takes the beacon's location on the captured image to provide an accurate six-DoF navigation solution. These navigation solutions from the object detector were evaluated on multiple metrics and then compared to navigation solutions provided by a VICON motion tracking system. Finally, Monte Carlo analysis, using the collinearity equations as a system model, evaluates an object detector's performance with various noise levels. [ABSTRACT FROM AUTHOR]