Search

There are two standard methods in instance segmentation: top-down and bottom-up. The top-down approach performs object detection to generate candidate proposals and then performs pixel-level segmentation for each proposal. It is accurate and flexible, capable of handling objects of different sizes and shapes. However, it is computationally complex and relies on object detection accuracy. The bottom-up approach first performs pixel-level clustering or segmentation and then combines candidate instances to obtain the final segmentation result. It can handle overlapping cases and has lower computational complexity, but it may need to localize accurately, and segment instances, and the segmentation granularity is coarser. In this paper, the Urban Street Scene Instance Segmentation (UISNet) algorithm is proposed. Firstly, the feature extraction network is the foundation of UISNet, which uses EfficientNet as the backbone network. Secondly, MPAFPN is the feature pyramid network part of UISNet, used for multi-scale feature fusion. By using EfficientNet and MPAFPN as the backbone network and bottleneck layers, the accuracy of UISNet is improved by 4% compared to ResNet and FPN. In the inference phase, this paper introduces an innovative dual-branch design that combines top-down and bottom-up strategies. One branch is the bounding box aggregation branch, which generates highdimensional information such as the shape and orientation of bounding boxes based on the FCOS Head. The other branch is the mask decoding branch, which creates mask prediction results. These two branches are fused using the Mask FCN Header to obtain the final instance segmentation result. With this dual-branch design, the model can effectively utilize the information from both top-down and bottom-up approaches, thereby improving the accuracy and robustness of instance segmentation. Through experimental comparisons, the proposed network model in this paper achieves the best performance in terms of accuracy compared to other instance segmentation networks, with an accuracy of 36.28%. Moreover, the proposed model performs better in urban street scenes, enhancing object detection and segmentation and offering more reliable and efficient solutions for applications such as autonomous driving and intelligent transportation. [ABSTRACT FROM AUTHOR]

Pedestrian behavior exhibits high levels of dynamism, and pedestrian trajectories are influenced not only by the pedestrians themselves, but also by interactions with surrounding objects. Efficiently understanding pedestrian motion behavior and modeling its interactions play a crucial role in fields like autonomous driving. Addressing issues related to dynamic feature extraction and interaction modeling in pedestrian trajectory prediction tasks, this paper introduces the conditional pedestrian trajectory prediction network (CSGAT-Net) based on semantic segmentation maps and spatiotemporal graph attention. CSGAT-Net models the physical environment and pedestrian behavior information in the scene as a semantic map, and it leverages graph attention networks to extract pedestrian interaction features. Finally, it predicts pedestrian future trajectories using a variational autoencoder. Comparative experiments conducted on publicly available datasets, ETH and UCY, show that our model exhibits favorable objective evaluation metrics and subjective prediction performance. Particularly, in terms of ADE and FDE metrics, CSGAT-Net outperforms current state-of-the-art methods, indicating that our model can reasonably and accurately predict pedestrian trajectories in different scenarios. [ABSTRACT FROM AUTHOR]

Localization and perception play an important role as the basis of autonomous Unmanned Aerial Vehicle (UAV) applications, providing the internal state of movements and the external understanding of environments. Simultaneous Localization And Mapping (SLAM), one of the critical techniques for localization and perception, is facing technical upgrading, due to the development of embedded hardware, multi-sensor technology, and artificial intelligence. This survey aims at the development of visual SLAM and the basis of UAV applications. The solutions to critical problems for visual SLAM are shown by reviewing state-of-the-art and newly presented algorithms, providing the research progression and direction in three essential aspects: real-time performance, texture-less environments, and dynamic environments. Visual–inertial fusion and learning-based enhancement are discussed for UAV localization and perception to illustrate their role in UAV applications. Subsequently, the trend of UAV localization and perception is shown. The algorithm components, camera configuration, and data processing methods are also introduced to give comprehensive preliminaries. In this paper, we provide coverage of visual SLAM and its related technologies over the past decade, with a specific focus on their applications in autonomous UAV applications. We summarize the current research, reveal potential problems, and outline future trends from academic and engineering perspectives. [ABSTRACT FROM AUTHOR]

Facing the significant challenge of 3D object detection in complex weather conditions and road environments, existing algorithms based on single-frame point cloud data struggle to achieve desirable results. These methods typically focus on spatial relationships within a single frame, overlooking the semantic correlations and spatiotemporal continuity between consecutive frames. This leads to discontinuities and abrupt changes in the detection outcomes. To address this issue, this paper proposes a multi-frame 3D object detection algorithm based on a deformable spatiotemporal Transformer. Specifically, a deformable cross-scale Transformer module is devised, incorporating a multi-scale offset mechanism that non-uniformly samples features at different scales, enhancing the spatial information aggregation capability of the output features. Simultaneously, to address the issue of feature misalignment during multi-frame feature fusion, a deformable cross-frame Transformer module is proposed. This module incorporates independently learnable offset parameters for different frame features, enabling the model to adaptively correlate dynamic features across multiple frames and improve the temporal information utilization of the model. A proposal-aware sampling algorithm is introduced to significantly increase the foreground point recall, further optimizing the efficiency of feature extraction. The obtained multi-scale and multi-frame voxel features are subjected to an adaptive fusion weight extraction module, referred to as the proposed mixed voxel set extraction module. This module allows the model to adaptively obtain mixed features containing both spatial and temporal information. The effectiveness of the proposed algorithm is validated on the KITTI, nuScenes, and self-collected urban datasets. The proposed algorithm achieves an average precision improvement of 2.1% over the latest multi-frame-based algorithms. [ABSTRACT FROM AUTHOR]

With the continuous advancement of autonomous driving and monitoring technologies, there is increasing attention on non-intrusive target monitoring and recognition. This paper proposes an ArcFace SE-attention model-agnostic meta-learning approach (AS-MAML) by integrating attention mechanisms into residual networks for pedestrian gait recognition using frequency-modulated continuous-wave (FMCW) millimeter-wave radar through meta-learning. We enhance the feature extraction capability of the base network using channel attention mechanisms and integrate the additive angular margin loss function (ArcFace loss) into the inner loop of MAML to constrain inner loop optimization and improve radar discrimination. Then, this network is used to classify small-sample micro-Doppler images obtained from millimeter-wave radar as the data source for pose recognition. Experimental tests were conducted on pose estimation and image classification tasks. The results demonstrate significant detection and recognition performance, with an accuracy of 94.5%, accompanied by a 95% confidence interval. Additionally, on the open-source dataset DIAT-μRadHAR, which is specially processed to increase classification difficulty, the network achieves a classification accuracy of 85.9%. [ABSTRACT FROM AUTHOR]

Point cloud registration is widely used in autonomous driving, SLAM, and 3D reconstruction, and it aims to align point clouds from different viewpoints or poses under the same coordinate system. However, point cloud registration is challenging in complex situations, such as a large initial pose difference, high noise, or incomplete overlap, which will cause point cloud registration failure or mismatching. To address the shortcomings of the existing registration algorithms, this paper designed a new coarse-to-fine registration two-stage point cloud registration network, CCRNet, which utilizes an end-to-end form to perform the registration task for point clouds. The multi-scale feature extraction module, coarse registration prediction module, and fine registration prediction module designed in this paper can robustly and accurately register two point clouds without iterations. CCRNet can link the feature information between two point clouds and solve the problems of high noise and incomplete overlap by using a soft correspondence matrix. In the standard dataset ModelNet40, in cases of large initial pose difference, high noise, and incomplete overlap, the accuracy of our method, compared with the second-best popular registration algorithm, was improved by 7.0%, 7.8%, and 22.7% on the MAE, respectively. Experiments showed that our CCRNet method has advantages in registration results in a variety of complex conditions. [ABSTRACT FROM AUTHOR]

Adverse weather conditions, such as snow-covered roads, represent a challenge for autonomous vehicle research. This is particularly challenging as it might cause misalignment between the longitudinal axis of the vehicle and the actual direction of travel. In this paper, we extend previous work in the field of autonomous vehicles on snow-covered roads and present a novel approach for side-slip angle estimation that combines perception with a hybrid artificial neural network pushing the prediction horizon beyond existing approaches. We exploited the feature extraction capabilities of convolutional neural networks and the dynamic time series relationship learning capabilities of gated recurrent units and combined them with a motion model to estimate the side-slip angle. Subsequently, we evaluated the model using the 3DCoAutoSim simulation platform, where we designed a suitable simulation environment with snowfall, friction, and car tracks in snow. The results revealed that our approach outperforms the baseline model for prediction horizons ⩾ 2 seconds. This extended prediction horizon has practical implications, by providing drivers and autonomous systems with more time to make informed decisions, thereby enhancing road safety. [ABSTRACT FROM AUTHOR]

Autonomous vehicle can calculate the movement track of surrounding vehicles according to the track prediction algorithm, and make response to reduce driving risk, while the traditional track prediction model will produce large errors in the case of long-term series prediction. To address this issue, this paper proposed a trajectory prediction model based on the Informer algorithm, and used the publicly available dataset NGSIM to conduct experimental analysison. Firstly, it filtered the original data by using symmetric exponential moving average method(sEMA), and added a joint normalization layer to the original Informer encoder to extract features from different vehicles, reducing the motion error between different vehicles, and improving the prediction accuracy by considering the speed information of the vehicle itself and the vehicle movement information of the surrounding environment. Finally, it got the vehicle trajectory distribution at the future time through the decoder. The results show that the trajectory prediction error of the model is less than 0.5 m. Through the analysis of MAE and MSE results of trajectory prediction, when the prediction time exceeds 0.3 s, the trajectory prediction effect of Informer model is obviously better than other algorithms, which verifies the effectiveness of the model and algorithm. [ABSTRACT FROM AUTHOR]

3D object detection, as an essential part of perception, plays a principal role in the autonomous driving system. The cost-competitive monocular 3D object detection has drawn increasing attention recently. However, it still suffers an inferior accuracy especially for occluded objects due to the limited camera view. Inspired by compositional models, in which an object is represented as a combination of multiple components, this paper proposes a new monocular 3D object detection method that decreases the impact of occlusion by utilizing an object's front and side cues. To do this, the features are extracted from a decoupled front and side representation and then fused by an attention-based module to obtain a more consistent feature distribution. An uncertainty-guided depth ensemble based on geometry is further applied to refine the depth prediction. Experiment results demonstrate that as compared to the conventional methods, the proposed method significantly improves the detection performance for occluded objects while still satisfying real-time efficiency, with the Average Precision on 40 recall positions (AP40), respectively, increasing by 10.23% for partly occluded objects and 12.22% for mostly occluded objects in the KITTI benchmark. The codes are released at https://github.com/kagurua/Front-Side-Det [ABSTRACT FROM AUTHOR]

In recent years, 3D object detection based on LiDAR point clouds is a key component of autonomous driving. In pursuit of enhancing the accuracy of 3D point cloud feature extraction and point cloud detection, this paper introduces a novel 3D object detection model, termed as Graph Self-Attention-RCNN (GA-RCNN). This model is designed to integrate voxel information and point location information, enhancing the quality of 3D object proposals while maintaining contextual accuracy. The first stage rectifies the previous approach that relied on local features for preselected boxes, overlooking crucial global contextual information. An improved method is suggested in this work, utilizing BEV to capture long-range dependencies via a cross-attention mechanism. The second stage addresses the overreliance on local neighborhood point feature extraction. The Graph Self-Attention Pooling method is proposed, characterized by its dynamic computation of contribution weights for inputs. This enhances the model's flexibility and generalization performance. Extensive evaluations on KITTI and Waymo datasets demonstrate GA-RCNN's superior accuracy compared to other methods, affirming its efficacy in 3D object detection. [ABSTRACT FROM AUTHOR]

Accurate and rapid response in complex driving scenarios is a challenging problem in autonomous driving. If a target is detected, the vehicle will not be able to react in time, resulting in fatal safety accidents. Therefore, the application of driver assistance systems requires a model that can accurately detect targets in complex scenes and respond quickly. In this paper, a lightweight feature extraction model, ShuffDet, is proposed to replace the CSPDark53 model used by YOLOX by improving the YOLOX algorithm. At the same time, an attention mechanism is introduced into the path aggregation feature pyramid network (PAFPN) to make the network focus more on important information in the network, thereby improving the accuracy of the model. This model, which combines two methods, is called ShuffYOLOX, and it can improve the accuracy of the model while keeping it lightweight. The performance of the ShuffYOLOX model on the KITTI dataset is tested in this paper, and the experimental results show that compared to the original network, the mean average precision (mAP) of the ShuffYOLOX model on the KITTI dataset reaches 92.20%. In addition, the number of parameters of the ShuffYOLOX model is reduced by 34.57%, the Gflops are reduced by 42.19%, and the FPS is increased by 65%. Therefore, the ShuffYOLOX model is very suitable for autonomous driving applications. [ABSTRACT FROM AUTHOR]

Autonomous vehicles have become a topic of interest in recent times due to the rapid advancement of automobile and computer vision technology. The ability of autonomous vehicles to drive safely and efficiently relies heavily on their ability to accurately recognize traffic signs. This makes traffic sign recognition a critical component of autonomous driving systems. To address this challenge, researchers have been exploring various approaches to traffic sign recognition, including machine learning and deep learning. Despite these efforts, the variability of traffic signs across different geographical regions, complex background scenes, and changes in illumination still poses significant challenges to the development of reliable traffic sign recognition systems. This paper provides a comprehensive overview of the latest advancements in the field of traffic sign recognition, covering various key areas, including preprocessing techniques, feature extraction methods, classification techniques, datasets, and performance evaluation. The paper also delves into the commonly used traffic sign recognition datasets and their associated challenges. Additionally, this paper sheds light on the limitations and future research prospects of traffic sign recognition. [ABSTRACT FROM AUTHOR]

In the research of robot systems, path planning and obstacle avoidance are important research directions, especially in unknown dynamic environments where flexibility and rapid decision makings are required. In this paper, a state attention network (SAN) was developed to extract features to represent the interaction between an intelligent robot and its obstacles. An auxiliary actor discriminator (AAD) was developed to calculate the probability of a collision. Goal-directed and gap-based navigation strategies were proposed to guide robotic exploration. The proposed policy was trained through simulated scenarios and updated by the Soft Actor-Critic (SAC) algorithm. The robot executed the action depending on the AAD output. Heuristic knowledge (HK) was developed to prevent blind exploration of the robot. Compared to other methods, adopting our approach in robot systems can help robots converge towards an optimal action strategy. Furthermore, it enables them to explore paths in unknown environments with fewer moving steps (showing a decrease of 33.9%) and achieve higher average rewards (showning an increase of 29.15%). [ABSTRACT FROM AUTHOR]

Object detection is a crucial component of the perception system in autonomous driving. However, the road scene presents a highly intricate environment where the visibility and characteristics of traffic targets are susceptible to attenuation and loss due to various complex road scenarios such as lighting conditions, weather conditions, time of day, background elements, and traffic density. Nevertheless, the current object detection network must exhibit more learning capabilities when detecting such targets. This also exacerbates the loss of features during the feature extraction and fusion process, significantly compromising the network's detection performance on traffic targets. This paper presents a novel methodology by which to overcome the concerns above, namely HRYNet. Firstly, a dual fusion gradual pyramid structure (DFGPN) is introduced, which employs a two-stage gradient fusion strategy to enhance the generation of more comprehensive multi-scale high-level semantic information, strengthen the interconnection between non-adjacent feature layers, and reduce the information gap that exists between them. HRYNet introduces an anti-interference feature extraction module, the residual multi-head self-attention mechanism (RMA). RMA enhances the target information by implementing a characteristic channel weighting policy, thereby reducing background interference and improving the attention capability of the network. Finally, the detection performance of HRYNet was evaluated by utilizing three datasets: the horizontally collected dataset BDD1000K, the UAV high-altitude dataset Visdrone, and a custom dataset. Experimental results demonstrate that HRYNet achieves a higher mAP_0.5 compared with YOLOv8s on the three datasets, with increases of 10.8%, 16.7%, and 5.5%, respectively. To optimize HRYNet for mobile devices, this study presents Lightweight HRYNet (LHRYNet), which effectively reduces the number of model parameters by 2 million. The results demonstrate that LHRYNet outperforms YOLOv8s in terms of mAP_0.5, with improvements of 6.7%, 10.9%, and 2.5% observed on the three datasets, respectively. [ABSTRACT FROM AUTHOR]

Train distance estimation in a turnout area is an important task for the autonomous driving of urban railway transit, since this function can assist trains in sensing the positions of other trains within the turnout area and prevent potential collision accidents. However, because of large incident angles on object surfaces and far distances, Lidar or stereo vision cannot provide satisfactory precision for such scenarios. In this paper, we propose a method for train distance estimation in a turnout area based on monocular vision: firstly, the side windows of trains in turnout areas are detected by instance segmentation based on YOLOv8; secondly, the vertical directions, the upper edges and lower edges of side windows of the train are extracted by feature extraction; finally, the distance to the target train is calculated with an appropriated pinhole camera model. The proposed method is validated by practical data captured from Hong Kong Metro Tsuen Wan Line. A dataset of 2477 images is built to train the instance segmentation neural network, and the network is able to attain an MIoU of 92.43% and a MPA of 97.47% for segmentation. The accuracy of train distance estimation is then evaluated in four typical turnout area scenarios with ground truth data from on-board Lidar. The experiment results indicate that the proposed method achieves a mean RMSE of 0.9523 m for train distance estimation in four typical turnout area scenarios, which is sufficient for determining the occupancy of crossover in turnout areas. [ABSTRACT FROM AUTHOR]

Pedestrian tracking is an important aspect of autonomous vehicles environment perception in a vehicle running environment. The performance of the existing pedestrian tracking algorithms is limited by the complex traffic environment, the changeable appearance characteristics of pedestrians and the frequent occlusion interaction, which leads to the insufficient accuracy and stability of tracking. Therefore, this paper proposes a detector–tracker integration framework for autonomous vehicle pedestrian tracking. Firstly, a pedestrian objects detector based on the improved YOLOv7 network was established. Space-to-Depth convolution layer was adopted to improve the backbone network of YOLOv7. Then, a novel appearance feature extraction network is proposed, which integrates the convolutional structural re-parameterization idea to construct a full-scale feature extraction block, which is the optimized DeepSORT tracker. Finally, experiments were carried out on MOT17 and MOT20 public datasets and driving video sequences, and the tracking performance of the proposed framework was evaluated by comparing it with the most advanced multi-object tracking algorithms. Quantitative analysis results show that the framework has high tracking accuracy. Compared with DeepSORT, MOTA improves by 2.3% in the MOT17 dataset and MOTA improves by 4.2% in the MOT20 dataset. Through qualitative evaluation on real driving video sequences, the framework proposed in this paper is robust in a variety of climate environments, and can be effectively applied to the pedestrian tracking of autonomous vehicles. [ABSTRACT FROM AUTHOR]

The precise localization of unmanned ground vehicles (UGVs) in industrial parks without prior GPS measurements presents a significant challenge. Simultaneous localization and mapping (SLAM) techniques can address this challenge by capturing environmental features, using sensors for real-time UGV localization. In order to increase the real-time localization accuracy and efficiency of UGVs, and to improve the robustness of UGVs' odometry within industrial parks—thereby addressing issues related to UGVs' motion control discontinuity and odometry drift—this paper proposes a tightly coupled LiDAR-IMU odometry method based on FAST-LIO2, integrating ground constraints and a novel feature extraction method. Additionally, a novel maintenance method of prior maps is proposed. The front-end module acquires the prior pose of the UGV by combining the detection and correction of relocation with point cloud registration. Then, the proposed maintenance method of prior maps is used to hierarchically and partitionally segregate and perform the real-time maintenance of the prior maps. At the back-end, real-time localization is achieved by the proposed tightly coupled LiDAR-IMU odometry that incorporates ground constraints. Furthermore, a feature extraction method based on the bidirectional-projection plane slope difference filter is proposed, enabling efficient and accurate point cloud feature extraction for edge, planar and ground points. Finally, the proposed method is evaluated, using self-collected datasets from industrial parks and the KITTI dataset. Our experimental results demonstrate that, compared to FAST-LIO2 and FAST-LIO2 with the curvature feature extraction method, the proposed method improved the odometry accuracy by 30.19% and 48.24% on the KITTI dataset. The efficiency of odometry was improved by 56.72% and 40.06%. When leveraging prior maps, the UGV achieved centimeter-level localization accuracy. The localization accuracy of the proposed method was improved by 46.367% compared to FAST-LIO2 on self-collected datasets, and the located efficiency was improved by 32.33%. The z-axis-located accuracy of the proposed method reached millimeter-level accuracy. The proposed prior map maintenance method reduced RAM usage by 64% compared to traditional methods. [ABSTRACT FROM AUTHOR]

With the rapid development of the autonomous driving industry, there is increasing research on related perception tasks. However, research on road surface traffic sign detection tasks is still limited. There are two main challenges to this task. First, when the target object's pixel ratio is small, the detection accuracy often decreases. Second, the existing publicly available road surface traffic sign datasets have limited image data. To address these issues, this paper proposes a new instance segmentation network, RTS R-CNN, for road surface traffic sign detection tasks based on Mask R-CNN. The network can accurately perceive road surface traffic signs and provide important information for the autonomous driving decision-making system. Specifically, CSPDarkNet53_ECA is proposed in the feature extraction stage to enhance the performance of deep convolutional networks by increasing inter-channel interactions. Second, to improve the network's detection accuracy for small target objects, GR-PAFPN is proposed in the feature fusion part, which uses a residual feature enhancement module (RFA) and atrous spatial pyramid pooling (ASPP) to optimize PAFPN and introduces a balanced feature pyramid module (BFP) to handle the imbalanced feature information at different resolutions. Finally, data augmentation is used to generate more data and prevent overfitting in specific scenarios. The proposed method has been tested on the open-source dataset Ceymo, achieving a Macro F1-score of 87.56%, which is 2.3% higher than the baseline method, while the inference speed reaches 23.5 FPS. [ABSTRACT FROM AUTHOR]

This paper proposes the design of a 360° map establishment and real-time simultaneous localization and mapping (SLAM) algorithm based on equirectangular projection. All equirectangular projection images with an aspect ratio of 2:1 are supported for input image types of the proposed system, allowing an unlimited number and arrangement of cameras. Firstly, the proposed system uses dual back-to-back fisheye cameras to capture 360° images, followed by the adoption of the perspective transformation with any yaw degree given to shrink the feature extraction area in order to reduce the computational time, as well as retain the 360° field of view. Secondly, the oriented fast and rotated brief (ORB) feature points extracted from perspective images with a GPU acceleration are used for tracking, mapping, and camera pose estimation in the system. The 360° binary map supports the functions of saving, loading, and online updating to enhance the flexibility, convenience, and stability of the 360° system. The proposed system is also implemented on an nVidia Jetson TX2 embedded platform with 1% accumulated RMS error of 250 m. The average performance of the proposed system achieves 20 frames per second (FPS) in the case with a single-fisheye camera of resolution 1024 × 768, and the system performs panoramic stitching and blending under 1416 × 708 resolution from a dual-fisheye camera at the same time. [ABSTRACT FROM AUTHOR]

LiDAR point cloud semantic segmentation algorithm is crucial to the environmental understanding of unmanned vehicles. At this stage, in autonomous vehicles, effectively integrating the complementary information of LiDAR and camera has become the focus of research. In this work, a network framework (called PI-Seg) for LiDAR point clouds semantic segmentation by fusing appearance features of RGB images is proposed. In this paper, the perspective projection module is introduced to align and synchronize point clouds with images to reduce appearance information loss. And an efficient and concise dual-flow feature extraction network is designed, a fusion module based on a continuous convolution structure is used for feature fusion, which effectively reduces the amount of parameters and runtime performance, and more suitable for autonomous driving scenarios. Finally, the fused features are added to the LiDAR point cloud features as the final output features, and the point cloud category label prediction is realized through the MLP network. The experimental results demonstrate that PI-Seg has a 5.3% higher mIoU score than SalsaNext, which is also a projection-based method, and still has a 1.4% performance improvement compared with the latest Cylinder3D algorithm, and in quantitative analyses the mAP value also has the best performance, showing that PI-Seg is better than other existing methods. [ABSTRACT FROM AUTHOR]

Real-time calculation of high-precision forward-backward pyramid LK optical flow on embedded computing platform has become the decisive factor of whether the algorithm can be used in realistic scenarios such as autonomous driving. This paper presented a feature extraction method based on mesh division and a new forward- backward constraint method. To address the load unbalance in the optical flow calculation process, this paper proposed a dynamic window-based pyramid model. It also reduced the calculation bit width to further improve the overall performance. Using the 720P videos in realistic scenarios, this method yields significant performance gained over the GPU optimized version of OpenCV by 4.1 times on the embedded GPU platform TX2 and achieved 30 fps performance. It helped the SLAM system increase the frame rate to 28 fps in real car scene . This method improves the accuracy of the pose and point cloud in realistic scenarios, meeting the real-time application requirements of vehicle scenarios. [ABSTRACT FROM AUTHOR]

High-accurate and real-time localization is the fundamental and challenging task for autonomous driving in a dynamic traffic environment. This paper presents a coordinated positioning strategy that is composed of semantic information and probabilistic data association, which improves the accuracy of SLAM in dynamic traffic settings. First, the improved semantic segmentation network, building on Fast-SCNN, uses the Res2net module instead of the Bottleneck in the global feature extraction to further explore the multi-scale granular features. It achieves the balance between segmentation accuracy and inference speed, leading to consistent performance gains on the coordinated localization task of this paper. Second, a novel scene descriptor combining geometric, semantic, and distributional information is proposed. These descriptors are made up of significant features and their surroundings, which may be unique to a traffic scene, and are used to improve data association quality. Finally, a probabilistic data association is created to find the best estimate using a maximum measurement expectation model. This approach assigns semantic labels to landmarks observed in the environment and is used to correct false negatives in data association. We have evaluated our system with ORB-SLAM2 and DynaSLAM, the most advanced algorithms, to demonstrate its advantages. On the KITTI dataset, the results reveal that our approach outperforms other methods in dynamic traffic situations, especially in highly dynamic scenes, with sub-meter average accuracy. [ABSTRACT FROM AUTHOR]

The recent boom of autonomous driving nowadays has made object detection in traffic scenes a hot topic of research. Designed to classify and locate instances in the image, this is a basic but challenging task in the computer vision field. With its powerful feature extraction abilities, which are vital for object detection, deep learning has expanded its application areas to this field during the past several years and thus achieved breakthroughs. However, even with such powerful approaches, traffic scenarios have their own specific challenges, such as real-time detection, changeable weather, and complex lighting conditions. This survey is dedicated to summarizing research and papers on applying deep learning to the transportation environment in recent years. More than 100 research papers are covered, and different aspects such as key generic object detection frameworks, categorized object detection applications in traffic scenario, evaluation metrics, and classified datasets are included. Some open research fields are also provided. We believe that it is the first survey focusing on deep learning-based object detection in traffic scenario. [ABSTRACT FROM AUTHOR]

Vision-based target detection and segmentation has been an important research content for environment perception in autonomous driving, but the mainstream target detection and segmentation algorithms have the problems of low detection accuracy and poor mask segmentation quality for multi-target detection and segmentation in complex traffic scenes. To address this problem, this paper improved the Mask R-CNN by replacing the backbone network ResNet with the ResNeXt network with group convolution to further improve the feature extraction capability of the model. Furthermore, a bottom-up path enhancement strategy was added to the Feature Pyramid Network (FPN) to achieve feature fusion, while an efficient channel attention module (ECA) was added to the backbone feature extraction network to optimize the high-level low resolution semantic information graph. Finally, the bounding box regression loss function smooth L1 loss was replaced by CIoU loss to speed up the model convergence and minimize the error. The experimental results showed that the improved Mask R-CNN algorithm achieved 62.62% mAP for target detection and 57.58% mAP for segmentation accuracy on the publicly available CityScapes autonomous driving dataset, which were 4.73% and 3.96%% better than the original Mask R-CNN algorithm, respectively. The migration experiments showed that it has good detection and segmentation effects in each traffic scenario of the publicly available BDD autonomous driving dataset. [ABSTRACT FROM AUTHOR]

Detecting object locations and semantic classes in an image, such as traffic signs, traffic lights, and guide signs, is the crucial problem for autonomous driving, known as object detection. However, stable object detection in complex real-world environments, such as urban environments, is still challenging because of clutter, time of day, blur etc., even with modern deep convolutional neural networks (DCNNs). On the other hand, a high definition (HD) map is a pre-built information resource for autonomous driving tasks, especially for controls. Besides controls, HD map utilization for detection tasks has been gaining attention in recent years, enabling us to stabilize detection even in complex real-world environments. However, it is challenging to use object information from an HD map as detection directly because the self-localization error affects the transformed object locations on the image coordinate system from the HD map's coordinate system. This paper explores incorporating HD map information into deep feature maps of a DCNN-based model. Two proposed modules implicitly make the feature extraction efficient and stable by utilizing HD map information. As a result of the experiments, the proposed module improved a modern model for challenging images of the urban area Shinjuku by 37 % in mAP, even in self-localization errors. [ABSTRACT FROM AUTHOR]

Advanced object detection methods always face high algorithmic complexity or low accuracy when used in pedestrian target detection for the autonomous driving system. This paper proposes a lightweight pedestrian detection approach called the YOLOv5s- G 2 network to address these issues. We apply Ghost and GhostC3 modules in the YOLOv5s- G 2 network to minimize computational cost during feature extraction while keeping the network's capability of extracting features intact. The YOLOv5s- G 2 network improves feature extraction accuracy by incorporating the Global Attention Mechanism (GAM) module. This application can extract relevant information for pedestrian target identification tasks and suppress irrelevant information, improving the unidentified problem of occluded and small targets by replacing the GIoU loss function used in the bounding box regression with the α -CIoU loss function. The YOLOv5s- G 2 network is evaluated on the WiderPerson dataset to ensure its efficacy. Our proposed YOLOv5s- G 2 network offers a 1.0% increase in detection accuracy and a 13.2% decrease in Floating Point Operations (FLOPs) compared to the existing YOLOv5s network. As a result, the YOLOv5s- G 2 network is preferable for pedestrian identification as it is both more lightweight and more accurate. [ABSTRACT FROM AUTHOR]

Infrared target detection has strong anti‐interference ability, long working distance and can work day and night. So it is widely used in military security and transportation fields, and infrared road object detection is critical in traffic checkpoints and autonomous driving. However, the target scale in infrared images changes greatly, small targets are difficult to detect, the poor image quality and low signal‐to‐noise ratio are still huge challenges in infrared target detection. This paper proposes an improved infrared target detection model I‐CenterNet based on the anchor‐free model CenterNet. The EfficientNetV2 with the channel attention mechanism is used instead of the traditional structure as the backbone network to enhance feature extraction. In order to reduce the noise of the input infrared image, Dilated‐Residual U‐net (DRUNet) is used. Meanwhile, feature pyramid and Sub‐Pixel are combined for multi‐scale feature fusion. Data enhancement is implemented to improve model performance. The experimental results show that the average detection accuracy of this model on the Flir infrared data set is 87.9%, and the average detection speed reaches 14.2 frames/s. [ABSTRACT FROM AUTHOR]

Simultaneous localization and mapping (SLAM) algorithm is a prerequisite for unmanned ground vehicle (UGV) localization, path planning, and navigation, which includes two essential components: frontend odometry and backend optimization. Frontend odometry tends to amplify the cumulative error continuously, leading to ghosting and drifting on the mapping results. However, loop closure detection (LCD) can be used to address this technical issue by significantly eliminating the cumulative error. The existing LCD methods decide whether a loop exists by constructing local or global descriptors and calculating the similarity between descriptors, which attaches great importance to the design of discriminative descriptors and effective similarity measurement mechanisms. In this paper, we first propose novel multi-channel descriptors (CMCD) to alleviate the lack of point cloud single information in the discriminative power of scene description. The distance, height, and intensity information of the point cloud is encoded into three independent channels of the shadow-casting region (bin) and then compressed it into a two-dimensional global descriptor. Next, an ORB-based dynamic threshold feature extraction algorithm (DTORB) is designed using objective 2D descriptors to describe the distributions of global and local point clouds. Then, a DTORB-based similarity measurement method is designed using the rotation-invariance and visualization characteristic of descriptor features to overcome the subjective tendency of the constant threshold ORB algorithm in descriptor feature extraction. Finally, verification is performed over KITTI odometry sequences and the campus datasets of Jilin University collected by us. The experimental results demonstrate the superior performance of our method to the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Object detection is a critical technology of environmental perception for autonomous driving vehicle. The Convolutional Neural Network has gradually become a powerful tool in the field of vehicle detection because of its powerful ability of feature extraction. In aiming to reach the balance between speed and accuracy of detection in complex traffic scenarios, this paper proposes an improved lightweight and high-performance vehicle–pedestrian detection algorithm based on the YOLOv4. Firstly, the backbone network CSPDarknet53 is replaced by MobileNetv2 to reduce the number of parameters and raise the capability of feature extraction. Secondly, the method of multi-scale feature fusion is used to realize the information interaction among different feature layers. Finally, a coordinate attention mechanism is added to focus on the region of interest in the image by way of weight adjustment. The experimental results show that this improved model has a great performance in vehicle–pedestrian detection in traffic scenarios. Experimental results on PASCAL VOC datasets show that the improved model's mAP is 85.79% and speed is 35FPS, which has an increase of 4.31% and 16.7% compared to YOLOv4. Furthermore, the improved YOLOv4 model maintains a great balance between detection accuracy and speed on different datasets, indicating that it can be applied to vehicle–pedestrian detection in traffic scenarios. [ABSTRACT FROM AUTHOR]

In this paper, we propose a 3D object detection method called MMAF-Net that is based on the multi-view and multi-stage adaptive fusion of RGB images and LiDAR point cloud data. This is an end-to-end architecture, which combines the characteristics of RGB images, the front view of point clouds based on reflection intensity, and the bird's eye view of point clouds. It also adopts a multi-stage fusion approach of "data-level fusion + feature-level fusion" to fully exploit the strength of multimodal information. Our proposed method addresses key challenges found in current 3D object detection methods for autonomous driving, including insufficient feature extraction from multimodal data, rudimentary fusion techniques, and sensitivity to distance and occlusion. To ensure the comprehensive integration of multimodal information, we present a series of targeted fusion methods. Firstly, we propose a novel input form that encodes dense point cloud reflectivity information into the image to enhance its representational power. Secondly, we design the Region Attention Adaptive Fusion module utilizing an attention mechanism to guide the network in adaptively adjusting the importance of different features. Finally, we extend the 2D DIOU (Distance Intersection over Union) loss function to 3D and develop a joint regression loss based on 3D_DIOU and SmoothL1 to optimize the similarity between detected and ground truth boxes. The experimental results on the KITTI dataset demonstrate that MMAF-Net effectively addresses the challenges posed by highly obscured or crowded scenes while maintaining real-time performance and improving the detection accuracy of smaller and more difficult objects that are occluded at far distances. • A novel object detection method is proposed integrating RGB images and LiDAR data. • This study tests the method on the KITTI dataset. • This method tackles highly obscured or crowded scenes for autonomous driving. • This method improves the detection accuracy of smaller and occluded targets far away. [ABSTRACT FROM AUTHOR]

Vehicle-trajectory prediction is essential for intelligent traffic systems (ITS), as it can help autonomous vehicles to plan a safe and efficient path. However, it is still a challenging task because existing studies have mainly focused on the spatial interactions of adjacent vehicles regardless of the temporal dependencies. In this paper, we propose a spatial-temporal attentive LSTM encoder–decoder model (STAM-LSTM) to predict vehicle trajectories. Specifically, the spatial attention mechanism is used to capture the spatial relationships among neighboring vehicles and then obtain the global spatial feature. Meanwhile, the temporal attention mechanism is designed to distinguish the effects of different historical time steps on future trajectory prediction. In addition, the motion feature of vehicles is extracted to reveal the influence of dynamic information on vehicle-trajectory prediction, and is combined with the local and global spatial features to represent the integrated features of the target vehicle at each historical moment. The experiments were conducted on public highway trajectory datasets—US-101 and I-80 in NGSIM—and the results demonstrate that our model achieves state-of-the-art prediction performance. [ABSTRACT FROM AUTHOR]

To solve the long-tail problem and improve the testing efficiency for autonomous navigation systems of unmanned surface vehicles (USVs), a visual image-based navigation scene complexity perception method is proposed. In this paper, we intend to accurately construct a mathematical model between navigation scene complexity and visual features from the analysis and processing of image textures. First, the typical complex elements are summarized, and the navigation scenes are divided into four levels according to whether they contain these typical elements. Second, the textural features are extracted using the gray level cogeneration matrix (GLCM) and Tamura coarseness, which are applied to construct the feature vectors of the navigation scenes. Furthermore, a novel paired bare bone particle swarm clustering (PBBPSC) method is proposed to classify the levels of complexity, and the exact value of the navigation scene complexity is calculated using the clustering result and an interval mapping method. By comparing different methods on the classical and self-collected datasets, the experimental results show that our proposed complexity perception method can not only better describe the level of complexity of navigation scenes but also obtain more accurate complexity values. [ABSTRACT FROM AUTHOR]

To solve the long-tail problem and improve the testing efficiency for autonomous navigation systems of unmanned surface vehicles (USVs), a visual image-based navigation scene complexity perception method is proposed. In this paper, we intend to accurately construct a mathematical model between navigation scene complexity and visual features from the analysis and processing of image textures. First, the typical complex elements are summarized, and the navigation scenes are divided into four levels according to whether they contain these typical elements. Second, the textural features are extracted using the gray level cogeneration matrix (GLCM) and Tamura coarseness, which are applied to construct the feature vectors of the navigation scenes. Furthermore, a novel paired bare bone particle swarm clustering (PBBPSC) method is proposed to classify the levels of complexity, and the exact value of the navigation scene complexity is calculated using the clustering result and an interval mapping method. By comparing different methods on the classical and self-collected datasets, the experimental results show that our proposed complexity perception method can not only better describe the level of complexity of navigation scenes but also obtain more accurate complexity values. [ABSTRACT FROM AUTHOR]

Three-dimensional object detection in the point cloud can provide more accurate object data for autonomous driving. In this paper, we propose a method named MA-MFFC that uses an attention mechanism and a multi-scale feature fusion network with ConvNeXt module to improve the accuracy of object detection. The multi-attention (MA) module contains point-channel attention and voxel attention, which are used in voxelization and 3D backbone. By considering the point-wise and channel-wise, the attention mechanism enhances the information of key points in voxels, suppresses background point clouds in voxelization, and improves the robustness of the network. The voxel attention module is used in the 3D backbone to obtain more robust and discriminative voxel features. The MFFC module contains the multi-scale feature fusion network and the ConvNeXt module; the multi-scale feature fusion network can extract rich feature information and improve the detection accuracy, and the convolutional layer is replaced with the ConvNeXt module to enhance the feature extraction capability of the network. The experimental results show that the average accuracy is 64.60% for pedestrians and 80.92% for cyclists on the KITTI dataset, which is 1.33% and 2.1% higher, respectively, compared with the baseline network, enabling more accurate detection and localization of more difficult objects. [ABSTRACT FROM AUTHOR]

Limitations are identified in the expressive capabilities of the deep feature extraction network employed in deep reinforcement learning (DRL), particularly in complex scenarios. Additionally, learning performance is negatively impacted by compound errors, with consideration given to the potential critical role of sample quality in learning outcomes. Therefore, how to improve the collision avoidance decision-making adaptive and effectiveness for autonomous ships navigating in various scenarios is crucial. To address these issues and overcome data acquisition difficulties, a method called rule-guided vision supervised learning (RGVSL) is proposed in this paper. Through a static collision avoidance decision-making task, a comparison is drawn between a deep feature extraction network and the Nature CNN in DQN, revealing shortcomings in the feature extraction of DRL. Additionally, a collision avoidance decision-making environment with the capability of generalization for maritime encounters is proposed, and the advantages of the method in terms of adaptability and learning cost are validated. Finally, it is demonstrated that the RGVSL method is equivalent to a DQN with γ set to 0, indicating a significant performance improvement without compound errors. Achieving an adaptive decision accuracy of over 90% in various encounter scenarios without retraining, this research substantially reduces learning costs. It can provide innovative and practical solutions for the technological development in the field of autonomous ship collision avoidance decision-making. • Revolutionizes DRL data by using explicit rules for diverse scenarios, boosting model performance. • Exposes Nature CNN's DQN inadequacy, emphasizing the need for effective feature expression in DRL. • RGVSL achieves 90% accuracy in diverse maritime scenarios, cutting retraining costs, enhancing performance. [ABSTRACT FROM AUTHOR]

6DoF camera localization is an important component of autonomous driving and navigation. Deep learning has achieved impressive results in localization, but its robustness in dynamic environments has not been adequately addressed. In this paper, we propose a framework based on hybrid attention mechanism which can be generally applied to existing CNN-based pose regressors to improve their robustness in dynamic environments. Specifically, we propose a novel Transformer Bottleneck (TBo) block including convolution, channel attention, and a position-aware self-attention mechanism, which extracts more geometrically robust features by capturing the corresponding long-term dependencies between pixels. Furthermore, we introduce shuffle attention (SA) before the pose regressor, which integrates feature information in both spatial and channel dimensions, forcing the network to learn geometrically robust features, reducing the effects of dynamic objects and illumination conditions to improve camera localization accuracy. We evaluate our method on commonly benchmarked indoor and outdoor datasets and the experimental results show that our proposed method can significantly improve localization performance compared compare favorably to contemporary pose regressors schemes. In addition, extensive ablation evaluations are conducted to prove the effectiveness of our proposed hybrid attention bottleneck block for pose regression networks. • We propose a novel Transformer Bottleneck block with self-attention and channel attention to overcome the limitations of convolution. This coupling allows them to be optimized in a mutually reinforcing manner, significantly improving fine-grained feature extraction for accurate localization. • We propose a novel end-to-end hybrid attention network for single image localization, which improves the accuracy and robustness of camera localization, especially in dynamic scenes. • We conduct extensive experiments in both indoor and outdoor dataset, which shows that our model performs better than the existing competitive methods. [ABSTRACT FROM AUTHOR]

Model-checking and verification using Kripke structures and computational tree logic* (CTL*) use abstractions from the model/process/application to create the state-transition graphs that verify the model behavior. This scheme of profiling the performance of a process imports that the depth of the process operation correlates with the level abstraction. However, because of state explosion problems, these abstractions tend to restrict the scope to create manageable execution states. Therefore, for context modeling, this procedure does not generate a fine-grained behavioral model as generated states limit the ability of the abstraction to capture the execution time interactions amongst the processes, the hardware, and the kernel. Hence, in this paper, we present an end-to-end framework that comprises auto-encoders and probabilistic models to understand the behavior of system processes and detect deviant behaviors. We test this framework with a publicly available dataset generated from an autonomous aerial vehicle (UAV) application and the results show that by creating a fine-grained model that exploits previously unharnessed properties of the system calls, we can create a dynamic anomaly detection framework that evolves as the threats change. [ABSTRACT FROM AUTHOR]

As a key environment perception technology of autonomous driving or driver assistance systems, lane detection is to ensure vehicles to drive safely in corresponding lane. However, existing lane detection algorithms for two-lane detection focus on using various filtering methods to reduce the impact of useless information, resulting in low accuracy and low efficiency. In this paper, a novel Adaptive Region of Interest (A-ROI) extraction method is proposed to improve the accuracy and real-time performance of the two-lane detection algorithm. Three key technologies are introduced to solve the problems. First, A-ROI, which only focuses on the lane where the vehicle is located, is applied to the Bird's-Eye-View image obtained by using Inverse Perspective Mapping (IPM). Next, based on Bayesian framework and Likelihood models, a lane feature extraction method with a lane-like feature filter is used for edge detection. Finally, an improved Random Sample Consensus (RANSAC) algorithm is introduced by using a filter that can remove noisy lane data. The performance of the proposed A-ROI method together with the improved lane detection method is evaluated via simulation of various scenarios. Experimental results show the proposed method has better accuracy and real-time performance than the traditional lane detection methods. [ABSTRACT FROM AUTHOR]

Autonomous driving has been among the most popular and challenging topics in the past few years. Among all modules in autonomous driving, High-definition (HD) map has drawn lots of attention in recent years due to its high precision and informative level in localization. Since localization is a significant module for automated vehicles to navigate an unknown environment, it has immediately become one of the most critical components of autonomous driving. Big organizations like HERE, NVIDIA, and TomTom have created HD maps for different scenes and purposes for autonomous driving. However, such HD maps are not open-source and are only available for internal research or automotive companies. Even though researchers have proposed various methods to create HD maps using different types of sensor data, there are few papers that review and summarize those methods. New researchers do not have a clear insight into the current state of HD map creation methods to work on their HD map research. Due to the reason above, reviewing, classifying, comparing, and summarizing the state-of-the-art techniques for HD map creation is necessary. This paper reviews recent HD map creation methods that leverage both 2D and 3D map generation. This review introduces the concept of HD maps and their usefulness in autonomous driving and gives a detailed overview of HD map creation methods. We will also discuss the limitations of the current HD map creation methods to motivate future research. Additionally, a chronological overview is created with the most recent HD map creation methods in this paper. [ABSTRACT FROM AUTHOR]

Accurately detect vehicles or pedestrians from 3D point clouds (3D object detection) is a fast developing research topic in autonomous driving and other domains. The fundamental component for feature extraction in 3D object detection is Set Abstraction (SA), which can downsample points while aggregating points to extract features. However, the current SA ignores the geometric and semantic properties of point clouds and may miss to detect remote small objects. In this paper, FocusSA is proposed, which consists two modules for enhancing useful feature extraction in the SA layer to improve 3D object detection accuracy. At first, Focused FPS (FocFPS) is proposed to evaluate the foreground and boundary scores of the points and reweighs the Furthest Point Sampling (FPS) using the evaluated scores to retain more contextual points in downsampling. Then a Geometry-aware Feature Extraction (GeoFE) module is proposed to add geometric information to enrich the awareness of geometric structure in feature aggregation. To evaluate the performances of the proposed methods, we conduct extensive experiments on three difficulty levels of Car class in KITTI dataset. The experimental results show that on "moderate" instances, our results outperform the state-of-the-art method by 1.08%. Moreover, FocusSA is easy to be plugged in popular architectures. [ABSTRACT FROM AUTHOR]

Road and sidewalk detection in urban scenarios is a challenging task because of the road imperfections and high sensor data bandwidth. Traditional free space and ground filter algorithms are not sensitive enough for small height differences. Camera-based or sensor-fusion solutions are widely used to classify drivable road from sidewalk or pavement. A LIDAR sensor contains all the necessary information from which the feature extraction can be done. Therefore, this paper focuses on LIDAR-based feature extraction. For road and sidewalk detection, the current paper presents a real-time (20 Hz+) solution. This solution can also be used for local path planning. Sidewalk edge detection is the combination of three algorithms working parallelly. To validate the result, the de facto standard benchmark dataset, KITTI, was used alongside our measurements. The data and the source code to reproduce the results are shared publicly on our GitHub repository. [ABSTRACT FROM AUTHOR]

Recently, 3D object detection by point cloud data processing has been applied to robotics and autonomous driving because of the popularity of the LiDAR sensors. Point cloud data contain the depth and geometric space information of an object as compared with the 2D images, and achieve high precision for classification and location. In the traditional processing of point cloud data, the disorder and sparsity of the points are significant problems. In addition, the traditional detector can only support processing a limited number of point clouds. Thus, it is difficult to detect objects using a large number of point clouds. However, the previous methods need to sample the point cloud data into a coarser type, so they cannot avoid the loss of information and the accuracy is affected, as seen in in PV-RCNN. In this paper, we propose a multiscale feature fusion detector called multiscale region proposal networks (MS-RPNs), which can provide multiscale prediction results for difficult category objects. Meanwhile, our method can improve the detection accuracy for smaller objects with the optimal processing of the multiscale feature extraction module. The efficiency and accuracy of the multiscale region proposal network on the KITTI 3D object detection datasets was evaluated using numerous experiments. [ABSTRACT FROM AUTHOR]

For autonomous driving systems, vehicle detection is an important part and challenging problem due to the complex traffic scenes and poor computing resources. This paper proposes an improved SSD(single shot mutlibox detector) algorithm for the fast detection of vehicles in traffic scenes. MobileNet v2 is selected as the backbone feature extraction network for SSD, which improves the real-time performance of the algorithm. To improve detection accuracy, the channel attention mechanism is utilized for feature weighting, and the deconvolution module is utilized to construct a bottom–top feature fusion structure. The experimental results show that the average precision of the proposed algorithm on the BDD100K and KITTI datasets is 82.59% and 84.83%, respectively. The single inference time of the algorithm is 73ms, which is only about 5/11 of the original model, realizing the improvement of inference speed and prediction accuracy. • Vision-based surrounding vehicle detection is utilized for intelligent driving environment sensing to ensure driving safety. • The proposed method has three improvements to the single shot mutlibox detector (SSD), including a lightweight backbone network, the attention mechanism, and the deconvolution module. • The proposed method simplifies hardware requirements, reduces computational costs, and meets the needs of practical application. • The proposed method could automatically detect vehicles in various complex traffic scenes. [ABSTRACT FROM AUTHOR]

When it comes to the accuracy of autonomous motion, it is necessary to consider object detection and recognition, especially for the robot application of the complex environment. This paper investigates novel dual-view 3D object detection networks combined with the Lidar point cloud and RGB image in engineering scenarios. The developed system is applied for autonomous vehicles that the detected objects are cars, cyclists, and pedestrians. Firstly, a feature extraction network based on the residual module is presented, and the specific features are from the RGB image. The point cloud is transformed into Bird's Eye View (BEV), and the BEV feature extraction network is built based on sparse convolution. Besides, the feature maps are input into the region proposal network (RPN) to obtain the optimal proposal so that the object classification and the bounding box regression are obtained. Finally, to evaluate the flexibility of the developed framework, extensive data sets are generated through the CARLA simulator and verified on the KITTI data set and unmanned motion platform (BIT-NAZA robot), indicating that the proposed networks can achieve satisfactory performance in the real-world scenario. • The dual-view object detection networks combined with the Lidar and RGB are designed. • The feature extraction networks are proposed for RGB based on the residual module. • The comparative experiments on KITTI datasets and the mobile robot are carried out. [ABSTRACT FROM AUTHOR]

Driver drowsiness is one of the leading causes of traffic accidents. This paper proposes a new method for classifying driver drowsiness using deep convolution neural networks trained by wavelet scalogram images of electrocardiogram (ECG) signals. Three different classes were defined for drowsiness based on video observation of driving tests performed in a simulator for manual and automated modes. The Bayesian optimization method is employed to optimize the hyperparameters of the designed neural networks, such as the learning rate and the number of neurons in every layer. To assess the results of the deep network method, heart rate variability (HRV) data is derived from the ECG signals, some features are extracted from this data, and finally, random forest and k-nearest neighbors (KNN) classifiers are used as two traditional methods to classify the drowsiness levels. Results show that the trained deep network achieves balanced accuracies of about 77% and 79% in the manual and automated modes, respectively. However, the best obtained balanced accuracies using traditional methods are about 62% and 64%. We conclude that designed deep networks working with wavelet scalogram images of ECG signals significantly outperform KNN and random forest classifiers which are trained on HRV-based features. [ABSTRACT FROM AUTHOR]

In recent years, self-supervised monocular depth estimation has gained popularity among researchers because it uses only a single camera at a much lower cost than the direct use of laser sensors to acquire depth. Although monocular self-supervised methods can obtain dense depths, the estimation accuracy needs to be further improved for better applications in scenarios such as autonomous driving and robot perception. In this paper, we innovatively combine soft attention and hard attention with two new ideas to improve self-supervised monocular depth estimation: (1) a soft attention module and (2) a hard attention strategy. We integrate the soft attention module in the model architecture to enhance feature extraction in both spatial and channel dimensions, adding only a small number of parameters. Unlike traditional fusion approaches, we use the hard attention strategy to enhance the fusion of generated multi-scale depth predictions. Further experiments demonstrate that our method can achieve the best self-supervised performance both on the standard KITTI benchmark and the Make3D dataset. [ABSTRACT FROM AUTHOR]