Search

Most pedestrian detection methods focus on bounding boxes based on fusing RGB with lidar. These methods do not relate to how the human eye perceives objects in the real world. Furthermore, lidar and vision can have difficulty detecting pedestrians in scattered environments, and radar can be used to overcome this problem. Therefore, the motivation of this work is to explore, as a preliminary step, the feasibility of fusing lidar, radar, and RGB for pedestrian detection that potentially can be used for autonomous driving that uses a fully connected convolutional neural network architecture for multimodal sensors. The core of the network is based on SegNet, a pixel-wise semantic segmentation network. In this context, lidar and radar were incorporated by transforming them from 3D pointclouds into 2D gray images with 16-bit depths, and RGB images were incorporated with three channels. The proposed architecture uses a single SegNet for each sensor reading, and the outputs are then applied to a fully connected neural network to fuse the three modalities of sensors. Afterwards, an up-sampling network is applied to recover the fused data. Additionally, a custom dataset of 60 images was proposed for training the architecture, with an additional 10 for evaluation and 10 for testing, giving a total of 80 images. The experiment results show a training mean pixel accuracy of 99.7% and a training mean intersection over union of 99.5%. Also, the testing mean of the IoU was 94.4%, and the testing pixel accuracy was 96.2%. These metric results have successfully demonstrated the effectiveness of using semantic segmentation for pedestrian detection under the modalities of three sensors. Despite some overfitting in the model during experimentation, it performed well in detecting people in test mode. Therefore, it is worth emphasizing that the focus of this work is to show that this method is feasible to be used, as it works regardless of the size of the dataset. Also, a bigger dataset would be necessary to achieve a more appropiate training. This method gives the advantage of detecting pedestrians as the human eye does, thereby resulting in less ambiguity. Additionally, this work has also proposed an extrinsic calibration matrix method for sensor alignment between radar and lidar based on singular value decomposition.

Recently, the research on monocular 3D target detection based on pseudo-LiDAR data has made some progress. In contrast to LiDAR-based algorithms, the robustness of pseudo-LiDAR methods is still inferior. After conducting in-depth experiments, we realized that the main limitations are due to the inaccuracy of the target position and the uncertainty in the depth distribution of the foreground target. These two problems arise from the inaccurate depth estimation. To deal with the aforementioned problems, we propose two innovative solutions. The first is a novel method based on joint image segmentation and geometric constraints, used to predict the target depth and provide the depth prediction confidence measure. The predicted target depth is fused with the overall depth of the scene and results in the optimal target position. For the second, we utilize the target scale, normalized with the Gaussian function, as a priori information. The uncertainty of depth distribution, which can be visualized as long-tail noise, is reduced. With the refined depth information, we convert the optimized depth map into the point cloud representation, called a pseudo-LiDAR point cloud. Finally, we input the pseudo-LiDAR point cloud to the LiDAR-based algorithm to detect the 3D target. We conducted extensive experiments on the challenging KITTI dataset. The results demonstrate that our proposed framework outperforms various state-of-the-art methods by more than 12.37% and 5.34% on the easy and hard settings of the KITTI validation subset, respectively. On the KITTI test set, our framework also outperformed state-of-the-art methods by 5.1% and 1.76% on the easy and hard settings, respectively.

In this paper, I propose a bird eye view image detection method for parking areas and collision risk areas at the same time in parking situations. Deep learning algorithms using area detection and semantic segmentation were used. The main architecture of the method described in this paper is based on a harmonic densely connected network and a cross-stage partial network. The dataset used for training was calibrated to four 190° wide-angle cameras to generate around view monitor (AVM) images based on the Chungbuk National University parking lot, and an experiment was performed based on this dataset. In the experimental results, the available parking area was visualized by detecting the parking line, parking area, and available driving area in the AVM images. Furthermore, the undetected area in the semantic segmentation as a collision risk area was visualized in order to obtain the results. According to the proposed attention CSPHarDNet model, the experimental results were 81.89% mIoU and 18.36 FPS in a NVIDIA Xavier environment. The results of this experiment demonstrated that algorithms can be used in real time in a parking situation and have better performance results compared to the conventional HarDNet.

Predicting the mobility patterns of vehicles together with their interactions among surrounding traffic objects is a critical task in autonomous driving systems. Existing graph neural network-based trajectory prediction models primarily capture the structural connectivity of network nodes (road objects) and assume equal priority to all neighbors of a node. However, in real-time traffic networks, the behavior of each vehicle is significantly influenced by its neighboring road objects and this influence is not uniform. This necessitates a neighbor interaction-aware trajectory prediction model that assumes non-uniform priority among neighboring nodes. In this article, we have designed a novel unified trajectory prediction model which is suitable for both highway and urban traffic conditions. The proposed approach seamlessly integrates multi-level context modeling using graph attention mechanisms, capturing and leveraging interactions and dependencies among objects at varied levels of proximity within a graph. Additionally, it employs an encoder–decoder long short-term memory architecture for long-term trajectory prediction, ensuring adaptability to different driving scenarios. The advanced graph attention mechanisms play a crucial role in modeling spatial dependencies between vehicles, allowing the proposed model to dynamically adapt to evolving interactions over time. The experimentations done on real-world trajectory datasets, namely, Next Generation Simulation US-101 highway dataset and diverse urban datasets such as ApolloScape and Argoverse demonstrate remarkable performance of MC-GATP in long-term trajectory prediction. The model showcases superior prediction accuracy, scalability, and computational efficiency for both highway and urban environments. [ABSTRACT FROM AUTHOR]

3D point clouds collected by low-channel light detection and ranging (LiDAR) are relatively sparse compared to high-channel LiDAR, which is considered costly. To address this, an outdoor large-scene point cloud reconstruction (LSPCR) technique based on transformer is proposed in this study. The LSPCR approach first projects the original sparse 3D point cloud onto a 2D range image; then, it enhances the resolution in the vertical direction of the 2D range image before converting the high-resolution range image back to a 3D point cloud as the final reconstructed point cloud data. Experiments were performed on the real-world KITTI dataset, and the results show that LSPCR achieves an average accuracy improvement of over 60% compared to non-deep-learning algorithms; it also achieves better performance compared to the latest deep-learning algorithms. Therefore, LSPCR is an effective solution for sparse point cloud reconstruction and addresses the challenges associated with high-resolution LiDAR point clouds. [ABSTRACT FROM AUTHOR]

An accurate panorama depth estimation result is crucial to risk perception in autonomous driving practice. In this paper, an innovative framework is presented to address the challenges of imperfect observation and projection fusion in panorama depth estimation, enabling the accurate capture of distances from surrounding images in driving scenarios. First, the Patch Filling method is proposed to alleviate the imperfect observation of panoramic depth in autonomous driving scenarios, which constructs a panoramic depth map based on the sparse distance data provided by the 3D point cloud. Then, in order to tackle the distortion challenge faced by outdoor panoramic images, a method for image context learning, ViT-Fuse, is proposed and specifically designed for equirectangular panoramic views. The experimental results show that the proposed ViT-Fuse reduces the estimation error by 9.15% on average in driving scenarios compared with the basic method and exhibits more robust and smoother results on the edge details of the depth estimation maps. [ABSTRACT FROM AUTHOR]

Object detection in road scenarios is crucial for intelligent transport systems and autonomous driving, but complex traffic conditions pose significant challenges. This paper introduces Z-You Only Look Once version 8 small (Z-YOLOv8s), designed to improve both accuracy and real-time efficiency under real-world uncertainties. By incorporating Revisiting Perspective Vision Transformer (RepViT) and C2f into the YOLOv8s framework, and integrating the Large Selective Kernel Network (LSKNet), the model enhances spatial feature extraction. Additionally, the YOLOv8s backbone is optimized with Space-to-Depth Convolution (SPD-Conv) for better small object detection. The Softpool-Spatial Pyramid Pooling Fast (SoftPool-SPPF) module ensures precise characteristic information preservation. Z-YOLOv8s improves mean average precision (mAP)@0.5 on the Berkeley Deep Drive 100 K (BDD100K) and Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) datasets by 7.3 % and 3.8 %, respectively. It also achieves accuracy increases of 5.7 % and 6.5 % in Average Precision (AP)-Small, and a real-time detection speed of 78.41 frames per second (FPS) on the BDD100K. Z-YOLOv8s balances detection precision and processing speed more effectively than other detectors, as demonstrated by experimental results and comparisons. • Designing a RepViTC2f module, to address occluded object challenges in traffic scenarios. • SoftPool-SPPF enhances small object detection by refining edge details. • Z-YOLOv8s balances speed and precision on BDD100K and KITTI datasets. [ABSTRACT FROM AUTHOR]

With the rapid advancement of information technology and intelligent systems, autonomous driving has garnered significant attention and research in recent years. Key technologies, such as Simultaneous Localization and Mapping (SLAM), Perception and Localization, and Scene Segmentation, have proven to be essential in this field. These technologies not only evolve independently, each with its own research focus and application paths, but also complement and rely on one another in various complex autonomous driving scenarios. This paper provides a comprehensive review of the development and current state of these technologies, along with a forecast of their future trends. [ABSTRACT FROM AUTHOR]

Recently, extensive research has been conducted on generating virtual training data in a digital twin-based simulator to reduce the time and cost associated with acquiring high-quality training data necessary for autonomous driving. In this study, we propose an efficient method of generating synthetic training datasets for autonomous driving by combining real-world and virtual training data. Specifically, we propose a method of implementing a digital twin-based autonomous driving simulator, collecting large amounts of virtual training data using its camera sensor, and generating synthetic training datasets by combining virtual and real-world training data in various ratios. The effectiveness of these datasets is then validated in deep learning applications, particularly for detecting traffic lights and signal information. Validation results indicate that synthetic training datasets significantly improve deep learning performance, provided they include a sufficient amount of real-world training data to avoid class imbalance issues. We conclude that the synthetic training datasets generated using a digital twin-based simulator are cost-effective and practical for deep learning applications. [ABSTRACT FROM AUTHOR]

Accurately classifying road signs is crucial for autonomous driving due to the high stakes involved in ensuring safety and compliance. As Convolutional Neural Networks (CNNs) have largely replaced traditional Machine Learning models in this domain, the demand for substantial training data has increased. This study aims to compare the performance of classical Machine Learning (ML) models and Deep Learning (DL) models under varying amounts of training data, particularly focusing on altered signs to mimic real-world conditions. We evaluated three classical models: Support Vector Machine (SVM), Random Forest, and Linear Discriminant Analysis (LDA), and one Deep Learning model: Convolutional Neural Network (CNN). Using the German Traffic Sign Recognition Benchmark (GTSRB) dataset, which includes approximately 40,000 German traffic signs, we introduced digital alterations to simulate conditions such as environmental wear or vandalism. Additionally, the Histogram of Oriented Gradients (HOG) descriptor was used to assist classical models. Bayesian optimization and k-fold cross-validation were employed for model fine-tuning and performance assessment. Our findings reveal a threshold in training data beyond which accuracy plateaus. Classical models showed a linear performance decrease under increasing alteration, while CNNs, despite being more robust to alterations, did not significantly outperform classical models in overall accuracy. Ultimately, classical Machine Learning models demonstrated performance comparable to CNNs under certain conditions, suggesting that effective road sign classification can be achieved with less computationally intensive approaches. [ABSTRACT FROM AUTHOR]

At present Deep Neural Networks (DNN) have a dominant role in the AI-driven Autonomous driving approaches. This paper focuses on the potential safety risks of deploying DNN classifiers in Advanced Driver Assistance System (ADAS) systems. In our experience, many theoretically sound AI-driven solutions tested and deployed in ADAS have shown serious safety flaws in practice. A brief review of practice and theory of automotive safety standards and related body of knowledge is presented. It is followed by a comparative analysis between DNN classifiers and safety standards developed in the automotive industry. The output of the study provides advice and recommendations for filling the current gaps within the complex and interrelated factors pertaining to the safety of Autonomous Road Vehicles (ARV). This study may assist ARV's safety, system, and technology providers during the design, development, and implementation life cycle. The contribution of this work is to highlight and link the learning rules enforced by risk factors when DNN classifiers are expected to provide a near real-time safer Vehicle Navigation Solution (VNS). [ABSTRACT FROM AUTHOR]

Autonomous driving pays considerable attention to pedestrian trajectory prediction as a crucial task. Constructing effective pedestrian trajectory prediction models depends heavily on utilizing the motion characteristics of pedestrians, along with their interactions among themselves and between themselves and their environment. However, traditional trajectory prediction models often fall short of capturing complex real-world scenarios. To address these challenges, this paper proposes an enhanced pedestrian trajectory prediction model, M2Tames, which incorporates comprehensive motion, interaction, and semantic context factors. M2Tames provides an interaction module (IM), which consists of an improved multi-head mask temporal attention mechanism (M2Tea) and an Interaction Inference Module (I2). M2Tea thoroughly characterizes the historical trajectories and potential interactions, while I2 determines the precise interaction types. Then, IM adaptively aggregates useful neighbor features to generate a more accurate interactive feature map and feeds it into the final layer of the U-Net encoder to fuse with the encoder's output. Furthermore, by adopting the U-Net architecture, M2Tames can learn and interpret scene semantic information, enhancing its understanding of the spatial relationships between pedestrians and their surroundings. These innovations improve the accuracy and adaptability of the model for predicting pedestrian trajectories. Finally, M2Tames is evaluated on the ETH/UCY and SDD datasets for short- and long-term settings, respectively. The results demonstrate that M2Tames outperforms the state-of-the-art model MSRL by 2.49% (ADE) and 8.77% (FDE) in the short-term setting and surpasses the optimum Y-Net by 6.89% (ADE) and 1.12% (FDE) in the long-term prediction. Excellent performance is also shown on the ETH/UCY datasets. [ABSTRACT FROM AUTHOR]

In the context of autonomous driving, real-time and accurate object detection is crucial for ensuring vehicle safety. To address the problem of severe missed detection and poor accuracy for small targets in complex traffic conditions, a novel object detection algorithm for automatic driving is proposed, named YED-YOLO. Firstly, based on the original algorithm, the efficient multiscale attention (EMA) module is introduced to enhance the attention to the underlying features. Then, the CSPDarknet53 to 2-stage FPN (C2f) module is improved to solve the problem of insufficient receptive fields by adding deformable convolutions (DCNs). Furthermore, a novel intersection over union (IoU) loss function calculation method is proposed, which improves the generalization ability and convergence speed of the model. Finally, the ablation and comparative experiments are conducted using the recall, precision and frames per second (FPS). The experimental results indicate that the algorithm proposed in this paper demonstrates substantial enhancement effet for smaller traffic participants such as pedestrians and cyclists, whose APs increase by 3% and 3.1%, respectively. [ABSTRACT FROM AUTHOR]

Accurate prediction of pedestrian trajectories is of great significance for intelligent monitoring and safe navigation of autonomous driving. The trajectory of pedestrians is not only affected by the pedestrians themselves, but also by the interaction with the surrounding pedestrians. In recent years, the introduction of attention mechanism has further enhanced the model's ability to focus on key information, making the prediction more accurate. Therefore, this study proposes a pedestrian trajectory prediction model combining self-attention mechanism and social behavior characteristics. This method uses the newly studied KAN network architecture to initially encode pedestrian movement information, adopts improved social pooling layer to model the spatial characteristics of pedestrians, adaptively pays attention to the most useful adjacent information, and provides suggestions for better spatial modeling. The self-attention module is used to further extract the motion features of the target pedestrians and the surrounding pedestrians at the current moment to obtain more connected spatial features, so that the model can process and understand the sequence data more effectively and efficiently, predict the future trajectory of the target pedestrians more accurately, and enhance the performance of the prediction process. Experiments on public data sets ETH and UCY show that the prediction accuracy of this model is significantly improved compared with other methods. [ABSTRACT FROM AUTHOR]

Accurate vehicle detection is crucial for the advancement of intelligent transportation systems, including autonomous driving and traffic monitoring. This paper presents a comparative analysis of two advanced deep learning models—YOLOv8 and YOLOv10—focusing on their efficacy in vehicle detection across multiple classes such as bicycles, buses, cars, motorcycles, and trucks. Using a range of performance metrics, including precision, recall, F1 score, and detailed confusion matrices, we evaluate the performance characteristics of each model.The findings reveal that YOLOv10 generally outperformed YOLOv8, particularly in detecting smaller and more complex vehicles like bicycles and trucks, which can be attributed to its architectural enhancements. Conversely, YOLOv8 showed a slight advantage in car detection, underscoring subtle differences in feature processing between the models. The performance for detecting buses and motorcycles was comparable, indicating robust features in both YOLO versions. This research contributes to the field by delineating the strengths and limitations of these models and providing insights into their practical applications in real-world scenarios. It enhances understanding of how different YOLO architectures can be optimized for specific vehicle detection tasks, thus supporting the development of more efficient and precise detection systems. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Chongqing University of Technology (Natural Science) is the property of Chongqing University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

At the current stage of autonomous driving, monitoring the behavior of safety stewards (drivers) is crucial to establishing liability in the event of an accident. However, there is currently no method for the quantitative assessment of safety steward behavior that is trusted by multiple stakeholders. In recent years, deep-learning-based methods can automatically detect abnormal behaviors with surveillance video, and blockchain as a decentralized and tamper-resistant distributed ledger technology is very suitable as a tool for providing evidence when determining liability. In this paper, a trusted supervision paradigm for autonomous driving (TSPAD) based on multimodal data authentication is proposed. Specifically, this paradigm consists of a deep learning model for driving abnormal behavior detection based on key frames adaptive selection and a blockchain system for multimodal data on-chaining and certificate storage. First, the deep-learning-based detection model enables the quantification of abnormal driving behavior and the selection of key frames. Second, the key frame selection and image compression coding balance the trade-off between the amount of information and efficiency in multiparty data sharing. Third, the blockchain-based data encryption sharing strategy ensures supervision and mutual trust among the regulatory authority, the logistic platform, and the enterprise in the driving process. [ABSTRACT FROM AUTHOR]

Object detection in road scenarios is crucial for intelligent transport systems and autonomous driving, but complex traffic conditions pose significant challenges. This paper introduces Z-You Only Look Once version 8 small (Z-YOLOv8s), designed to improve both accuracy and real-time efficiency under real-world uncertainties. By incorporating Revisiting Perspective Vision Transformer (RepViT) and C2f into the YOLOv8s framework, and integrating the Large Selective Kernel Network (LSKNet), the model enhances spatial feature extraction. Additionally, the YOLOv8s backbone is optimized with Space-to-Depth Convolution (SPD-Conv) for better small object detection. The Softpool-Spatial Pyramid Pooling Fast (SoftPool-SPPF) module ensures precise characteristic information preservation. Z-YOLOv8s improves mean average precision (mAP)@0.5 on the Berkeley Deep Drive 100 K (BDD100K) and Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) datasets by 7.3 % and 3.8 %, respectively. It also achieves accuracy increases of 5.7 % and 6.5 % in Average Precision (AP)-Small, and a real-time detection speed of 78.41 frames per second (FPS) on the BDD100K. Z-YOLOv8s balances detection precision and processing speed more effectively than other detectors, as demonstrated by experimental results and comparisons.

Trajectory prediction is highly essential for accurate navigation. Existing deep learning-based approaches always encounter serious performance degradation when facing shifted data or unseen scenarios. For learning transferable representations across different scenarios, the promising pretraining technique is applied to trajectory prediction tasks. However, relevant studies employ point-level masking mechanisms, which cannot capture local motion information across multiple time steps. Additionally, for trajectory data that couples multiple motion states, extracting the temporal dependencies within each state sequence remains highly challenging. To tackle this issue, we propose a channel-independent pretrained network via tokenized patching for efficient vehicle trajectory prediction, and it is composed of tokenized patch masking, channel-independent extractor (CiE), and state decoupling-mixing (SDM). Specifically, first of all, based on the designed tokenized patching scheme, TPM is established to represent local information and long-term relations in masked sequences. Then, through a series of weight-shared dense layers, CiE is designed to capture the individual dependencies among state sequences in an unsupervised pretraining manner. Moreover, by decoupling the complicated trajectory into pseudo-state representations, SDM is proposed to independently reconstruct the state sequences and further carry out representation mixing operations, to realize available trajectory predictions. Finally, extensive experiments show that our framework is effective and achieves the state-of-the-art performance on the INTERACTION and Argoverse2 datasets. [ABSTRACT FROM AUTHOR]

Accurate vehicle detection is crucial for the advancement of intelligent transportation systems, including autonomous driving and traffic monitoring. This paper presents a comparative analysis of two advanced deep learning models—YOLOv8 and YOLOv10—focusing on their efficacy in vehicle detection across multiple classes such as bicycles, buses, cars, motorcycles, and trucks. Using a range of performance metrics, including precision, recall, F1 score, and detailed confusion matrices, we evaluate the performance characteristics of each model.The findings reveal that YOLOv10 generally outperformed YOLOv8, particularly in detecting smaller and more complex vehicles like bicycles and trucks, which can be attributed to its architectural enhancements. Conversely, YOLOv8 showed a slight advantage in car detection, underscoring subtle differences in feature processing between the models. The performance for detecting buses and motorcycles was comparable, indicating robust features in both YOLO versions. This research contributes to the field by delineating the strengths and limitations of these models and providing insights into their practical applications in real-world scenarios. It enhances understanding of how different YOLO architectures can be optimized for specific vehicle detection tasks, thus supporting the development of more efficient and precise detection systems.

Deep learning approaches have been gaining importance in several applications. However, the widespread use of these methods in safety-critical domains, such as Autonomous Driving, is still dependent on their reliability and trustworthiness. The goal of this paper is to provide a review of deep learning-based uncertainty methods and their applications to support perception tasks for Autonomous Driving. We detail significant Uncertainty Quantification and calibration methods, and their contributions and limitations, as well as important metrics and concepts. We present an overview of the state of the art of out-of-distribution detection and active learning, where uncertainty estimates are commonly applied. We show how these methods have been applied in the automotive context, providing a comprehensive analysis of reliable AI for Autonomous Driving. Finally, challenges and opportunities for future work are discussed for each topic. [ABSTRACT FROM AUTHOR]

Visual object tracking is an important technology in camera-based sensor networks, which has a wide range of practicability in auto-drive systems. A transformer is a deep learning model that adopts the mechanism of self-attention, and it differentially weights the significance of each part of the input data. It has been widely applied in the field of visual tracking. Unfortunately, the security of the transformer model is unclear. It causes such transformer-based applications to be exposed to security threats. In this work, the security of the transformer model was investigated with an important component of autonomous driving, i.e., visual tracking. Such deep-learning-based visual tracking is vulnerable to adversarial attacks, and thus, adversarial attacks were implemented as the security threats to conduct the investigation. First, adversarial examples were generated on top of video sequences to degrade the tracking performance, and the frame-by-frame temporal motion was taken into consideration when generating perturbations over the depicted tracking results. Then, the influence of perturbations on performance was sequentially investigated and analyzed. Finally, numerous experiments on OTB100, VOT2018, and GOT-10k data sets demonstrated that the executed adversarial examples were effective on the performance drops of the transformer-based visual tracking. White-box attacks showed the highest effectiveness, where the attack success rates exceeded 90% against transformer-based trackers. [ABSTRACT FROM AUTHOR]

Autonomous vehicles must maintain awareness of their surroundings, encompassing other vehicles, pedestrians, traffic signs, and varying road conditions. Among numerous objectives, the most crucial one is the detection of vehicles and pedestrians. Despite advancements, complex traffic scenarios still pose challenges in target detection. Variations in target size and occlusion often lead to misjudgments and missed detections. To address these issues, an enhanced vehicle and pedestrian detection model, YOLOv8nRLD, is proposed. It integrates several innovations to enhance target detection accuracy. Firstly, the introduction of the Receptive-Field Convolutional Block Attention Module enhances the model's ability to detect and emphasize salient features, facilitating more precise target localization and identification. Additionally, the Spatial Pyramid Pooling Fast module within the backbone feature extraction network is refined with the inclusion of the Large Separated Kernel Attention module. This augmentation significantly boosts the network's feature extraction capabilities. Furthermore, the adoption of a dynamic target detection head, termed Dynamic Head, incorporating attention mechanisms and diverse perception capabilities, enhances the model's feature expression capacity. Experimental validation on the BDD100K and Cityscapes datasets demonstrates notable performance enhancements, with mean Average Precision increasing by 4% and 2.5%, respectively. Ablation experiments confirm the effectiveness of individual modules, underscoring the method's efficacy and versatility in vehicle and pedestrian detection tasks. [ABSTRACT FROM AUTHOR]

Self-localization and pose registration are required for sound operation of next generation autonomous vehicles under uncertain environments. Thus, precise localization and mapping are crucial tasks in odometry, planning and other downstream processing. In order to reduce information loss in preprocessing, we propose leveraging LiDAR-based localization and mapping (LOAM) with point cloud-based deep learning instead of convolutional neural network (CNN) based methods that require cylindrical projection. The normal distribution transform (NDT) algorithm is then used to refine the former coarse pose estimation from the deep learning model. The results demonstrate that the proposed method is comparable in performance to recent benchmark studies. We also explore the possibility of using Product Quantization to improve NDT internal neighborhood searching by using high-level features as fingerprints. [ABSTRACT FROM AUTHOR]

Few prior works study self-driving cars by deep learning with IoT collaboration. SDC-Net, which is an end-to-end multitask self-driving car camera cocoon IoT-based system, is one of the research areas that tackles this direction. However, by design, SDC-Net is not able to identify the accident locations; it only classifies whether a scene is a crash scene or not. In this work, we introduce an enhanced design for the SDC-Net system by (1) replacing the classification network with a detection one, (2) adapting our benchmark dataset labels built on the CARLA simulator to include the vehicles' bounding boxes while keeping the same training, validation, and testing samples, and (3) modifying the shared information via IoT to include the accident location. We keep the same path planning and automatic emergency braking network, the digital automation platform, and the input representations to formulate the comparative study. The SDC-Net++ system is proposed to (1) output the relevant control actions, especially in case of accidents: accelerate, decelerate, maneuver, and brake, and (2) share the most critical information to the connected vehicles via IoT, especially the accident locations. A comparative study is also conducted between SDC-Net and SDC-Net++ with the same input representations: front camera only, panorama and bird's eye views, and with single-task networks, crash avoidance only, and multitask networks. The multitask network with a BEV input representation outperforms the nearest representation in precision, recall, f1-score, and accuracy by more than 15.134%, 12.046%, 13.593%, and 5%, respectively. The SDC-Net++ multitask network with BEV outperforms SDC-Net multitask with BEV in precision, recall, f1-score, accuracy, and average MSE by more than 2.201%, 2.8%, 2.505%, 2%, and 18.677%, respectively. [ABSTRACT FROM AUTHOR]

In the realm of autonomous vehicle technology, the multimodal vehicle detection network (MVDNet) represents a significant leap forward, particularly in the challenging context of weather conditions. This paper focuses on the enhancement of MVDNet through the integration of a multi-head attention layer, aimed at refining its performance. The integrated multi-head attention layer in the MVDNet model is a pivotal modification, advancing the network's ability to process and fuse multimodal sensor information more efficiently. The paper validates the improved performance of MVDNet with multi-head attention through comprehensive testing, which includes a training dataset derived from the Oxford Radar RobotCar. The results clearly demonstrate that the multi-head MVDNet outperforms the other related conventional models, particularly in the average precision (AP) of estimation, under challenging environmental conditions. The proposed multi-head MVDNet not only contributes significantly to the field of autonomous vehicle detection but also underscores the potential of sophisticated sensor fusion techniques in overcoming environmental limitations. [ABSTRACT FROM AUTHOR]

With the rapid advancement of information technology and intelligent systems, autonomous driving has garnered significant attention and research in recent years. Key technologies, such as Simultaneous Localization and Mapping (SLAM), Perception and Localization, and Scene Segmentation, have proven to be essential in this field. These technologies not only evolve independently, each with its own research focus and application paths, but also complement and rely on one another in various complex autonomous driving scenarios. This paper provides a comprehensive review of the development and current state of these technologies, along with a forecast of their future trends.

Autonomous driving, as a pivotal technology in modern transportation, is progressively transforming the modalities of human mobility. In this domain, vehicle detection is a significant research direction that involves the intersection of multiple disciplines, including sensor technology and computer vision. In recent years, many excellent vehicle detection methods have been reported, but few studies have focused on summarizing and analyzing these algorithms. This work provides a comprehensive review of existing vehicle detection algorithms and discusses their practical applications in the field of autonomous driving. First, we provide a brief description of the tasks, evaluation metrics, and datasets for vehicle detection. Second, more than 200 classical and latest vehicle detection algorithms are summarized in detail, including those based on machine vision, LiDAR, millimeter-wave radar, and sensor fusion. Finally, this article discusses the strengths and limitations of different algorithms and sensors, and proposes future trends. [ABSTRACT FROM AUTHOR]

Copyright of Automotive Engineer (1674-6546) is the property of Auto Engineering Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Adaptive cruise control (ACC) enables efficient, safe, and intelligent vehicle control by autonomously adjusting speed and ensuring a safe following distance from the vehicle in front. This paper proposes a novel adaptive cruise system, namely the Safety-First Reinforcement Learning Adaptive Cruise Control (SFRL-ACC). This system aims to leverage the model-free nature and high real-time inference efficiency of Deep Reinforcement Learning (DRL) to overcome the challenges of modeling difficulties and lower computational efficiency faced by current optimization control-based ACC methods while simultaneously maintaining safety advantages and optimizing ride comfort. Firstly, we transform the ACC problem into a safe DRL formulation Constrained Markov Decision Process (CMDP) by carefully designing state, action, reward, and cost functions. Subsequently, we propose the Projected Constrained Policy Optimization (PCPO)-based ACC Algorithm SFRL-ACC, which is specifically tailored to solve the CMDP problem. PCPO incorporates safety constraints that further restrict the trust region formed by the Kullback–Leibler (KL) divergence, facilitating DRL policy updates that maximize performance while keeping safety costs within their limit bounds. Finally, we train an SFRL-ACC policy and compare its computation time, traffic efficiency, ride comfort, and safety with state-of-the-art MPC-based ACC control methods. The experimental results prove the superiority of the proposed method in the aforementioned performance aspects. [ABSTRACT FROM AUTHOR]

In the field of intelligent connected vehicles, the precise and real-time identification of speed bumps is critically important for the safety of autonomous driving. To address the issue that existing visual perception algorithms struggle to simultaneously maintain identification accuracy and real-time performance amidst image distortion and complex environmental conditions, this study proposes an enhanced lightweight neural network framework, YOLOv5-FPNet. This framework strengthens perception capabilities in two key phases: feature extraction and loss constraint. Firstly, FPNet, based on FasterNet and Dynamic Snake Convolution, is developed to adaptively extract structural features of distorted speed bumps with accuracy. Subsequently, the C3-SFC module is proposed to augment the adaptability of the neck and head components to distorted features. Furthermore, the SimAM attention mechanism is embedded within the backbone to enhance the ability of key feature extraction. Finally, an adaptive loss function, Inner–WiseIoU, based on a dynamic non-monotonic focusing mechanism, is designed to improve the generalization and fitting ability of bounding boxes. Experimental evaluations on a custom speed bumps dataset demonstrate the superior performance of FPNet, with significant improvements in key metrics such as the mAP, mAP50_95, and FPS by 38.76%, 143.15%, and 51.23%, respectively, compared to conventional lightweight neural networks. Ablation studies confirm the effectiveness of the proposed improvements. This research provides a fast and accurate speed bump detection solution for autonomous vehicles, offering theoretical insights for obstacle recognition in intelligent vehicle systems. [ABSTRACT FROM AUTHOR]

This paper provides an in-depth review of deep learning techniques to address the challenges of odometry and global ego-localization using frequency modulated continuous wave (FMCW) radar sensors. In particular, we focus on the prediction of odometry, which involves the determination of the ego-motion of a system by external sensors, and loop closure detection, which concentrates on the determination of the ego-position typically on an existing map. We initially emphasize the significance of these tasks in the context of radar sensors and underscore the motivations behind them. The subsequent sections delve into the practical implementation of deep learning approaches, strategically designed to effectively address the aforementioned challenges. We primarily focus on spinning and automotive radar configurations within the domain of autonomous driving. Additionally, we introduce publicly available datasets that have been instrumental in addressing these challenges and analyze the importance and struggles of current methods used for radar based odometry and localization. In conclusion, this paper highlights the distinctions between the addressed tasks and other radar perception applications, while also discussing their differences from challenges posed by alternative sensor modalities. The findings contribute to the ongoing discourse on advancing radar sensor capabilities through the application of deep learning methodologies, particularly in the context of enhancing odometry and ego-localization for autonomous driving applications. [ABSTRACT FROM AUTHOR]

The article outlines various approaches to developing a fuzzy decision algorithm designed for monitoring and issuing warnings about driver drowsiness. This algorithm is based on analyzing EOG (electrooculography) signals and eye state images with the aim of preventing accidents. The drowsiness warning system comprises key components that learn about, analyze and make decisions regarding the driver's alertness status. The outcomes of this analysis can then trigger warnings if the driver is identified as being in a drowsy state. Driver drowsiness is characterized by a gradual decline in attention to the road and traffic, diminishing driving skills and an increase in reaction time, all contributing to a higher risk of accidents. In cases where the driver does not respond to the warnings, the ADAS (advanced driver assistance systems) system should intervene, assuming control of the vehicle's commands. [ABSTRACT FROM AUTHOR]

The given text discusses a road segmentation method called MSAN that uses multi-modal scale-aware attentions to segment roads in real-time for autonomous driving. The method combines RGB and depth information to improve prediction results and outperforms single modality models. The MSAN architecture consists of two attention modules, one for feature fusion across modalities and scales and another for scale-aware fusion. The experiments show that MSAN achieves high performance and efficiency on multiple datasets. However, the method relies heavily on labeled data, which limits its adaptability to different scenarios. Future research aims to reduce data dependency through few-shot learning strategies. [Extracted from the article]

As autonomous vehicles get closer to our daily lives, the need for architectures that function as redundant pipelines is becoming increasingly critical. To address this issue without compromising the budget, researchers aim to avoid duplicating high-cost sensors such as LiDARs. In this work, we propose using monocular cameras, which are already essential for some modules of the autonomous platform, for 3D scene understanding. While many methods for depth estimation using single images have been proposed in the literature, they usually rely on complex neural network ensembles that extract dense feature maps, resulting in a high computational cost. Instead, we propose a novel and inherently efficient method for obtaining depth images that replace tangled neural architectures with attention mechanisms applied to basic encoder–decoder models. We evaluate our method on the KITTI public dataset and in real-world experiments on our automated vehicle. The obtained results prove the viability of our approach, which can compete with intricate state-of-the-art methods while outperforming most alternatives based on attention mechanisms. [ABSTRACT FROM AUTHOR]

Monocular 3D object detection (Mono3OD) is a challenging yet cost-effective vision task in the fields of autonomous driving and mobile robotics. The lack of reliable depth information makes obtaining accurate 3D positional information extremely difficult. In recent years, center-guided monocular 3D object detectors have directly regressed the absolute depth of the object center based on 2D detection. However, this approach heavily relies on local semantic information, ignoring contextual spatial cues and global-to-local visual correlations. Moreover, visual variations in the scene can lead to inevitable depth prediction errors for objects at different scales. To address these limitations, we propose a Mono3OD framework based on scene-level adaptive instance depth estimation (MonoSAID). Firstly, the continuous depth is discretized into multiple bins, and the width distribution of depth bins is adaptively generated based on scene-level contextual semantic information. Then, by establishing the correlation between global contextual semantic feature information and local semantic features of instances, and using the probability distribution representation of local instance features and the linear combination of bin centers distributions to solve the depth problem. In addition, a multi-scale spatial perception attention module is designed to extract attention maps of various scales through pyramid pooling operations. This design enhances the model’s receptive field and multi-scale spatial perception capabilities, thereby improving its ability to model target objects. We conducted extensive experiments on the KITTI dataset and the Waymo dataset. The results show that MonoSAID can effectively improve the 3D detection accuracy and robustness, and our method achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

The application of large field-of-view (FoV) cameras equipped with fish-eye lenses brings notable advantages to various real-world computer vision applications, including autonomous driving. While deep learning has proven successful in conventional computer vision applications using regular perspective images, its potential in fish-eye camera contexts remains largely unexplored due to limited datasets for fully supervised learning. Semi-supervised learning comes as a potential solution to manage this challenge. In this study, we explore and benchmark two popular semi-supervised methods from the perspective image domain for fish-eye image segmentation. We further introduce FishSegSSL, a novel fish-eye image segmentation framework featuring three semi-supervised components: pseudo-label filtering, dynamic confidence thresholding, and robust strong augmentation. Evaluation on the WoodScape dataset, collected from vehicle-mounted fish-eye cameras, demonstrates that our proposed method enhances the model's performance by up to 10.49% over fully supervised methods using the same amount of labeled data. Our method also improves the existing image segmentation methods by 2.34%. To the best of our knowledge, this is the first work on semi-supervised semantic segmentation on fish-eye images. Additionally, we conduct a comprehensive ablation study and sensitivity analysis to showcase the efficacy of each proposed method in this research. [ABSTRACT FROM AUTHOR]

Abstract The advancement of autonomous driving heavily relies on the ability to accurate lane lines detection. As deep learning and computer vision technologies evolve, a variety of deep learning-based methods for lane line detection have been proposed by researchers in the field. However, owing to the simple appearance of lane lines and the lack of distinctive features, it is easy for other objects with similar local appearances to interfere with the process of detecting lane lines. The precision of lane line detection is limited by the unpredictable quantity and diversity of lane lines. To address the aforementioned challenges, we propose a novel deep learning approach for lane line detection. This method leverages the Swin Transformer in conjunction with LaneNet (called ST-LaneNet). The experience results showed that the true positive detection rate can reach 97.53% for easy lanes and 96.83% for difficult lanes (such as scenes with severe occlusion and extreme lighting conditions), which can better accomplish the objective of detecting lane lines. In 1000 detection samples, the average detection accuracy can reach 97.83%, the average inference time per image can reach 17.8 ms, and the average number of frames per second can reach 64.8 Hz. The programming scripts and associated models for this project can be accessed openly at the following GitHub repository: https://github.com/Duane711/Lane-line-detection-ST-LaneNet .

Visual Place Recognition (VPR) is a fundamental task in robotics and computer vision, where the ability to recognize locations from visual inputs is crucial for autonomous navigation systems. Traditional methods, which rely on handcrafted features or standard convolutional neural networks (CNNs), struggle with environmental changes that significantly alter a place’s appearance. Recent advancements in deep learning have improved VPR by focusing on deep-learned features, enhancing robustness under varying conditions. However, these methods often overlook saliency cues, leading to inefficiencies in dynamic scenes. To address these limitations, we propose an improved MixVPR model that incorporates both self-attention and cross-attention mechanisms through a spatial-wise hybrid attention mechanism. This enhancement integrates spatial saliency cues into the global image embedding, improving accuracy and reliability. We also utilize the DINOv2 visual transformer for robust feature extraction. Extensive experiments on mainstream VPR benchmarks demonstrate that our method achieves superior performance while maintaining computational efficiency. Ablation studies and visualizations further validate the contributions of our attention mechanisms to the model’s performance improvement.