Search

• Integration of ConvNeXt V2′s backbone enables extraction of critical multi-scale features from the frontal view. • The Enhanced BEV Features module facilitates feature fusion, yielding richer and more semantically informative representations. • Dynamicallyformer introduces a novel architecture for feature space transformation, employing dynamic learnable encoding and deformable attention mechanisms to achieve precise conversion from the frontal view to Bird's Eye View (BEV) via a novel representation of 3D reference points. Lane detection plays a crucial role in autonomous driving technology, and there have been many recent advancements in 3D lane detection methods. One common approach is simplifying the problem by transforming images into Bird's Eye View (BEV) space. However, existing methods still have limitations, particularly in accurately identifying lanes in many complex autonomous driving scenarios, such as slopes and extreme weather conditions. Therefore, this paper proposes a dynamically updated lane detection method called DynamicallyLane. This method utilizes ConvNeXt V2-N deformations as the backbone for feature extraction, employs dynamic learnable encoding and deformable attention mechanisms, achieves precise conversion from the frontal view to BEV through a novel representation of 3D reference points, and utilizes the Enhanced BEV Features module for feature fusion, obtaining richer and more semantically informative feature representations to facilitate the model learning process better. Experimental results demonstrate that DynamicallyLane achieves an F-score of 56.2 % on the OpenLane dataset and performs excellently on the Apollo 3D Synthetic dataset. Our code is available at https://github.com/Tafble/lane_detection. [Display omitted] [ABSTRACT FROM AUTHOR]

Few-class object detection is a critical task in numerous scenes, such as autonomous driving and intelligent surveillance. The current researches mainly focus on the correlation or decoupling between classification and regression subtasks in object detection. However, they rarely take advantage of the potential of re-decoupling the classification subtask. In this paper, we propose to re-decouple the classification branch in object detection for few-class scenes by reducing multi-classification features to multiple binary-classification features. Since multi-classification losses cannot supervise the network to learn decoupled binary-classification features, we introduce a single-class loss to supervise decoupled multiple binary-classification branches. In particular, we propose a basic feature degradation head (FD-Head) structure that decouples the classification branches and applies binary-classification loss to encourage each branch to learn only the degraded single-class features. In addition, based on the mutual exclusion between classes, we propose a mutual exclusion constraint (FD-Head-M) module to constrain the scores of all classes, promoting the detector performance. Finally, we replace the original convolution with more powerful feature extraction modules to form the enhanced FD-Head (FD-Head-E). Notably, our method can be used as a universal module and embedded into the existing object detectors to boost their performance. When applying our method to typical object detectors, it experimentally achieves performance gains of 1.2–2.2%, 1.7–2.5% on the KITTI-3, SeaShips datasets respectively. When using ResNet50 as the backbone network, our method gains an accuracy of 45.9% on the MS COCO dataset. • A feature degradation head for few-class scenes is proposed. • Guiding all sub-branches to learn more significant single-class features. • A mutually exclusive constraint module is proposed to reduce false detection. • Feature degradation head has also been extended to multi-class scenes. [ABSTRACT FROM AUTHOR]

Point cloud preprocessing lays the foundation for the realization of autonomous vehicles (AVs) as it is the backbone of 3D LiDAR simultaneous localization and mapping (SLAM). Matching feature points selection based on multiple classifiers from preprocessing techniques may significantly increase the chances of the good matching result, and thus reduces drift error accumulation. In this paper, a series of point cloud preprocessing and feature extraction methods were proposed, where LiDAR sensor is used only. Experiments indicate that our ground point segmentation algorithm is efficient, comparable to state-of-the-art methods, and even outperforms the general approaches when measured with certain metrics. Improvement in extracting edge features with vertical clustering can ensure stability and geometrical characteristics of features. With the implementation of the proposed point cloud preprocessing techniques on well-known pose estimation framework such as LeGO-LOAM, higher accuracy with the reduction in both rotation and translation error in most dataset sequences is achieved. Finally, the proposed algorithm is examined and evaluated via KITTI, Semantic-KITTI, and our own VLP-16 campus datasets. [ABSTRACT FROM AUTHOR]

Loop closure detection is a crucial technique supporting localization and navigation in autonomous vehicles. Existing research focuses on feature extraction in global scenes while neglecting considerations for local dense environments. In such local scenes, there are a large number of buildings, vehicles, and traffic signs, characterized by abundant objects, dense distribution, and interlaced near and far. The current methods only employ a single strategy for constructing descriptors, which fails to provide a detailed representation of the feature distribution in dense scenes, leading to inadequate discrimination of descriptors. Therefore, this paper proposes a multi-information point cloud descriptor to address the aforementioned issues. This descriptor integrates three types of environmental features: object density, region density, and distance, enhancing the recognition capability in local dense scenes. Additionally, we incorporated wavelet transforms and invariant moments from the image domain, designing wavelet invariant moments with rotation and translation invariance. This approach resolves the issue of point cloud mismatch caused by LiDAR viewpoint variations. In the experimental part, We collected data from dense scenes and conducted targeted experiments, demonstrating that our method achieves excellent loop closure detection performance in these scenes. Finally, the method is applied to a complete SLAM system, achieving accurate mapping. [ABSTRACT FROM AUTHOR]

In this paper, we propose a 3D object detection method called MMAF-Net that is based on the multi-view and multi-stage adaptive fusion of RGB images and LiDAR point cloud data. This is an end-to-end architecture, which combines the characteristics of RGB images, the front view of point clouds based on reflection intensity, and the bird's eye view of point clouds. It also adopts a multi-stage fusion approach of "data-level fusion + feature-level fusion" to fully exploit the strength of multimodal information. Our proposed method addresses key challenges found in current 3D object detection methods for autonomous driving, including insufficient feature extraction from multimodal data, rudimentary fusion techniques, and sensitivity to distance and occlusion. To ensure the comprehensive integration of multimodal information, we present a series of targeted fusion methods. Firstly, we propose a novel input form that encodes dense point cloud reflectivity information into the image to enhance its representational power. Secondly, we design the Region Attention Adaptive Fusion module utilizing an attention mechanism to guide the network in adaptively adjusting the importance of different features. Finally, we extend the 2D DIOU (Distance Intersection over Union) loss function to 3D and develop a joint regression loss based on 3D_DIOU and SmoothL1 to optimize the similarity between detected and ground truth boxes. The experimental results on the KITTI dataset demonstrate that MMAF-Net effectively addresses the challenges posed by highly obscured or crowded scenes while maintaining real-time performance and improving the detection accuracy of smaller and more difficult objects that are occluded at far distances. • A novel object detection method is proposed integrating RGB images and LiDAR data. • This study tests the method on the KITTI dataset. • This method tackles highly obscured or crowded scenes for autonomous driving. • This method improves the detection accuracy of smaller and occluded targets far away. [ABSTRACT FROM AUTHOR]

Limitations are identified in the expressive capabilities of the deep feature extraction network employed in deep reinforcement learning (DRL), particularly in complex scenarios. Additionally, learning performance is negatively impacted by compound errors, with consideration given to the potential critical role of sample quality in learning outcomes. Therefore, how to improve the collision avoidance decision-making adaptive and effectiveness for autonomous ships navigating in various scenarios is crucial. To address these issues and overcome data acquisition difficulties, a method called rule-guided vision supervised learning (RGVSL) is proposed in this paper. Through a static collision avoidance decision-making task, a comparison is drawn between a deep feature extraction network and the Nature CNN in DQN, revealing shortcomings in the feature extraction of DRL. Additionally, a collision avoidance decision-making environment with the capability of generalization for maritime encounters is proposed, and the advantages of the method in terms of adaptability and learning cost are validated. Finally, it is demonstrated that the RGVSL method is equivalent to a DQN with γ set to 0, indicating a significant performance improvement without compound errors. Achieving an adaptive decision accuracy of over 90% in various encounter scenarios without retraining, this research substantially reduces learning costs. It can provide innovative and practical solutions for the technological development in the field of autonomous ship collision avoidance decision-making. • Revolutionizes DRL data by using explicit rules for diverse scenarios, boosting model performance. • Exposes Nature CNN's DQN inadequacy, emphasizing the need for effective feature expression in DRL. • RGVSL achieves 90% accuracy in diverse maritime scenarios, cutting retraining costs, enhancing performance. [ABSTRACT FROM AUTHOR]

6DoF camera localization is an important component of autonomous driving and navigation. Deep learning has achieved impressive results in localization, but its robustness in dynamic environments has not been adequately addressed. In this paper, we propose a framework based on hybrid attention mechanism which can be generally applied to existing CNN-based pose regressors to improve their robustness in dynamic environments. Specifically, we propose a novel Transformer Bottleneck (TBo) block including convolution, channel attention, and a position-aware self-attention mechanism, which extracts more geometrically robust features by capturing the corresponding long-term dependencies between pixels. Furthermore, we introduce shuffle attention (SA) before the pose regressor, which integrates feature information in both spatial and channel dimensions, forcing the network to learn geometrically robust features, reducing the effects of dynamic objects and illumination conditions to improve camera localization accuracy. We evaluate our method on commonly benchmarked indoor and outdoor datasets and the experimental results show that our proposed method can significantly improve localization performance compared compare favorably to contemporary pose regressors schemes. In addition, extensive ablation evaluations are conducted to prove the effectiveness of our proposed hybrid attention bottleneck block for pose regression networks. • We propose a novel Transformer Bottleneck block with self-attention and channel attention to overcome the limitations of convolution. This coupling allows them to be optimized in a mutually reinforcing manner, significantly improving fine-grained feature extraction for accurate localization. • We propose a novel end-to-end hybrid attention network for single image localization, which improves the accuracy and robustness of camera localization, especially in dynamic scenes. • We conduct extensive experiments in both indoor and outdoor dataset, which shows that our model performs better than the existing competitive methods. [ABSTRACT FROM AUTHOR]