Search

Facing the significant challenge of 3D object detection in complex weather conditions and road environments, existing algorithms based on single-frame point cloud data struggle to achieve desirable results. These methods typically focus on spatial relationships within a single frame, overlooking the semantic correlations and spatiotemporal continuity between consecutive frames. This leads to discontinuities and abrupt changes in the detection outcomes. To address this issue, this paper proposes a multi-frame 3D object detection algorithm based on a deformable spatiotemporal Transformer. Specifically, a deformable cross-scale Transformer module is devised, incorporating a multi-scale offset mechanism that non-uniformly samples features at different scales, enhancing the spatial information aggregation capability of the output features. Simultaneously, to address the issue of feature misalignment during multi-frame feature fusion, a deformable cross-frame Transformer module is proposed. This module incorporates independently learnable offset parameters for different frame features, enabling the model to adaptively correlate dynamic features across multiple frames and improve the temporal information utilization of the model. A proposal-aware sampling algorithm is introduced to significantly increase the foreground point recall, further optimizing the efficiency of feature extraction. The obtained multi-scale and multi-frame voxel features are subjected to an adaptive fusion weight extraction module, referred to as the proposed mixed voxel set extraction module. This module allows the model to adaptively obtain mixed features containing both spatial and temporal information. The effectiveness of the proposed algorithm is validated on the KITTI, nuScenes, and self-collected urban datasets. The proposed algorithm achieves an average precision improvement of 2.1% over the latest multi-frame-based algorithms. [ABSTRACT FROM AUTHOR]

In recent years, 3D object detection based on LiDAR point clouds is a key component of autonomous driving. In pursuit of enhancing the accuracy of 3D point cloud feature extraction and point cloud detection, this paper introduces a novel 3D object detection model, termed as Graph Self-Attention-RCNN (GA-RCNN). This model is designed to integrate voxel information and point location information, enhancing the quality of 3D object proposals while maintaining contextual accuracy. The first stage rectifies the previous approach that relied on local features for preselected boxes, overlooking crucial global contextual information. An improved method is suggested in this work, utilizing BEV to capture long-range dependencies via a cross-attention mechanism. The second stage addresses the overreliance on local neighborhood point feature extraction. The Graph Self-Attention Pooling method is proposed, characterized by its dynamic computation of contribution weights for inputs. This enhances the model's flexibility and generalization performance. Extensive evaluations on KITTI and Waymo datasets demonstrate GA-RCNN's superior accuracy compared to other methods, affirming its efficacy in 3D object detection. [ABSTRACT FROM AUTHOR]

In this paper, we propose a 3D object detection method called MMAF-Net that is based on the multi-view and multi-stage adaptive fusion of RGB images and LiDAR point cloud data. This is an end-to-end architecture, which combines the characteristics of RGB images, the front view of point clouds based on reflection intensity, and the bird's eye view of point clouds. It also adopts a multi-stage fusion approach of "data-level fusion + feature-level fusion" to fully exploit the strength of multimodal information. Our proposed method addresses key challenges found in current 3D object detection methods for autonomous driving, including insufficient feature extraction from multimodal data, rudimentary fusion techniques, and sensitivity to distance and occlusion. To ensure the comprehensive integration of multimodal information, we present a series of targeted fusion methods. Firstly, we propose a novel input form that encodes dense point cloud reflectivity information into the image to enhance its representational power. Secondly, we design the Region Attention Adaptive Fusion module utilizing an attention mechanism to guide the network in adaptively adjusting the importance of different features. Finally, we extend the 2D DIOU (Distance Intersection over Union) loss function to 3D and develop a joint regression loss based on 3D_DIOU and SmoothL1 to optimize the similarity between detected and ground truth boxes. The experimental results on the KITTI dataset demonstrate that MMAF-Net effectively addresses the challenges posed by highly obscured or crowded scenes while maintaining real-time performance and improving the detection accuracy of smaller and more difficult objects that are occluded at far distances. • A novel object detection method is proposed integrating RGB images and LiDAR data. • This study tests the method on the KITTI dataset. • This method tackles highly obscured or crowded scenes for autonomous driving. • This method improves the detection accuracy of smaller and occluded targets far away. [ABSTRACT FROM AUTHOR]