Back to Search
Start Over
Towards efficient multi-modal 3D object detection: Homogeneous sparse fuse network.
- Source :
-
Expert Systems with Applications . Dec2024, Vol. 256, pN.PAG-N.PAG. 1p. - Publication Year :
- 2024
-
Abstract
- LiDAR-only 3D detection methods struggle with the sparsity of point clouds. To overcome this issue, multi-modal methods have been proposed, but their fusion is a challenge due to the heterogeneous representation of images and point clouds. This paper proposes a novel multi-modal framework, Homogeneous Sparse Fusion (HS-Fusion), which generates pseudo point clouds from depth completion. The proposed framework introduces a 3D foreground-aware middle extractor that efficiently extracts high-responding foreground features from sparse point cloud data. This module can be integrated into existing sparse convolutional neural networks. Furthermore, the proposed homogeneous attentive fusion enables cross-modality consistency fusion. Finally, the proposed HS-Fusion can simultaneously combine 2D image features and 3D geometric features of pseudo point clouds using multi-representation feature extraction. The proposed network has been found to attain better performance on the 3D object detection benchmarks. In particular, the proposed model demonstrates a 4.02% improvement in accuracy compared to the pure model. Moreover, its inference speed surpasses that of other models, thus further validating the efficacy of HS-Fusion. • LiDAR-based 3D detection faces point cloud sparsity challenges. • A novel Homogeneous Sparse Fusion multi-modal approach is introduced. • Homogeneous Sparse Fusion adaptively extracts foreground features. • Cross-modality consistency fusion is achieved with multi-representation. • Homogeneous Sparse Fusion excels in accuracy and speed. [ABSTRACT FROM AUTHOR]
Details
- Language :
- English
- ISSN :
- 09574174
- Volume :
- 256
- Database :
- Academic Search Index
- Journal :
- Expert Systems with Applications
- Publication Type :
- Academic Journal
- Accession number :
- 179365162
- Full Text :
- https://doi.org/10.1016/j.eswa.2024.124945