Boosting CNN-Based Pedestrian Detection via 3D LiDAR Fusion in Autonomous Driving

Robust pedestrian detection has been treated as one of the main pursuits for excellent autonomous driving. Recently, some convolutional neural networks (CNN) based detectors have made large progress for this goal, such as Faster R-CNN. However, the performance of them still needs a large space to be boosted, even owning the complex learning architectures. In this paper, we novelly introduce the 3D LiDAR sensor to boost the CNN-based pedestrian detection. Facing the heterogeneous and asynchronous properties of two different sensors, we firstly introduce an accurate calibration method for visual and LiDAR sensors. Then, some physically geometrical clues acquired by 3D LiDAR are explored to eliminate the erroneous pedestrian proposals generated by the state-of-the-art CNN-based detectors. Exhaustive experiments verified the superiority of the proposed method.