A multi-task Faster R-CNN method for 3D vehicle detection based on a single image.

Vehicle detection is an important part of robot environmental perception. In this paper, a 3D vehicle detection method using a single image is proposed to generate the 3D space coordinate information of the object using monocular vision for autonomous driving. The proposed method works under the multi-task framework and integrates 2D object detection, 3D object detection, orientation estimation and key point detection into one unified deep convolution neural network (DCNN) which could be trained by end-to-end learning. Besides, our proposed method is built by modifying Fast R-CNN using multi-task learning, and thus our proposed method is named multi-task Faster R-CNN (MT-Faster R-CNN). The experiments on KITTI dataset are conducted to evaluate our proposed method and the other 3D vehicle detection methods. The experimental results demonstrate that our proposed method is competitive and could significantly assist autonomous driving. • Our proposed MT-Faster R-CNN could be trained by end-to-end learning, which makes it more effective for optimizing. • Our proposed network can simultaneously output 2D and 3D detection results based on single image. • Extensive experiments show that we outperform our baseline with a large margin on KITTI dataset. [ABSTRACT FROM AUTHOR]