Search

Data augmentation plays a crucial role in training a CNN-based detector. Most previous approaches were based on using a combination of general image-processing operations and could only produce limited plausible image variations. Recently, GAN (Generative Adversarial Network) based methods have shown compelling visual results. However, they are prone to fail at preserving image-objects and maintaining translation consistency when faced with large and complex domain shifts, such as day-to-night. In this paper, we propose AugGAN, a GAN-based data augmenter which could transform on-road driving images to a desired domain while image-objects would be well-preserved. The contribution of this work is three-fold: (1) we design a structure-aware unpaired image-to-image translation network which learns the latent data transformation across different domains while artifacts in the transformed images are greatly reduced; (2) we quantitatively prove that the domain adaptation capability of a vehicle detector is not limited by its training data; (3) our object-preserving network provides significant performance gain in the difficult day-to-night case in terms of vehicle detection. AugGAN could generate more visually plausible images compared to competing methods on different on-road image translation tasks across domains. In addition, we quantitatively evaluate different methods by training Faster R-CNN and YOLO with datasets generated from the transformed results and demonstrate significant improvement on the object detection accuracies by using the proposed AugGAN model.

Vehicle detection is a fundamental function required for advanced driver assistance systems. Extensive research has shown that good performance can be obtained on public datasets by various state-of-the-art approaches, especially the deep learning methods. However, those methods are mostly two-stage approaches which inevitably require extensive computing resources and are hard to be deployed on an embedded computing platform with real-time computing performance. We introduce a single-stage vehicle detector which can work in real-time on NVIDIA DrivePX2 platform. The main contributions of this paper are threefold. We propose a detection scheme which includes multi-scale features and multi-anchor boxes to improve the accuracy of a single-stage detector. Secondly, a new data augmentation strategy is proposed to systematically generate a lot of vehicle training images whose appearances are randomly truncated, so our detector is trained to detect partially-seen vehicles better. Thirdly, we present a multi-stage image-based online hard example mining (MSI-OHEM) framework specifically designed for single-stage detectors. MSI-OHEM performs fine-tuning on hard examples and the ones with slightly-insufficient IOU that are considered true positives. Compared to other classical object detectors, the proposed detector achieves very competitive result in terms of average precision (AP) and computational speed. For the newly-defined vehicle class (car+bus) on VOC2007 test, our detector, using MobileNetV2, GoogLeNet, Inception-v2 and ResNet-50 as basenets, achieves 85.35%/85.62%/86.49%/87.81% AP and runs at 64/58/48/28 FPS on NVIDIA DrivePX2, respectively.

This paper presents a traffic light detection system based on convolutional neural networks and lidar data. The proposed approach contains two stages. In the first detection stage, the map information is adopted to assist the object detection, and two cameras with different focal lengths are used to detect traffic lights at different distances. In the following recognition stage, we combine detector and classifier to deal with the problem of many light states. A dataset is created with urban road scenes in Taiwan. The experiments have demonstrated the advance in traffic light detection and recognition.

In this paper, we introduce Boost-CNN, a robust stop-line detector that can detect objects (stop line) with competitive tradeoff between speed and accuracy. Boost-CNN consists of an AdaBoost classifier and a CNN. The former is our region proposal generator and it is further combined with the later to be a stop-line detector. In addition, an automatic hard mining method is proposed to reduce the number of false alarm. Our proposed detector achieves 91.5% in accuracy and has 100 FPS performance in test time (performed on NVIDIA DIGITS DevBox and Titan X GPU).

This paper presents a fast vehicle detector which can be deployed on NVIDIA DrivePX2 under real-time constraints. The network predicts bounding boxes with different aspect ratio and scale priors from the specifically-designed prediction module given concatenated multi-scale feature map. A new data augmentation strategy is proposed to systematically generate a lot of vehicle training images whose appearance is randomly truncated so our detector could detect occluded vehicles better. Besides, we propose a non-region-based online hard example mining framework which performs fine-tuning by picking (1) hard examples and (2) detection results with insufficient IOU. Compared to other classical object detectors, this work achieves very competitive result in terms of average precision (AP) and computational speed. For the newly-defined vehicle class (car+bus) on VOC2007 test, our detector achieves 85.32 AP and runs at 48 FPS and 30 FPS on NVIDIA Titan X & GP106 (DrivePX2), respectively.

Advances in video technology have enabled its wide adoption in the auto industry. Today, many vehicles are equipped with backup, front-looking, and side-looking cameras that allow the driver to easily monitor the traffic around the vehicle for enhanced safety. This paper reports our research on evaluating many existing object tracking techniques, and proposes a new tracker design and its application for 3D environmental mapping in vehicular technology applications. The contribution of our research is 4-fold: (1) We evaluate a large collection of state-of-the-art trackers using multiple criteria relevant to vehicular technology applications, (2) we show how to derive useful evaluation metrics from public-domain, real-world driving videos that do not come with ground-truth information on pixel tracking, (3) we propose a new tracker that is geared specifically for vehicular technology application and show that it achieves tracking accuracy that outperforms SIFT and is on-par with state-of-the-art optical-flow tracking algorithm, which has the best accuracy in our evaluation. Furthermore, we show that our tracker is 600 times more efficient than optical flow and 7 times more efficient than SIFT, and (4) we validated our new tracker design for 3D environmental map building application and showed that the new tracker can obtain comparable results as SIFT but with a significant saving in runtime.

This paper proposes a new intelligent surveillance system for parking lot management in underground environments. The feature based image stitching and image blending techniques are used to let video streams captured by multiple cameras be fused into single output whose blind area could be easily seen through. Such a system allows that anyone can immediately understand what is going on in the whole monitored area with a simple glance and without prior geometrical knowledge of the place because the blind area is translucent. The image stitching processes consists of four main parts:(1) feature correspondence detection and fundamental matrix estimation, (2) outlier filtering scheme based on the epipolar geometry and RANSAC, (3) computing the homography matrices between each pair of images, (4) projectively warping the images with their corresponding homography matrices, and conducting image fusion with the non-overlapping parts of the warped images. Finally, the process of translucent blending is applied to eliminate the blind areas inside the stitched image, and then the corresponding cameras behind the occluding pillar provide pixels for translucentizing. The authors have implemented the preliminary system with six surveillance cameras at underground parking lot environments, and experiment results of real world video sequences have been performed to verify the proposed design.

the goal of the project is to design intelligent and robust image-processing and augmented-reality algorithms for driver assistance and enhanced vehicular safety. In particular, the focuses were two-fold: (1) realizing the abilities to identify and localize in a vehicle's on-board video the sweeping windshield wipers during raining days and (2) designing and implementing an in-painting technique to remove the image of the windshield wipers and replace it with the corresponding pixels (not blocked by the wipers) from an adjacent frame.

This paper proposes a new algorithm to eliminate the wiper interference in a vehicle's onboard video to improve the detection rate of vision-based Advanced Driver Assistance Systems (ADAS), such as Forward Collision Warning (FCW) Systems. During raining days, the windshield wipers periodically and partially block the appearance of obstacles on the road that are to be detected in these early warning systems, and hence, impair their performance. The wiper pixels could be in-painted via (1) localizing the pixels belonging to the wipers and (2) replacing the wiper pixels with non-wiper pixels extracted from an adjacent video frame with no such blockage. Finally, since it is impossible to obtain the images captured in the same scenario with and without wiper interference, a quantitatively analyzing framework which blends wipers into non-wiper images is also proposed to quantitatively assess the detection rate of data with wipers, without wipers and with wipers but eliminated by our algorithm. The experimental results show that the wiper pixels being segmented and in-painted blend in unobtrusively with the surrounding pixels and the FCW detection rate is indeed improved.

This paper proposes a new vehicle blind spot elimination system which utilizes the on-board videos captured from other vehicles and the host vehicle. Such information is exchanged by WAVE/DSRC devices to achieve collaborative safety. The preceding vehicle which fully or partially blocks the field of view of the host vehicle could be translucentized in the video captured by the host vehicle and the driving environment of the front vehicle could be then visually checked by the host driver.

This paper proposes a monocular vision-based obstacle detection algorithm for parking assistance applications of advance safety vehicle by a rear camera. In order to efficiently detect various moving and stationary obstacles behind the vehicle, the feature of corner into the rear obstacles is firstly estimated by the Features from Accelerated Segment Test (FAST) corner detection methods. Then, the inverse perspective mapping (IPM) image can be used to determine whether every detected feature belongs to an obstacle candidate or to the ground. Based on these results, the segmentation and identification strategies are also proposed in order to determine the degree of collision risk and to filter out the non-hazardous candidates. Finally, the correct obstacle regions in IPM transformed image can be easily and quickly extracted. The system can provide a vision-based alert to the driver, helping to avoid collisions with obstacles behind the host vehicle. Through extensive experiments, we have shown that the rear obstacle detection system in typical urban situations can be used to efficiently extract obstacle regions markings. The proposed algorithm achieves high detecting rate and low computing power and is successfully implemented in ADI-BF561 600MHz dual core DSP.

Rain removal from an image/video is a challenging problem and has been recently investigated extensively. In our previous work, we have proposed the first single-image-based rain streak removal framework via properly formulating it as an image decomposition problem based on morphological component analysis (MCA) solved by performing dictionary learning and sparse coding. However, in this previous work, the dictionary learning process cannot be fully automatic, where the two dictionaries used for rain removal were selected heuristically or by human intervention. In this paper, we extend our previous work to propose an automatic self-learning-based rain streak removal framework for single image. We propose to automatically self-learn the two dictionaries used for rain removal without additional information or any assumption. We then extend our single-image-based method to video-based rain removal in a static scene by exploiting the temporal information of successive frames and reusing the dictionaries learned by the former frame(s) in a video while maintaining the temporal consistency of the video. As a result, the rain component can be successfully removed from the image/video while preserving most original details. Experimental results demonstrate the efficacy of the proposed algorithm.