A Fusion Approach to Detect Traffic Signs Using Registered Color Images and Noisy Airborne LiDAR Data

Traffic sign detection is considered as one of the active research topics in transportation and computer vision. The previous works mainly focus on detecting traffic signs in images or in mobile light detection and ranging (LiDAR) data. In this paper, we propose a novel deep learning method to accurately detect traffic signs by fusing the complementary features from registered airborne geo-referenced color images and noisy airborne LiDAR data. Specifically, we first segment the airborne color images to road and non-road segments by integrating various local features in an inequality constraint quadratic optimization model. Next, we find the corresponding road regions in LiDAR data and extract high elevated objects above the road. We then segment the extracted objects to different regions corresponding to traffic sign candidates using Euclidean distance-based clustering. Finally, we find the corresponding traffic sign candidates in color images, extract their deep features, and represent them in a convex optimization model to classify the candidates. A set of extensive experiments have been carried out on the airborne geo-referenced color images and noisy airborne LiDAR data captured by Utah State University from I-15 highway. The results show the effectiveness of the proposed method in detecting traffic signs.