Fusion of Multiple Sensor Types in Computer Vision Systems

This research provides an analysis of several approaches to the fusion of multiple dissimilar sensors to supplement simple color vision detection and recognition. Nonvisible sensor systems can enhance computer vision systems. The study investigates the use of thermal infrared (IR) sensors in combination with color data for object detection and recognition. The authors analyze several types of high-level and low-level sensor fusion to compare error rates with raw color and raw IR error rates in detection and recognition of vehicles in a scene. Principal components analysis is used to reduce the dimensionality of sensor input data to discard nonessential data, while preserving data important to classification. One recognition method showing promise is to exploit the strength of nonvisible information (e.g., low light, shadows, etc.) to reduce the search space for color data by replacing the V channel in the HSV color sensor data with IR. For detection, one method showing promise is the replacement or averaging of the dominant color channel with IR.
The original document contains color images.