Dual camera system for synchronous imaging in visible and non-visible spectral regions

A dual-camera system comprised of a visible camera and a long wavelength infrared camera system is presented. This solution for synchronous imaging in visible and non-visible spectral regions has important applications in localising and identifying objects with different thermal signatures. A detailed description of the design process and implementation of the dual-camera system is presented, highlighting all aspects of the image sensors used, their control and interfacing requirement and the necessary image processing to combine the acquired images into a single multi-spectrum image. The developed system consists of an acrylic box containing both sensors, an electronic interface board for conditioning and readout of the infrared sensor, and an image processing system that also handles network connection and communication with client programs by acting as a server. A field programmable gate array (FPGA) device is used to implement the readout protocols for the two sensor modules. Apart from the readout function, the FPGA is also a hardware accelerator which can be combined with a host central processing unit (CPU) to form an Open Computing Language (OpenCL) framework for a heterogeneous platform to perform complex image processing functions. As part of the research, geometric calibration of this multi-sensor system is conducted to achieve the intrinsic and extrinsic parameters of the cameras, for the purpose of remove lens distortion effect. Stereo camera calibration and stereo image rectification are performed to obtain the relative position and orientation of the sensors. A method is proposed to match the two views from separate cameras using triangulation such that a combinational image can be formed, which consist of information from the visible and infrared band. With the acceleration of OpenCL framework, this system can give a continuous stream of image pairs for 10.15 images/s. In the work of radiometric calibration of the uncooled infrared camera, a method which creates a temperature-controlled environment has been used, and this method does not require expensive blackbody equipment. From the image data analysis, a temperature model has been built to represent the relationship between the target surface temperature and the pixel value on the infrared image. A novel colour map is employed on the infrared image that can achieve better performance in the imaging modality than the conventional colour maps.