Investigation of colour constancy using blind signal separation and physics-based image modelling

Colour is an important property in image and video processing; it is used for the segmentation, classification, and recognition of objects. The observed colour of a surface, as captured by an imaging sensor, can be affected by factors such as specular reflection, illumination variation and shadows which can lead to erroneous colour identification. This creates a need for techniques that are able to extract an illumination invariant descriptor of the surface reflectance of an object, such techniques would enable the development of image and video processing systems which are able to identify the actual colour of an object, independent of illumination variations. Thus achieving what is referred to as colour constancy. This research aims to investigate the effectiveness of applying blind signal separation integrated with a physical model of image formation into a framework for achieving colour constancy. The particular model considered in this study is the dichromatic reflection model. This model has been used in approaches to colour constancy developed by other researchers. However, most of these approaches use mixed image components (i.e. composed of specular and diffuse components) in order to estimate illumination and consequently achieve colour constancy. In addition, most of these approaches require the segmentation of the image into regions which correspond to different colours on the multi-coloured surfaces, in high specular reflection (highlight) areas of the image. Correct segmentation of multi-coloured surfaces is difficult to achieve. This thesis proposes an alternative approach embodied in a framework which integrates blind signal separation and dichromatic model of image formation. Unlike the conventional approaches, by using blind signal separation, the illumination can be estimated more accurately using the explicitly separated specular image component and colour constancy is achieved by utilising the explicitly separated diffuse image component only. In addition, by using the blind signal separation the multi-coloured surfaces segmentation problem can be avoided. The research questions addressed by this research are “how should blind signal separation be integrated with the dichromatic model?” and “how does the proposed framework perform in the context of achieving colour constancy?” A novel colour constancy framework is developed in this thesis, and experimental findings about the performance of the framework are reported. Unlike the existing work, the proposed framework includes a new method to estimate the illumination spectral power distribution (ISPD) by using an explicitly extracted specular component of images. Furthermore, the proposed framework includes a new method for estimating the surface spectral reflectance using an explicitly extracted diffuse component, instead of mixed image components which are used by other researchers. The framework consists of three stages which are: the separation of image components, the ISPD estimation and the estimation of surface spectral reflectance. The methodology exploited to evaluate the performance of the framework involves the development of algorithms, their implementation in software, and their assessment using well-designed experiments anchored on quantitative performance measurement methods. The goodness-of-fit coefficient (GFC) is used to evaluate the performance of the framework, by measuring the degree of similarity between the estimated spectral distribution and a known reference. Values of GFC range between 0 and 1; a higher value representing a higher degree of similarity. Using an image data set generated by the author, compared to the manufacturer’s specifications, the estimated ISPD has an average GFC value equal to 0.9830 and 0.9215 for two light sources with colour temperature of 5500 K and 2900 K, respectively. The average GFC of the estimated ISPD improves significantly by 2.9% when the explicit specular image component is used instead of mixed image components. Furthermore, using Foster et al’s image data set (a set of hyperspectral images of natural scenes which was collected by Foster, Nascimento, and Amano), the ISPD is estimated using the mixed image components for other light sources with different colour temperatures. The results show that the estimated ISPD has an average value of the GFC equal to 0.9986 compared to the measured illumination. Using the data set collected by the author of this thesis, the surface spectral reflectance is estimated at individual pixels of an object illuminated by two alternative light sources with colour temperatures of 5500 K and 2900 K. A comparative assessment shows that the spectral reflectance, estimated for each given surface, has almost the same spectral signature for the two light sources. The comparison between the surface spectral reflectance estimates corresponding to the two light sources gives an average GFC value which ranges from 0.9611 to 0.9887, depending on the type of the blind separation technique that is used (i.e. the spatially constrained FastICA technique and the technique developed by Umeyama and Godin). Given that the surface spectral reflectance is the output of the last stage of the framework, which depends on the output of the previous two stages, therefore the GFC measured for surface spectral reflectance reflects the performance of the whole framework. The high GFC values mean that the estimates of surface reflectance under the two light sources are very similar, despite the differences between the two illuminants. This similarity implies that the extracted surface reflectance is significantly independent of illumination characteristics, hence showing that the proposed framework achieved a significant degree of colour constancy. Moreover, the observed results show a statistically significant improvement in the accuracy of the estimated surface spectral reflectance by 2.6% in terms of average GFC value when the explicitly extracted diffuse image component is used instead of the mixed image components. Compared to the surface spectral reflectance measurements included in Foster et al’s image data set, the surface spectral reflectance estimated using the mixed image components has an average GFC value equal to 0.9608.