Search

Previous studies suggest that to achieve color constancy, the human visual system makes use of multiple cues, including a priori assumptions about the illumination (“daylight priors”). Specular highlights have been proposed to aid constancy, but the evidence for their usefulness is mixed. Here, we used a novel cue-combination approach to test whether the presence of specular highlights or the validity of a daylight prior improves illumination chromaticity estimates, inferred from achromatic settings, to determine whether and under which conditions either cue contributes to color constancy. Observers made achromatic settings within three-dimensional rendered scenes containing matte or glossy shapes, illuminated by either daylight or nondaylight illuminations. We assessed both the variability of these settings and their accuracy, in terms of the standard color constancy index (CCI). When a spectrally uniform background was present, neither CCIs nor variability improved with specular highlights or daylight illuminants (Experiment 1). When a Mondrian background was introduced, CCIs decreased overall but were higher for scenes containing glossy, as opposed to matte, shapes (Experiments 2 and 3). There was no overall reduction in variability of settings and no benefit for scenes illuminated by daylights. Taken together, these results suggest that the human visual system indeed uses specular highlights to improve color constancy but only when other cues, such as from the local surround, are weakened.

Diminished Reality (DR) is a video editing technique that alters reality by removing certain objects. It can be used as a preliminary step in Augmented Reality to replace real objects by virtual ones with different sizes and shapes. It can also be used solely, for example, in the case of virtually emptying a furnished apartment. The general approach of DR consists in three main steps. First, an inpainting technique is applied to a target region in the image to coherently remove an object. The image corresponds to a keyframe of the video stream. Second, the resulting inpainted region is transmitted to the next frames of the video stream by copying pixel intensities with respect to the camera pose and scene geometry. This consists in estimating the camera orientation and position in 3D which can be obtained by a Simultaneous Localization and Mapping (SLAM) technique. Third, the target region is updated with respect to the lighting change in the scene.In this thesis, we focused on the third step of the DR pipeline. Although many DR applications have been proposed in the literature, few are the ones who dealt with light change in the scene. Most of past work assumes that the surface is Lambertian and therefore perfectly diffuse. However, this is often not true, especially in indoor environments. By identifying specular highlights as the main cause for lighting change in the target region, we proposed two main approaches to address this problem.First, we proposed a specularity propagation method applied to real-time DR. Using the DR pipeline mentioned earlier, we integrated an interpolation function based on Thin-Plate Splines (TPS) in order to estimate the change ratios of the pixel intensities in the target region. This function is constrained by a number of specularity properties to achieve a plausible reconstruction of the specular highlights in the video stream. Our approach was tested on several real-time videos and achieved coherent reproduction of specularities in the context of DR.Second, we addressed the lighting problem in DR and AR as an inverse rendering problem. To do so, we analyzed the image components as described in light reflection models. In Computer Graphics, local illumination models such as Phong’s are used to render synthetic images in real-time. In this case, the parameters of the model are set by the user as inputs along with the scene’s geometry, the light source configuration and the camera pose. However, in a Mixed Reality (MR) application, the parameters of the model are unknown and have to be set in concordance with the real image from the camera. So, in this case we want to solve an inverse local illumination problem where the input is the real image. The output is the model’s parameters along with the light source configuration, the scene’s geometry and the camera pose. In this thesis, we proposed an exhaustive evaluation of the well-posedness of this problem with a focus on the specular highlights. The camera pose and the scene’s geometry are estimated using the SLAM approach and the rest of the unknown parameters are estimated by minimizing a photometric cost. We showed that we can invert a local illumination model from the observation of a single specular highlight. Therefore, in the context of AR and DR applications, we do not need to know the number of light sources in the scene a priori since each specularity is processed separately. This also opens many perspectives for similar inversion problems like camera localization., La réalité diminuée (RD) est une technique de montage vidéo qui modifie la réalité en supprimant certains objets. Il peut être utilisé comme étape préliminaire en réalité augmentée pour remplacer des objets réels par des objets virtuels de différentes tailles et formes. Il peut également être utilisé, par exemple, dans le cas de la réaménagement virtuel d'un appartement meublé. L'approche générale de RD consiste en trois étapes principales. Tout d'abord, une technique d'inpainting est appliquée à une région cible dans l'image pour retirer de manière cohérente un objet. Ceci correspond à une image clé du flux vidéo. Deuxièmement, la région modifiée résultante est transmise aux images suivantes du flux vidéo en copiant les intensités de pixels par rapport à la pose de la caméra et à la géométrie de la scène. Cela consiste à estimer l'orientation et la position de la caméra en 3D qui peuvent être obtenues par une technique de localisation et de cartographie simultanées (SLAM). Troisièmement, la région cible est mise à jour par rapport au changement d'illumination dans la scène.Dans cette thèse, nous nous sommes concentrés sur la troisième étape du pipeline DR. Bien que de nombreuses applications RD aient été proposées dans la littérature, rares sont celles qui ont traité des changements de lumière dans la scène. La plupart des travaux passés supposent que la surface est Lambertienne et donc parfaitement diffuse. Cependant, ce n'est souvent pas vrai, en particulier dans les environnements intérieurs. En identifiant les reflets spéculaires comme la principale cause du changement d'illumination dans la région cible, nous avons proposé deux approches principales pour résoudre ce problème.Dans un premier temps, nous avons proposé une méthode de propagation temps-réel de la spécularité appliquée à la RD. En utilisant le pipeline RD mentionné précédemment, nous avons intégré une fonction d'interpolation basée sur des splines à plaques minces (TPS) afin d'estimer les rapports de changement des intensités de pixels dans la région cible. Cette fonction est contrainte à un nombre de propriétés spécifiques à la forme de la spécularité pour obtenir une reconstruction plausible des reflets spéculaires dans le flux vidéo. Notre approche a été testée sur plusieurs vidéos en temps réel et a réalisé une reproduction cohérente des spécularités dans le contexte de la RD.Deuxièmement, nous avons abordé le problème d'illumination en RD et RA comme un problème de rendu inverse. Pour ce faire, nous avons analysé les composants de l'image comme décrit dans les modèles de réflexion de la lumière. En infographie, des modèles d'illumination locales tels que Phong sont utilisés pour rendre des images synthétiques en temps réel. Dans ce cas, les paramètres du modèle sont définis par l'utilisateur en tant qu'entrées en plus de la géométrie de la scène, la configuration de la source lumineuse et la pose de la caméra. Cependant, dans une application de réalité mixte (MR), les paramètres du modèle sont inconnus et doivent être définis en concordance avec l'image réelle de la caméra. Donc, dans ce cas, nous voulons résoudre un problème d'inversion de l’illumination locale où l'entrée est l'image réelle. La sortie correspond aux paramètres du modèle ainsi qu'à la configuration de la source lumineuse, à la géométrie de la scène et à la pose de la caméra. Dans cette thèse, nous avons proposé une évaluation exhaustive du conditionnement de ce problème en mettant l'accent sur les reflets spéculaires. La pose de la caméra et la géométrie de la scène sont estimées à l'aide de l'approche SLAM et les autres paramètres inconnus sont estimés en minimisant un coût photométrique. Nous avons montré que nous pouvons inverser un modèle d’illumination locale à partir de l'observation d'une seule spécularité. (...)

The material property of glossiness, which is attributed to many objects in our daily life, is physically independent of the objects' color. However, perceived glossiness can change with the contrast between the highlight and the area around the specular highlight. Hitherto, experiments mainly investigated gloss on unicolored surfaces. It is well known that the context in which a surface is embedded can influence its perceived lightness. Here we investigated whether similar contextual effects exist also for gloss perception by presenting single surfaces containing two different colors. We tested the influence of the second color on participants' gloss judgments with both real surfaces and photographs of those surfaces. In both conditions, participants were influenced by the second color on the surface even though they were asked to ignore it. We found contrasting contextual effects on the bicolored surfaces. However, when explicitly asked to rate the global gloss on the bicolored surfaces, participants took both parts of the surface equally into account.

Purpose Live intra-operative functional imaging has multiple potential clinical applications, such as localization of ischemia, assessment of organ transplantation success and perfusion monitoring. Recent research has shown that live monitoring of functional tissue properties, such as tissue oxygenation and blood volume fraction, is possible using multispectral imaging in laparoscopic surgery. While the illuminant spectrum is typically kept constant in laparoscopic surgery and can thus be estimated from preoperative calibration images, a key challenge in open surgery originates from the dynamic changes of lighting conditions. Methods The present paper addresses this challenge with a novel approach to light source calibration based on specular highlight analysis. It involves the acquisition of low-exposure time images serving as a basis for recovering the illuminant spectrum from pixels that contain a dominant specular reflectance component. Results Comprehensive in silico and in vivo experiments with a range of different light sources demonstrate that our approach enables an accurate and robust recovery of the illuminant spectrum in the field of view of the camera, which results in reduced errors with respect to the estimation of functional tissue properties. Our approach further outperforms state-of-the-art methods proposed in the field of computer vision. Conclusion Our results suggest that low-exposure multispectral images are well suited for light source calibration via specular highlight analysis. This work thus provides an important first step toward live functional imaging in open surgery.

Complex specular surfaces under sharp point lighting show a fascinating glinty appearance, but rendering it is an unsolved problem. Using Monte Carlo pixel sampling for this purpose is impractical: the energy is concentrated in tiny highlights that take up a minuscule fraction of the pixel. We instead compute an accurate solution using a completely different deterministic approach. Our method considers the true distribution of normals on a surface patch seen through a single pixel, which can be highly complex. We show how to evaluate this distribution efficiently, assuming a Gaussian pixel footprint and Gaussian intrinsic roughness. We also take advantage of hierarchical pruning of position-normal space to rapidly find tex-els that might contribute to a given normal distribution evaluation. Our results show complex, temporally varying glints from materials such as bumpy plastics, brushed and scratched metals, metallic paint and ocean waves. Copyright © ACM.