Shape from shading under relaxed assumptions

Shape-from-shading is a classical problem in computer vision. The aim is to recover 3D surface shape from a single image of an object, based on a photometric analysis of the pattern of shading. Since the amount of light reflected by a surface is a function of the direction of the incident light and the viewer, relative to the local surface normal, image intensity conveys information about surface orientation and hence, indirectly, surface height. This thesis aims to relax some of the assumptions typically made in the shape-from-shading literature. Our aim is to move shape-fromshading away from the overly simplistic assumptions which have limited its applicability to images captured in lab conditions. In particular, we focus on assumptions of surface smoothness, point source illumination and reconstruction in the surface normal domain. In Chapter 3, we relax the assumption of a smooth surface. We exploit a psychology-inspired heuristic that pixels need only have a similar surface orientation if they are both in close proximity and have a similar intensity. This leads to an adaptive smoothing process which is able to preserve fine surface structure. We adapt a geometric shape-from-shading framework to overcome the problem of normals “flipping” between solutions which alternately satisfy data-closeness and smoothness terms. Under the classical assumption of point source illumination, we show that our method significantly outperforms a number of previously reported methods. In Chapter 4, we relax the assumption of illumination being provided by a single point light source. Specifically, we consider environment illumination in which lighting is represented by a spherical function which describes the incident radiance from all directions in the scene. We use the well known result that Lambertian reflectance acts like a low pass filter and hence the convolution of environment lighting and surface reflectance can be efficiently represented using a low order spherical harmonic. With an order-1 approximation, we show how the image irradiance equation can be solved as a quadratically-constrained linear least squares optimisation. The global optimum is found using the method of Lagrange multipliers. The order-2 case is non-convex and prone to converge on local minima if solved using local optimisation. We reformulate the problem as a bilinear system of equations which leads to an efficient and robust solution method. In both cases, we incorporate a structure-preserving smoothness constraint based on ideas from Chapter 3 to regularise the problem. In Chapter 5, we continue with the relaxed illumination assumption (i.e. we model environment illumination), but we develop algorithms which operate in the domain of surface height rather than surface normals. This has the advantage of reducing the dimensionality of the problem at the expense of increased complexity. Moreover, integrability is implicitly enforced via the problem formulation. We describe two contributions. The first is a linear method for recovering surface height directly from images formed by taking ratios between colour channels. In this case, the nonlinear normalisation term is factored out. This allows us to form a linear system of equations relating image intensity and surface height via a finite difference approximation to the surface gradient. Finally, we relax the assumption that the object must be globally convex (i.e. contains no self occlusions). We show that self occluded intensity can be related to unoccluded intensity via a quadratic inequality constraint. This is too weak a constraint to be used for shape-from-shading on its own. However, we use it to develop an occlusion-sensitive surface integration algorithm. We show that the problem can be formulated as a convex optimisation and solved using semidefinite programming.