Geometrically guided exemplar-based inpainting

Exemplar-based methods have proven their efficiency for the reconstruction of missing parts in a digital image. Texture as well as local geometry are often very well restored by such methods. Some applications, however, require the ability to reconstruct nonlocal geometric features, e.g., long edges. In order to do so, we propose to first compute a geometric sketch, which is then interpolated and used as a guide for the global reconstruction. In comparison with other related approaches, the originality of our work relies on the following points: (1) The geometric sketch computation is parameter-free and based on level lines, which provides a complete, reliable, and stable representation of the image. (2) The completion of the geometric sketch is fully automatic. It is done using a new—and interesting on its own—geometric inpainting approach that interpolates level lines with Euler spirals. Euler spirals are natural curves for shape completion and have been used already for edge completion and inpainting. It is the first time, however, that these curves are used for completing the whole level lines structure. (3) The general reconstruction is performed using a guided version of a classical exemplar-based method. However, we do not constrain the exemplar-based reconstruction to strictly follow the geometric guide. We actually use a new metric between blocks that consists of the sum of the classical ${{\mathrm L}^2}$ metric between any two blocks of the general image plus an ${{\mathrm L}^2}$ metric between the corresponding blocks in the completed geometric image. This is equivalent to a Lagrangian relaxation of a strictly guided reconstruction. We discuss in the paper the details of the method and some related mathematical issues, and we illustrate its efficiency on several examples.