Search

Generally applied to a painting for protection purposes, a varnish layer becomes yellow over time, making the painting undergo an appearance change. Upon this change, the conservators start a process that entails removing the old layer of varnish and applying a new one. As widely discussed in the literature, helping the conservators through supplying them with the probable outcome of the varnish removal can be of great value to them, aiding in the decision making process regarding varnish removal. This help can be realized through virtual cleaning, which in simple terms, refers to simulation of the cleaning process outcome. There have been different approaches devised to tackle the problem of virtual cleaning, each of which tries to develop a method that virtually cleans the artwork in a more accurate manner. Although successful in some senses, the majority of them do not possess a high level of accuracy. Prior approaches suffer from a range of shortcomings such as a reliance on identifying locations of specific colors on the painting, the need to access a large set of training data, or their lack of applicability to a wide range of paintings. In this work, we develop a Deep Generative Network to virtually clean the artwork. Using this method, only a small area of the painting needs to be physically cleaned prior to virtual cleaning. Using the cleaned and uncleaned versions of this small area, the entire unvarnished painting can be estimated. It should be noted that this estimation is performed in the spectral reflectance domain and herein it is applied to hyperspectral imagery of the work. The model is first applied to a Macbeth ColorChecker target (as a proof of concept) and then to real data of a small impressionist panel by Georges Seurat (known as 'Haymakers at Montfermeil' or just 'Haymakers'). The Macbeth ColorChecker is simulated in both varnished and unvarnished forms, but in the case of the 'Haymakers', we have real hyperspectral imagery belonging to both states. The results of applying the Deep Generative Network show that the proposed method has done a better job virtually cleaning the artwork compared to a physics-based method in the literature. The results are presented through visualization in the sRGB color space and also by computing Euclidean distance and spectral angle (calculated in the spectral reflectance domain) between the virtually cleaned artwork and the physically cleaned one. The ultimate goal of our virtual cleaning algorithm is to enable pigment mapping and identification after virtual cleaning of the artwork in a more accurate manner, even before the process of physical cleaning. [ABSTRACT FROM AUTHOR]

A varnish layer that is applied to a painting, generally to protect it, yellows over time, deteriorating the original look of the painting. This prompts conservators to undertake a cleaning process to remove the old varnish and apply a new one. Providing the conservators with the likely appearance of the painting before the cleaning process starts can be helpful to them, which can be done through virtual cleaning. Virtual cleaning is simply the simulation of the cleaning process. Previous works in this area required the method to have access to black and white paint regions, or physically removing the varnish first at a few spots. Through looking at the problem of virtual cleaning differently, we try to address those shortcomings. To do so, we propose using a convolutional neural network (CNN) to tackle the problem of virtual cleaning. The CNN is trained on artificially yellowed images of people, urban and rural areas, and color charts, as well as their original versions. The network is then applied to various paintings with similar scene content. The results of the method are first compared to the only physical model in the virtual cleaning field. We compare the outputs from the proposed method and the physical model by visualization as well as a quantitative measure that calculates the spectral similarity between the outputs and the reference images. These results show that the proposed method outperforms the physical model. The CNN is also applied to images of the Mona Lisa and The Virgin and Child with Saint Anne, both painted by Leonardo da Vinci. Results show both a qualitative and quantitative improvement in the color quality of the resulting image compared to their reference images. The CNN developed here is also compared to a CNN that has been developed for the purpose of image colorization in the literature to demonstrate the effectiveness of the CNN devised here, showing that the CNN architecture herein leads to a better result. The novelty of the work proposed herein lies in two premises. First, the accuracy of the method, which is demonstrated through comparison with the only physical approach derived until now. Second is the generalizability of the method which is shown through blindly applying the method to two famous works of art for which no information but an RGB image of the uncleaned artwork is known. [ABSTRACT FROM AUTHOR]

Spectral imaging modalities, including reflectance and X-ray fluorescence, play an important role in conservation science. In reflectance hyperspectral imaging, the data are classified into areas having similar spectra and turned into labeled pigment maps using spectral features and fusing with other information. Direct classification and labeling remain challenging because many paints are intimate pigment mixtures that require a non-linear unmixing model for a robust solution. Neural networks have been successful in modeling non-linear mixtures in remote sensing with large training datasets. For paintings, however, existing spectral databases are small and do not encompass the diversity encountered. Given that painting practices are relatively consistent within schools of artistic practices, we tested the suitability of using reflectance spectra from a subgroup of well-characterized paintings to build a large database to train a one-dimensional (spectral) convolutional neural network. The labeled pigment maps produced were found to be robust within similar styles of paintings. [ABSTRACT FROM AUTHOR]