Search

To address the challenge of sparse old photo datasets, we apply transfer learning to image inpainting tasks. Specifically, we improve a two-stage image inpainting network that focuses on collaborative subtasks. We also design a transform module based on the cross-aggregation of windows to improve long-distance contextual information acquisition in image inpainting and enhance the integrity of images in terms of structure and texture. Our improved two-stage network has a significantly better repair performance compared to that of the current common inpainting methods. We further apply transfer learning techniques by utilizing the improved two-stage image inpainting network as the base network and decoupling the generator into a feature extractor and classifier, which consist of an encoder and a decoder, respectively. We obtain a domain-invariant feature extractor through minimax game training using source and target domain data. This feature extractor can be combined with the original encoder to restore old photo images. To verify the effectiveness of our approach, we conducted comparative experiments. Our results show that the PSNR, SSIM, and FID indexes of the model using transfer learning are 11.8%, 2.96%, and 44.4% higher than those without transfer learning, respectively. These findings suggest that applying transfer learning techniques can be an effective solution to address the challenge of sparse old photo datasets in image inpainting tasks. [ABSTRACT FROM AUTHOR]

To overcome the fault of convolutional networks, which can be over-smooth, blurred, or discontinuous, a novel transformer network with cross-window aggregated attention is proposed. Our network as a whole is constructed as a generative adversarial network model, and by embedding the Window Aggregation Transformer (WAT) module, we improve the information aggregation between windows without increasing the computational complexity and effectively obtain the image long-range dependencies to solve the problem that convolutional operations are limited by local feature extraction. First, the encoder extracts the multi-scale features of the image with convolution kernels of different scales; second, the feature maps of different scales are input into a WAT module to realize the aggregation between feature information and finally, these features are reconstructed by the decoder, and then, the generated image is input into the global discriminator, in which the discrimination between real and fake images is completed. It is experimentally verified that our designed Transformer window attention network is able to make the structured texture of the restored images richer and more natural when performing the restoration task of large broken or structurally complex images. [ABSTRACT FROM AUTHOR]