Back to Search
Start Over
Generalized image outpainting with U-transformer.
- Source :
-
Neural Networks . May2023, Vol. 162, p1-10. 10p. - Publication Year :
- 2023
-
Abstract
- In this paper, we develop a novel transformer-based generative adversarial neural network called U-Transformer for generalized image outpainting problems. Different from most present image outpainting methods conducting horizontal extrapolation, our generalized image outpainting could extrapolate visual context all-side around a given image with plausible structure and details even for complicated scenery, building, and art images. Specifically, we design a generator as an encoder-to-decoder structure embedded with the popular Swin Transformer blocks. As such, our novel neural network can better cope with image long-range dependencies which are crucially important for generalized image outpainting. We propose additionally a U-shaped structure and multi-view Temporal Spatial Predictor (TSP) module to reinforce image self-reconstruction as well as unknown-part prediction smoothly and realistically. By adjusting the predicting step in the TSP module in the testing stage, we can generate arbitrary outpainting size given the input sub-image. We experimentally demonstrate that our proposed method could produce visually appealing results for generalized image outpainting against the state-of-the-art image outpainting approaches. • To our best knowledge, the proposed U-Transformer is the first transformer-based image outpainting framework able to obtain global features and keep high resolutions. • The TSP module leverages the multi-view LSTM network and self-attention blocks which can transfer incomplete latent features and support the generation of arbitrary output resolutions. • U-Transformer produces visually appealing results against the state-of-the-art image outpainting networks. It also outperforms the competitors in most terms of quantitative results. [ABSTRACT FROM AUTHOR]
- Subjects :
- *GENERATIVE adversarial networks
*VISUAL cryptography
Subjects
Details
- Language :
- English
- ISSN :
- 08936080
- Volume :
- 162
- Database :
- Academic Search Index
- Journal :
- Neural Networks
- Publication Type :
- Academic Journal
- Accession number :
- 163229517
- Full Text :
- https://doi.org/10.1016/j.neunet.2023.02.021