51. Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping
- Author
-
Huan Fu, Dacheng Tao, Kayhan Batmanghelich, Kun Zhang, Chaohui Wang, Mingming Gong, UBTECH Sydney Artificial IntelIigence Centre, The University of Sydney, Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pennsylvania Commonwealth System of Higher Education (PCSHE)-Pennsylvania Commonwealth System of Higher Education (PCSHE), Laboratoire d'Informatique Gaspard-Monge (LIGM), École des Ponts ParisTech (ENPC)-Centre National de la Recherche Scientifique (CNRS)-Université Gustave Eiffel, Department of Philosophy, Carnegie Mellon University [Pittsburgh] (CMU), and This research was supported by Australian Research Council Projects FL-170100117, DP-180103424, and IH-180100002. This work was partially supported by SAP SE and CNRS INS2I-JCJC-INVISANA. This work is partially supported by NIH Award Number 1R01HL141813-01, NSF 1839332 Tripod+X, and SAP SE. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU used for this research. We were also grateful for the computational resources provided by Pittsburgh Super Computing grant number TGASC170024.
- Subjects
FOS: Computer and information sciences ,Computer science ,business.industry ,Computer Vision and Pattern Recognition (cs.CV) ,Deep learning ,05 social sciences ,Geometric transformation ,Computer Science - Computer Vision and Pattern Recognition ,Function (mathematics) ,010501 environmental sciences ,01 natural sciences ,Article ,Domain (software engineering) ,Image (mathematics) ,Constraint (information theory) ,Consistency (database systems) ,0502 economics and business ,[INFO]Computer Science [cs] ,Artificial intelligence ,050207 economics ,business ,Algorithm ,Transformation geometry ,0105 earth and related environmental sciences - Abstract
Unsupervised domain mapping aims to learn a function to translate domain X to Y by a function GXY in the absence of paired examples. Finding the optimal GXY without paired data is an ill-posed problem, so appropriate constraints are required to obtain reasonable solutions. One of the most prominent constraints is cycle consistency, which enforces the translated image by GXY to be translated back to the input image by an inverse mapping GYX. While cycle consistency requires the simultaneous training of GXY and GY X, recent studies have shown that one-sided domain mapping can be achieved by preserving pairwise distances between images. Although cycle consistency and distance preservation successfully constrain the solution space, they overlook the special properties that simple geometric transformations do not change the semantic structure of images. Based on this special property, we develop a geometry-consistent generative adversarial network (GcGAN), which enables one-sided unsupervised domain mapping. GcGAN takes the original image and its counterpart image transformed by a predefined geometric transformation as inputs and generates two images in the new domain coupled with the corresponding geometry-consistency constraint. The geometry-consistency constraint reduces the space of possible solutions while keep the correct solutions in the search space. Quantitative and qualitative comparisons with the baseline (GAN alone) and the state-of-the-art methods including CycleGAN and DistanceGAN demonstrate the effectiveness of our method.
- Published
- 2019