OccGAN: Semantic image augmentation for driving scenes.

• The OccGAN structure is a semantic augmentation method on Cityscapes. • The Rationality Module utilizes prior knowledge to implant occluders. • The Authenticity Module ensures the plausibility by a generative adversarial network. • Our Method improves the performance of several SOTA algorithms. Difficult images with complicated environments and occlusion have significant impacts on the performance of algorithms. They obey the long-tail distribution in the widely used datasets, which results in rare samples being overwhelmed during training. This paper presents a new approach to generate plausible occluded images with annotation as a kind of data augmentation with scenes semantics. To achieve this task, we proposed the Occlusion-based Generative Adversarial Network (OccGAN) structure, which consists of a Rationality Module and an Authenticity Module. The Rationality Module generated preliminary occluded samples under the guidance of prior semantic knowledge. And the Authenticity Module is a generative adversarial structure to ensure the reality of the produced images. Qualitative results of the visualization process are given to verify the ablation study. Experiments on the semantic segmentation task indicate that several state-of-the-art algorithms combined with our OccGAN such as DRN, Deeplabv3+, PSPNet and ResNet-38, have boosts on IoU class scores and IoU category scores successfully. [ABSTRACT FROM AUTHOR]