A novel stereo image self-inpainting network for autonomous robots.

Recently, vision-based methods have exhibited promising prospects in assigning autonomous robots more capabilities for better environment perception. Since visual sensors are easily affected under extreme conditions, current image inpainting methods based on CNNs jointly with generative adversarial networks (GANs) usually generate patches quite different from the ground truth (GT), which is harmful to autonomous robots. In this paper, we propose a novel multiscale feature alignment module with an early fusion strategy to align the left and right feature maps to better capture the motion cues between them. Then, the aligned features are fused to fill holes in the left image. To aggregate the multiscale feature maps dynamically, we propose a multiscale feature aggregation module based on an attention mechanism, of which the fusion module is designed as a symmetrical architecture to adaptively incorporate the complementary contextual correlations from different feature branches. In addition, a spatial attention module able to capture the correlations among pixels is introduced into our network to enhance the inpainting capacity and generate more refined details. To evaluate the effectiveness of our proposed method, many experiments are conducted on a stereo image dataset. The quantitative and qualitative results show that our method significantly outperforms the recent state-of-the-art image inpainting methods while running over 22 fps on a single NVIDIA RTX2080Ti GPU. • A new image inpainting method which utilizes additional image as constraints to assist the inpainting process is proposed. • The multi-scale feature alignment module based on deformable convolution can realize the feature alignment of different images. • Implicitly detect and restore damaged regions by exploiting attention maps. • Image textures and features are refined through a long-range spatial attention mechanism. [ABSTRACT FROM AUTHOR]