CNN-based pre-processing and multi-frame-based view transformation for fisheye camera-based AVM system

The edges of the wide angle (WA) image generally have poor definition and resolution, which often causes deterioration of the around view monitor (AVM) image quality. This paper proposes a convolutional neural network (CNN)-based preprocessing and a multi-frame-based view transformation to solve this problem, and presents an AVM system based on these methods. First, we analyze the general distortion characteristics of the WA image, and propose a preprocessing using the CNN learning model based on the analysis result. Next, in the view transformation (VT) of the outer edge of the WA image, the inherent problem of low pixel density is solved through motion compensation and hole filling using adjacent frames. Experimental results show that the AVM images by the proposed methods are superior to general AVM images in terms of objective image quality as well as subjective image quality.