Search

Image demosaicking has been an essential and challenging problem among the most crucial steps of image processing behind image sensors. Due to the rapid development of intelligent processors based on deep learning, several demosaicking methods based on a convolutional neural network (CNN) have been proposed. However, it is difficult for their networks to run in real-time on edge computing devices with a large number of model parameters. This paper presents a compact demosaicking neural network based on the UNet++ structure. The network inserts densely connected layer blocks and adopts Gaussian smoothing layers instead of down-sampling operations before the backbone network. The densely connected blocks can extract mosaic image features efficiently by utilizing the correlation between feature maps. Furthermore, the block adopts depthwise separable convolutions to reduce the model parameters, the Gaussian smoothing layer can expand the receptive fields without down-sampling image size and discarding image information. The size constraints on the input and output images can also be relaxed, and the quality of demosaicked images is improved. Experiment results show that the proposed network can improve the running speed by 42% compared with the fastest CNN-based method and achieve comparable reconstruction quality as it on four mainstream datasets. Besides, when we carry out the inference processing on the demosaicked images on typical deep CNN networks, Mobilenet v1 and SSD, the accuracy can also achieve 85.83% (top 5) and 75.44% (mAP), which performs comparably to the existing methods. The proposed network has the highest computing efficiency and lowest parameter number through all methods, demonstrating that it is well suitable for applications on modern edge computing devices.

This paper presents a fast and effective polarization image demosaicking algorithm, which explores inter-channel dependency of Stokes parameters for the minimization of residual aliasing artifacts after cubic spline interpolation. A guided filtering approach is used for denoising. An optimization based on the confidence level of the aforementioned guided filtering, the correlations between the demosaicked image and input, as well as the total intensity, angle and degree of linear polarization, is constructed and solved with Newton’s method. Experimental results demonstrate that the proposed algorithm can surpass the existing methods in terms of both objective root mean squared error and structural similarity index by at least 36.0% and 3.4%, respectively, and by close visual inspection of the clarity of objects in the angle and degree of linear polarization images. The proposed algorithm consists of only convolutions and elementwise operations, making it fast and parallelizable for efficient GPU acceleration. An image of size 512×612×4 can be processed within 10 s on i7-6700k CPU, and gains further 5 times speedup with M4000M GPU. Ministry of Education (MOE) Accepted version This work was supported in part by the Nanjing University of Aeronautics and Astronautics, Nanjing, China, under Grant 56Y AH18043 and Grant PAC19009 and in part by the Singapore Ministry of Education AcRF Tier 1 under Grant MOE2018-T1- 001-131, RG87/18-(S).

Interpolation from a Color Filter Array (CFA) is the most common method for obtaining full color image data. Its success relies on the smart combination of a CFA and a demosaicing algorithm. Demosaicing on the one hand has been extensively studied. Algorithmic development in the past 20 years ranges from simple linear interpolation to modern neural-network-based (NN) approaches that encode the prior knowledge of millions of training images to fill in missing data in an inconspicious way. CFA design, on the other hand, is less well studied, although still recognized to strongly impact demosaicing performance. This is because demosaicing algorithms are typically limited to one particular CFA pattern, impeding straightforward CFA comparison. This is starting to change with newer classes of demosaicing that may be considered generic or CFA-agnostic. In this study, by comparing performance of two state-of-the-art generic algorithms, we evaluate the potential of modern CFA-demosaicing. We test the hypothesis that, with the increasing power of NN-based demosaicing, the influence of optimal CFA design on system performance decreases. This hypothesis is supported with the experimental results. Such a finding would herald the possibility of relaxing CFA requirements, providing more freedom in the CFA design choice and producing high-quality cameras.

In this paper, we propose a weights-based image demosaicking algorithm which is based on the Bayer pattern color filter array (CFA). When reconstructing the missing G components, the proposed algorithm uses weights based on posteriori gradients to mitigate color artifacts and distortions. Furthermore, the proposed algorithm makes full use of the correlation of R&ndash, B channels in high frequency when interpolating R/B values at B/R positions. Experimental results show that the proposed algorithm is superior to previous similar algorithms in composite peak signal-to-noise ratio (CPSNR) and subjective visual effect. The biggest advantage of the proposed algorithm is the use of posteriori gradients and the correlation of R&ndash, B channels in high frequency.

Based on a deep understanding of all-phase digital filter (APDF) design and all-phase biorthogonal transform (APBT), this paper will further study the windowed all-phase digital filter (WAPDF) and windowed all-phase biorthogonal transform (WAPBT), discuss the principle of the WAPBT, and provide a unified construction method of the all-phase transform (APT). Based on a type of orthogonal transform, i.e., discrete sine transform (DST), an interpolation filter called an all-phase DST (APDST) filter is constructed and used for image demosaicking, it is compared with bilinear interpolation and all-phase inverse discrete cosine transform (APIDCT) interpolation filters, to test its performance in image interpolation and provide analysis and discussion. The experimental results show that APIDCT and APDST filters with a size of 7 &times, 7 are similar in interpolation performance, but better than the bilinear interpolation method. In addition to its use in image interpolation demosaicking, the low-pass filter designed in this paper can also be widely used in image interpolation, image denoising, image resizing, and other fields of image processing.

In this paper, we propose a new demosaicking algorithm which uses eight-directional weights based on the gradient of color difference (EWGCD) for Bayer image demosaicking. To obtain the interpolation of green (G) pixels, the eight-directional G pixel values are first estimated in red (R)/blue (B) pixels. This estimate is used to calculate the color difference in R/B pixels of the Bayer image in diagonal directions. However, in horizontal and vertical directions, the new estimated G pixels are defined to obtain the color difference. The eight-directional weights of estimated G pixels can be obtained by considering the gradient of the color difference and the gradient of the RGB pixels of the Bayer image. Therefore, the eight-directional weighted values and the first estimated G pixel values are combined to obtain the full G image. Compared with six similar algorithms using the same eighteen McMaster images, the results of the experiment demonstrate that the proposed algorithm has a better performance not only in the subjective visual measurement but also in the assessments of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) index measurement.

In this paper, we propose an effective directional Bayer color filter array (CFA) demosaicking algorithm based on residual interpolation (RI). The proposed directional interpolation algorithm aims to reduce computational complexity and get more accurate interpolated pixel values in the complex edge areas. We use the horizontal and vertical weights to combine and smooth color difference estimations. Compared with four directional weights in minimized Laplacian residual interpolation, the proposed algorithm not only guarantees the quality of color images but also reduces the computational complexity. Generally, the directional estimations may be inaccurately calculated because of the false edge information in irregular edges. We alleviate it by using a new method to calculate the directional color difference estimations. Experimental results show that the proposed algorithm provides outstanding performance compared with some previous algorithms, especially in the complex edge areas. In addition, it has lower computational complexity and better visual effect.