Search

Multispectral imaging systems with a multispectral filter array (MSFA) provide an affordable and portable way to capture multispectral images (MSIs) that have a variety of applications in different fields. These systems generate raw images initially and require an effective multispectral image demosaicking technique for reconstructing the MSI from the raw data. These demosaicking methods require proper usage of the spectral correlation available between the bands of MSI to generate high quality MSI. However, the existing demosaicking methods only partially utilize this spectral correlation, as they use spectral correlation only in the initial estimation of MSI. In this work, we utilize the spectral correlation between bands iteratively and finally enhance the quality of the generated image using median filtering based image enhancement. The exploratory results on the two standard datasets publicise the quality of the presented method on the various metrics. [ABSTRACT FROM AUTHOR]

Demosaicking is a popular scientific area that is being explored by a vast number of scientists. Current digital imaging technologies capture colour images with a single monochrome sensor. In addition, the colour images were captured using a sensor coupled with a Colour Filter Array (CFA). Furthermore, the demosaicking procedure is required to obtain a full-colour image. Image denoising and image demosaicking are the two important image restoration techniques, which have increased popularity in recent years. Finding a suitable strategy for multiple image restoration is critical for researchers. Hence, a deep learning (DL) based image denoising and image demosaicking is developed in this research. Moreover, the Autoregressive Circle Wave Optimization (ACWO) based Demosaicking Convolutional Neural Network (DMCNN) is designed for image demosaicking. The Quantum Wavelet Transform (QWT) is used in the image denoising process. Similarly, Quantum Wavelet Transform (QWT) is used to analyse the abrupt changes in the input image with noise. The transformed image is then subjected to a thresholding technique, which determines an appropriate threshold range. Once the threshold range has been determined, soft thresholding is applied to the resulting wavelet coefficients. After that, the extraction and reconstruction of the original image is carried out using the Inverse Quantum Wavelet Transform (IQWT). Finally, the fused image is created by combining the results of both processes using a weighted average. The denoised and demosaicked images are combined using the weighted average technique. Furthermore, the proposed QWT+DMCNN-ACWO model provided the ideal values of Peak signal-to-noise ratio (PSNR), Second derivative like measure of enhancement (SDME), Structural Similarity Index (SSIM), Figure of Merit (FOM) of 0.890, and computational time of 49.549 dB, 59.53 dB, 0.963, 0.890, and 0.571, respectively. [ABSTRACT FROM AUTHOR]

• The Pearson correlation coefficient was employed to model the relationship between polarization channels, and the impact of the degree of polarization on the correlation was characterized. • A novel interpolation method was proposed based on this correlation and achieves a higher Peak Signal-to-Noise ratio (PSNR) and improved visual effects. Division of Focal Plane (DoFP) polarimeters have demonstrated significant potential due to their ability to perform real-time imaging coupled with a simple structure. However, their sparse polarization sampling introduces instantaneous Field of View (IFoV) errors and challenges for image reconstruction quality. Existing demosaicking methods primarily consider individual polarization channels, limiting reconstruction precision. In this work, we propose a novel demosaicking approach leveraging correlations between polarization channels. Moreover, a direction criterion based on joint-channel-gradient is incorporated to reconstruct more accurate boundary features. Our method, as a balancing strategy, can be easily combined with other algorithms for further optimization. Evaluations using a publicly available database and actual DoFP polarimetric data demonstrate significant quality improvements over other advanced methods. Reconstructed images exhibit a noticeable enhancement of polarization-dependent details. [ABSTRACT FROM AUTHOR]

Polarimetric imaging has been considered one of the crucial research fields in tactical object recognition in the scene. The present paper addresses the detection and classification of different objects in the scene based on data of polarization cameras. Initially, a pre-treatment phase is conducted where the angle and the degree of polarization of each region of the scene are obtained by polarimetric images. After that stage, stokes-based parameters are extracted from the polarimetric images. In addition, Mueller matrix decomposition is also applied to extract attributes. In the classification step, we applied several machine-learning algorithms. These classification methods are mainly chosen due to their (i) high generalization performance with nonlinear data and (ii) powerfulness in big data separation. Finally, we applied these strategies to the publicly available PolaBot database from Bourgogne Franche-Comte University. The obtained results indicated that the proposed strategy accurately detected and differentiated between different classes in the scene. [ABSTRACT FROM AUTHOR]

In thisletter, we propose a new generic multispectral image demosaicking algorithm using adaptive spectral correlation. Our proposed algorithm defines the spectrally localized average image (SLAI) corresponding to each spectral band, which has a strong spectral correlation with the pixel values of the corresponding spectral band in the raw image captured using a single sensor. The proposed algorithm uses the SLAI to estimate the missing pixel values of the spectral bands. Experimental results reveal that our algorithm outperforms other state-of-the-art generic multispectral image demosaicking algorithms in terms of objective and subjective evaluations. [ABSTRACT FROM AUTHOR]

This paper develops an integrity protection technique based on image demosaicking. To provide the reversible property, the rebuilt components in color pixels are used for secret data embedding. The random value and the hashed result of the most significant high-order bits of the color components are employed to generate the authentication code for each rebuilt component. In addition, the optimal single bit map block truncation coding (OSBMBTC) technique is adopted for generating recovery codes. A block-based image recovery procedure is designed to reconstruct the modified areas. Experimental results reveal that the illegally tampered objects in the test images can be found in these tests even when 87.5% pixels are modified. From the results, the image quality of the recovery image is acceptable when 50% of pixels are modified. Moreover, the demosaicked image can be reversibly constructed when the embedded image is not tampered. [ABSTRACT FROM AUTHOR]

In this work, we show that an image reconstruction from a burst of individually demosaicked RAW captures propagates demosaicking artifacts throughout the image processing pipeline. Hence, we propose a joint regularization scheme for burst denoising and demosaicking. We model the burst alignment functions and the color filter array sampling functions into one linear operator. Then, we formulate the individual burst reconstruction and the demosaicking problems into a three-color-channel optimization problem. We introduce a crosschannel prior to the solution of this optimization problem and develop a numerical solver via alternating direction method of multipliers. Moreover, our proposed method avoids the complexity of alignment estimation as a preprocessing step for burst reconstruction. It relies on a phase correlation approach in the Fourier’s domain to efficiently find the relative translation, rotation, and scale among the burst captures and to perform warping accordingly. As a result of these steps, the proposed joint burst denoising and demosaicking solution improves the quality of reconstructed images by a considerable margin compared to existing image model-based methods.

As an important part of image processing, image demosaicking has been widely researched. It is especially necessary to propose an efficient interpolation algorithm with good visual quality and performance. To improve the limitations of residual interpolation (RI), based on RI algorithm, minimalized-Laplacian RI (MLRI), and iterative RI (IRI), this paper focuses on adaptive RI (ARI) and proposes an improved ARI (IARI) algorithm which obtains more distinct R, G, and B colors in the images. The proposed scheme fully considers the brightness information and edge information of the image. Since the ARI algorithm is not completely adaptive, IARI algorithm executes ARI algorithm twice on R and B components according to the directional difference, which surely achieves an adaptive algorithm for all color components. Experimental results show that the improved method has better performance than other four existing methods both in subjective assessment and objective assessment, especially in the complex edge area and color brightness recovery. [ABSTRACT FROM AUTHOR]

This paper presents a fast and effective polarization image demosaicking algorithm, which explores inter-channel dependency of Stokes parameters for the minimization of residual aliasing artifacts after cubic spline interpolation. A guided filtering approach is used for denoising. An optimization based on the confidence level of the aforementioned guided filtering, the correlations between the demosaicked image and input, as well as the total intensity, angle and degree of linear polarization, is constructed and solved with Newton’s method. Experimental results demonstrate that the proposed algorithm can surpass the existing methods in terms of both objective root mean squared error and structural similarity index by at least 36.0% and 3.4%, respectively, and by close visual inspection of the clarity of objects in the angle and degree of linear polarization images. The proposed algorithm consists of only convolutions and element-wise operations, making it fast and parallelizable for efficient GPU acceleration. An image of size $512\times 612\times 4$ can be processed within 10 s on i7-6700k CPU, and gains further 5 times speedup with M4000M GPU. [ABSTRACT FROM AUTHOR]

Area scan cameras are widely used in industrial manufacturing for quality controlling, production monitoring, or defect inspection. The area scan cameras are normally integrated in a single sensor covered by the Bayer color filter array for RGB signal capturing, and the raw image with one channel for each pixel is obtained in one exposure. Therefore, it is significant to reconstruct the other two channels at each pixel which is called demosaicking. In this article, one hybrid demosaicking algorithm is proposed based on fuzzy edge strength and residual interpolation (RI) method. We first demosaic the G channel with one edge-directed filter and refine the high textured details by using fuzzy edge strength interpolation. Then, the RI is employed to calculate R and B channels, where G is used as the guide image for computing residuals. In the experiment, we evaluate our proposed method with nine prevalent interpolation-based algorithms on the IMAX and Kodak image datasets. The experimental results demonstrate that the proposed algorithm outperform the others by synthesizing the performance in peak signal-to-noise ratio, structural similarity, visual comparison, and computational time. [ABSTRACT FROM AUTHOR]

In this article, we introduce a variational demosaicking model that restores full-color images from sampled data acquired with a RGBW color filter array (CFA). The proposed model employs the concept of the low-dimensional patch manifold model (LDMM) in Shi et al. (J Sci Comput 75(2):638–656, 2018) and inter-channel correlation terms. The LDMM enables regions without color data to be filled out smoothly from given sparse data, while conserving textures. Moreover, the inter-correlation terms defined in the gradient domain help diminish color artifacts in demosaicked images. We also present an efficient iterative algorithm for solving the proposed model. Numerical experiments validate the effectiveness of our model for demosaicking images acquired with RGBW CFAs as compared to the state-of-the-art models. [ABSTRACT FROM AUTHOR]

Image demosaicking has been an essential and challenging problem among the most crucial steps of image processing behind image sensors. Due to the rapid development of intelligent processors based on deep learning, several demosaicking methods based on a convolutional neural network (CNN) have been proposed. However, it is difficult for their networks to run in real-time on edge computing devices with a large number of model parameters. This paper presents a compact demosaicking neural network based on the UNet++ structure. The network inserts densely connected layer blocks and adopts Gaussian smoothing layers instead of down-sampling operations before the backbone network. The densely connected blocks can extract mosaic image features efficiently by utilizing the correlation between feature maps. Furthermore, the block adopts depthwise separable convolutions to reduce the model parameters; the Gaussian smoothing layer can expand the receptive fields without down-sampling image size and discarding image information. The size constraints on the input and output images can also be relaxed, and the quality of demosaicked images is improved. Experiment results show that the proposed network can improve the running speed by 42% compared with the fastest CNN-based method and achieve comparable reconstruction quality as it on four mainstream datasets. Besides, when we carry out the inference processing on the demosaicked images on typical deep CNN networks, Mobilenet v1 and SSD, the accuracy can also achieve 85.83% (top 5) and 75.44% (mAP), which performs comparably to the existing methods. The proposed network has the highest computing efficiency and lowest parameter number through all methods, demonstrating that it is well suitable for applications on modern edge computing devices.

Multispectral images with more than three spectral bands of electromagnetic spectrum are highly demanded and useful in various domains but practical use of these images are limited due to size and cost of multispectral imaging systems. The use of multispectral image demosaicking (MSID) and single sensor multispectral filter array (MSFA) have been explored to reduce the size and cost of these systems. However single sensor based MSFA demosaicked images suffer from quality degradation and thus their usage may be adversely affected. In this paper, we examine the efficacy of new multispectral imaging (MSI) systems using limited but more than one MSFA sensors. We compare the performance of single-MSFA sensor based MSI system with 2-sensor and 4-sensor based proposed MSI systems using different MSID methods and analyse on the basis of peak-signalto- noise ratio (PSNR) and structural similarity (SSIM) metrics. [ABSTRACT FROM AUTHOR]

In this paper, we propose an effective directional Bayer color filter array (CFA) demosaicking algorithm based on residual interpolation (RI). The proposed directional interpolation algorithm aims to reduce computational complexity and get more accurate interpolated pixel values in the complex edge areas. We use the horizontal and vertical weights to combine and smooth color difference estimations. Compared with four directional weights in minimized Laplacian residual interpolation, the proposed algorithm not only guarantees the quality of color images but also reduces the computational complexity. Generally, the directional estimations may be inaccurately calculated because of the false edge information in irregular edges. We alleviate it by using a new method to calculate the directional color difference estimations. Experimental results show that the proposed algorithm provides outstanding performance compared with some previous algorithms, especially in the complex edge areas. In addition, it has lower computational complexity and better visual effect. [ABSTRACT FROM AUTHOR]

In this paper, we propose a weights-based image demosaicking algorithm which is based on the Bayer pattern color filter array (CFA). When reconstructing the missing G components, the proposed algorithm uses weights based on posteriori gradients to mitigate color artifacts and distortions. Furthermore, the proposed algorithm makes full use of the correlation of R−B channels in high frequency when interpolating R/B values at B/R positions. Experimental results show that the proposed algorithm is superior to previous similar algorithms in composite peak signal-to-noise ratio (CPSNR) and subjective visual effect. The biggest advantage of the proposed algorithm is the use of posteriori gradients and the correlation of R−B channels in high frequency.

Digital cameras generally use a single image sensor which surface is covered by a color filter array. The Color Filter Array (CFA) limits each sensor pixel by sampling one of the three primary color values (red, green or blue), whereas the other two missing color values would be acquired by the post-processing procedure called demosaicking. From the noisy CFA data, the full color images are reconstructed through an imaging pipeline of demosaicking and denoising. However, image denoising in the RGB space has expensive computation cost. In this paper, to increase the efficiency and the color fidelity, we propose a novel joint denoising and demosaicking strategy to reconstruct the noiseless full color image from the input noisy CFA data. The low-rank approximation technique is first used to remove the noise from CFA data. Then, image demosaicking using both color difference space and signal correlation are applied to the denoised CFA data to obtain the noise-less full color image. The experimental results show that the proposed algorithm not only improves the quality of full color image but also outperforms the existing state-of-the-art methods both subjectively and objectively. [ABSTRACT FROM AUTHOR]

Based on a deep understanding of all-phase digital filter (APDF) design and all-phase biorthogonal transform (APBT), this paper will further study the windowed all-phase digital filter (WAPDF) and windowed all-phase biorthogonal transform (WAPBT), discuss the principle of the WAPBT, and provide a unified construction method of the all-phase transform (APT). Based on a type of orthogonal transform, i.e., discrete sine transform (DST), an interpolation filter called an all-phase DST (APDST) filter is constructed and used for image demosaicking; it is compared with bilinear interpolation and all-phase inverse discrete cosine transform (APIDCT) interpolation filters, to test its performance in image interpolation and provide analysis and discussion. The experimental results show that APIDCT and APDST filters with a size of 7 × 7 are similar in interpolation performance, but better than the bilinear interpolation method. In addition to its use in image interpolation demosaicking, the low-pass filter designed in this paper can also be widely used in image interpolation, image denoising, image resizing, and other fields of image processing.

In this paper, we propose a new demosaicking algorithm which uses eight-directional weights based on the gradient of color difference (EWGCD) for Bayer image demosaicking. To obtain the interpolation of green (G) pixels, the eight-directional G pixel values are first estimated in red (R)/blue (B) pixels. This estimate is used to calculate the color difference in R/B pixels of the Bayer image in diagonal directions. However, in horizontal and vertical directions, the new estimated G pixels are defined to obtain the color difference. The eight-directional weights of estimated G pixels can be obtained by considering the gradient of the color difference and the gradient of the RGB pixels of the Bayer image. Therefore, the eight-directional weighted values and the first estimated G pixel values are combined to obtain the full G image. Compared with six similar algorithms using the same eighteen McMaster images, the results of the experiment demonstrate that the proposed algorithm has a better performance not only in the subjective visual measurement but also in the assessments of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) index measurement.

In this paper, we propose an effective directional Bayer color filter array (CFA) demosaicking algorithm based on residual interpolation (RI). The proposed directional interpolation algorithm aims to reduce computational complexity and get more accurate interpolated pixel values in the complex edge areas. We use the horizontal and vertical weights to combine and smooth color difference estimations. Compared with four directional weights in minimized Laplacian residual interpolation, the proposed algorithm not only guarantees the quality of color images but also reduces the computational complexity. Generally, the directional estimations may be inaccurately calculated because of the false edge information in irregular edges. We alleviate it by using a new method to calculate the directional color difference estimations. Experimental results show that the proposed algorithm provides outstanding performance compared with some previous algorithms, especially in the complex edge areas. In addition, it has lower computational complexity and better visual effect.

Single imaging sensor-based multispectral imaging systems (MSISs) facilitate snapshot imaging with a multispectral filter array (MSFA). These single sensor-based MSISs need an effective image demosaicking method to generate the full-multispectral image from the mosaicked image captured by an imaging sensor with MSFA help. This paper proposes an effective and generic multispectral image demosaicking (MSID) method based on the concept of the pseudo-panchromatic (PPAN) image. The proposed MSID method accepts the mosaicked image capture with the help of preferred binary tree-based MSFA patterns. We estimate the PPAN image from the mosaicked image by utilizing the spatial filters developed carefully based on the spectral band's relative placement in the binary tree-based MSFA pattern. Our proposed MSID method utilizes the spectral correlation between the PPAN image and the mosaicked image to generate the multispectral image. The proposed generic MSID method surpasses the existing generic MSID methods in different image quality parameters on the multispectral images from two benchmark multispectral datasets. • PPAN image exhibits strong spectral correlation with each spectral bands. • Use spatial filters to generate PPAN image from the mosaicked image. • Inter-band correlation in multiple directions achieves better performance in MSID. [ABSTRACT FROM AUTHOR]

This paper presents a new algorithm for demosaicking images captured through a color filter array (CFA). The objective of the CFA demosaicking is to render a full color image from the mosaicked image. This is commonly achieved by estimating the missing color information from the surrounding observed pixels. In this paper, we integrate the observation that color images have a strong intrachannel spatial correlation in low-frequency components and a dominant interchannel correlation in the high-frequency components. A new framework is proposed to utilize this information where the missing pixels in each color channel are estimated from the wavelet subbands. A modified median-filtering operation is then applied in the subband domain. The algorithm is adaptive and produces superior full-resolution images when compared with other methods. [ABSTRACT FROM AUTHOR]

The author begins by discussing the image formation process. The demosaicking methods are examined in three groups: the first group consists of heuristic approaches. The second group formulates demosaicking as a restoration problem. The third group is a generalization that uses the spectral filtering model given in Wandell. [ABSTRACT FROM PUBLISHER]

Image demosaicking has been an essential and challenging problem among the most crucial steps of image processing behind image sensors. Due to the rapid development of intelligent processors based on deep learning, several demosaicking methods based on a convolutional neural network (CNN) have been proposed. However, it is difficult for their networks to run in real-time on edge computing devices with a large number of model parameters. This paper presents a compact demosaicking neural network based on the UNet++ structure. The network inserts densely connected layer blocks and adopts Gaussian smoothing layers instead of down-sampling operations before the backbone network. The densely connected blocks can extract mosaic image features efficiently by utilizing the correlation between feature maps. Furthermore, the block adopts depthwise separable convolutions to reduce the model parameters; the Gaussian smoothing layer can expand the receptive fields without down-sampling image size and discarding image information. The size constraints on the input and output images can also be relaxed, and the quality of demosaicked images is improved. Experiment results show that the proposed network can improve the running speed by 42% compared with the fastest CNN-based method and achieve comparable reconstruction quality as it on four mainstream datasets. Besides, when we carry out the inference processing on the demosaicked images on typical deep CNN networks, Mobilenet v1 and SSD, the accuracy can also achieve 85.83% (top 5) and 75.44% (mAP), which performs comparably to the existing methods. The proposed network has the highest computing efficiency and lowest parameter number through all methods, demonstrating that it is well suitable for applications on modern edge computing devices. [ABSTRACT FROM AUTHOR]

• We proposed a new spectral signal reconstruction method for a multispectral imaging capturing system with nine spectral bands. • We design a new filter array in a 4×4 MSFA pattern and propose a multispectral demosaicking algorithm for recovering the sparse spectral data. • The experimental results demonstrate that our proposed demosaicking algorithm outperforms the other methods in PSNR, SSIM, and subjective evaluation. Multispectral filter array (MSFA) imaging with one single sensor is a portable and inexpensive means of acquiring spectral image which is widely used for object detection, material analysis and mechanical system diagnosis. The most challenging task for MSFA imaging is the multispectral demosaicking with the aim of reconstructing the captured raw/mosaic image, especially for the systems with many bands which result in higher sparseness of the raw data. In this paper, we present a 9-band MSFA imaging system in a repetitive 4 × 4 filter array on a single sensor, and propose a demosaicking algorithm for reconstructing the raw spectral image. Within the 4 × 4 MSFA pattern, the fifth spectral band takes up half of the total spatial position while the remaining eight bands occupy 1/16 respectively. To reconstruct the sparse raw data, we first recover the fifth band by propagating the neighboring sampled pixels to the unsampled position using the image gradients, and then employ the reconstructed fifth band as a guided image to demosaick the other bands with the guided filter and residual interpolation. Finally, we estimate the spectral reflectance values from the multispectral image and the characterization matrix. In the experiment, we evaluate the performance of the 9-band imaging system with the binary tree-based edge-sensing (BTES) algorithm, compressed sensing (CS) algorithm, and our proposed demosaicking algorithm. The experiment results demonstrate that our demosaicking algorithm not only outperforms BTES and CS algorithms in terms of objective image quality, e.g., PSNR values and spectral errors, but also reduces the demosaicking artifacts in terms of subjective evaluations. [ABSTRACT FROM AUTHOR]

In this paper, we propose a weights-based image demosaicking algorithm which is based on the Bayer pattern color filter array (CFA). When reconstructing the missing G components, the proposed algorithm uses weights based on posteriori gradients to mitigate color artifacts and distortions. Furthermore, the proposed algorithm makes full use of the correlation of R–B channels in high frequency when interpolating R/B values at B/R positions. Experimental results show that the proposed algorithm is superior to previous similar algorithms in composite peak signal-to-noise ratio (CPSNR) and subjective visual effect. The biggest advantage of the proposed algorithm is the use of posteriori gradients and the correlation of R–B channels in high frequency. [ABSTRACT FROM AUTHOR]

Based on a deep understanding of all-phase digital filter (APDF) design and all-phase biorthogonal transform (APBT), this paper will further study the windowed all-phase digital filter (WAPDF) and windowed all-phase biorthogonal transform (WAPBT), discuss the principle of the WAPBT, and provide a unified construction method of the all-phase transform (APT). Based on a type of orthogonal transform, i.e., discrete sine transform (DST), an interpolation filter called an all-phase DST (APDST) filter is constructed and used for image demosaicking; it is compared with bilinear interpolation and all-phase inverse discrete cosine transform (APIDCT) interpolation filters, to test its performance in image interpolation and provide analysis and discussion. The experimental results show that APIDCT and APDST filters with a size of 7 × 7 are similar in interpolation performance, but better than the bilinear interpolation method. In addition to its use in image interpolation demosaicking, the low-pass filter designed in this paper can also be widely used in image interpolation, image denoising, image resizing, and other fields of image processing. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new demosaicking algorithm which uses eight-directional weights based on the gradient of color difference (EWGCD) for Bayer image demosaicking. To obtain the interpolation of green (G) pixels, the eight-directional G pixel values are first estimated in red (R)/blue (B) pixels. This estimate is used to calculate the color difference in R/B pixels of the Bayer image in diagonal directions. However, in horizontal and vertical directions, the new estimated G pixels are defined to obtain the color difference. The eight-directional weights of estimated G pixels can be obtained by considering the gradient of the color difference and the gradient of the RGB pixels of the Bayer image. Therefore, the eight-directional weighted values and the first estimated G pixel values are combined to obtain the full G image. Compared with six similar algorithms using the same eighteen McMaster images, the results of the experiment demonstrate that the proposed algorithm has a better performance not only in the subjective visual measurement but also in the assessments of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) index measurement. [ABSTRACT FROM AUTHOR]