567 results on '"demosaicing"'
Search Results
2. An Edge-Preserving Regularization Model for the Demosaicing of Noisy Color Images.
- Author
-
Boccuto, Antonio, Gerace, Ivan, Giorgetti, Valentina, Martinelli, Francesca, and Tonazzini, Anna
- Abstract
This paper proposes an edge-preserving regularization technique to solve the color image demosaicing problem in the realistic case of noisy data. We enforce intra-channel local smoothness of the intensity (low-frequency components) and inter-channel local similarity of the depth of object borders and textures (high-frequency components). Discontinuities of both the low-frequency and high-frequency components are accounted for implicitly, i.e., through suitable functions of the proper derivatives. For the treatment of even the finest image details, derivatives of first, second, and third orders are considered. The solution to the demosaicing problem is defined as the minimizer of an energy function, accounting for all these constraints plus a data fidelity term. This non-convex energy is minimized via an iterative deterministic algorithm, applied to a family of approximating functions, each implicitly referring to geometrically consistent image edges. Our method is general because it does not refer to any specific color filter array. However, to allow quantitative comparisons with other published results, we tested it in the case of the Bayer CFA and on the Kodak 24-image dataset, the McMaster (IMAX) 18-image dataset, the Microsoft Demosaicing Canon 57-image dataset, and the Microsoft Demosaicing Panasonic 500-image dataset. The comparisons with some of the most recent demosaicing algorithms show the good performance of our method in both the noiseless and noisy cases. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
3. The Effect of Varying the Light Spectrum of a Scene on the Localisation of Photogrammetric Features.
- Author
-
Burdziakowski, Pawel
- Subjects
- *
MONOCHROMATIC light , *LARGE deviations (Mathematics) , *ALGORITHMS , *COLOR , *PHOTOGRAMMETRY , *DIGITAL photogrammetry , *IMAGE registration - Abstract
In modern digital photogrammetry, an image is usually registered via a digital matrix with an array of colour filters. From the registration of the image until feature points are detected on the image, the image is subjected to a series of calculations, i.e., demosaicing and conversion to greyscale, among others. These algorithms respond differently to the varying light spectrum of the scene, which consequently results in the feature location changing. In this study, the effect of scene illumination on the localisation of a feature in an image is presented. The demosaicing and greyscale conversion algorithms that produce the largest and smallest deviation of the feature from the reference point were assessed. Twelve different illumination settings from polychromatic light to monochromatic light were developed and performed, and five different demosaicing algorithms and five different methods of converting a colour image to greyscale were analysed. A total of 300 different cases were examined. As the study shows, the lowest deviation in the polychromatic light domain was achieved for light with a colour temperature of 5600 K and 5000 K, while in the monochromatic light domain, it was achieved for light with a green colour. Demosaicing methods have a significant effect on the localisation of a feature, and so the smallest feature deviation was achieved for smooth hue-type demosaicing, while for greyscale conversion, it was achieved for the mean type. Demosaicing and greyscale conversion methods for monochrome light had no effect. The article discusses the problem and concludes with recommendations and suggestions in the area of illuminating the scene with artificial light and the application of the algorithms, in order to achieve the highest accuracy using photogrammetric methods. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
4. How to best combine demosaicing and denoising?
- Author
-
Guo, Yu, Jin, Qiyu, Morel, Jean-Michel, and Facciolo, Gabriele
- Subjects
CONVOLUTIONAL neural networks ,IMAGE denoising ,MATHEMATICAL analysis ,IMAGE reconstruction - Abstract
Image demosaicing and denoising play a critical role in the raw imaging pipeline. These processes have often been treated as independent, without considering their interactions. Indeed, most classic denoising methods handle noisy RGB images, not raw images. Conversely, most demosaicing methods address the demosaicing of noise free images. The real problem is to jointly denoise and demosaic noisy raw images. But the question of how to proceed is still not clarified. In this paper, we carry out extensive experiments and a mathematical analysis to tackle this problem by low complexity algorithms. Indeed, both problems have only been addressed jointly by end-to-end heavy-weight convolutional neural networks (CNNs), which are currently incompatible with low-power portable imaging devices and remain by nature domain (or device) dependent. Our study leads us to conclude that, with moderate noise, demosaicing should be applied first, followed by denoising. This requires a simple adaptation of classic denoising algorithms to demosaiced noise, which we justify and specify. Although our main conclusion is 'demosaic first, then denoise, ' we also discover that for high noise, there is a moderate PSNR gain by a more complex strategy: partial CFA denoising followed by demosaicing and by a second denoising on the RGB image. These surprising results are obtained by a black-box optimization of the pipeline, which could be applied to any other pipeline. We validate our results on simulated and real noisy CFA images obtained from several benchmarks. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
5. SDAT: Sub-Dataset Alternation Training for Improved Image Demosaicing
- Author
-
Yuval Becker, Raz Z. Nossek, and Tomer Peleg
- Subjects
Demosaicing ,image restoration ,inductive bias ,Electrical engineering. Electronics. Nuclear engineering ,TK1-9971 - Abstract
Image demosaicing is an important step in the image processing pipeline for digital cameras. In data centric approaches, such as deep learning, the distribution of the dataset used for training can impose a bias on the networks' outcome. For example, in natural images most patches are smooth, and high-content patches are much rarer. This can lead to a bias in the performance of demosaicing algorithms. Most deep learning approaches address this challenge by utilizing specific losses or designing special network architectures. We propose a novel approach SDAT, Sub-Dataset Alternation Training, that tackles the problem from a training protocol perspective. SDAT is comprised of two essential phases. In the initial phase, we employ a method to create sub-datasets from the entire dataset, each inducing a distinct bias. The subsequent phase involves an alternating training process, which uses the derived sub-datasets in addition to training also on the entire dataset. SDAT can be applied regardless of the chosen architecture as demonstrated by various experiments we conducted for the demosaicing task. The experiments are performed across a range of architecture sizes and types, namely CNNs and transformers. We show improved performance in all cases. We are also able to achieve state-of-the-art results on three highly popular image demosaicing benchmarks.
- Published
- 2024
- Full Text
- View/download PDF
6. A Snapshot Multi-Spectral Demosaicing Method for Multi-Spectral Filter Array Images Based on Channel Attention Network.
- Author
-
Zhang, Xuejun, Dai, Yidan, Zhang, Geng, Zhang, Xuemin, and Hu, Bingliang
- Subjects
- *
CONVOLUTIONAL neural networks , *IMAGING systems , *SPECTRAL imaging , *IMAGE reconstruction , *MULTISPECTRAL imaging - Abstract
Multi-spectral imaging technologies have made great progress in the past few decades. The development of snapshot cameras equipped with a specific multi-spectral filter array (MSFA) allow dynamic scenes to be captured on a miniaturized platform across multiple spectral bands, opening up extensive applications in quantitative and visualized analysis. However, a snapshot camera based on MSFA captures a single band per pixel; thus, the other spectral band components of pixels are all missed. The raw images, which are captured by snapshot multi-spectral imaging systems, require a reconstruction procedure called demosaicing to estimate a fully defined multi-spectral image (MSI). With increasing spectral bands, the challenge of demosaicing becomes more difficult. Furthermore, the existing demosaicing methods will produce adverse artifacts and aliasing because of the adverse effects of spatial interpolation and the inadequacy of the number of layers in the network structure. In this paper, a novel multi-spectral demosaicing method based on a deep convolution neural network (CNN) is proposed for the reconstruction of full-resolution multi-spectral images from raw MSFA-based spectral mosaic images. The CNN is integrated with the channel attention mechanism to protect important channel features. We verify the merits of the proposed method using 5 × 5 raw mosaic images on synthetic as well as real-world data. The experimental results show that the proposed method outperforms the existing demosaicing methods in terms of spatial details and spectral fidelity. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
7. Pyramid Attention Network for Image Restoration.
- Author
-
Mei, Yiqun, Fan, Yuchen, Zhang, Yulun, Yu, Jiahui, Zhou, Yuqian, Liu, Ding, Fu, Yun, Huang, Thomas S., and Shi, Humphrey
- Subjects
- *
PYRAMIDS , *IMAGE denoising , *IMAGE reconstruction , *ATTENTION , *PROBLEM solving - Abstract
Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales. However, recent advanced deep convolutional neural network-based methods for image restoration do not take full advantage of self-similarities by relying on self-attention neural modules that only process information at the same scale. To solve this problem, we present a novel Pyramid Attention module for image restoration, which captures long-range feature correspondences from a multi-scale feature pyramid. Inspired by the fact that corruptions, such as noise or compression artifacts, drop drastically at coarser image scales, our attention module is designed to be able to borrow clean signals from their "clean" correspondences at the coarser levels. The proposed pyramid attention module is a generic building block that can be flexibly integrated into various neural architectures. Its effectiveness is validated through extensive experiments on multiple image restoration tasks: image denoising, demosaicing, compression artifact reduction, and super resolution. Without any bells and whistles, our PANet (pyramid attention module with simple network backbones) can produce state-of-the-art results with superior accuracy and visual quality. Our code is available at https://github.com/SHI-Labs/Pyramid-Attention-Networks [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
8. PyNET-Q×Q: An Efficient PyNET Variant for Q×Q Bayer Pattern Demosaicing in CMOS Image Sensors
- Author
-
Minhyeok Cho, Haechang Lee, Hyunwoo Je, Kijeong Kim, Dongil Ryu, and Albert No
- Subjects
Bayer filter ,color filter array (CFA) ,demosaicing ,image signal processor (ISP) ,knowledge distillation ,non-Bayer CFA ,Electrical engineering. Electronics. Nuclear engineering ,TK1-9971 - Abstract
Deep learning-based image signal processor (ISP) models for mobile cameras can generate high-quality images that rival those of professional DSLR cameras. However, their computational demands often make them unsuitable for mobile settings. Additionally, modern mobile cameras employ non-Bayer color filter arrays (CFA) such as Quad Bayer, Nona Bayer, and $\text{Q}\times \text{Q}$ Bayer to enhance image quality, yet most existing deep learning-based ISP (or demosaicing) models focus primarily on standard Bayer CFAs. In this study, we present PyNET- $\text{Q}\times \text{Q}$ , a lightweight demosaicing model specifically designed for $\text{Q}\times \text{Q}$ Bayer CFA patterns, which is derived from the original PyNET. We also propose a knowledge distillation method called progressive distillation to train the reduced network more effectively. Consequently, PyNET- $\text{Q}\times \text{Q}$ contains less than 2.5% of the parameters of the original PyNET while preserving its performance. Experiments using $\text{Q}\times \text{Q}$ images captured by a prototype $\text{Q}\times \text{Q}$ camera sensor show that PyNET- $\text{Q}\times \text{Q}$ outperforms existing conventional algorithms in terms of texture and edge reconstruction, despite its significantly reduced parameter count. Code and partial datasets can be found at https://github.com/Minhyeok01/PyNET-QxQ.
- Published
- 2023
- Full Text
- View/download PDF
9. Simultaneous Color Restoration and Depth Estimation in Light Field Imaging
- Author
-
Yongwei Li, Filiberto Pla, Marten Sjostrom, and Ruben Fernandez-Beltran
- Subjects
Light field ,demosaicing ,depth estimation ,Markov random field ,graph model ,Electrical engineering. Electronics. Nuclear engineering ,TK1-9971 - Abstract
Recent studies in the light field imaging have shown the potential and advantages of different light field information processes. In most of the existing techniques, the processing pipeline of light field has been treated in a step-by-step manner, and each step is considered to be independent from the others. For example, in light field color demosaicing, inferring the scene geometry is treated as an irrelevant and negligible task, and vice versa. Such processing techniques may fail due to the inherent connection among different steps, and result in both corrupted post-processing and defective pre-processing results. In this paper, we address the interaction between color interpolation and depth estimation in light field, and propose a probabilistic approach to handle these two processing steps jointly. This probabilistic framework is based on a Markov Random Fields —Collaborative Graph Model for simultaneous Demosaicing and Depth Estimation (CGMDD)—to explore the color-depth interdependence from general light field sampling. Experimental results show that both image interpolation quality and depth estimation can benefit from their interaction, mainly for processes such as image demosaicing which are shown to be sensitive to depth information, especially for light field sampling with large baselines.
- Published
- 2022
- Full Text
- View/download PDF
10. Multispectral single chip reconstruction using DNNs with application to open neurosurgery
- Author
-
Göb Stephan, Götz Theresa Ida, and Wittenberg Thomas
- Subjects
spectral reconstruction ,debayering ,demosaicing ,dcnn ,open neurosurgery ,Medicine - Abstract
Multispectral imaging devices incorporating up to 256 different spectral channels have recently become available for various healthcare applications, as e.g. laparoscopy, gastroscopy, dermatology or perfusion imaging for wound analysis. Currently, the use of such devices is limited due to very high investment costs and slow capture times. To compensate these shortcomings, single sensors with spectral masking on the pixel level have been proposed. Hence, adequate spectral reconstruction methods are needed. Within this work, two deep convolutional neural networks (DCNN) architectures for spectral image reconstruction from single sensors are compared with each other. Training of the networks is based on a huge collection of different MSI imagestacks, which have been subsampled, simulating 16-channel single sensors with spectral masking. We define a training, validation and test set (‘HITgoC’) resulting in 351 training (631.128 sub-images), 99 validation (163.272 sub-images) and 51 test images. For the application in the field of neurosurgery an additional testing set of 36 image stacks from the Nimbus data collection is used, depicting MSI brain data during open surgery. Two DCNN architectures were compared to bilinear interpolation (BI) and an intensity difference (ID) algorithm. The DCNNs (ResNet-Shinoda) were trained on HITgoC and consist of a preprocessing step using BI or ID and a refinement part using a ResNet structure. Similarity measures used were PSNR, SSIM and MSE between predicted and reference images. We calculated the similarity measures for HitgoC and Nimbus data and determined differences of the mean similarity measure values achieved with the ResNet-ID and baseline algorithms such as BI algorithm and ResNet-Shinoda. The proposed method achieved better results against BI in SSIM (.0644 vs. .0252), PSNR (15.3 dB vs. 9.1 dB) and 1-MSE*100 (.0855 vs. .0273) and compared to ResNet-Shinoda in SSIM (.0103 vs. .0074), PSNR (3.8 dB vs. 3.6 dB) and 1-MSE*100 (.0075 vs. .0047) for HITgoC/Nimbus. In this study, significantly better results for spectral reconstruction in MSI images of open neurosurgery was achieved using a combination of ID-interpolation and ResNet structure compared to standard methods.
- Published
- 2021
- Full Text
- View/download PDF
11. Deep Demosaicing for Polarimetric Filter Array Cameras.
- Author
-
Pistellato, Mara, Bergamasco, Filippo, Fatima, Tehreem, and Torsello, Andrea
- Subjects
- *
POLARIMETRY , *CAMERAS , *STOKES parameters , *IMAGE color analysis , *LIQUID crystal displays - Abstract
Polarisation Filter Array (PFA) cameras allow the analysis of light polarisation state in a simple and cost-effective manner. Such filter arrays work as the Bayer pattern for colour cameras, sharing similar advantages and drawbacks. Among the others, the raw image must be demosaiced considering the local variations of the PFA and the characteristics of the imaged scene. Non-linear effects, like the cross-talk among neighbouring pixels, are difficult to explicitly model and suggest the potential advantage of a data-driven learning approach. However, the PFA cannot be removed from the sensor, making it difficult to acquire the ground-truth polarization state for training. In this work we propose a novel CNN-based model which directly demosaics the raw camera image to a per-pixel Stokes vector. Our contribution is twofold. First, we propose a network architecture composed by a sequence of Mosaiced Convolutions operating coherently with the local arrangement of the different filters. Second, we introduce a new method, employing a consumer LCD screen, to effectively acquire real-world data for training. The process is designed to be invariant by monitor gamma and external lighting conditions. We extensively compared our method against algorithmic and learning-based demosaicing techniques, obtaining a consistently lower error especially in terms of polarisation angle. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
12. Deep Joint Demosaicing and High Dynamic Range Imaging Within a Single Shot.
- Author
-
Xu, Yilun, Liu, Ziyang, Wu, Xingming, Chen, Weihai, Wen, Changyun, and Li, Zhengguo
- Subjects
- *
HIGH dynamic range imaging , *BAYER process , *STATIC VAR compensators , *IMAGE color analysis , *DEEP learning - Abstract
Spatially varying exposure (SVE) is a promising choice for high-dynamic-range (HDR) imaging (HDRI). The SVE-based HDRI, which is called single-shot HDRI, is an efficient solution to avoid ghosting artifacts. However, it is very challenging to restore a full-resolution HDR image from a real-world image with SVE because: a) only one-third of pixels with varying exposures are captured by camera in a Bayer pattern, b) some of the captured pixels are over- and under-exposed. For the former challenge, a spatially varying convolution (SVC) is designed to process the Bayer images carried with varying exposures. For the latter one, an exposure-guidance method is proposed against the interference from over- and under-exposed pixels. Finally, a joint demosaicing and HDRI deep learning framework is formalized to include the two novel components and to realize an end-to-end single-shot HDRI. Experiments indicate that the proposed end-to-end framework avoids the problem of cumulative errors and surpasses the related state-of-the-art methods. Related codes and datasets will be provided at https://github.com/yilun-xu/SVEHDRI/. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
13. On the use of polychromatic cameras for high spatial resolution spectral dose measurements.
- Author
-
Cloutier, E, Beaulieu, L, and Archambault, L
- Subjects
- *
SPATIAL resolution , *INTERPOLATION algorithms , *CAMERAS , *SIGNAL-to-noise ratio , *SCINTILLATION counters , *STANDARD deviations , *MEASUREMENT , *SCINTILLATORS - Abstract
Objective. Despite the demonstrated benefits of hyperspectral formalism for stem effect corrections in the context of fiber dose measurements, this approach has not been yet translated into volumetric measurements where cameras are typically used for their distinguishing spatial resolution. This work investigates demosaicing algorithms for polychromatic cameras based spectral imaging. Approach. The scintillation and Cherenkov signals produced in a radioluminescent phantom are imaged by a polychromatic camera and isolated using the spectral formalism. To do so, five demosaicing algorithms are investigated from calibration to measurements: a clustering method and four interpolation algorithms. The resulting accuracy of scintillation and Cherenkov images is evaluated with measurements of the differences (mean ± standard deviation) between the obtained and expected signals from profiles drawn across a scintillation spot. Signal-to-noise ratio and signal-to-background ratio are further measured and compared in the resulting scintillation images. Finally, the resulting differences on the scintillation signal from a 0.2 × 0.2 cm2 region-of-interest (ROI) were reported. Main results. Clustering, OpenCV, bilinear, Malvar and Menon demosaicing algorithms respectively yielded differences of 3 ± 5%, 1 ± 3%, 1 ± 3%, 1 ± 2% and 2 ± 4% in the resulting scintillation images. For the Cherenkov images, all algorithms provided differences below 1%. All methods enabled measurements over the detectability (SBR > 2) and sensitivity (SNR > 5) thresholds with the bilinear algorithm providing the best SNR value. Clustering, OpenCV, bilinear, Malvar and Menon demosaicing algorithms respectively provided differences on the ROI analysis of 7 ± 5%, 3 ± 2%, 3 ± 2%, 4 ± 2%, 7 ± 3%. Significance. Radioluminescent signals can accurately be isolated using a single polychromatic camera. Moreover, demosaicing using a bilinear kernel provided the best results and enabled Cherenkov signal subtraction while preserving the full spatial resolution of the camera. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
14. Data-Driven Convolutional Model for Digital Color Image Demosaicing.
- Author
-
de Gioia, Francesco and Fanucci, Luca
- Subjects
COLOR filter arrays ,PROBLEM solving ,IMAGE reconstruction ,INTERPOLATION algorithms ,RANDOM measures ,DIGITAL cameras ,LIGHT filters - Abstract
Modern digital cameras use specific arrangement of Color Filter Array to sample light wavelength corresponding to visible colors. The most common Color Filter Array is the Bayer filter that samples only one color per pixel. To recover the full resolution image, an interpolation algorithm can be used. This process is called demosaicing and it is one of the first processing stages of a digital imaging pipeline. We introduce a novel data-driven model for demosaicing that takes into account the different requirements for reconstruction of the image Luma and Chrominance channels. The final model is a parallel composition of two reconstruction networks with individual architecture and trained with distinct loss functions. In order to solve the overfitting problem, we prepared a dataset that contains groups of patches that share common chromatic and spectral characteristics. We reported the reconstruction error on noise-free images and measured the effect of random noise and quantization noise in the demosaicing reconstruction. To test our model performance, we implemented the network on NVIDIA Jetson Nano, obtaining an end-to-end running time of less than one second for a full frame 12 MPixel image. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
15. MCFD: A Hardware-Efficient Noniterative Multicue Fusion Demosaicing Algorithm.
- Author
-
Yang, Xiaodong, Zhou, Wengang, and Li, Houqiang
- Subjects
- *
IMAGE processing , *COLOR filter arrays , *IMAGE color analysis , *ALGORITHMS - Abstract
Color demosaicing is an essential step in the camera image processing pipeline, especially on hardware platforms for real-time video applications. Though many demosaicing algorithms have been proposed in the past two decades, there remains a substantial gap between industrial needs and academic research. On one hand, industry requires a high perceptual quality, artifact-free, and low-cost demosaicing algorithm that is noniterative and small window based ready to be implemented on a hardware platform with a limited line buffer. On the other hand, academia is targeting high PSNR/SSIM, with the computation cost and line buffer receiving second priority, and often a frame buffer and iterative operation are used. The cost, large line buffer and iteration requirement make most existing demosaicing algorithms inapplicable on hardware platforms. In this paper, we introduce a novel low-cost demosaicing algorithm to narrow the gap. We keep the operation window size and computation cost as the first priority and achieve both high PSNR/SSIM and visual perceptual quality compared to previous state-of-the-art methods on standard test datasets. We fully investigate the a priori knowledge of natural scene raw images and find several key cues that are beneficial to demosaicing. Our demosaicing algorithm is a smart fusion of these useful cues. Furthermore, we solve typical demosaicing issues that occur in many traditional methods, including false color artifacts, T-section closing, and zippering artifacts. The proposed method has no learning stage, no iteration operation, a small line buffer and a limited number of parameters, so it can easily be applied to hardware platforms. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
16. Convolutional neural network advances in demosaicing for fluorescent cancer imaging with color–near-infrared sensors.
- Author
-
Jin, Yifei, Kondov, Borislav, Kondov, Goran, Singhal, Sunil, Nie, Shuming, and Gruev, Viktor
- Subjects
- *
CONVOLUTIONAL neural networks , *COMPUTER-assisted surgery , *IMAGE intensifiers , *ONCOLOGIC surgery , *GRAPHICS processing units - Abstract
Single-chip imaging devices featuring vertically stacked photodiodes and pixelated spectral filters are advancing multi-dye imaging methods for cancer surgeries, though this innovation comes with a compromise in spatial resolution. To mitigate this drawback, we developed a deep convolutional neural network (CNN) aimed at demosaicing the color and near-infrared (NIR) channels, with its performance validated on both pre-clinical and clinical datasets. We introduce an optimized deep CNN designed for demosaicing both color and NIR images obtained using a hexachromatic imaging sensor. A residual CNN was fine-tuned and trained on a dataset of color images and subsequently assessed on a series of dual-channel, color, and NIR images to demonstrate its enhanced performance compared with traditional bilinear interpolation. Our optimized CNN for demosaicing color and NIR images achieves a reduction in the mean square error by 37% for color and 40% for NIR, respectively, and enhances the structural dissimilarity index by 37% across both imaging modalities in pre-clinical data. In clinical datasets, the network improves the mean square error by 35% in color images and 42% in NIR images while enhancing the structural dissimilarity index by 39% in both imaging modalities. We showcase enhancements in image resolution for both color and NIR modalities through the use of an optimized CNN tailored for a hexachromatic image sensor. With the ongoing advancements in graphics card computational power, our approach delivers significant improvements in resolution that are feasible for real-time execution in surgical environments. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
17. A new multi-picture architecture for learned video deinterlacing and demosaicing with parallel deformable convolution and self-attention blocks.
- Author
-
Ji, Ronglei and Tekalp, A. Murat
- Subjects
- *
SUPERVISED learning , *MISSING data (Statistics) , *CUBES , *PARALLEL programming , *VIDEOS , *SPEECH synthesis - Abstract
Despite the fact real-world video deinterlacing and demosaicing are well-suited to supervised learning from synthetically degraded data because the degradation models are known and fixed, learned video deinterlacing and demosaicing have received much less attention compared to denoising and super-resolution tasks. We propose a new multi-picture architecture for video deinterlacing or demosaicing by aligning multiple supporting pictures with missing data to a reference picture to be reconstructed, benefiting from both local and global spatio-temporal correlations in the feature space using modified deformable convolution blocks and a novel residual efficient top- k self-attention (kSA) block, respectively. Separate reconstruction blocks are used to estimate different types of missing data. Our extensive experimental results, on synthetic or real-world datasets, demonstrate that the proposed novel architecture provides superior results that significantly exceed the state-of-the-art for both tasks in terms of PSNR, SSIM, and perceptual quality. Ablation studies are provided to justify and show the benefit of each novel modification made to the deformable convolution and residual efficient kSA blocks. Code is available: https://github.com/KUIS-AI-Tekalp-Research-Group/Video-Deinterlacing. • Our model achieves state-of-the-art results for deinterlacing and demosaicing tasks. • We propose modified deformable convolution (DfConv) block to extract local features. • Our efficient residual top-k self-attention (EkSA) block extracts global features. • We additively combine local and global spatio-temporal features computed in parallel. • Separate reconstruction blocks are used for each field/channel to be reconstructed. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
18. Gradient-Based Feature Extraction From Raw Bayer Pattern Images.
- Author
-
Zhou, Wei, Zhang, Ling, Gao, Shengyu, and Lou, Xin
- Subjects
- *
PATTERN matching , *FEATURE extraction , *COLOR filter arrays , *COMPUTER vision , *SIGNAL processing , *IMAGE color analysis , *COMPUTATIONAL complexity - Abstract
In this paper, the impact of demosaicing on gradient extraction is studied and a gradient-based feature extraction pipeline based on raw Bayer pattern images is proposed. It is shown both theoretically and experimentally that the Bayer pattern images are applicable to the central difference gradient-based feature extraction algorithms with negligible performance degradation, as long as the arrangement of color filter array (CFA) patterns matches the gradient operators. The color difference constancy assumption, which is widely used in various demosaicing algorithms, is applied in the proposed Bayer pattern image-based gradient extraction pipeline. Experimental results show that the gradients extracted from Bayer pattern images are robust enough to be used in histogram of oriented gradients (HOG)-based pedestrian detection algorithms and shift-invariant feature transform (SIFT)-based matching algorithms. By skipping most of the steps in the image signal processing (ISP) pipeline, the computational complexity and power consumption of a computer vision system can be reduced significantly. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
19. Wavelet-based color modification detection based on variance ratio
- Author
-
Jong Ju Jeon and Il Kyu Eom
- Subjects
Color modification ,Image forgery ,Color difference ,Demosaicing ,Variance ratio ,Wavelet transform ,Electronics ,TK7800-8360 - Abstract
Abstract Color modification is one of the popular image forgery techniques. It can be used to eliminate criminal evidence in various ways, such as modifying the color of a car used in a crime. If the color of a digital image is modified, the locations of the interpolated and original samples may be changed. Because the original and interpolated pixels have different statistical characteristics, these differences can serve as a basic clue for estimating the degree of color modification. It is assumed that the variance of original samples is greater than that of the interpolated samples. Therefore, we present a novel algorithm for color modification estimation using the variance ratio of color difference images in the wavelet domain. The color difference model is used to emphasize the differences between the original and interpolated samples. For color difference images, we execute a wavelet transform and use the highest frequency subband to calculate variances. We define a variance ratio measurement to quantify the level of color modification. Additionally, changed color local regions can be efficiently detected using the proposed algorithm. Experimental results demonstrate that the proposed method generates accurate estimation results for detecting color modification. Compared to the conventional method, our method provides superior color modification detection performance.
- Published
- 2018
- Full Text
- View/download PDF
20. Demosaicing by Differentiable Deep Restoration.
- Author
-
Tang, Jie, Li, Jian, Tan, Ping, and Kim, Byung-Gyu
- Subjects
COLOR filter arrays ,CONVOLUTIONAL neural networks ,DIGITAL cameras ,LIGHT filters ,IMAGE processing - Abstract
A mosaic of color filter arrays (CFAs) is commonly used in digital cameras as a spectrally selective filter to capture color images. The captured raw image is then processed by a demosaicing algorithm to recover the full-color image. In this paper, we formulate demosaicing as a restoration problem and solve it by minimizing the difference between the input raw image and the sampled full-color result. This under-constrained minimization is then solved with a novel convolutional neural network that estimates a linear subspace for the result at local image patches. In this way, the result in an image patch is determined by a few combination coefficients of the subspace bases, which makes the minimization problem tractable. This approach further allows joint learning of the CFA and demosaicing network. We demonstrate the superior performance of the proposed method by comparing it with state-of-the-art methods in both settings of noise-free and noisy data. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
21. I3D: a new dataset for testing denoising and demosaicing algorithms.
- Author
-
Bonanomi, Cristian, Balletti, Simone, Lecca, Michela, Anisetti, Marco, Rizzi, Alessandro, and Damiani, Ernesto
- Subjects
DIGITAL cameras ,IMAGE processing ,ALGORITHMS - Abstract
In this paper we present a dataset of images to test the performance of image processing algorithms, in particular demosaicing and denoising methods. Despite the plethora of demosaicing and denoising algorithms present in the literature, only few benchmarks are available to test their performance, and most of them are quite old, thus inadequate to represent the images captured by modern devices. The proposed dataset is composed by twenty 16 bit-depth images that can be used to test full-reference image quality metrics. More specifically, twelve pictures have been synthetically created by means of 2D or 3D softwares, while eight images have been captured by a high-end digital camera. [ABSTRACT FROM AUTHOR]
- Published
- 2020
- Full Text
- View/download PDF
22. Digital image forensic approach based on the second-order statistical analysis of CFA artifacts.
- Author
-
Singh, Gurinder and Singh, Kulbir
- Subjects
DIGITAL images ,ONLINE social networks ,STATISTICS ,ARCHAEOLOGY methodology ,MARKOV processes ,IMAGE processing - Abstract
The truthfulness of digital images can be evaluated by investigating the CFA artifacts introduced due to the interpolation process of the image acquisition phase. In this paper, an image tampering detection technique is proposed by exposing the CFA artifacts in difference domain through the higher-order statistical analysis based on the Markov transition probability matrix (MTPM). Firstly, the given image is re-interpolated with most commonly used four Bayer CFA patterns. The re-interpolation process is performed by using bilinear interpolation scheme for simplicity purpose. Then, the difference between the given image and its re-interpolated versions is evaluated to analyze the CFA inconsistencies. The target difference image is selected corresponding to the maximum sum which is further processed to evaluate the MTPM based second-order statistical feature. The recommended approach is assessed on different images from UCID dataset and various social networking websites based on scalar-based and SVM/machine learning based forensic detectors. The experiment results confirm that the projected method offers improved efficiency in comparison to the existing techniques based on different forgery scenarios. [ABSTRACT FROM AUTHOR]
- Published
- 2020
- Full Text
- View/download PDF
23. Design of multispectral array imaging system based on depth-guided network.
- Author
-
Yan, Gangqi, Song, Yansong, Zhang, Bo, Liang, Zonglin, Piao, Mingxu, Dong, Keyan, Zhang, Lei, Liu, Tianci, Wang, Yanbai, Li, Xinghang, and Hu, Wenyi
- Subjects
- *
MULTISPECTRAL imaging , *IMAGING systems , *SPECTRAL imaging , *IMAGE reconstruction - Abstract
• Image reconstruction techniques in the multispectral domain were investigated. • Design of an eight-band multispectral filter array imaging system. • A deep guidance network modeling algorithm is proposed. • Outperforms other existing methods in both quantitative and qualitative results. Imaging techniques using multispectral filter arrays (MSFA)have become a research hotspot with the rapid development of spectroscopic techniques. Among them, exploiting the correlation of color channels in the raw data and reconstructing raw images with high sparsity is a bottleneck and constraint in multi-band MSFA imaging systems. Therefore, this paper proposes a 4 × 4 eight-band MSFA imaging system containing a high sampling rate all-pass band. The all-pass band with a 1/2 high sampling rate contains rich color texture information to provide more features. A depth-guided reconstruction network (DGRN), including a depth-guided model (DGM) and a channel adaptive convolution model (CACM), is established to reconstruct the original spectral images. DGM extracts the color texture information of all-pass band images as the guide feature, which is combined with the initially processed eight-band shallow features to be the input of CACM to assign different guide features to different bands adaptively for learning and aggregation. The spatial correlation and spectral correlation of multiple bands are jointly learned using spectral and spatial properties to make the network flexible for MSFA imaging systems. The experimental results show that the method can effectively remove the artifacts of reconstructed images and improve the edge texture clarity. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
24. Residual Interpolation Integrated Pixel-by-Pixel Adaptive Iterative Process for Division of Focal Plane Polarimeters
- Author
-
Jie Yang, Weiqi Jin, Su Qiu, Fuduo Xue, and Meishu Wang
- Subjects
demosaicing ,division of focal plane polarimeters ,iteration ,residual interpolation ,Chemical technology ,TP1-1185 - Abstract
Residual interpolations are effective methods to reduce the instantaneous field-of-view error of division of focal plane (DoFP) polarimeters. However, their guide-image selection strategies are improper, and do not consider the DoFP polarimeters’ spatial sampling modes. Thus, we propose a residual interpolation method with a new guide-image selection strategy based on the spatial layout of the pixeled polarizer array to improve the sampling rate of the guide image. The interpolation performance is also improved by the proposed pixel-by-pixel, adaptive iterative process and the weighted average fusion of the results of the minimized residual and minimized Laplacian energy guide filters. Visual and objective evaluations demonstrate the proposed method’s superiority to the existing state-of-the-art methods. The proposed method proves that considering the spatial layout of the pixeled polarizer array on the physical level is vital to improving the performance of interpolation methods for DoFP polarimeters.
- Published
- 2022
- Full Text
- View/download PDF
25. Noise-Resistant Demosaicing with Deep Image Prior Network and Random RGBW Color Filter Array
- Author
-
Edwin Kurniawan, Yunjin Park, and Sukho Lee
- Subjects
color filter array ,deep image prior ,demosaicing ,deep learning ,Chemical technology ,TP1-1185 - Abstract
In this paper, we propose a deep-image-prior-based demosaicing method for a random RGBW color filter array (CFA). The color reconstruction from the random RGBW CFA is performed by the deep image prior network, which uses only the RGBW CFA image as the training data. To our knowledge, this work is a first attempt to reconstruct the color image with a neural network using only a single RGBW CFA in the training. Due to the White pixels in the RGBW CFA, more light is transmitted through the CFA than in the case with the conventional RGB CFA. As the image sensor can detect more light, the signal-to-noise-ratio (SNR) increases and the proposed demosaicing method can reconstruct the color image with a higher visual quality than other existing demosaicking methods, especially in the presence of noise. We propose a loss function that can train the deep image prior (DIP) network to reconstruct the colors from the White pixels as well as from the red, green, and blue pixels in the RGBW CFA. Apart from using the DIP network, no additional complex reconstruction algorithms are required for the demosaicing. The proposed demosaicing method becomes useful in situations when the noise becomes a major problem, for example, in low light conditions. Experimental results show the validity of the proposed method for joint demosaicing and denoising.
- Published
- 2022
- Full Text
- View/download PDF
26. Color transfer of an image in texture decomposition using demosaicing algorithm
- Author
-
Sreelatha, Tammineni, Lebaka, Sivaprasad, and Prathap, P.
- Published
- 2018
- Full Text
- View/download PDF
27. Data-Driven Convolutional Model for Digital Color Image Demosaicing
- Author
-
Francesco de Gioia and Luca Fanucci
- Subjects
demosaicing ,bayer filter ,color filter array ,convolutional neural network ,image processing ,Technology ,Engineering (General). Civil engineering (General) ,TA1-2040 ,Biology (General) ,QH301-705.5 ,Physics ,QC1-999 ,Chemistry ,QD1-999 - Abstract
Modern digital cameras use specific arrangement of Color Filter Array to sample light wavelength corresponding to visible colors. The most common Color Filter Array is the Bayer filter that samples only one color per pixel. To recover the full resolution image, an interpolation algorithm can be used. This process is called demosaicing and it is one of the first processing stages of a digital imaging pipeline. We introduce a novel data-driven model for demosaicing that takes into account the different requirements for reconstruction of the image Luma and Chrominance channels. The final model is a parallel composition of two reconstruction networks with individual architecture and trained with distinct loss functions. In order to solve the overfitting problem, we prepared a dataset that contains groups of patches that share common chromatic and spectral characteristics. We reported the reconstruction error on noise-free images and measured the effect of random noise and quantization noise in the demosaicing reconstruction. To test our model performance, we implemented the network on NVIDIA Jetson Nano, obtaining an end-to-end running time of less than one second for a full frame 12 MPixel image.
- Published
- 2021
- Full Text
- View/download PDF
28. A Brief Review of Some Interesting Mars Rover Image Enhancement Projects
- Author
-
Chiman Kwan
- Subjects
Mars rover Curiosity ,Mastcam ,multispectral images ,perceptually lossless compression ,demosaicing ,stereo images ,Electronic computers. Computer science ,QA75.5-76.95 - Abstract
The Curiosity rover has landed on Mars since 2012. One of the instruments onboard the rover is a pair of multispectral cameras known as Mastcams, which act as eyes of the rover. In this paper, we summarize our recent studies on some interesting image processing projects for Mastcams. In particular, we will address perceptually lossless compression of Mastcam images, debayering and resolution enhancement of Mastcam images, high resolution stereo and disparity map generation using fused Mastcam images, and improved performance of anomaly detection and pixel clustering using combined left and right Mastcam images. The main goal of this review paper is to raise public awareness about these interesting Mastcam projects and also stimulate interests in the research community to further develop new algorithms for those applications.
- Published
- 2021
- Full Text
- View/download PDF
29. Estimation of Bayer CFA pattern configuration based on singular value decomposition
- Author
-
Jong Ju Jeon, Hyun Jun Shin, and Il Kyu Eom
- Subjects
Color filter array ,Bayer pattern identification ,Forgery detection ,Singular value ,Color difference image ,Demosaicing ,Electronics ,TK7800-8360 - Abstract
Abstract An image sensor can measure only one color per pixel through the color filter array. Missing pixels are estimated using an interpolation process. For this reason, a captured pixel and interpolated pixel have different statistical characteristics. Because the pattern of a color filter array is changed when the image is manipulated or forged, this pattern change can be a clue to detect image forgery. However, the majority of forgery detection algorithms assume that they know the color filter array pattern. Therefore, estimating the configuration of the color filter array can have an important role as a precondition for image forgery detection. In this paper, we propose an efficient algorithm for estimating the Bayer color filter array configuration. We first construct a color difference image to reflect the characteristics of different demosaicing methods. To identify the color filter array pattern, we employ singular value decomposition. The truncated sum of the singular values is used to distinguish the color filter array pattern. Experimental results confirm that the proposed method generates acceptable estimation results in identifying color filter array patterns. Compared with conventional methods, the proposed method provides superior performance.
- Published
- 2017
- Full Text
- View/download PDF
30. Further Improvement of Debayering Performance of RGBW Color Filter Arrays Using Deep Learning and Pansharpening Techniques.
- Author
-
Chiman Kwan and Chou, Bryan
- Subjects
DEEP learning ,COLOR filter arrays ,DIGITAL images ,DIGITAL cameras ,HIGH resolution imaging - Abstract
The RGBW color filter arrays (CFA), also known as CFA2.0, contains R, G, B, and white (W) pixels. It is a 4 × 4 pattern that has 8 white pixels, 4 green pixels, 2 red pixels, and 2 blue pixels. The pattern repeats itself over the whole image. In an earlier conference paper, we cast the demosaicing process for CFA2.0 as a pansharpening problem. That formulation is modular and allows us to insert different pansharpening algorithms for demosaicing. New algorithms in interpolation and demosaicing can also be used. In this paper, we propose a new enhancement of our earlier approach by integrating a deep learning-based algorithm into the framework. Extensive experiments using IMAX and Kodak images clearly demonstrated that the new approach improved the demosaicing performance even further. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF
31. Optimization of Demosaicing Algorithm for Autofluorescence Imaging System.
- Author
-
Qin, Shijia, Wang, Zhiwu, Yan, Guozheng, Kuang, Shuai, Cheng, Hao, and Xiao, Jie
- Abstract
Autofluorescence imaging (AFI) systems are widely used in the detection of precancerous lesions. Fluorescence images of precancerous tissue are usually red (R) or blue (B), so this kind of system has high requirement for colour recovery, especially in R and B channels. Besides, AFI system requires bulk data transmission with no time delay. Existing colour recovery algorithms focus more on green (G) channel, overlooking R and B channels. Although the state-of-art demosaicing algorithms can perform well in colour recovery, they often have high computational cost and high hardware requirements. We propose an efficient interpolation algorithm with low complexity to solve the problem. When calculating R and B channel values, we innovatively propose the diagonal direction to select the interpolation direction, and apply colour difference law to make full use of the correlation between colour channels. The experimental results show that the peak signal-to-noise ratios (PSNRs) of G, R and B channels reach 37.54, 37.40 and 38.22 dB, respectively, which shows good performance in recovery of R and B channels. In conclusion, the algorithm proposed in this paper can be used as an alternative to the existing demosaicing algorithms for AFI system. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF
32. Deep Joint Demosaicing and High Dynamic Range Imaging Within a Single Shot
- Author
-
Changyun Wen, Zhengguo Li, Xingming Wu, Yilun Xu, Weihai Chen, Ziyang Liu, and School of Electrical and Electronic Engineering
- Subjects
FOS: Computer and information sciences ,Bayer filter ,Demosaicing ,Pixel ,business.industry ,Computer science ,Computer Vision and Pattern Recognition (cs.CV) ,Image and Video Processing (eess.IV) ,Computer Science - Computer Vision and Pattern Recognition ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Electrical Engineering and Systems Science - Image and Video Processing ,Convolution ,HDRi ,Interference (communication) ,High-dynamic-range imaging ,FOS: Electrical engineering, electronic engineering, information engineering ,Electrical and electronic engineering [Engineering] ,Media Technology ,Computer vision ,Artificial intelligence ,Electrical and Electronic Engineering ,Ghosting ,business ,Spatially Varying Exposure - Abstract
Spatially varying exposure (SVE) is a promising choice for high-dynamic-range (HDR) imaging (HDRI). The SVE-based HDRI, which is called single-shot HDRI, is an efficient solution to avoid ghosting artifacts. However, it is very challenging to restore a full-resolution HDR image from a real-world image with SVE because: a) only one-third of pixels with varying exposures are captured by camera in a Bayer pattern, b) some of the captured pixels are over- and under-exposed. For the former challenge, a spatially varying convolution (SVC) is designed to process the Bayer images carried with varying exposures. For the latter one, an exposure-guidance method is proposed against the interference from over- and under-exposed pixels. Finally, a joint demosaicing and HDRI deep learning framework is formalized to include the two novel components and to realize an end-to-end single-shot HDRI. Experiments indicate that the proposed end-to-end framework avoids the problem of cumulative errors and surpasses the related state-of-the-art methods., 15 pages, 17 figures
- Published
- 2022
- Full Text
- View/download PDF
33. A Two-Stage Convolutional Neural Network for Joint Demosaicking and Super-Resolution
- Author
-
Pak Lun Kevin Ding, Kan Chang, Yufei Tan, Baoxin Li, and Hengxin Li
- Subjects
Demosaicing ,business.industry ,Computer science ,Media Technology ,Pattern recognition ,Stage (hydrology) ,Artificial intelligence ,Electrical and Electronic Engineering ,Joint (audio engineering) ,business ,Superresolution ,Convolutional neural network - Published
- 2022
- Full Text
- View/download PDF
34. Efficient colour filter array demosaicking with prior error reduction
- Author
-
M. S. Safna Asiq and W. R. Sam Emmanuel
- Subjects
Colour image ,Demosaicing ,General Computer Science ,Pixel ,Computer science ,business.industry ,Colour filter array ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Process (computing) ,020206 networking & telecommunications ,02 engineering and technology ,Residual ,Image (mathematics) ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Computer vision ,Artificial intelligence ,Error reduction ,business - Abstract
A single sensor camera captures only one intensity value in a pixel known as the raw/unprocessed image. The raw or incomplete image is reconstructed to a full colour image by the process called demosaicking. The proposed work introduces an error efficient demosaicking algorithm. The efficient prior error reduction technique helps to obtain better results. The demosaicked green image is a guide to the residual demosaicking process. The conventional demosaicking is replaced by the residual demosaicking. The standard Kodak and McMaster datasets are employed in the experimental analysis. The proposed methodology has produced optimal performance with reduced error compared to the conventional demosaicking algorithms.
- Published
- 2022
- Full Text
- View/download PDF
35. Demosaicing by Differentiable Deep Restoration
- Author
-
Jie Tang, Jian Li, and Ping Tan
- Subjects
color filter array ,demosaicing ,color restoration ,convolutional neural networks ,closed-form optimization ,Technology ,Engineering (General). Civil engineering (General) ,TA1-2040 ,Biology (General) ,QH301-705.5 ,Physics ,QC1-999 ,Chemistry ,QD1-999 - Abstract
A mosaic of color filter arrays (CFAs) is commonly used in digital cameras as a spectrally selective filter to capture color images. The captured raw image is then processed by a demosaicing algorithm to recover the full-color image. In this paper, we formulate demosaicing as a restoration problem and solve it by minimizing the difference between the input raw image and the sampled full-color result. This under-constrained minimization is then solved with a novel convolutional neural network that estimates a linear subspace for the result at local image patches. In this way, the result in an image patch is determined by a few combination coefficients of the subspace bases, which makes the minimization problem tractable. This approach further allows joint learning of the CFA and demosaicing network. We demonstrate the superior performance of the proposed method by comparing it with state-of-the-art methods in both settings of noise-free and noisy data.
- Published
- 2021
- Full Text
- View/download PDF
36. Joint Demosaicing and Denoising Based on Interchannel Nonlocal Mean Weighted Moving Least Squares Method
- Author
-
Yeahwon Kim, Hohyung Ryu, Sunmi Lee, and Yeon Ju Lee
- Subjects
moving least squares approximation ,demosaicing ,patch similarity ,color filter array ,Chemical technology ,TP1-1185 - Abstract
Nowadays, the sizes of pixel sensors in digital cameras are decreasing as the resolution of the image sensor increases. Due to the decreased size, the pixel sensors receive less light energy, which makes it more sensitive to thermal noise. Even a small amount of noise in the color filter array (CFA) can have a significant effect on the reconstruction of the color image, as two-thirds of the missing data would have to be reconstructed from noisy data; because of this, direct denoising would need to be performed on the raw CFA to obtain a high-resolution color image. In this paper, we propose an interchannel nonlocal weighted moving least square method for the noise removal of the raw CFA. The proposed method is our first attempt of applying a two dimensional (2-D) polynomial approximation to denoising the CFA. Previous works make use of 2-D linear or directional 1-D polynomial approximations. The reason that 2-D polynomial approximation methods have not been applied to this problem is the difficulty of the weight control in the 2-D polynomial approximation method, as a small amount of noise can have a large effect on the approximated 2-D shape. This makes CFA denoising more important, as the approximated 2-D shape has to be reconstructed from only one-third of the original data. To address this problem, we propose a method that reconstructs the approximated 2-D shapes corresponding to the RGB color channels based on the measure of the similarities of the patches directly on the CFA. By doing so, the interchannel information is incorporated into the denoising scheme, which results in a well-controlled and higher order of polynomial approximation of the color channels. Compared to other nonlocal-mean-based denoising methods, the proposed method uses an extra reproducing constraint, which guarantees a certain degree of the approximation order; therefore, the proposed method can reduce the number of false reconstruction artifacts that often occur in nonlocal-mean-based denoising methods. Experimental results demonstrate the performance of the proposed algorithm.
- Published
- 2020
- Full Text
- View/download PDF
37. Demosaicing of CFA 3.0 with Applications to Low Lighting Images
- Author
-
Chiman Kwan, Jude Larkin, and Bulent Ayhan
- Subjects
debayering ,demosaicing ,color filter array (CFA) ,RGBW pattern ,Bayer pattern ,CFA 1.0 ,Chemical technology ,TP1-1185 - Abstract
Low lighting images usually contain Poisson noise, which is pixel amplitude-dependent. More panchromatic or white pixels in a color filter array (CFA) are believed to help the demosaicing performance in dark environments. In this paper, we first introduce a CFA pattern known as CFA 3.0 that has 75% white pixels, 12.5% green pixels, and 6.25% of red and blue pixels. We then present algorithms to demosaic this CFA, and demonstrate its performance for normal and low lighting images. In addition, a comparative study was performed to evaluate the demosaicing performance of three CFAs, namely the Bayer pattern (CFA 1.0), the Kodak CFA 2.0, and the proposed CFA 3.0. Using a clean Kodak dataset with 12 images, we emulated low lighting conditions by introducing Poisson noise into the clean images. In our experiments, normal and low lighting images were used. For the low lighting conditions, images with signal-to-noise (SNR) of 10 dBs and 20 dBs were studied. We observed that the demosaicing performance in low lighting conditions was improved when there are more white pixels. Moreover, denoising can further enhance the demosaicing performance for all CFAs. The most important finding is that CFA 3.0 performs better than CFA 1.0, but is slightly inferior to CFA 2.0, in low lighting images.
- Published
- 2020
- Full Text
- View/download PDF
38. Efficient Training Procedures for Multi-Spectral Demosaicing
- Author
-
Ivana Shopovska, Ljubomir Jovanov, and Wilfried Philips
- Subjects
RGB ,NIR ,multispectral ,demosaicing ,deep learning ,data sampling ,Chemical technology ,TP1-1185 - Abstract
The simultaneous acquisition of multi-spectral images on a single sensor can be efficiently performed by single shot capture using a mutli-spectral filter array. This paper focused on the demosaicing of color and near-infrared bands and relied on a convolutional neural network (CNN). To train the deep learning model robustly and accurately, it is necessary to provide enough training data, with sufficient variability. We focused on the design of an efficient training procedure by discovering an optimal training dataset. We propose two data selection strategies, motivated by slightly different concepts. The general term that will be used for the proposed models trained using data selection is data selection-based multi-spectral demosaicing (DSMD). The first idea is clustering-based data selection (DSMD-C), with the goal to discover a representative subset with a high variance so as to train a robust model. The second is an adaptive-based data selection (DSMD-A), a self-guided approach that selects new data based on the current model accuracy. We performed a controlled experimental evaluation of the proposed training strategies and the results show that a careful selection of data does benefit the speed and accuracy of training. We are still able to achieve high reconstruction accuracy with a lightweight model.
- Published
- 2020
- Full Text
- View/download PDF
39. Joint Demosaicing and Denoising Based on a Variational Deep Image Prior Neural Network
- Author
-
Yunjin Park, Sukho Lee, Byeongseon Jeong, and Jungho Yoon
- Subjects
color filter array ,deep image prior ,demosaicing ,deep learning ,Chemical technology ,TP1-1185 - Abstract
A joint demosaicing and denoising task refers to the task of simultaneously reconstructing and denoising a color image from a patterned image obtained by a monochrome image sensor with a color filter array. Recently, inspired by the success of deep learning in many image processing tasks, there has been research to apply convolutional neural networks (CNNs) to the task of joint demosaicing and denoising. However, such CNNs need many training data to be trained, and work well only for patterned images which have the same amount of noise they have been trained on. In this paper, we propose a variational deep image prior network for joint demosaicing and denoising which can be trained on a single patterned image and works for patterned images with different levels of noise. We also propose a new RGB color filter array (CFA) which works better with the proposed network than the conventional Bayer CFA. Mathematical justifications of why the variational deep image prior network suits the task of joint demosaicing and denoising are also given, and experimental results verify the performance of the proposed method.
- Published
- 2020
- Full Text
- View/download PDF
40. Polarization Image Demosaicking via Nonlocal Sparse Tensor Factorization
- Author
-
Hanwen Yu, Jianlai Chen, Junchao Zhang, Buge Liang, Mengdao Xing, and Degui Yang
- Subjects
Demosaicing ,Computer science ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,General Earth and Planetary Sciences ,Polarimeter ,Tensor ,Electrical and Electronic Engineering ,Cube ,Polarization (waves) ,Neural coding ,Algorithm ,Image resolution ,Image (mathematics) - Abstract
Division-of-focal-plane (DoFP) polarimeter provides a way for snapshot acquisition, making it available to simultaneously record polarization measurements at different orientations. This polarization imaging system has gained more attention in the last few years and is promising to be used in the fields of computer vision and remote sensing. However, this system suffers from the degradation of spatial resolution. To reconstruct polarization information at full resolution, polarization image demosaicking is indispensable. To address polarization image demosaicking issue while preserving the essential structure of polarization data, a sparse tensor factorization-based model is proposed. For a target cube, its similar cubes are first grouped together as a tensor. Then, its compact dictionary and sparse core tensor are learned by factorizing the tensor using sparse coding. Moreover, the correlation among different polarization orientations and the nonlocal self-similarity are adopted to boost the performance. Experimental results on synthetic and real-world data demonstrate that our proposed model outperforms several state-of-the-art methods in terms of both quantitative measurements and visual quality.
- Published
- 2022
- Full Text
- View/download PDF
41. Image reconstruction for color contact image sensor (CIS).
- Author
-
Lu, Xuan, Ren, Jiayu, Wang, Dingwen, Deng, Dexiang, and Shi, Wenxuan
- Abstract
The color contact image sensor is often used to capture the surface of some materials for the defect detection in industry. However, the special imaging mode leads a special image pattern of the color contact image sensor. This pattern of the sensor can be used to increase the resolution of the image, while none of the algorithms is able to properly process it, recently. This paper presents an approach for the reconstruction of the color contact image sensor. We combine the sparse prior that often used in super-resolution and the inter-channel correlation prior that the majority of image demosaicing algorithms used to solve this problem. Extensive experiments on simulated image and the real image captured by color contact image sensor show that our method achieves good results in terms of both objective and human visual evaluations. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF
42. Hardware-efficient color correlation-adaptive demosaicing with multifiltering.
- Author
-
Lee, Seung Hyun, Choi, Dong Yoon, and Song, Byung Cheol
- Subjects
- *
COLOR image processing , *COLORS , *COLOR filter arrays - Abstract
Color demosaicing is a key image processing step aiming to reconstruct the missing pixels from a recorded raw image that has a color filter array (CFA) pattern. The color correlation -- based guided filters, such as minimized-Laplacian residual interpolation (MLRI), are known as the state-of-the-art demosaicing techniques. However, in the conventional guided filter-based techniques, the artifacts are generated in areas with low color correlation. Furthermore, a large number of line memories are required in the hardware implementation of a conventional guided filter-based technique because of the large effective field size. To overcome these two problems, we propose a color correlation -- adaptive demosaicing algorithm that selectively applies a specific intracolor demosaicing to regions with low color correlation. We also propose an algorithm structure that reduces the effective field size in the vertical direction to reduce the number of line memories, while maintaining the image quality performance during the hardware implementation. The experimental results show that the proposed scheme can reduce the line memory to one-third while showing marginal performance degradation compared to the state-of-the-art MLRI weighted framework in terms of the color peak signal-to-noise ratio for the IMAX datasets. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF
43. A Markov based image forgery detection approach by analyzing CFA artifacts.
- Author
-
Singh, Amneet, Singh, Gurinder, and Singh, Kulbir
- Subjects
COLOR filter arrays ,MARKOV random fields ,INTERPOLATION ,DIGITAL image processing ,FORGERY - Abstract
The image acquisition device, the light is filtered through a Color Filter Array (CFA), where each pixel captures only one color (from Red, Green, and Blue), while others are calibrated. This process is known as interpolation process, and the artifacts introduced are called CFA or interpolation artifacts. The structure of these artifacts in the image is disturbed while a forgery is introduced in an image. In this paper, a high-order statistical approach is proposed to detect the inconsistencies in the artifacts of different parts of the image to expose any forgery present. The Markov Transition Probability Matrix (MTPM) is employed to develop various features that will detect the presence or absence of CFA artifacts in a particular region of the image. The Markov random process is applied because it provides an enhanced efficiency and reduced computational complexity for the forgery detection model. The algorithm is tested on 2 × 2 pixel block of the image which provides the results of a fine quality. There is no prior information of the location of the forged region of the image. The algorithm is tested on various images, taken from various social networking websites. The proposed forgery detection technique outperforms the existing state-of-the-art techniques for the different forgery scenarios by providing an average accuracy of 90.58%. [ABSTRACT FROM AUTHOR]
- Published
- 2018
- Full Text
- View/download PDF
44. Demosaicing enhancement using pixel-level fusion.
- Author
-
Kwan, Chiman, Chou, Bryan, Kwan, Li-Yun M., Larkin, Jude, Ayhan, Bulent, Bell, James F., and Kerner, Hannah
- Abstract
Bayer pattern has been widely used in commercial digital cameras. In NASA’s mast camera (Mastcams) onboard the Mars rover Curiosity, Bayer pattern has also been used in capturing the RGB bands. It is well known that debayering, also known as demosaicing in the literature, introduces artifacts such as false colors and zipper edges. In this paper, we first present four fusion approaches, including weighted and the well-known alpha-trimmed mean filtering approaches. Each fusion approach combines demosaicing results from seven debayering algorithms in the literature, which are selected based on their performance mentioned in other survey papers and the availability of open source codes. Second, we present debayering results using two benchmark image data sets: IMAX and Kodak. It was observed that none of the seven algorithms in the literature can yield the best performance in terms of peak signal-to-noise ratio (PSNR), CIELAB score, and subjective evaluation. Although the fusion algorithms are simple, it turns out that the debayering performance can be improved quite dramatically after fusion based on our extensive evaluations. In particular, the average PSNR improvements of the weighted fusion algorithm over the best individual method are 1.1 dB for the IMAX database and 1.8 dB for the Kodak database, respectively. Third, we applied the various algorithms to 36 actual Mastcam images. Subjective evaluation indicates that the fusion algorithms still work well, but not as good as the existing debayering algorithm used by NASA. [ABSTRACT FROM AUTHOR]
- Published
- 2018
- Full Text
- View/download PDF
45. Optimized Multi-Spectral Filter Array Based Imaging of Natural Scenes.
- Author
-
Yuqi Li, Aditi Majumder, Hao Zhang, and Gopi, M.
- Abstract
Multi-spectral imaging using a camera with more than three channels is an efficient method to acquire and reconstruct spectral data and is used extensively in tasks like object recognition, relighted rendering, and color constancy. Recently developed methods are used to only guide content-dependent filter selection where the set of spectral reflectances to be recovered are known a priori. We present the first content-independent spectral imaging pipeline that allows optimal selection of multiple channels. We also present algorithms for optimal placement of the channels in the color filter array yielding an efficient demosaicing order resulting in accurate spectral recovery of natural reflectance functions. These reflectance functions have the property that their power spectrum statistically exhibits a power-law behavior. Using this property, we propose power-law based error descriptors that are minimized to optimize the imaging pipeline. We extensively verify our models and optimizations using large sets of commercially available wide-band filters to demonstrate the greater accuracy and efficiency of our multi-spectral imaging pipeline over existing methods. [ABSTRACT FROM AUTHOR]
- Published
- 2018
- Full Text
- View/download PDF
46. Deep Demosaicking with Luminance and Chrominance Estimations
- Author
-
Taishi Iriyama, Hisashi Aomori, Tsuyoshi Otake, and Masatoshi Sato
- Subjects
Demosaicing ,business.industry ,Computer science ,Chrominance ,Computer vision ,Artificial intelligence ,business ,Luminance - Published
- 2021
- Full Text
- View/download PDF
47. Predictive Filter Flow Network for Universal Demosaicking
- Author
-
Tsuyoshi Otake, Taishi Iriyama, Hisashi Aomori, Masatoshi Sato, and Daiki Arai
- Subjects
Demosaicing ,Filter (video) ,business.industry ,Computer science ,Computer vision ,Artificial intelligence ,Flow network ,business - Published
- 2021
- Full Text
- View/download PDF
48. Toward blind joint demosaicing and denoising of raw color filter array data
- Author
-
Fangfang Wu, Xin Li, Weisheng Dong, Guangming Shi, Zhonglong Zheng, and Tao Huang
- Subjects
0209 industrial biotechnology ,Demosaicing ,Color image ,Computer science ,business.industry ,Cognitive Neuroscience ,Pipeline (computing) ,Noise reduction ,Pattern recognition ,02 engineering and technology ,Computer Science Applications ,Noise ,020901 industrial engineering & automation ,Artificial Intelligence ,Gamma correction ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Color filter array ,Artificial intelligence ,Joint (audio engineering) ,business - Abstract
Raw color-filter-array (CFA) data collected in the real world are often noisy and signal-dependent, which makes it difficult to recover the full-resolution noise-free color image. Denoising and demosaicing are two popular tools developed for noisy CFA data in modern color imaging pipeline. However, most existing works on joint demosaicing and denoising (JDD) are based on ad hoc assumptions about image degradation process; while in practice little is known about noise statistics (e.g., noise level) and processing pipeline (e.g., gamma correction). We advocate a blind formulation of joint demosaicing and denoising (bJDD) problem in this paper and present a novel divide-and-conquer approach toward blind reconstruction from noisy raw CFA data. Instead of making over-simplified assumptions about noise statistics, we propose to develop a more realistic Poisson-Gaussian noise model for simulating noisy raw CFA data in the real world. We also introduce a sub-network to adaptively estimate the noise level map from the noisy input, which will provide supplementary information to the deep model for non-blind JDD. Finally, we have adopted a generative adversarial network (GAN) based network for further perceptual optimization. Our extensive experimental results have shown convincingly improved performance over existing state-of-the-art methods in terms of both subjective and objective quality metrics.
- Published
- 2021
- Full Text
- View/download PDF
49. NTSDCN: New Three-Stage Deep Convolutional Image Demosaicking Network
- Author
-
Yan Wang, Ruiqin Xiong, Shiying Yin, Zhan Ma, Shuyuan Zhu, and Bing Zeng
- Subjects
Demosaicing ,Computer science ,business.industry ,Feature extraction ,Pattern recognition ,Iterative reconstruction ,Residual ,Convolutional neural network ,Convolution ,Media Technology ,Code (cryptography) ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,Interpolation - Abstract
In this letter, we compose a new three-stage deep convolutional neural network (NTSDCN) for image demosaicking, and it consists of our proposed Laplacian energy-constrained local residual unit (LC-LRU) and a feature-guided prior fusion unit (FG-PFU). Specifically, the LC-LRU is used to refine the learning target of the specific residual blocks in the network and enhance the dominant information of the residual features. The FG-PFU is designed to guide the feature extraction of the red (R) and blue (B) channels by utilizing prior information from the reconstructed green (G) channel. In our proposed NTSDCN, we recover the G channel image in the first stage with the CFA image and reconstruct the R and B images in the second stage. Finally, we fine-tune the resulting R, G and B images in the third stage to compose a full-color RGB image. The experimental results show that our proposed method achieves better performance than the state-of-the-art methods. The code is available at https://github.com/wyannn/NTSDCN .
- Published
- 2021
- Full Text
- View/download PDF
50. Noise Removal in the Developing Process of Digital Negatives
- Author
-
Marek Szczepański and Filip Giemza
- Subjects
demosaicing ,interpolation ,color filter array ,bayer filter ,image filtering ,image noise ,Chemical technology ,TP1-1185 - Abstract
Most modern color digital cameras are equipped with a single image sensor with a color filter array (CFA). One of the most important stages of preprocessing is noise reduction. Most research related to this topic ignores the problem associated with the actual color image acquisition process and assumes that we are processing the image in the sRGB space. In the presented paper, the real process of developing raw images obtained from the CFA sensor was analyzed. As part of the work, a diverse database of test images in the form of a digital negative and its reference version was prepared. The main problem posed in the work was the location of the denoising and demosaicing algorithms in the entire raw image processing pipeline. For this purpose, all stages of processing the digital negative are reproduced. The process of noise generation in the image sensors was also simulated, parameterizing it with ISO sensitivity for a specific CMOS sensor. In this work, we tested commonly used algorithms based on the idea of non-local means, such as NLM or BM3D, in combination with various techniques of interpolation of CFA sensor data. Our experiments have shown that the use of noise reduction methods directly on the raw sensor data, improves the final result only in the case of highly disturbed images, which corresponds to the process of image acquisition in difficult lighting conditions.
- Published
- 2020
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.