Search

Salient object detection is a method of finding an object within an image that a person determines to be important and is expected to focus on. Various features are used to compute the visual saliency, and in general, the color and luminance of the scene are widely used among the spatial features. However, humans perceive the same color and luminance differently depending on the influence of the surrounding environment. As the human visual system (HVS) operates through a very complex mechanism, both neurobiological and psychological aspects must be considered for the accurate detection of salient objects. To reflect this characteristic in the saliency detection process, we have proposed two pre-processing methods to apply to the input image. First, we applied a bilateral filter to improve the segmentation results by smoothing the image so that only the overall context of the image remains while preserving the important borders of the image. Second, although the amount of light is the same, it can be perceived with a difference in the brightness owing to the influence of the surrounding environment. Therefore, we applied oriented difference-of-Gaussians (ODOG) and locally normalized ODOG (LODOG) filters that adjust the input image by predicting the brightness as perceived by humans. Experiments on five public benchmark datasets for which ground truth exists show that our proposed method further improves the performance of previous state-of-the-art methods.

Super-Resolution (SR) is a fundamental computer vision task that aims to obtain a high-resolution clean image from the given low-resolution counterpart. This paper reviews the NTIRE 2021 Challenge on Video Super-Resolution. We present evaluation results from two competition tracks as well as the proposed solutions. Track 1 aims to develop conventional video SR methods focusing on the restoration quality. Track 2 assumes a more challenging environment with lower frame rates, casting spatio-temporal SR problem. In each competition, 247 and 223 participants have registered, respectively. During the final testing phase, 14 teams competed in each track to achieve state-of-the-art performance on video SR tasks.

International audience; This paper reviews the Challenge on Image Demoireing that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2020. Demoireing is a difficult task of removing moire patterns from an image to reveal an underlying clean image. The challenge was divided into two tracks. Track 1 targeted the single image demoireing problem, which seeks to remove moire patterns from a single image. Track 2 focused on the burst demoireing problem, where a set of degraded moire images of the same scene were provided as input, with the goal of producing a single demoired image as output. The methods were ranked in terms of their fidelity, measured using the peak signal-to-noise ratio (PSNR) between the ground truth clean images and the restored images produced by the participants' methods. The tracks had 142 and 99 registered participants, respectively, with a total of 14 and 6 submissions in the final testing stage. The entries span the current state-of-the-art in image and burst image demoireing problems. © 2020 IEEE.

Motion blur is one of the most common degradation artifacts in dynamic scene photography. This paper reviews the NTIRE 2020 Challenge on Image and Video Deblurring. In this challenge, we present the evaluation results from 3 competition tracks as well as the proposed solutions. Track 1 aims to develop single-image deblurring methods focusing on restoration quality. On Track 2, the image deblurring methods are executed on a mobile platform to find the balance of the running speed and the restoration accuracy. Track 3 targets developing video deblurring methods that exploit the temporal relation between input frames. In each competition, there were 163, 135, and 102 registered participants and in the final testing phase, 9, 4, and 7 teams competed. The winning methods demonstrate the state-of-the-art performance on image and video deblurring tasks.

Many deep-learning-based image/video denoising models have been developed, and recently, several approaches for training a denoising neural network without using clean images have been proposed. However, Noise2Noise method requires paired noisy data, and obtaining them is occasionally difficult, whereas other existing models trained using unpaired noisy data deliver limited performance. Obtaining an accurate optical flow from noisy videos is also a difficult task because conventional optical flow estimation methods are primarily focused on estimating the optical flow using clean videos. This study proposes a new framework to fine-tune video denoising and optical flow estimation networks using unpaired noisy videos. These two networks are jointly trained to realize synergy; an improvement in the denoising performance increases the accuracy of the flow estimation, and an improvement in the flow-estimation performance enhances the quality of the training data for the denoiser. Our experimental results reveal that proposed approach outperforms the existing training schemes in video denoising and also provides accurate optical flows even when the videos contain a considerable amount of noise.

In this paper, a novel algorithm is proposed using speckle reducing anisotropic diffusion (SRAD) and gradient domain guided image filtering (GDGIF) to reduce speckle in synthetic aperture radar (SAR) images. SRAD is suitable for reducing multiplicative noise in SAR images because it can directly process log-compressed data. Since GDGIF has edge-aware weighting, it is adaptively applied to SRAD result images to additionally reduce speckle noise. Experimental results demonstrate that the proposed algorithm, compared to existing filtering methods, shows excellent speckle noise reduction performance and a low computational complexity.

Synthetic aperture radar (SAR) images are difficult to analyze due to speckle noise, which is a characteristic of multiplicative noise. Over the last few decades, a number of studies have been performed regarding the removal of speckle noise. However, the existing studies exhibit edge information loss when removing speckle noise. In this paper, we propose an algorithm using speckle reducing anisotropic diffusion (SRAD), soft thresholding, and a guided filter to effectively remove speckle noise from SAR images while preserving edge information. The proposed algorithm first obtains a filtered image by applying an SRAD filter to a noise image. To further remove the multiplicative noise remaining in the filtered image, a logarithmic transformation is applied to convert it into additive noise. The filtering image was decomposed into multiresolution images using discrete wavelet transform (DWT). Soft thresholding and a guided filter were used for each of the high-frequency subimages and the low-frequency subimage. Then, an inverse DWT and an exponential transform are applied to the denoised image. The experimental results indicate that the proposed algorithm shows better performance than the conventional filtering method in terms of both objective and subjective performances.

In this paper, we propose a new demosaicking system that uses post-processing technique to remove zipper artifacts that are primarily caused by improper interpolation in a quincunx-sampled green plane. We use a high-order approximation to predict accurate pixel values in the weighted average-based green channel interpolation. We also perform prediction using a color difference model in the red and blue channel interpolation process. As a well-designed demosaicking system too cannot prevent the occurrence of ON–OFF pattern artifacts and the resulting false colors, we propose post-processing for zipper artifact detection and removal to improve the demosaicked image. After performing the just noticeable difference model-based zipper artifact detection in the luminance image of the demosaicked image, the clamping-based zipper artifact removal process is performed in the red, green, and blue planes. Experimental results demonstrate that the proposed demosaicking system improves both the objective and subjective image qualities compared to the conventional state-of-the-art demosaicking algorithms.

This paper proposes a thumbnail quality enhancement algorithm that uses a predefined weight table. Conventional thumbnail extraction algorithms in high efficiency video coding use a simple downsampling method to produce thumbnail images with low complexity, resulting in thumbnail quality deterioration. The proposed algorithm estimates the original average values of each thumbnail pixel using a weighted average value of several pixels, based on intra-mode direction. The proposed method improves the visual quality of thumbnail images while maintaining low complexity.

In this paper, we introduce an efficient image demosaicking method using a bilateral filter and directional differentiation with consideration of both the spatial closeness and the similarity between the interpolated pixel and the neighbor pixels. Spatial closeness is considered as spatial locality. We utilize an adaptive weighted average to estimate the missing pixel value, where the adaptive weight is calculated based on three components: directional differentiation, similarity between the pixel and each of its neighbor pixels, and spatial locality. The experimental results show that the proposed method outperforms existing approaches in both objective and subjective performance.

This paper reviews the first AIM challenge on mapping camera RAW to RGB images with the focus on proposed solutions and results. The participating teams were solving a real-world photo enhancement problem, where the goal was to map the original low-quality RAW images from the Huawei P20 device to the same photos captured with the Canon 5D DSLR camera. The considered problem embraced a number of computer vision subtasks, such as image demosaicing, denoising, gamma correction, image resolution and sharpness enhancement, etc. The target metric used in this challenge combined fidelity scores (PSNR and SSIM) with solutions' perceptual results measured in a user study. The proposed solutions significantly improved baseline results, defining the state-of-the-art for RAW to RGB image restoration.

Video frame interpolation has been studied for a long time; however, it is still a difficult low-level vision task. Owing to the improved performance of optical flow estimation, frame-interpolation studies based on optical flow are actively conducted. However, the existing methods are generally tested using high-fps sequences and developed for 2× upscaling or generating multiple frames with a single estimator. This paper proposes a 4× video-interpolation framework that aims to convert 15-fps to 60-fps videos based on a structure comprising flow estimation followed by an enhancement network. We improve the performance by training specialized flow estimators for each direction and frame position. Furthermore, we use the original frames and flow maps as additional inputs for the enhancement network to improve the subjective image quality. Consequently, the proposed network interpolates high-quality frames with a fast runtime and demonstrates its superiority in the AIM 2019 video temporal super-resolution challenge. The associated code is available at https://github.com/SonghyunYu/PoSNet.

Videos contain various types and strengths of motions that may look unnaturally discontinuous in time when the recorded frame rate is low. This paper reviews the first AIM challenge on video temporal super-resolution (frame interpolation) with a focus on the proposed solutions and results. From low-frame-rate (15 fps) video sequences, the challenge participants are asked to submit higher-frame-rate (60 fps) video sequences by estimating temporally intermediate frames. We employ the REDS_VTSR dataset derived from diverse videos captured in a hand-held camera for training and evaluation purposes. The competition had 62 registered participants, and a total of 8 teams competed in the final testing phase. The challenge winning methods achieve the state-of-the-art in video temporal super-resolution.

It is difficult to apply most video temporal super-resolution studies for real-world scenes because they are optimized for a specific range of characteristics. In this paper, we propose a video temporal super-resolution method that is tolerant to motion diversity and noise. Our proposed method improves its robustness by fine-tuning the pre-trained SPyNet that is trained for videos with simple motions and moderate conditions. Moreover, our proposed network learns to accurately synthesize two frames generated by a backward warping function without requiring any additional information using the architecture of a modified DHDN. This enables our proposed method to efficiently synthesize two warped frames by saving the computational complexity for pre-training and extracting the additional information. Finally, we apply the self-ensemble method, which is commonly used in studies on image processing but not on video processing. The application of the self-ensemble method enables our network to generate stable output frames with improved quality without any additional training. Our proposed network proved its performance by ranking 5th in the AIM 2019 video temporal super-resolution challenge; the performance gap between our proposed network and the 3rd-and 4th-ranked solutions was very small. The source code and pre-trained models are available at https://github.com/BumjunPark/DVTSR.

In the process of obtaining synthetic aperture radar (SAR) images, speckle noise exhibited in the SAR images. Speckle noise in the SAR images degrades an image quality and become difficult to interpret the image. Speckle noise reduction technique is a crucial step for using various application of SAR images. In this paper, we proposed a novel algorithm based on speckle reducing anisotropic diffusion (SRAD) and weighted least squares (WLS) filter for removing speckle noise and preserving edge information. The SRAD filter is applied as a preprocessing filter. A logarithmic transformation employed to convert a multiplicative noise remaining in the SRAD filter result image. The multiplicative noise changed as an additive noise. The additive noise in the SRAD filtering result image removed by the WLS filter. We can obtain a despeckled image employing an exponential transform. Experimental results demonstrate that the proposed method exhibits the best speckle noise reduction and edge preservation performance than the conventional filtering techniques.

This paper reviews the NTIRE 2019 challenge on real image denoising with focus on the proposed methods and their results. The challenge has two tracks for quantitatively evaluating image denoising performance in (1) the Bayer-pattern raw-RGB and (2) the standard RGB (sRGB) color spaces. The tracks had 216 and 220 registered participants, respectively. A total of 15 teams, proposing 17 methods, competed in the final phase of the challenge. The proposed methods by the 15 teams represent the current state-of-the-art performance in image denoising targeting real noisy images.

최근 넓은 동적 영역 이미지 기술은 컴퓨터 그래픽 분야에서 화제다. 본 논문에서는 가중 최소자승(가중회귀분석) 최적화 체계에 기반하여 넓은 동적 영역 이미지를 처리하는 톤매핑 알고리듬을 제안한다. 제안하는 방법은 시각적 후광 현상을 피하는 동시에 기존의 디스플레이에서 더 지각적인 넓은 동적 영역 이미지들을 보여주기 위해 가중 최소자승 필터링과 iCAM06모델을 결합한다. 제안된 알고리듬은 먼저 넓은 동적 영역 이미지를 base layer와 detail layer로 나눈다. Base layer는 큰 규모의 변화량을 가지고 있으며 가중 최소자승 필터링을 사용하여 얻어지고 iCAM06 모델을 포함한다. 다음으로, 인간의 시각 체계에 따라 base layer는 적응적으로 압축된다. 압축 시에는 base layer만 대비를 줄이고 detail layer을 보존한다. 본 논문에서는 객관적 화질 평가와 주관적 화질 평가를 통하여 제안하는 알고리듬을 적용한 이미지가 기존의 알고리듬을 적용한 이미지들에 비해 원본 넓은 동적 영역 이미지에 더 유사하다는 것을 보여준다.

In this paper, we propose a four-direction residual interpolation (FDRI) method for color filter array interpolation. The proposed algorithm exploits a guided filtering process to generate the tentative image. The residual image is generated by exploiting the tentative and original images. We use an FDRI algorithm to more accurately estimate the missing pixel values; the estimated image is adaptively combined with a joint inverse gradient weight. Based on the experimental results, the proposed method provides a superior performance in terms of objective and subjective quality compared with the conventional state-of-the-art demosaicking methods.

Guided image filter (GIF) is proposed for manipulating the high dynamic range (HDR) image. In the conventional algorithm, it is used to divide an image into a base layer and a detail layer and then to multiply the detail layer by compression function for enhancing the detail information of the image. However, in most cases, an image displays the detail features and edge information of the objects in different scales. That is to say, all the detail features of the objects cannot be clearly reflected under a certain scale. The multi-scale decomposition utilizes more a scale extracts the edges and details of the image, the main idea is to filter the image through multi-scale decomposition to get a basic layer and several detail layers, which are respectively processed and synthesized at the output. In this paper, an GIF multi-scale decomposition for HDR image is introduced. The experimental results show that the proposed algorithm has better edge preserving performance than the conventional algorithm.

In this paper, we propose an algorithm that improves the weight map part consisting of signal strength, signal structure, and mean intensity. The patch-based conventional weight map causes the brightness of the image to be shifted to one side, resulting in loss of image information, unexpected artifacts, and an overall unbalance in image brightness. In this study, we propose a novel algorithm by improving the weight map. First, the order-statistic filter using maximum values. Second, the unsharp masking filter using Laplacian. Third, the linear combination using gamma transformation. The proposed algorithm prevents the loss of image information by reducing the over-saturation of the image, accurate representation of dark and bright areas by increasing contrast, and preserve the detail such as the edge. Through subjective and objective experimental results, it is confirmed that the proposed algorithm shows better performance than the conventional algorithms.

Ultrasound imaging devices are more secure and real-time than other medical diagnosis devices such as X-ray, CT, and MRI. However, due to speckle noise, it exhibits lower resolution and image quality than other than medical diagnostic devices. Therefore, it is difficult to diagnose lesions using ultrasound images due to this noise. In this paper, we propose an algorithm using speckle reducing anisotropic diffusion (SRAD) and a guided filter for speckle noise reduction and edge preservation. First, speckle, which is multiplicative noise, is removed by using SRAD. The remaining noise in the filtered image is converted to additive noise using logarithmic transformation. The additional noise remaining in the filtered image is further removed using a guided filter. Finally, the noise-free image obtained using exponential transform. Experimental results show that the proposed algorithm exhibits an excellent speckle noise reduction and edge preservation performance than the conventional filtering techniques.

This paper presents a novel multidirectional weighted interpolation algorithm for color filter array interpolation. Our proposed method has two contributions to demosaicking. First, different from conventional interpolation methods based on two directions or four directions, the proposed method exploits to greater degree correlations among neighboring pixels along eight directions to improve the interpolation performance. Second, we propose an efficient postprocessing method to reduce interpolation artifacts based on the color difference planes. Compared with conventional state-of-the-art demosaicking algorithms, our experimental results show the proposed algorithm provides superior performance in both objective and subjective image quality. Furthermore, this implementation has moderate computational complexity.

This paper presents a piecewise image denoising with multi-scale block region detector based on quadtree structure for effective image restoration. Proposed piecewise image denoising method suggests multi-scale block region detector (MBRD) by dividing whole pixels of a noisy image into three parts, with regional characteristics: strong variation region, weak variation region, and flat region. These regions are classified according to total pixels variation between multi-scale blocks and are applied principal component analysis with local pixel grouping, bilateral filtering, and structure-preserving image decomposition operator called relative total variation. The performance of proposed method is evaluated by Experimental results. we can observe that region detection results generated by the detector seems to be well classified along the characteristics of regions. In addition, the piecewise image denoising provides the positive gain with regard to PSNR performance. In the visual evaluation, details and edges are preserved efficiently over the each region; therefore, the proposed method effectively reduces the noise and it proves that it improves the performance of denoising by the restoration process according to the region characteristics.

We propose an adaptive spatial postprocessing algorithm for using in block-based discrete cosine transform (BDCT) coded images. The proposed method is comprised of three procedural steps: a thresholding step, a model classification step, and a deblocking filtering step. First, we apply adaptive thresholding to extract the pixel vector containing the blocking artifacts. This threshold has a strong correlation with image quality and the standard deviation of the pixel vector. Next, we update block types using a simple rule, thus influencing deblocking performance. Our research proposes that the activity of the pixel vector can be used to categorize the pixel vector model. With the pixel vector model determined, we are then able to apply a suitable filter to each model with different local properties. For example, we are able to utilize a directional filter to reduce ringing artifacts in edge regions. With various images and bit-rate conditions, images deblocked by the proposed method exhibit both significant visual quality improvement and PSNR gain, with fairly low computational complexity.

Demosaicking is an important step in generating a full-color image from a color filter array image captured by a single-sensor digital camera. In this paper, we propose a new demosaicking algorithm which uses Taylor series and an adaptive fusion strategy. For generating directional predictors, we present a new interpolator based on Taylor series. The directional predictors are used together with spatial and spectral correlations. Using these predictors and the adaptive fusion strategy improves the green channel interpolation, in turn helping to reconstruct the red and blue channel interpolations. Experiments comparing the proposed algorithm to some other demosaicking algorithms show that the proposed algorithm provides superior performance in terms of both objective and subjective image quality.

To generate a high resolution image from a low resolution one, interpolation plays a crucial role. However, conventional interpolation methods including edge-based interpolation methods have some drawbacks such as the limited number of edge directions, imprecise edge detection, and inefficient interpolation. To overcome these shortcomings, we propose a new edge-directed interpolation method, which has three aims: various edge directions, reliable edge detection, and outstanding interpolation. Since the number of candidate edge directions in the proposed method is flexible, we can use several edges included in the high resolution image. To accurately determine the edge direction, we use autocorrelation of neighboring pixels for candidate directions based on the duality between a high resolution image and its corresponding low resolution image. For the interpolation step, we utilize a Blackman–Harris windowed-sinc weighted average filter where we use correlation values obtained in the edge detection step as weights. Experimental results show that the proposed method outperforms conventional methods in terms of both the subjective and the objective results.

In this paper, we present a new depth upsampler in which an upsampled depth map is computed at each pixel as the average of neighboring pixels, weighted by color and depth intensity filters. The proposed method features two parameters, an adaptive smoothing parameter and a control parameter. The adaptive smoothing parameter is determined based on the ratio between a depth map and its corresponding color image. The adaptive smoothing parameter is used to control the dynamic range of the color-range filter. The control parameter assigns a larger weighting factor to pixels in the object to which a missing pixel belongs. In a comparison with five existing upsamplers, the proposed method outperforms all five in terms of both objective and subjective quality.

In this paper, we present a novel color image demosaicking algorithm using a voting-based edge direction detection method and a directional weighted interpolation method. By introducing the voting strategy, the interpolation direction of the center missing color component can be determined accurately. Along the determined interpolation direction, the center missing color component is interpolated using the gradient weighted interpolation method by exploring the intra-channel gradient correlation of the neighboring pixels. As compared with the latest demosaicking algorithms, experiments show that the proposed algorithm provides superior performance in terms of both objective and subjective image qualities.

Synthetic aperture radar (SAR) images map Earth’s surface at high resolution, regardless of the weather conditions or sunshine phenomena. Therefore, SAR images have applications in various fields. Speckle noise, which has the characteristic of multiplicative noise, degrades the image quality of SAR images, which causes information loss. This study proposes a speckle noise reduction algorithm while using the speckle reducing anisotropic diffusion (SRAD) filter, discrete wavelet transform (DWT), soft threshold, improved guided filter (IGF), and guided filter (GF), with the aim of removing speckle noise. First, the SRAD filter is applied to the SAR images, and a logarithmic transform is used to convert multiplicative noise in the resulting SRAD image into additive noise. A two-level DWT is used to divide the resulting SRAD image into one low-frequency and six high-frequency sub-band images. To remove the additive noise and preserve edge information, horizontal and vertical sub-band images employ the soft threshold; the diagonal sub-band images employ the IGF; while, the low- frequency sub-band image removes additive noise using the GF. The experiments used both standard and real SAR images. The experimental results reveal that the proposed method, in comparison to state-of-the art methods, obtains excellent speckle noise removal, while preserving the edges and maintaining low computational complexity.

A new bilateral frame rate upconversion algorithm using the image texture complexity characteristic is proposed. In the proposed algorithm, as the cost function is adjusted according to the image texture complexity, the accuracy of motion estimation (ME) is improved. In the experimental results, the proposed algorithm outperforms the conventional bilateral ME process by 0.32 dB.

The existing thumbnail extraction method generates a thumbnail by reducing the reconstructed frame after a full decoding process. This method is complex and requires considerable time, particularly for higher resolution video. To alleviate these issues, a fast thumbnail extraction method which utilises an intra-prediction mode for high-efficiency video coding (HEVC) is presented. The proposed method reconstructs only the 4×4 boundary pixels needed for a thumbnail image based on the intra-prediction mode. Experimental results indicate that the proposed method significantly reduces the computational complexity and extraction time for a thumbnail image. The visual quality of thumbnail images obtained with this method does not differ significantly from that of images extracted after a full decoding process.

In this paper, we present a perception-based tone mapping technique using histogram modification for displaying high dynamic range image. HDR (high dynamic range) tone mapping algorithms are used to display HDR image on LDR (low dynamic rnage) devices. Although perception-based tone mapping methods provides better performance, it dose not always produce good results for a wide variety of images. The proposed method reduces dynamic range by using the perception-based tone mapping function and histogram modification. A derivative of perception-based tone mapping function is used as constraint function of histogram and additional compensation process is performed. This method not only improves contrast by adopting different constraints on each pixel value, but also preserves more visual details. In order to prevent over enhancement, histogram modification technique is applied. Furthermore, it can control the rate of image contrast using control parameters. Subjective and objective evaluations show that proposed algorithm is better than existing algorithms.

In this paper, a local weighted interpolation method for intra-field deinterlacing is proposed as an improved version of the DCS (deinterlacing with awareness of closeness and similarity) algorithm. The original DCS method is derived from bilateral filter which takes the local spatial closeness and pixel similarity into account when calculating the weight of interpolation. The proposed algorithm achieves three improvements: 1) instead of the line average, a more accurate interpolation filter is used to estimate the center missing pixel; 2) the center-independent interpolation method is proposed to replace the center-dependent interpolation strategy; 3) the adaptive weighted interpolation method is used to improve the accuracy of interpolation. Experimental results show that the proposed algorithm provides superior performance in terms of both objective and subjective image qualities when compared with other conventional benchmarks, including DCS algorithms with low complexity.

In this paper, a motion adaptive deinterlacing algorithm is proposed. It consists of three parts: (1) modified edge-based line average, (2) pixel-based consequent five-field motion detection, and (3) block-based local characteristic for detecting true motion and calculating the motion intensity by using an improved method which is able to detect the inner part of moving objects precisely as well as to reduce the risk of false detection caused by intrinsic noises in the image. Depending on the detected motion activity level, it combines spatial and temporal methods with weighting factor. Simulations conducted on several video sequences indicate that the performance of the proposed method is superior to the conventional methods in terms of both subjective and objective video quality.

Nowadays, video technology has been successfully improved creating tremendous results. As video technology improve, multimedia devices and demands from users are diversified. Therefore, a video codec used in these devices should support various displays with different resolutions. The technology to generate a higher resolution image from the associated low-resolution image is called interpolation. Interpolation is generally performed in either the spatial domain or the DCT domain. To use the advantages of both domains, we have proposed the new adaptive interpolation algorithm based on edge direction, which adaptively exploits the advantages of both domains. The experimental results demonstrate that our algorithm performs well in terms of PSNR and reduces the blocking artifacts.

In this paper, we proposed tone mapping method which uses threshold function. It was impossible for conventional Multi-Scale Retinex algorithm to express detail parts due to damage in bright region of dynamic range at compression processing. We reduce the damage, which occurs in tone mapping process, by setting the threshold of Log Scale image in this algorithm. The experimental tests demonstrated that proposed algorithm reserves detailed information and significantly improves the quality of image, compared to conventional algorithms.

Nowadays many camera-based advanced driver assistance systems (ADAS) have been introduced to assist the drivers and ensure their safety under various driving conditions. One of the problems faced by drivers is the faded scene visibility and lower contrast while driving in foggy conditions. In this paper, we present a novel approach to provide a solution to this problem by employing deep neural networks. We assume that the fog in an image can be mathematically modeled by an unknown complex function and we utilize the deep neural network to approximate the corresponding mathematical model for the fog. The advantages of our technique are as follows: (i) its real-time operation and (ii) being based on minimal input, that is, a single image, and exhibiting robustness/generalization for various unseen image data. Experiments carried out on various synthetic images indicate that our proposed technique has the abilities to approximate the corresponding fog function reasonably and remove it for better visibility and safety.

In this paper, an error surface-considered modeling algorithm for quarter-pixel motion estimation during video encoding is presented. We previously proposed two algorithms: a model-based quarter-pixel motion estimation (MBQME) algorithm and a hierarchical model-based quarter-pixel motion estimation (HMBQME) algorithm. MBQME is an interpolation-free algorithm that has a minimum motion estimation time, while HMBQME has selective interpolation according to the decision process. Consequently, the peak signal-to-noise ratio (PSNR) for HMBQME is better than that of MBQME, but the motion estimation time is also increased. As an alternative method, we propose an error surface-considered modeling algorithm. In this scheme, the tendency of the error surface is first assessed. Using the strength of the edge at the error surface, we can classify the error surface region as plain or textured. For plain regions, interpolation-free and simple-structured modeling is appropriate for the quarter-pixel motion estimation method. In this case, we modified conventional mathematical modeling algorithm suitable for plain region. For textured regions, additional interpolation is needed for more accurate modeling. We calculate the half-pixel SAD values and perform more accurate modeling so as to find the best motion vector (MV). The experimental results show that the proposed scheme has better PSNR performance than any previous algorithms with minimum motion estimation time.

In most digital cameras, images are obtained by a sensor overlaid by the color filter array (CFA) such as Bayer, demanding a demosaicking procedure to rebuild the full resolution color images. However, due to the nature of sensor, it is necessary to consider denoising step to remove the noise. In this paper, we analyze demosaicking and denoising jointly and show that the proposed method can solve the denoising issue by simple manner, well suppress different level of noises. The proposed algorithm yields comparable performances measured by several image quality assessment (CPSNR, SCIELAB, and FSIM), while the computational cost is low.

Motion estimation is a core process within video coding schemes, because it enables the transmission and storage of video signals, while using a lower bit rate. Various methods have been proposed for motion estimation. A full search algorithm is considered to be the optimal method, but it suffers from a high computational burden. In order to reduce the computations, this paper proposes an adaptive two-step edgebased partial distortion search algorithm. The proposed algorithm reduces computations by reducing the total search point and adjusting the search range. The proposed algorithm is 147 times faster than full search (FS), 12 times faster than normalized partial distortion search, and 1.75 times faster than a two-step edge based partial distortion search (TS-EPDS). It also shows a high video quality, in comparison with FS, in terms of PSNR. The proposed algorithm is suitable for real-time implementation of high-quality digital video applications.

Most demosaicking solutions focus on improving the accuracy of direction estimation for interpolation, and they achieve a highly precise gradient estimation performance. Direction estimation techniques achieve great improvement in objective image quality; however, in terms of subjective image quality there are remaining demosaicking problems to solve. An adaptive window algorithm to reduce demosaicking artefacts by exploiting the colour similarity of neighbouring pixels is introduced. Experimental results confirm that the proposed method produces the best colour peak signal-to-noise ratio and FSIM performance, and achieves great improvement in subjective image quality.

In this paper, we propose an advanced deinterlacing method which uses filters to estimate the edge direction using luminance information. Subsequently, we are able to obtain the luminance values at for missing pixels. The fuzzy logic concept for image processing is discussed with regard to fuzzy membership function representation and fuzzy inference procedures. The fuzzy if-then rules are employed to conduct the determining edge direction. The use of a different membership function for different direction enables the filter to independently characterize separate influences on pixel variation. Simulation results demonstrate that the proposed method has an enhanced performance, both visually and in terms of the peak signal-to-noise ratio, compared with those of conventional deinterlacing methods.

We present a direct image resizing method in the compressed domain. Our method reduces the ringing effect for blocks with vertical or horizontal edge. Moreover, our method omits the inverse discrete cosine transform (IDCT) operation required to represent the image in the spatial domain. We first address an estimation scheme for the sums of the columns and rows within the doubled block. We then propose an IDCT-free block representation scheme using the pseudo-inverse. On the basis of IDCT-free block representation, the ringing effect was significantly reduced when compared to the conventional algorithms such as Dugad-Ahuja and Mukherjee-Mitra. Finally, we propose a hybrid algorithm that uses both the Dugad-Ahuja method and the proposed method. We apply our method only to categorized blocks with vertical or horizontal edge. Otherwise, the Dugad-Ahuja method is used. The proposed algorithm does not require IDCT, which therefore makes it easy to implement in hardware. Furthermore, due to the significant reduction of the ringing effect, our method yields better performance in terms of both objective and subjective views than the conventional methods.

A fuzzy if-then rule-based intra-field deinterlacing method using geometric duality is presented in this paper. The proposed method is a content-based hybrid scheme switching between the well-known edge-based linear average method and the proposed geometric duality-based deinterlacing method. Conventional deinterlacing methods usually employ edge-based interpolation techniques within pixel-based estimations. However, they are somewhat sensitive to noise and intensity variations in the image. Moreover, their performance is visually unacceptable due to their failure to estimate edge direction. To reduce this sensitivity, the proposed algorithm investigates features from low-resolution images, and applies them to high-resolution images to calculate the missing pixels. We analyzed properties of the missing pixels and modeled them using geometric regularity. Depending on the features of the region, the missing pixels were interpolated in different ways. The proposed algorithm is computationally feasible and promises to be a good candidate for a low-cost hardware interpolator.

This paper proposes a fine edge-preserving deinterlacing algorithm that is based on local gradient features. The proposed algorithm uses horizontal and vertical gradient operators to find the various edges, and then determines the dominant edge using the surrounding edges of the interpolating pixel, to increase the edge direction accuracy. The proposed method uses different reference pixels and weighting factors depending on the dominant edge direction. Experimental results demonstrate that the proposed method outperforms the conventional deinterlacing methods.

Video deinterlacing can be realized using time-space interpolation filters. To improve video deinterlacing quality with respect to missing pixels on moving diagonal lines, we developed a fuzzy concept that utilizes deinterlacing methods. The proposed algorithm consists of three parts. The first part is fuzzy rule-assisted edge-preserving-based deinterlacing (FED), which has an edge-preserving unit that utilizes fuzzy theory to find the most accurate edge direction and interpolates the missing pixels. Using the introduced gradients in the interpolation, the vertical resolution in the deinterlaced image is subjectively concealed. The second part is weighted fuzzy-reasoning-assisted deinterlacing (WFD), which works in the spatio-temporal domain. In this part, the computed weights are considered and multiplied by the candidate deinterlaced pixels, which successively build approximations of the deinterlaced sequence. The third part of the proposed algorithm is weighted fuzzy switching filtering, which analyzes the suitability of each method on system performance and uses a switching algorithm between FED and WFD. The outcome of image interpolation can be adjusted continuously by varying the setting of the membership function for fuzzy inference. Compared with conventional image interpolation methods, the algorithm presented in this paper provides improved edge quality in the deinterlaced image without the introduction of evident artifacts.