Search

Video deblurring, which aims at restoring the sharp video from blurry video, is drawing increasing attention in the field of computer vision. In this paper, a method called Multi-Attention Convolutional Neural Network (MACNN) consisting of the temporal-spatial attention module, the frame channel attention module, and the feature extraction-reconstruction module is proposed. First, we use the temporal-spatial attention module and the frame channel attention module to capture features with temporal and spatial information existing across neighboring frames. Then, these captured features are fused and reconstructed to restore the sharp frame. Last but not least, we train MACNN together with a content loss and a perceptual loss in an end-to-end manner to recover realistic video details. Both quantitative and qualitative evaluation results on standard benchmarks demonstrate the proposed MACNN is superior to the state-of-the-art methods in terms of accuracy, efficiency, and visual effect.

Space target super resolution is a domain-specific single image super resolution problem aiming to help distinguish the satellite and spacecrafts from numerous space debris. Compared to other object super resolution problem, images for space target are always in low quality with varies of degradation condition, as a result of long distance and motion blur, which significantly reduce the manual classification reliability, especially for these small targets, e.g., satellite payloads. To address this challenge, we present an end-to-end super-resolution and deblurring network (SRDN). Concretely, focusing on the low-resolution (LR) space target images with blind motion blur, we integrate the super-resolution and deblur function together, improving the image quality bya unified generative adversarial network (GAN)- based framework. We implement a deblur module by using contrastive learning to extract degradation feature, and add symmetrical down and up sampling modules to the super resolution network in order to restore texture information, while shortcut connections are redesigned to maintain the global similarity. Extensive experiments on the public satellite dataset BUAA-SID-share1.5 demonstrate that our network outperforms the state-of-the-art super-resolution and deblur methods.

Deep learning-based blind image deblurring plays an essential role in solving image blur since all existing kernels are limited in modeling the real world blur. Thus far, researchers focus on powerful models to handle the deblurring problem and achieve decent results. For this work, in a new aspect, we discover the great opportunity for image enhancement (e.g., deblurring) directly from RAW images and investigate novel neural network structures benefiting RAW-based learning. However, to the best of our knowledge, there is no available RAW image deblurring dataset. Therefore, we built a new dataset containing both RAW images and processed sRGB images and design a new model to utilize the unique characteristics of RAW images. The proposed deblurring model, trained solely from RAW images, achieves the state-of-art performance and outweighs those trained on processed sRGB images. Furthermore, with fine-tuning, the proposed model, trained on our new dataset, can generalize to other sensors. Additionally, by a series of experiments, we demonstrate that existing deblurring models can also be improved by training on the RAW images in our new dataset. Ultimately, we show a new venue for further opportunities based on the devised novel raw-based deblurring method and the brand-new Deblur-RAW dataset., Comment: IEEE Transactions on Multimedia

The amount of multimedia data, such as images and videos, has been increasing rapidly with the development of various imaging devices and the Internet, bringing more stress and challenges to information storage and transmission. The redundancy in images can be reduced to decrease data size via lossy compression, such as the most widely used standard Joint Photographic Experts Group (JPEG). However, the decompressed images generally suffer from various artifacts (e.g., blocking, banding, ringing, and blurring) due to the loss of information, especially at high compression ratios. This article presents a feature-enriched deep convolutional neural network for compression artifacts reduction (FeCarNet, for short). Taking the dense network as the backbone, FeCarNet enriches features to gain valuable information via introducing multi-scale dilated convolutions, along with the efficient 1 x 1 convolution for lowering both parameter complexity and computation cost. Meanwhile, to make full use of different levels of features in FeCarNet, a fusion block that consists of attention-based channel recalibration and dimension reduction is developed for local and global feature fusion. Furthermore, short and long residual connections both in the feature and pixel domains are combined to build a multi-level residual structure, thereby benefiting the network training and performance. In addition, aiming at reducing computation complexity further, pixel-shuffle-based image downsampling and upsampling layers are, respectively, arranged at the head and tail of the FeCarNet, which also enlarges the receptive field of the whole network. Experimental results show the superiority of FeCarNet over state-of-the-art compression artifacts reduction approaches in terms of both restoration capacity and model complexity. The applications of FeCarNet on several computer vision tasks, including image deblurring, edge detection, image segmentation, and object detection, demonstrate the effectiveness of FeCarNet further.

The computer-based approach is required for the non-invasive detection of chronic liver diseases that are asymptomatic, progressive, and potentially fatal in nature. In this study, we review the computer-aided diagnosis of hepatic lesions in view of diffuse- and focal liver disorders. This survey mainly focuses on three image acquisition modalities: ultrasonography, computed tomography, and magnetic resonance imaging. We present the insightful analysis with pros and cons for each preliminary step, particularly preprocessing, attribute analysis, and classification techniques to accomplish clinical diagnostic tasks. In preprocessing, we explore and compare commonly used denoising, deblurring and segmentation methods. Denoising is mainly performed with nonlinear models. In contrast, deep neural networks are frequently applied for deblurring and automatic segmentation of region-of-interest. In attribute analysis, the most common approach comprises texture properties. For classification, the support vector machine is mainly utilized across three image acquisition modalities. However, comparative analysis shows the best performance is obtained by deep learning-based convolutional neural networks. Considering biopsy samples or pathological factors such as overall stage, margin, and differentiation can be helpful for improving the prediction performance. In addition, technique breakthrough is expected soon with advances in machine learning models to address data limitation problems and improve the prediction performance.

Deblurring can not only provide visually more pleasant pictures and make photography more convenient, but also can improve the performance of objection detection as well as tracking. However, removing dynamic scene blur from images is a non-trivial task as it is difficult to model the non-uniform blur mathematically. Several methods first use single or multiple images to estimate optical flow (which is treated as an approximation of blur kernels) and then adopt non-blind deblurring algorithms to reconstruct the sharp images. However, these methods cannot be trained in an end-to-end manner and are usually computationally expensive. In this paper, we explore optical flow to remove dynamic scene blur by using the multi-scale spatially variant recurrent neural network (RNN). We utilize FlowNets to estimate optical flow from two consecutive images in different scales. The estimated optical flow provides the RNN weights in different scales so that the weights can better help RNNs to remove blur in the feature spaces. Finally, we develop a convolutional neural network (CNN) to restore the sharp images from the deblurred features. Both quantitative and qualitative evaluations on the benchmark datasets demonstrate that the proposed method performs favorably against state-of-the-art algorithms in terms of accuracy, speed, and model size., accepted by tcsvt

Camera-based multi-class (1D and 2D) barcode detectors that can help in decoding barcodes in different complex scenes have huge potential applications in situations where Internet of Things (IoT) is combined with Artificial Intelligence (AI) and Augmented Reality (AR). The decoding rate in such applications under real-life complex scenes is greatly affected by two major factors: First, we cannot accurately localize the barcodes, and secondly, we cannot decode the blur samples. In this paper, we firstly propose a barcode localization algorithm that is capable of regressing 4 vertices of barcodes accurately. Our localization method comprises an anchor-free approach that outputs multi-scale output prediction maps. These segmentation-like maps of each scale are then further divided into three types of maps (classification, centerness, and 8-D regression). 8-D localization result of barcodes along-with classification result is then obtained after post-processing. Secondly, we propose a conditional generative adversarial network-based model for deblurring blur QR codes. Extensive decoding experiments on a challenging complex scene dataset show that our localization and deblurring methods can contribute to improving the decoding rate of existing barcode decoders.

State-of-the-art face restoration methods employ deep convolutional neural networks (CNNs) to learn a mapping between degraded and sharp facial patterns by exploring local appearance knowledge. However, most of these methods do not well exploit facial structures and identity information, and only deal with task-specific face restoration (e.g., face super-resolution or deblurring). In this paper, we propose cross-tasks and cross-models plug-and-play 3D facial priors to explicitly embed the network with the sharp facial structures for general face restoration tasks. Our 3D priors are the first to explore 3D morphable knowledge based on the fusion of parametric descriptions of face attributes (e.g., identity, facial expression, texture, illumination, and face pose). Furthermore, the priors can easily be incorporated into any network and are very efficient in improving the performance and accelerating the convergence speed. Firstly, a 3D face rendering branch is set up to obtain 3D priors of salient facial structures and identity knowledge. Secondly, for better exploiting this hierarchical information (i.e., intensity similarity, 3D facial structure, and identity content), a spatial attention module is designed for the image restoration problems. Extensive face restoration experiments including face super-resolution and deblurring demonstrate that the proposed 3D priors achieve superior face restoration results over the state-of-the-art algorithms.

Current deep deblurring methods pay main attention to learning a transferring network to transfer synthetic blurred images to clean ones. Though achieving significant performance on the training datasets, they still suffer from a weaker generalization capability from training datasets to others with different synthetic blurs, thus resulting in significantly inferior performance on testing datasets. In order to alleviate this problem, we propose a latent contrastive model, Blur Distilling and Information Reconstruction Networks (BDIRNet), to learn image prior and improve the robustness of deep deblurring. The proposed BDIRNet consists of a blur removing network (DistillNet) and a reconstruction network (RecNet). Two kinds of images with almost the same information but different qualities are input into DistillNet to extract identical structure information via contrast latent information and purify the perturbations from other unimportant information like blur. While the RecNet is utilized to reconstruct sharp images based on the extracted information. In addition, inside the DistillNet and RecNet, a statistical anti-interference distilling (SAID) and anti-interference reconstruction (SAIR) modules are proposed to further enhance the robustness of our methods, respectively. Extensive experiments on different datasets show that the proposed methods achieve improved and robust results compared to recent state-of-the-art methods.

Blind image deblurring is a severely ill-posed problem in low-level vision. The success of blind image deblurring relies on statistical priors and well-designed regularizers to obtain a clear image. However, the prior-based method is time-consuming due to a lot of nonlinear calculations. To improve efficiency, this work proposes an enhanced salient edge selection for blind image deblurring. Different from the previous methods that only focus on the salient edge and ignore the image structure, the image sharpening operator is adopted to guide the finer image structure when salient edges provide strong edge information for blur kernel estimation. We find that the latent image restoration can be enhanced by joining the salient edge selection and an image sharpening operator, and the quality of the recovered kernel is hereby improved. By imposing L2 constraints on the salient edge and sharpening operator terms, a new energy function is introduced and an effective alternating optimization strategy is explored. Extensive experiments have been conducted, which shows that our method is more effective compared with state-of-the-art methods.

Motion blur is a common artifact in image processing, specifically in e-health services, which is caused by the motion of a camera or scene. In linear motion cases, the blur kernel, i.e., the function that simulates the linear motion blur process, depends on the length and direction of blur, called linear motion blur parameters. The estimation of blur parameters is a vital and sensitive stage in the process of reconstructing a sharp version of a motion blurred image, i.e., image deblurring. The estimation of blur parameters can also be used in e-health services. Since medical images may be blurry, this method can be used to estimate the blur parameters and then take an action to enhance the image. In this paper, some methods are proposed for estimating the linear motion blur parameters based on the extraction of features from the given single blurred image. The motion blur direction is estimated using the Radon transform of the spectrum of the blurred image. To estimate the motion blur length, the relation between a blur metric, called NIDCT (Noise-Immune Discrete Cosine Transform-based), and the motion blur length is applied. Experiments performed in this study showed that the NIDCT blur metric and the blur length have a monotonic relation. Indeed, an increase in blur length leads to increase in the blurriness value estimated via the NIDCT blur metric. This relation is applied to estimate the motion blur. The efficiency of the proposed method is demonstrated by performing some quantitative and qualitative experiments.

Deblurring images with outliers has always been a significantly challenging problem. Previous methods mainly involved complex operations, such as outlier and light streak detection, or sophisticated image priors for blur-kernel estimation, which increased the difficulty of deblurring images. Therefore, we developed a simple, yet efficient, blind deblurring algorithm in this study for handling images with outliers. To eliminate the impact of outliers during the kernel estimation process, we employed a robust Welsch loss function to characterize the data-fidelity term of our model. We observed that this function could extract significant edges successfully. Therefore, the image regularization term was also described by the same function. Using this unified robust function to describe our model can considerably reduce the complexity of the algorithm. Moreover, we derived a flexible weight function from the Welsch function to further improve the efficiency of our algorithm. To finally obtain accurate latent images, we developed a robust non-blind deblurring approach based on this flexible weight function. The experimental results indicate that our approach outperforms state-of-the-art methods in deblurring images with or without outliers. Compared with the method specifically for outliers, the recovery performance of our method can be improved by 12.9% (considering a dataset with impulse noise), and the execution efficiency is about 1.5 times faster.

Sparse based representation is being used extensively for image restoration. The dictionary learningthrough patch extraction is central to the sparse based schemes. In the process of dictionary learning,a large number of patches will be extracted from high quality images and dictionary will be formed.Hence, over-complete dictionaries will be built. To overcome the complexity associated with overcompletedictionaries many schemes were proposed. Of them, the adaptive sparse domain is thepopular one. Many variations of adaptive sparse domain schemes were proposed. Selection of obviouspatches is common to all. In all these schemes, individual patches will be considered as the basic entityand will be used. This is the reason for the complexity involved in sparse representation. In this paper,to avoid the complexity, the patches are grouped according to the similarity among the patches. Inaddition to reduce the complexity the proposed cluster based scheme considers the self-similarity ofthe patches involved. Hence better performance with less complexity is possible with the proposedschemes. In the process of testing, in addition to uniform blur and Gaussian blur, a combination of thetwo blurs is also considered.

High noise and low spatial resolution are two key confounding factors that limit the qualitative and quantitative accuracy of PET images. Artificial intelligence models for image denoising and deblurring are becoming increasingly popular for the postreconstruction enhancement of PET images. We present a detailed review of recent efforts for artificial intelligence-based PET image enhancement with a focus on network architectures, data types, loss functions, and evaluation metrics. We also highlight emerging areas in this field that are quickly gaining popularity, identify barriers to large-scale adoption of artificial intelligence models for PET image enhancement, and discuss future directions.

Satellite imagery acquired from optical remote sensing systems is often suffered due to several types of blur, such as atmospheric turbulence blur, motion blur, and defocus. Any kind of blur degrades the image quality, as it reduces the sharpness of edges and texture, and hence, spatial resolution is also reduced. This degradation poses a challenge for further automated analysis using such blurred images. Image deconvolution methods are conventionally applied to blurry images to estimate the blur and restore the original image. In this inverse and ill-posed problem, the restoration quality relies on the correct estimation of the point spread function that caused blur in the image. We aim to restore the blurred satellite imagery, corrupted by Gaussian blur, using the deep learning framework and hence prove the efficacy of the proposed method over the traditional methods. This paper presents an investigative analysis of the satellite image deblurring problem and simultaneously tackles the problem of low-resolution satellite imagery. A deep convolutional neural network (CNN) architecture is proposed to remove the Gaussian blur artifacts from images and increase their resolution. Experimental evaluation is performed on a hyperspectral image dataset consisting of 8 different terrain types. Results obtained after deblurring the images are compared with state-of-the-art methods based on both subjective and objective image quality measures: peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). The image quality of deblurred images obtained by the proposed deep CNN method demonstrates improved performance over some of the existing CNN methods.

Most image enhancement methods focused on restoration of digitized textual documents are limited to cases where the text information is still preserved in the input image, which may often not be the case. In this work, we propose a novel generative document restoration method which allows conditioning the restoration on a guiding signal in the form of target text transcription and which does not need paired high- and low-quality images for training. We introduce a neural network architecture with an implicit text-to-image alignment module. We demonstrate good results on inpainting, debinarization and deblurring tasks, and we show that the trained models can be used to manually alter text in document images. A user study shows that that human observers confuse the outputs of the proposed enhancement method with reference high-quality images in as many as 30% of cases.

In recent years, the image deblurring task has attracted more and more researchers’ attention. Many researchers are devoted to eliminating motion blur by using a “coarse-to-fine” architecture. This architecture can effectively eliminate motion blur caused by a simple relative displacement. But when using the architecture directly for the face image deblurring task, there would exist some problems. For example, complex network structures make the model difficult to be trained, and a large number of parameters need to be calculated would result in expensive runtime. Since details of images can’t be completely restored, the quality of images will deteriorate, and deblurring visual effect is poor, so in order to solve these issues, combining the “coarse-to-fine” architecture with the “stacked” architecture, we propose a new network architecture-“Stack-based Scale-recurrent Network” for the face image deblurring task. The ConvLSTM network is employed to build a model. We achieves the purpose of improving the visual clarity and the quality of restored face images. And we use the improved encoder-decoder to enhance network performance. Compared with the other deblurring methods, our method can restore clear face images with higher-quality and clearer vision.

As a well-known ill-conditional problem in the image processing field, image deblurring has become a hot topic recently. The prior-based blind image deblurring methods have recently shown promising effectiveness. A lot of advanced algorithms such as dark channel prior, bright channel prior, and local maximum gradient prior are time-consuming since nonlinear operators are involved. Presented in this paper is a fast blind image deblurring algorithm which uses the simplified extreme channel prior (SECP) and gradient cepstrum. The inspiration for this work comes from the fact that the simplified bright channel prior (SBCP) of the clear image has fewer non-one elements than the blurred one. We propose a novel SECP based on the proposed SBCP and the simplified dark channel prior (SDCP). By enforcing the $$L_{0}$$ norm constraint to the terms involving SECP and incorporating them into the traditional deblurring framework, an effective optimization scheme is explored. Furthermore, gradient cepstrum is used to determine the size of the initial kernel and restrain excessive iterations in each scale. Experimental results illustrate that our algorithm outperforms the state-of-the-art deblurring algorithms in terms of computational efficiency and deblurring effect on both benchmark datasets and real-world blur scenes.

Blurred images are difficult to avoid in situations when minor Atmospheric turbulence or camera movement results in low-quality images. We propose a system that takes a blurred image as input and produces a deblurred image by utilizing various filtering techniques. Additionally, we utilize the Siamese Network to match local image segments. A Siamese Neural Network model is used that is trained to account for image matching in the spatial domain. The best-matched image returned by the model is then further used for Signal-to-Noise ratio and Point Spread Function estimation. The Wiener filter is then used to deblur the image. Finally, the results of the deblurring techniques with existing algorithms are compared and it is shown that the error in deblurring an image using the techniques presented in this paper is considerably lesser than other techniques.

Video deblurring is a challenging low-level vision task due to variant blur artifacts caused by factors such as depth variations, high-speed movements and camera shakes. Although significant efforts have been devoted to addressing this task, two challenges of capturing temporal patterns and spatial topologies still remain. In this paper, an attention-based interframe compensation scheme is proposed to address the first challenge. The proposed scheme replaces frames in blurry sequences with newly restored frames, and estimates temporal patterns among the replaced sequence to restore the whole sequence. After each replacement, an attention block is employed to exploit dependencies among restored and blurry frames to capture stable temporal patterns. To tackle the second challenge, we propose an adaptive residual block that dynamically fuses multi-level features via learning location-specific weights. Comprehensive experimental results demonstrate that the proposed method achieves state-of-the-art performance in terms of accuracy, visual effect and model size.

For the success of video deblurring, it is essential to utilize information from neighboring frames. Most state-of-the-art video deblurring methods adopt motion compensation between video frames to aggregate information from multiple frames that can help deblur a target frame. However, the motion compensation methods adopted by previous deblurring methods are not blur-invariant, and consequently, their accuracy is limited for blurry frames with different blur amounts. To alleviate this problem, we propose two novel approaches to deblur videos by effectively aggregating information from multiple video frames. First, we present blur-invariant motion estimation learning to improve motion estimation accuracy between blurry frames. Second, for motion compensation, instead of aligning frames by warping with estimated motions, we use a pixel volume that contains candidate sharp pixels to resolve motion estimation errors. We combine these two processes to propose an effective recurrent video deblurring network that fully exploits deblurred previous frames. Experiments show that our method achieves the state-of-the-art performance both quantitatively and qualitatively compared to recent methods that use deep learning., 17 pages, Camera-ready version for ACM Transactions on Graphics (TOG) 2021

Out-of-focus blurring of the QR code is very common in mobile Internet systems, which often causes failure of authentication as a result of a misreading of the information hence adversely affects the operation of the system. To tackle this difficulty, this work firstly introduced an edge prior information, which is the average distance between the center point and the edge of the clear QR code images in the same batch. It is motivated by the theoretical analysis and the practical observation of the theory of CMOS image sensing, optics information, blur invariants, and the invariance of the center of the diffuse light spots. After obtaining the edge prior information, combining the iterative image and the center point of the binary image, the proposed method can accurately estimate the parameter of the out-of-focus blur kernel. Furthermore, we obtain the sharp image by Wiener filter, a non-blind image deblurring algorithm. By this, it avoids excessive redundant calculations. Experimental results validate that the proposed method has great practical utility in terms of deblurring quality, robustness, and computational efficiency, which is suitable for barcode application systems, e.g., warehouse, logistics, and automated production.

Active image sensors can only record scenes at low frame rates. The dynamic vision sensor (DVS) can capture high-speed scenes under challenging lighting conditions. However, DVS discards absolute light intensity information. The dynamic and active pixel vision image sensor (DAVIS) can output events and low-frame-rate grayscale images simultaneously. This paper proposes a high frame rate video reconstruction and deblurring algorithm based on events and low-speed blurred image sequences from DAVIS. First, we visualized events as a sequence of event images and performed image denoising process. Then, we used the extended 3D partial recursive search (E-3DPRS) method to generate high-performance motion vector field (MVF) between adjacent denoising event images with low computational complexity. MVF reflects the object motion trajectory with high time resolution. Furthermore, MVF is used as a priori information to restore potentially sharp images, which alleviates the highly ill-posed deblurring process. At last, we reconstructed a high frame rate sharp video based on a sharp image sequence and MVFs. Extensive experiments show that our proposed algorithm outperforms the existing state-of-the-art methods and achieves better average feature similarity and structural similarity.

Video deblurring is still a challenging low-level vision task since spatio-temporal characteristics across both the spatial and temporal domains are difficult to model. In this article, to model the temporal information, we develop a non-local block which estimates inter-frame similarity and inter-frame difference. Specially, for modeling the spatial characteristics and restoring sharp frame details, we propose a recursive block that iteratively refines feature maps generated at the last iteration. In addition, a novel temporal loss function is introduced to ensure the temporal consistency of generated frames. Experimental results on public datasets demonstrate that our method achieves state-of-the-art performance both quantitatively and qualitatively.

Blind image deblurring is a long standing challenging problem in image processing and low-level vision. Recently, sophisticated priors such as dark channel prior, extreme channel prior, and local maximum gradient prior, have shown promising effectiveness. However, these methods are computationally expensive. Meanwhile, since these priors involved subproblems cannot be solved explicitly, approximate solution is commonly used, which limits the best exploitation of their capability. To address these problems, this work firstly proposes a simplified sparsity prior of local minimal pixels, namely patch-wise minimal pixels (PMP). The PMP of clear images is much more sparse than that of blurred ones, and hence is very effective in discriminating between clear and blurred images. Then, a novel algorithm is designed to efficiently exploit the sparsity of PMP in deblurring. The new algorithm flexibly imposes sparsity inducing on the PMP under the MAP framework rather than directly uses the half quadratic splitting algorithm. By this, it avoids non-rigorous approximation solution in existing algorithms, while being much more computationally efficient. Extensive experiments demonstrate that the proposed algorithm can achieve better practical stability compared with state-of-the-arts. In terms of deblurring quality, robustness and computational efficiency, the new algorithm is superior to state-of-the-arts. Code for reproducing the results of the new method is available at https://github.com/FWen/deblur-pmp.git., Comment: 14 pages, 16 figures

Image deblurring is a classic and important problem in industrial fields, such as aviation photo restoration, object recognition in robotics, and autonomous vehicles. Blurry images in real-world scenarios consist of mixed blurring types, such as a natural motion blurring owing to shaking of the camera. Fast deblurring does not deblur the entire image because it is not the best option. Considering the computational costs, it is also better to have an alternative kernel to deblur different objects at a high-semantic level. To achieve better image restoration quality, it is also beneficial to combine the blurring category location and important structural information in terms of specific artifacts and degree of blurring. The goal of blind image deblurring is to restore sharpness from the unknown blurring kernel of an image. Recent deblurring methods tend to reconstruct prior knowledge, neglecting the influence of blur estimation and visual fidelity on image details and structure. Generative adversarial networks(GANs) have recently been attracting considerable attention from both academia and industry because GAN can perfectly generate new data with the same statistics as the training set. Therefore, this study proposes a generative neural architecture and an edge attention algorithm developed to restore vivid multimedia patches. Joint edge generation and image restoration techniques are designed to solve the low-level multimedia retrieval. This multipath refinement fusion network (MRFNet) can not only perform deblurring of images directly but also individual the frames separately from videos. Ablation experiments validate that our generative adversarial network MRFNet performs better in joint training than in multimodel. Compared to other GAN methods, our two-phase method exhibited state-of-the-art performance in terms of speed and accuracy as well as has a significant visual improvement.

The dynamic non-uniform blur caused by Radio Frequency Identification (RFID) multi-label motion seriously affects the identification and location of labels. It is an ill-posed inverse problem for that the blur kernel and sharp image are unknown. The traditional method of removing the blur is very time-consuming. In this work, we propose Multi-scale Recursive Codec Network based on the Authority Parameter (MRCN-AP) to deblur RFID multi-label images in a vision-based RFID multi-label 3D measurement system. This network is composed of a stack of three encoder-decoder subnets of different scales, which restores the blurry image in an end-to-end manner, and extracts the detail edge on each scale effectively from coarse to fine. The proposed authority parameters reduce the parameters memory of redundant networks and improve the speed of the deblurring network. Also, we propose new large-scale RFID multi-label blur-sharp image pairs captured by the dual CCD camera. The proposed model is implemented on an extended dataset. We prove that our method improves the speed by at least 0.55 s, and also increases Peak Signal to Noise Ratio (PSNR) by 2.43dB. Besides, better visual effects are obtained by MRCN-AP deblurring network for RFID multi-label image, which is more conducive to subsequent positioning and optimization.

Image deblurring is an important problem encountered in many image restoration tasks. To remove the motion blur of images captured from dynamic scenes, various Convolutional Neural Networks (CNNs) based methods are developed to restore the latent sharp image via an end-to-end trainable. However, these CNNs-based methods cannot restore enough structure details as no significant information is provided by the blurry input only. In this work, we propose a reference-guided deep deblurring method by incorporating the high-quality reference image into the deep network for better deblurring effect. Concretely, the correlation between the blurry input and the reference image is computed in the high-level feature space, and further represented by the similarity maps. To pick up the most relevant similarity maps to the input, the selective attention module is designed, and the selected similarity maps are decoded to reconstruct the deblurred sharp image. Extensive experimental results on two public datasets demonstrate that the proposed method achieves superior performance to existing methods quantitatively and qualitatively. More ablation experiments also validate that the favourable deblurred results can still be obtained even if the reference image is not similar with the input image.

In this paper, a novel weighted anisotropic total variational (WATV) model is proposed for image restoration. In this model, a weight function is defined to process degraded images, which is introduced into the $$\ell _{1}$$ norm-based regularization. In order to efficiently compute the restored images, the alternating direction method of multipliers (ADMM) is explored and the according convergence is analyzed briefly. The proposed model is applied to recover images with blurring and noise. Compared experiments of deblurring, denoising as well as restoration are conducted to show the effectiveness and efficiency of the proposed WATV model.

In image-related applications, the recorded images are the blurry version of the original image that usually depicts any scene. Due to the optical aberrations, atmospheric distortions, and motion of objects in the scene, blur occurs in the images and degrades image quality. Various image deblurring methods are modeled in the existing works, but accurately recovering the exact true image from the single recorded image or the set of images still results in challenging medical imaging applications. Thus, an effective image deblurring method named Exponential-Ant Cuckoo Search Optimization (Exponential-ACSO) is developed in this research to find the spatial information from the blurred image. The proposed Exponential-ACSO is designed by integrating the Exponential Weighted Moving Average (EWMA) with Ant Lion Optimization (ALO) and Cuckoo Search (CS) algorithm. The computation of new pixel values for noisy pixels makes the image deblurring process more robust and accurate. The objective function is considered to find the optimal fitness value for the parameters of kernel estimation. However, the proposed Exponential-ACSO showed better performance for the metrics, like peak signal-to-noise ratio (PSNR), second derivative like the measure of enhancement (SDME), and structural similarity index (SSIM) with the values 29.756, 33.562, and 0.6988, respectively.

Objects moving at high speed along complex trajectories often appear in videos, especially videos of sports. Such objects travel a considerable distance during exposure time of a single frame, and therefore, their position in the frame is not well defined. They appear as semi-transparent streaks due to the motion blur and cannot be reliably tracked by general trackers. We propose a novel approach called Tracking by Deblatting based on the observation that motion blur is directly related to the intra-frame trajectory of an object. Blur is estimated by solving two intertwined inverse problems, blind deblurring and image matting, which we call deblatting. By postprocessing, non-causal Tracking by Deblatting estimates continuous, complete, and accurate object trajectories for the whole sequence. Tracked objects are precisely localized with higher temporal resolution than by conventional trackers. Energy minimization by dynamic programming is used to detect abrupt changes of motion, called bounces. High-order polynomials are then fitted to smooth trajectory segments between bounces. The output is a continuous trajectory function that assigns location for every real-valued time stamp from zero to the number of frames. The proposed algorithm was evaluated on a newly created dataset of videos from a high-speed camera using a novel Trajectory-IoU metric that generalizes the traditional Intersection over Union and measures the accuracy of the intra-frame trajectory. The proposed method outperforms the baselines both in recall and trajectory accuracy. Additionally, we show that from the trajectory function precise physical calculations are possible, such as radius, gravity, and sub-frame object velocity. Velocity estimation is compared to the high-speed camera measurements and radars. Results show high performance of the proposed method in terms of Trajectory-IoU, recall, and velocity estimation.

In this paper, exploratory and innovative research is done on the implementation technique of artistic color virtual reality for similarity image recovery. Based on similarity images, a nonlocal natural image before the regular term is proposed to deal with the single-image blind deblurring problem. This paper designs a new artistic color virtual reality realization technology based on similarity image restoration, which exploits the low-rank property between nonlocal similarity blocks in images and combines a strong convex term to enhance the convexity of the artistic color virtual reality model. We analyze virtual reality interaction design from the perspective of art color design, sort out the concept and content of design, analyze the elements, design principles, and evaluation criteria included in virtual reality interaction art color design, and explore the conceptual principles of virtual reality interaction art color design. A full understanding of the characteristics of the medium of the virtual reality interaction can help us to better use this medium as a tool to create works that aim to bring higher quality and experiential feeling with a perceptual communication method that is closer to natural interaction. Combining the power of technology, artistic colourful thinking, and a design approach paves the way forward. The study shows that virtual reality technology can effectively improve the status quo and promote the cultivation of professional practice ability in art color design, which is conducive to the cultivation of applied design talents.

The method of blind deblurring based on machine learning can effectively deal with the blurred images in the real world. However, existing multi-level architectures can lead to problems such as the inability to reserve edges, the expected introduction of artifacts and ghosts when deblurring. Multi methods have found that using L 0 norm to realize image sparse representation can help keeping main structure of images. In this paper, an edge extraction module based on L 0 sparse representation is proposed to preserve the edge of images, which is embedded in a multi-scale recurrent network(SRN). When the current scale transmits information to the next scale, edge enhancement is performed using the edge extraction module. Furthermore, considering the correlation among pixels and the correlation among channels, we introduce dual-attention mechanism into the encoder-decoder structure. The deblurring experiment was carried out on the GOPRO dataset. Comparing with 5 state-of-art methods qualitatively and quantitatively, the experimental results show that the proposed method can better preserve the image edges and effectively avoid the artifact of the image. And the peak signal-to-noise ratio of the proposed method are improved compared with other methods.

The physical process used in CCD cameras for image formation makes it imperative to simultaneously upsample and deblur the captured images. In this paper, we provide an efficient scheme to solve this problem through a nonlinear end-to-end mapping carried out by a novel deep light-weight residual neural network. The proposed network is designed based on two main modules, namely, image upsampling and image deblurring, aimed for carrying out simultaneously the two tasks involved with the problem. The proposed network employs a residual block with a capacity of generating features in multiple receptive fields and enhancing the network’s representational capability. The proposed network is extensively experimented using benchmark datasets for image restoration.

Face images captured in unconstrained settings may suffer from one or multiple degradations, which would degrade the visual aesthetics of images and the performance of face recognition methods. However, many current methods only focus on a specific degradation or restoring the images without considering face identity. To address these problems, an identity-preservation-based deep learning method is proposed for super-resolving blurry face images. First, an extra recognition module is designed and integrated with the restoration module to extract different levels of identity-related and semantic features. Second, an assemble loss function is developed to use the identity preservation information as regularization and prior to guide the restoration and recognition process. Finally, qualitative and quantitative evaluations are conducted to demonstrate the effectiveness of the proposed method for face recovery and face recognition. The results indicate that facial identity can serve as an effective prior to face image restoration.

In dynamic scene deblurring, recent neural network–based methods have been very successful. But with the improvement of deep deblurring performance, network structure and learning become more complicated. Compared with large‐scale network parameters and complex network structures, an iterative multi‐scale residual network to achieve a more effective parameter sharing scheme is proposed. In each iterative unit, fast multi‐scale residual blocks to replace superimposed convolutional layers or classic residual blocks are used. On the basis of preventing model overfitting, the receptive field of the network is increased. At the same time, the gated recurrent unit is introduced to connect modules of different stages. The model does not rely on the estimation of the blur kernel and directly generates sharp images in an end‐to‐end manner. The experimental structure on the benchmark dataset and real‐world images showed that this method has better quality than the existing methods in terms of large‐scale blur and subjective perception effects, both in quantitative and qualitative terms.

Video-deblurring has achieved excellent results by using deep learning approaches. How to capture the dynamic spatio-temporal information in the videos is crucial on deblurring. In this paper, we propose a two-stream DeblurGAN which combines a 3D stream with a 2D stream to deblur. The 3D convolution provides spatial and temporal invariance to restore the foreground of frames, while the 2D convolution is sufficient to deal with spatial features, given a relatively consistant background. Thus, our model takes advantage of the great processing power of the 3D stream to handle the foreground which usually contains more dynamical motion blur, and the advantage of the simplicity of the 2D stream to handle the mostly consistent background. We have the full advantage of combining both the 3D convolution and the 2D convolution. Then we take the two-stream model as the generator and adopt the adversarial learning. We test our model on the VideoDeblurring and GOPRO datasets, and compare with other methods which we have listed. Our method outperforms others in the Peak Signal-to-Noise Ratio (PSNR), especially shows a good performance handling the foreground with obvious motion blur.

Extraction of news text captions aims at a digital understanding of what is happening in a specific region during a certain period that helps in better communication between different nations because we can easily translate the plain text from one language to another. Moving text captions causes blurry effects that are a significant cause of text quality impairments in the news channels. Most of the existing text caption detection models do not address this problem in a way that captures the different dynamic motion of captions, gathers a full news story among several frames in the sequence, resolves the blurring effect of text motion, offers a language-independent model, or provides it as an end-to-end solution for the community to use. We process the frames coming in sequence and extract edge features using either the Hough transform or our color-based technique. We verify text existence using a Convolutional Neural Network (CNN) text detection pre-trained model. We analyze the caption motion status using hybrid pre-trained Recurrent Neural Network (RNN) of Long Short-Term Memory (LSTM) type model and correlation-based model. In case the motion is determined to be horizontal rotation, there are two problems. First, it means that text keeps rotating with no stop resulting in a high blurring effect that affects the text quality and consequently resulting in low character recognition accuracy. Second, there are successive news stories which are separated by the channel logo or long spaces. We managed to solve the first problem by deblurring the text image using either Bicubic Spline Interpolation (BSI) technique or the Denoising Autoencoder Neural Network (DANN). We solved the second problem using a Point Feature Matching (PFM) technique to match the existing channel logo with the channels’ logo database (ground truth). We evaluate our framework using Abbyy® SDK as a standalone tool used for text recognition supporting different languages.

This paper introduces an efficient deblurring image method based on a convolution-based and an iterative concept. Our method does not require specific conditions on images, so it can be widely applied for unspecific generic images. The kernel estimation is firstly performed and then will be used to estimate a latent image in each iteration. The final deblurred image is obtained from the convolution of the blurred image with the final estimated kernel. However, image deblurring is an ill-posed problem due to the nonuniqueness of solutions. Therefore, we propose a smoothing function, unlike previous approaches that applied piecewise functions on estimating a latent image. In our approach, we employ L2-regularization on intensity and gradient prior to converging to a solution of the deblurring problem. Moreover, our work is based on the quadratic splitting method. It guarantees that each subproblem has a closed-form solution. Various experiments on synthesized and real-world images confirm that our approach outperforms several existing methods, especially on the images corrupted by noises. Moreover, our method gives more reasonable and more natural deblurred images than those of other methods.

Nuclear radiation-contaminated video deblurring is an important issue of the robot vision system and has been widely studied. In this paper, a hybrid radiation-contaminated video frame enhancement ...

Blind image deblurring is a conundrum because there are infinitely many pairs of latent image and blur kernel. To get a stable and reasonable deblurred image, proper prior knowledge of the latent image and the blur kernel is urgently required. Different from the recent works on the statistical observations of the difference between the blurred image and the clean one, our method is built on the surface-aware strategy arising from the intrinsic geometrical consideration. This approach facilitates the blur kernel estimation due to the preserved sharp edges in the intermediate latent image. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods on deblurring the text and natural images. Moreover, our method can achieve attractive results in some challenging cases, such as low-illumination images with large saturated regions and impulse noise. A direct extension of our method to the non-uniform deblurring problem also validates the effectiveness of the surface-aware prior.

Out-of-focus blur is a common image degradation phenomenon that occurs in case of lens defocusing. The out-of-focus blur kernel is usually modeled as a Gaussian function or a uniform disk in previous work. In this paper, we propose that it can be more accurately depicted using the generalized Gaussian (GG) function. This is motivated by the theoretical analysis of the out-of-focus blur and the practical observation of real blur kernels. We show that as the out-of-focus blur kernels are of specific shapes, the GG function can be further simplified to a single-parameter model. We estimate the parameter of the GG blur kernel from image patches containing step edges, and obtain the clear image by non-blind image deblurring. Experimental results validate that the proposed GG blur kernel estimation algorithm outperforms the state-of-the-art ones deploying either parametric (disk and Gaussian) or nonparametric kernels, and consequently benefits the image deblurring process.

The purpose of image motion deblur is to recover the underlying clear image from the corresponding blur image. In most traditional methods, the image recovery task is formulated as a problem of blur core estimation and use a priori to calculate. In this paper we proposes a generative adversarial network(GAN) model based on the mobilenet-V3 network structure to meet the needs of motion blurred image recovery on mobile devices. Based on traditional evaluation indicators, we propose a new evaluation metric on mobile device. Extensive experiments show that our method is superior to the competing methods.

We investigate two aspects of network architecture design for dynamic scene deblurring: (1) Learning blur characteristics and their location in dynamic scenes, which corresponds to learning what and where to attend in the channel and spatial axes, respectively. In this regard, we design an attention-adaptive module (AAM), the innovation of which is that it adaptively determines the arrangement of channel and spatial attention modules (i.e., sequentially or in parallel). Ablation experiments verified the effectiveness of the AAM by incorporating it into existing deblurring convolutional neural network (CNN) architectures. (2) Intuitively, geometric variations are widely observed in objects in dynamic scenes because different spatial regions are blurred by different motion kernels. However, owing to the fixed geometric structures in their modules, regular CNNs fail to adapt to these variations. Accordingly, we propose a deformable convolutional module (DCM) to handle geometric variations. Preliminary experiments demonstrated that incorporating the AAM and DCM into existing deblurring models can significantly improve performance. Moreover, it was empirically verified that an encoder–decoder ResBlock network incorporating the proposed modules compares favorably with state-of-the-art methods.

The current gold standard in solving image processing and computer vision tasks is using supervised learning of deep neural networks (DNNs), requiring large-scale datasets of input-output pairs. In many scenarios in which the output is an image – e.g., medical image analysis, image denoising, deblurring, super-resolution, dehazing, segmentation and optical flow estimation – the collection of labelled image pairs for training is either time-consuming or limited to simple degradation models. Indeed, there is an increasing body of work targeted at weakly supervised training, accompanied with different unsupervised loss functions. This work dives into the regime of Deep-Energy , a task-driven training approach that substitutes the generic loss with minimization of energy functions using DNNs. Such energy functions, often formulated as a combination of a data-fidelity term along with an application-specific prior, are essentially unsupervised as they do not assume knowledge of the output image. As opposed to classic energy minimization, where computationally-intensive inference is performed for each new image, our network, once trained, can compute the output with a single forward-pass operation. By incorporating application-specific domain knowledge to the loss function, we are able to use real-world images, thus decreasing the dependency on pixel-wise labelled data or synthetic datasets. We demonstrate our approach on three different applications: seeded segmentation, image matting and single image dehazing, showing clear benefits of both speedup and improved accuracy versus the classical energy minimization approach, and competitive performance with respect to fully supervised alternatives.

The papers in this special issue focus on deep learning for image/video restoration and compression. The huge success of deep-learning–based approaches in computer vision has inspired research in learned solutions to classic image/video processing problems, such as denoising, deblurring, dehazing, deraining, super-resolution (SR), and compression. Hence, learning-based methods have emerged as a promising nonlinear signal-processing framework for image/ video restoration and compression. Recent works have shown that learned models can achieve significant performance gains, especially in terms of perceptual quality measures, over traditional methods. Hence, the state of the art in image restoration and compression is getting redefined. This special issue covers the state of the art in learned image/video restoration and compression to promote further progress in innovative architectures and training methods for effective and efficient networks for image/video restoration and compression.

Smoke removal has been a challenging problem which refers to the process of retrieving clear images from the smoky ones. It has an extensive demand while the performances of existing image desmoking methods have limitations owing to the homogeneous medium based model, hand-crafted features, and insufficient datasets. Unlike the existing desmoking methods using an atmospheric scattering model, the proposed method utilizes a general deblurring model. Inspired by Generative Adversarial Network (GAN), we propose an end-to-end attentive DesmokeGAN which implements the visual attention into the generative network to effectively learn the smoke features and their surroundings. The architecture of critic network is identical to PatchGAN by adding multi-component loss function to the image patch. Due to the scarce of various and quality smoke datasets, a graphics rendering engine is used to synthesize the smoky images. The quantitative and qualitative results show that the designed framework performs better than the recent state-of-the-art desmoking approaches on both synthetic and real images in indoor and outdoor scenes, especially for the thick smoky images.

The performance of objective image quality assessment (IQA) models has been evaluated primarily by comparing model predictions to human quality judgments. Perceptual datasets gathered for this purpose have provided useful benchmarks for improving IQA methods, but their heavy use creates a risk of overfitting. Here, we perform a large-scale comparison of IQA models in terms of their use as objectives for the optimization of image processing algorithms. Specifically, we use eleven full-reference IQA models to train deep neural networks for four low-level vision tasks: denoising, deblurring, super-resolution, and compression. Subjective testing on the optimized images allows us to rank the competing models in terms of their perceptual performance, elucidate their relative advantages and disadvantages in these tasks, and propose a set of desirable properties for incorporation into future IQA models.

The images distortion leads to the loss of image information and the degradation of perceptual quality. To solve this problem, we investigate a novel mixed distortion image enhancement method based on the parallel network combines deep residual and reinforcement learning. The no-reference image quality assessment algorithm is used to determine the type and level of distorted images accurately. According to the type of distortion, the mixed distortion images enter one of the subsequent parallel joint learning networks automatically. In the joint learning framework, we prepare different residual networks to handle specialized restoration assignments including deblurring, denoising, or JPEG compression. Simultaneously, reinforcement learning then learns a policy to select the next best restoration tasks to progressively restore the quality of a corrupted image. Our method is capable of restoring images corrupted with complex mixed distortions in a more parameter-efficient manner in comparison to conventional networks. The extensive experiments on synthetic and real-world images validate the superior performances of the proposed method.

As a fundamental preprocessing technique, remote sensing image motion deblurring is important for visual understanding tasks. Most conventional approaches formulate the image motion deblurring task as a kernel estimation. Because the kernel estimation is a highly ill-posed problem, many priors have been applied to model the images and kernels. Even though these methods have obtained relatively better performances, they are usually time-consuming and not robust for different conditions. To address this problem, we propose a multitask learning mechanism for remote sensing image motion deblurring in this article, which contains an image restoration subtask and an image texture complexity recognition one. First, we consider the image motion deblurring problem as a domain transformation problem, from the blurred domain to a clear one. Specifically, the blurred domain represents the data space consisted of blurring images, and the definition of clear domain is similar. Second, we design a novel weighted attention map loss to enhance the reconstruction capability of the restoration subbranch for difficult local regions. Third, based on the restoration subbranch, a recognition subbranch is incorporated into the framework to guide the deblurring process, which provides the auxiliary texture complexity information to help the optimization of restoration subbranch. Additionally, in order to optimize the proposed network, we construct three large-scale datasets, and each sample in the dataset contains a clear image, a blurred image, and its texture label obtained by corresponding texture complexity. Finally, the experimental results on three constructed datasets demonstrate the robustness and the effectiveness of the proposed method.