Search

Underwater captured images are usually degraded by low contrast, hazy, and blurry due to absorbing and scattering, which limits their analyses and applications. To address these problems, a red channel prior guided variational framework is proposed based on the complete underwater image formation model (UIFM). Unlike most of the existing methods that only consider the direct transmission and backscattering components, we additionally include forward scattering component into the UIFM. In the proposed variational framework, we successfully incorporate the normalized total variation item and sparse prior knowledge of blur kernel together. In addition, we perform the estimation of blur kernel by varying image resolution in a coarse-to-fine manner to avoid local minima. Moreover, for solving the generated non-smooth optimization problem, we employ the alternating direction method of multipliers (ADMM) to accelerate the whole progress. Experimental results demonstrate that the proposed method has a good performance on dehazing and deblurring. Extensive qualitative and quantitative comparisons further validate its superiority against the other state-of-the-art algorithms. The code is available online at: https://github.com/Hou-Guojia.

Pattern recognition techniques form the heart of most, if not all, incoherent linear shift-invariant systems. When an object is recorded using a camera, the object information is sampled by the point spread function (PSF) of the system, replacing every object point with the PSF in the sensor. The PSF is a sharp Kronecker Delta-like function when the numerical aperture (NA) is large with no aberrations. When the NA is small, and the system has aberrations, the PSF appears blurred. In the case of aberrations, if the PSF is known, then the blurred object image can be deblurred by scanning the PSF over the recorded object intensity pattern and looking for pattern matching conditions through a mathematical process called correlation. Deep learning-based image classification for computer vision applications gained attention in recent years. The classification probability is highly dependent on the quality of images as even a minor blur can significantly alter the image classification results. In this study, a recently developed deblurring method, the Lucy-Richardson-Rosen algorithm (LR2A), was implemented to computationally refocus images recorded in the presence of spatio-spectral aberrations. The performance of LR2A was compared against the parent techniques: Lucy-Richardson algorithm and non-linear reconstruction. LR2A exhibited a superior deblurring capability even in extreme cases of spatio-spectral aberrations. Experimental results of deblurring a picture recorded using high-resolution smartphone cameras are presented. LR2A was implemented to significantly improve the performances of the widely used deep convolutional neural networks for image classification.

In this paper, we present an extensive review on deep learning methods for image restoration tasks. Deep learning techniques, led by convolutional neural networks, have received a great deal of attention in almost all areas of image processing, especially in image classification. However, image restoration is a fundamental and challenging topic and plays significant roles in image processing, understanding and representation. It typically addresses image deblurring, denoising, dehazing and super-resolution. There are substantial differences in the approaches and mechanisms in deep learning methods for image restoration. Discriminative learning based methods are able to deal with issues of learning a restoration mapping function effectively, while optimisation models based methods can further enhance the performance with certain learning constraints. In this paper, we offer a comparative study of deep learning techniques in image denoising, deblurring, dehazing, and super-resolution, and summarise the principles involved in these tasks from various supervised deep network architectures, residual or skip connection and receptive field to unsupervised autoencoder mechanisms. Image quality criteria are also reviewed and their roles in image restoration are assessed. Based on our analysis, we further present an efficient network for deblurring and a couple of multi-objective training functions for super-resolution restoration tasks. The proposed methods are compared extensively with the state-of-the-art methods with both quantitative and qualitative analyses. Finally, we point out potential challenges and directions for future research.

Video deblurring, which aims at restoring the sharp video from blurry video, is drawing increasing attention in the field of computer vision. In this paper, a method called Multi-Attention Convolutional Neural Network (MACNN) consisting of the temporal-spatial attention module, the frame channel attention module, and the feature extraction-reconstruction module is proposed. First, we use the temporal-spatial attention module and the frame channel attention module to capture features with temporal and spatial information existing across neighboring frames. Then, these captured features are fused and reconstructed to restore the sharp frame. Last but not least, we train MACNN together with a content loss and a perceptual loss in an end-to-end manner to recover realistic video details. Both quantitative and qualitative evaluation results on standard benchmarks demonstrate the proposed MACNN is superior to the state-of-the-art methods in terms of accuracy, efficiency, and visual effect.

ORB-SLAM2 is a feature-based sparse visual simultaneous localization and mapping (SLAM) system. After obtaining images and related data from sensors and cameras, it can extract useful information from them, and perform real-time positioning and map construction. This technology is now widely used in the fields of unmanned driving, smart home and factory operations, and it also provides a relatively stable foundation for the creation and development of many new SLAM systems. However, the current ORB-SLAM2 system can only work stably and normally when the camera is moving steadily and slowly. When the camera moves too fast or violently bumps, the image taken by the camera will inevitably be blurred, which will easily cause the "Track Lost" of the ORB-SLAM2 system. In order to solve this problem, this thesis combines the ORB-SLAM2 system with a deblurring algorithm based on deep learning, and uses the SLAM-oriented data set to optimize the deep learning network. This thesis also adds a blur detection module to ORB-SLAM2, which enables it to automatically distinguish blurry images. The experimental results demonstrated that this work detects and deblurs the blurred image, successfully avoiding part of the "Track Lost", and improving the accuracy of ORB-SLAM2 mapping.

Documents often exhibit various forms of degradation, which make it hard to be read and substantially deteriorate the performance of an OCR system. In this paper, we propose an effective end-to-end framework named Document Enhancement Generative Adversarial Networks (DE-GAN) that uses the conditional GANs (cGANs) to restore severely degraded document images. To the best of our knowledge, this practice has not been studied within the context of generative adversarial deep networks. We demonstrate that, in different tasks (document clean up, binarization, deblurring and watermark removal), DE-GAN can produce an enhanced version of the degraded document with a high quality. In addition, our approach provides consistent improvements compared to state-of-the-art methods over the widely used DIBCO 2013, DIBCO 2017 and H-DIBCO 2018 datasets, proving its ability to restore a degraded document image to its ideal condition. The obtained results on a wide variety of degradation reveal the flexibility of the proposed model to be exploited in other document enhancement problems., Accepted in IEEE TPAMI

This article introduces a novel model that restores color images degraded by blurring and mixed random-valued (RV) impulse and Gaussian noise. The model includes a data-fidelity term that consists of RV impulse noise and Gaussian noise components, and total variation regularization. Specifically, the data-fidelity term uses the l 0 - and l 2 -norms to represent the RV impulse and Gaussian noise components respectively. This results in an appropriate separation of these two noise components and thus enables simultaneous elimination of the mixed noise. Moreover, total variation regularization allows sufficient denoising in homogeneous regions while maintaining edges. To solve the nonconvex and nonsmooth model, we adopt a proximal alternating minimization approach and the alternating direction method of multipliers. These techniques contribute to an iterative algorithm with a convergence analysis provided. The experimental results demonstrate the effectiveness of the proposed model compared to other existing models, as measured by visual inspection and image quality assessments.

Space target super resolution is a domain-specific single image super resolution problem aiming to help distinguish the satellite and spacecrafts from numerous space debris. Compared to other object super resolution problem, images for space target are always in low quality with varies of degradation condition, as a result of long distance and motion blur, which significantly reduce the manual classification reliability, especially for these small targets, e.g., satellite payloads. To address this challenge, we present an end-to-end super-resolution and deblurring network (SRDN). Concretely, focusing on the low-resolution (LR) space target images with blind motion blur, we integrate the super-resolution and deblur function together, improving the image quality bya unified generative adversarial network (GAN)- based framework. We implement a deblur module by using contrastive learning to extract degradation feature, and add symmetrical down and up sampling modules to the super resolution network in order to restore texture information, while shortcut connections are redesigned to maintain the global similarity. Extensive experiments on the public satellite dataset BUAA-SID-share1.5 demonstrate that our network outperforms the state-of-the-art super-resolution and deblur methods.

Deep learning-based blind image deblurring plays an essential role in solving image blur since all existing kernels are limited in modeling the real world blur. Thus far, researchers focus on powerful models to handle the deblurring problem and achieve decent results. For this work, in a new aspect, we discover the great opportunity for image enhancement (e.g., deblurring) directly from RAW images and investigate novel neural network structures benefiting RAW-based learning. However, to the best of our knowledge, there is no available RAW image deblurring dataset. Therefore, we built a new dataset containing both RAW images and processed sRGB images and design a new model to utilize the unique characteristics of RAW images. The proposed deblurring model, trained solely from RAW images, achieves the state-of-art performance and outweighs those trained on processed sRGB images. Furthermore, with fine-tuning, the proposed model, trained on our new dataset, can generalize to other sensors. Additionally, by a series of experiments, we demonstrate that existing deblurring models can also be improved by training on the RAW images in our new dataset. Ultimately, we show a new venue for further opportunities based on the devised novel raw-based deblurring method and the brand-new Deblur-RAW dataset., Comment: IEEE Transactions on Multimedia

The amount of multimedia data, such as images and videos, has been increasing rapidly with the development of various imaging devices and the Internet, bringing more stress and challenges to information storage and transmission. The redundancy in images can be reduced to decrease data size via lossy compression, such as the most widely used standard Joint Photographic Experts Group (JPEG). However, the decompressed images generally suffer from various artifacts (e.g., blocking, banding, ringing, and blurring) due to the loss of information, especially at high compression ratios. This article presents a feature-enriched deep convolutional neural network for compression artifacts reduction (FeCarNet, for short). Taking the dense network as the backbone, FeCarNet enriches features to gain valuable information via introducing multi-scale dilated convolutions, along with the efficient 1 x 1 convolution for lowering both parameter complexity and computation cost. Meanwhile, to make full use of different levels of features in FeCarNet, a fusion block that consists of attention-based channel recalibration and dimension reduction is developed for local and global feature fusion. Furthermore, short and long residual connections both in the feature and pixel domains are combined to build a multi-level residual structure, thereby benefiting the network training and performance. In addition, aiming at reducing computation complexity further, pixel-shuffle-based image downsampling and upsampling layers are, respectively, arranged at the head and tail of the FeCarNet, which also enlarges the receptive field of the whole network. Experimental results show the superiority of FeCarNet over state-of-the-art compression artifacts reduction approaches in terms of both restoration capacity and model complexity. The applications of FeCarNet on several computer vision tasks, including image deblurring, edge detection, image segmentation, and object detection, demonstrate the effectiveness of FeCarNet further.

X-ray computed tomography (CT) enables nondestructive evaluation and dimensional metrology using the reconstructed volumes of measured objects. Although the abundant applications of industrial X-ray CT exist, a high-quality reconstructed volume requires a long measurement time owing to sharp transmission images from dense views. Intensive X-rays with the exposure kept can be used to reduce the measurement time. However, the quality of the measured objects inevitably deteriorates as a result of the so-called penumbra effect caused by large focal spots in X-ray sources. This paper proposes rotational fine-tuning (RFT) for the acceleration of CT scanning with the quality kept. First, sharp transmission images from sparse views are obtained in appropriate imaging conditions. Subsequently, blurry transmission images from dense views are obtained using a larger focal spot size and a shorter exposure time. The acquired blurry images are deblurred by convolutional neural networks fine-tuned using several pairs of sharp and blurry images obtained at the corresponding projection angles, and linear interpolation integrates the deblurred images to generate the final RFT output. The proposed method indirectly reduces the measurement time because no lengthy acquisition time is required, and fine-tuning and blurring are rapid when using GPUs. In the experiments, the comparison of PSNRs between blurry and deblurred reconstructed volumes is shown. To reveal the effect of the proposed method on dimensional metrology, the improvement in the surface deviations of the stepped cylinder is also shown. Finally, the improvement of the pore spaces in a porous aluminum is shown to assess the aspect of nondestructive evaluation. Overall, experimental results demonstrate that the proposed method can perform the acceleration of X-ray CT scanning with the quality kept.

The computer-based approach is required for the non-invasive detection of chronic liver diseases that are asymptomatic, progressive, and potentially fatal in nature. In this study, we review the computer-aided diagnosis of hepatic lesions in view of diffuse- and focal liver disorders. This survey mainly focuses on three image acquisition modalities: ultrasonography, computed tomography, and magnetic resonance imaging. We present the insightful analysis with pros and cons for each preliminary step, particularly preprocessing, attribute analysis, and classification techniques to accomplish clinical diagnostic tasks. In preprocessing, we explore and compare commonly used denoising, deblurring and segmentation methods. Denoising is mainly performed with nonlinear models. In contrast, deep neural networks are frequently applied for deblurring and automatic segmentation of region-of-interest. In attribute analysis, the most common approach comprises texture properties. For classification, the support vector machine is mainly utilized across three image acquisition modalities. However, comparative analysis shows the best performance is obtained by deep learning-based convolutional neural networks. Considering biopsy samples or pathological factors such as overall stage, margin, and differentiation can be helpful for improving the prediction performance. In addition, technique breakthrough is expected soon with advances in machine learning models to address data limitation problems and improve the prediction performance.

Different from the object motion blur, the defocus blur is caused by the limitation of the cameras' depth of field. The defocus amount can be characterized by the parameter of point spread function and thus forms a defocus map. In this paper, we propose a new network architecture called Defocus Image Deblurring Auxiliary Learning Net (DID-ANet), which is specifically designed for single image defocus deblurring by using defocus map estimation as auxiliary task to improve the deblurring result. To facilitate the training of the network, we build a novel and large-scale dataset for single image defocus deblurring, which contains the defocus images, the defocus maps and the all-sharp images. To the best of our knowledge, the new dataset is the first large-scale defocus deblurring dataset for training deep networks. Moreover, the experimental results demonstrate that the proposed DID-ANet outperforms the state-of-the-art methods for both tasks of defocus image deblurring and defocus map estimation, both quantitatively and qualitatively. The dataset, code, and model is available on GitHub: https://github.com/xytmhy/DID-ANet-Defocus-Deblurring.

Deblurring can not only provide visually more pleasant pictures and make photography more convenient, but also can improve the performance of objection detection as well as tracking. However, removing dynamic scene blur from images is a non-trivial task as it is difficult to model the non-uniform blur mathematically. Several methods first use single or multiple images to estimate optical flow (which is treated as an approximation of blur kernels) and then adopt non-blind deblurring algorithms to reconstruct the sharp images. However, these methods cannot be trained in an end-to-end manner and are usually computationally expensive. In this paper, we explore optical flow to remove dynamic scene blur by using the multi-scale spatially variant recurrent neural network (RNN). We utilize FlowNets to estimate optical flow from two consecutive images in different scales. The estimated optical flow provides the RNN weights in different scales so that the weights can better help RNNs to remove blur in the feature spaces. Finally, we develop a convolutional neural network (CNN) to restore the sharp images from the deblurred features. Both quantitative and qualitative evaluations on the benchmark datasets demonstrate that the proposed method performs favorably against state-of-the-art algorithms in terms of accuracy, speed, and model size., accepted by tcsvt

Camera-based multi-class (1D and 2D) barcode detectors that can help in decoding barcodes in different complex scenes have huge potential applications in situations where Internet of Things (IoT) is combined with Artificial Intelligence (AI) and Augmented Reality (AR). The decoding rate in such applications under real-life complex scenes is greatly affected by two major factors: First, we cannot accurately localize the barcodes, and secondly, we cannot decode the blur samples. In this paper, we firstly propose a barcode localization algorithm that is capable of regressing 4 vertices of barcodes accurately. Our localization method comprises an anchor-free approach that outputs multi-scale output prediction maps. These segmentation-like maps of each scale are then further divided into three types of maps (classification, centerness, and 8-D regression). 8-D localization result of barcodes along-with classification result is then obtained after post-processing. Secondly, we propose a conditional generative adversarial network-based model for deblurring blur QR codes. Extensive decoding experiments on a challenging complex scene dataset show that our localization and deblurring methods can contribute to improving the decoding rate of existing barcode decoders.

State-of-the-art face restoration methods employ deep convolutional neural networks (CNNs) to learn a mapping between degraded and sharp facial patterns by exploring local appearance knowledge. However, most of these methods do not well exploit facial structures and identity information, and only deal with task-specific face restoration (e.g., face super-resolution or deblurring). In this paper, we propose cross-tasks and cross-models plug-and-play 3D facial priors to explicitly embed the network with the sharp facial structures for general face restoration tasks. Our 3D priors are the first to explore 3D morphable knowledge based on the fusion of parametric descriptions of face attributes (e.g., identity, facial expression, texture, illumination, and face pose). Furthermore, the priors can easily be incorporated into any network and are very efficient in improving the performance and accelerating the convergence speed. Firstly, a 3D face rendering branch is set up to obtain 3D priors of salient facial structures and identity knowledge. Secondly, for better exploiting this hierarchical information (i.e., intensity similarity, 3D facial structure, and identity content), a spatial attention module is designed for the image restoration problems. Extensive face restoration experiments including face super-resolution and deblurring demonstrate that the proposed 3D priors achieve superior face restoration results over the state-of-the-art algorithms.

Current deep deblurring methods pay main attention to learning a transferring network to transfer synthetic blurred images to clean ones. Though achieving significant performance on the training datasets, they still suffer from a weaker generalization capability from training datasets to others with different synthetic blurs, thus resulting in significantly inferior performance on testing datasets. In order to alleviate this problem, we propose a latent contrastive model, Blur Distilling and Information Reconstruction Networks (BDIRNet), to learn image prior and improve the robustness of deep deblurring. The proposed BDIRNet consists of a blur removing network (DistillNet) and a reconstruction network (RecNet). Two kinds of images with almost the same information but different qualities are input into DistillNet to extract identical structure information via contrast latent information and purify the perturbations from other unimportant information like blur. While the RecNet is utilized to reconstruct sharp images based on the extracted information. In addition, inside the DistillNet and RecNet, a statistical anti-interference distilling (SAID) and anti-interference reconstruction (SAIR) modules are proposed to further enhance the robustness of our methods, respectively. Extensive experiments on different datasets show that the proposed methods achieve improved and robust results compared to recent state-of-the-art methods.

In this paper, we introduce a new iterative forward-backward splitting method with an error for solving the variational inclusion problem of the sum of two monotone operators in real Hilbert spaces. We suggest and analyze this method under some mild appropriate conditions imposed on the parameters such that another strong convergence theorem for these problem is obtained. We also apply our main result to improve the fast iterative shrinkage thresholding algorithm (IFISTA) with an error for solving the image deblurring problem. Finally, we provide numerical experiments to illustrate the convergence behavior and show the effectiveness of the sequence constructed by the inertial technique to the fast processing with high performance and the fast convergence with good performance of IFISTA.

Blind image deblurring is a severely ill-posed problem in low-level vision. The success of blind image deblurring relies on statistical priors and well-designed regularizers to obtain a clear image. However, the prior-based method is time-consuming due to a lot of nonlinear calculations. To improve efficiency, this work proposes an enhanced salient edge selection for blind image deblurring. Different from the previous methods that only focus on the salient edge and ignore the image structure, the image sharpening operator is adopted to guide the finer image structure when salient edges provide strong edge information for blur kernel estimation. We find that the latent image restoration can be enhanced by joining the salient edge selection and an image sharpening operator, and the quality of the recovered kernel is hereby improved. By imposing L2 constraints on the salient edge and sharpening operator terms, a new energy function is introduced and an effective alternating optimization strategy is explored. Extensive experiments have been conducted, which shows that our method is more effective compared with state-of-the-art methods.

Motion blur is a common artifact in image processing, specifically in e-health services, which is caused by the motion of a camera or scene. In linear motion cases, the blur kernel, i.e., the function that simulates the linear motion blur process, depends on the length and direction of blur, called linear motion blur parameters. The estimation of blur parameters is a vital and sensitive stage in the process of reconstructing a sharp version of a motion blurred image, i.e., image deblurring. The estimation of blur parameters can also be used in e-health services. Since medical images may be blurry, this method can be used to estimate the blur parameters and then take an action to enhance the image. In this paper, some methods are proposed for estimating the linear motion blur parameters based on the extraction of features from the given single blurred image. The motion blur direction is estimated using the Radon transform of the spectrum of the blurred image. To estimate the motion blur length, the relation between a blur metric, called NIDCT (Noise-Immune Discrete Cosine Transform-based), and the motion blur length is applied. Experiments performed in this study showed that the NIDCT blur metric and the blur length have a monotonic relation. Indeed, an increase in blur length leads to increase in the blurriness value estimated via the NIDCT blur metric. This relation is applied to estimate the motion blur. The efficiency of the proposed method is demonstrated by performing some quantitative and qualitative experiments.

Underwater explorations and probes have now become frequent for marine discovery and endangered resources protection. The decrease in natural light with an increase in water depth and the characteristic of the medium to absorb and scatter light pose crucial challenges to underwater vision systems. Autonomous Underwater Vehicles (AUVs) depend upon their imaging systems for navigation and environmental resource exploration. This paper proposes DeepRecog—an integrated underwater image deblurring and object recognition framework for AUV vision systems. The principle behind the image deblurring module involves a threefold approach consisting of CNNs, adaptive and transformative filters. The ensemble object detection and recognition module identifies marine life and other frequently existent underwater assets from AUV images and achieves mean Average Precision (mAP) of 0.95 and was found to be 6.42% more precise than YOLOv3, 8.43% more than FasterRCNN + VGG16 and 15.78% more than FasterRCNN. This framework was created with the purpose of providing real-time detection and recognition with minimal delay. The system can also be employed for former images acquired from AUVs and hopes to facilitate efficient solutions for marine image post-processing.

Deblurring images with outliers has always been a significantly challenging problem. Previous methods mainly involved complex operations, such as outlier and light streak detection, or sophisticated image priors for blur-kernel estimation, which increased the difficulty of deblurring images. Therefore, we developed a simple, yet efficient, blind deblurring algorithm in this study for handling images with outliers. To eliminate the impact of outliers during the kernel estimation process, we employed a robust Welsch loss function to characterize the data-fidelity term of our model. We observed that this function could extract significant edges successfully. Therefore, the image regularization term was also described by the same function. Using this unified robust function to describe our model can considerably reduce the complexity of the algorithm. Moreover, we derived a flexible weight function from the Welsch function to further improve the efficiency of our algorithm. To finally obtain accurate latent images, we developed a robust non-blind deblurring approach based on this flexible weight function. The experimental results indicate that our approach outperforms state-of-the-art methods in deblurring images with or without outliers. Compared with the method specifically for outliers, the recovery performance of our method can be improved by 12.9% (considering a dataset with impulse noise), and the execution efficiency is about 1.5 times faster.

Sparse based representation is being used extensively for image restoration. The dictionary learningthrough patch extraction is central to the sparse based schemes. In the process of dictionary learning,a large number of patches will be extracted from high quality images and dictionary will be formed.Hence, over-complete dictionaries will be built. To overcome the complexity associated with overcompletedictionaries many schemes were proposed. Of them, the adaptive sparse domain is thepopular one. Many variations of adaptive sparse domain schemes were proposed. Selection of obviouspatches is common to all. In all these schemes, individual patches will be considered as the basic entityand will be used. This is the reason for the complexity involved in sparse representation. In this paper,to avoid the complexity, the patches are grouped according to the similarity among the patches. Inaddition to reduce the complexity the proposed cluster based scheme considers the self-similarity ofthe patches involved. Hence better performance with less complexity is possible with the proposedschemes. In the process of testing, in addition to uniform blur and Gaussian blur, a combination of thetwo blurs is also considered.

Blind image deconvolution is one of the most challenging problems in image restoration. Inspired by the work on sparsity constraint and deblurring of blind motion, we propose a model with fractiona...

High noise and low spatial resolution are two key confounding factors that limit the qualitative and quantitative accuracy of PET images. Artificial intelligence models for image denoising and deblurring are becoming increasingly popular for the postreconstruction enhancement of PET images. We present a detailed review of recent efforts for artificial intelligence-based PET image enhancement with a focus on network architectures, data types, loss functions, and evaluation metrics. We also highlight emerging areas in this field that are quickly gaining popularity, identify barriers to large-scale adoption of artificial intelligence models for PET image enhancement, and discuss future directions.

Satellite imagery acquired from optical remote sensing systems is often suffered due to several types of blur, such as atmospheric turbulence blur, motion blur, and defocus. Any kind of blur degrades the image quality, as it reduces the sharpness of edges and texture, and hence, spatial resolution is also reduced. This degradation poses a challenge for further automated analysis using such blurred images. Image deconvolution methods are conventionally applied to blurry images to estimate the blur and restore the original image. In this inverse and ill-posed problem, the restoration quality relies on the correct estimation of the point spread function that caused blur in the image. We aim to restore the blurred satellite imagery, corrupted by Gaussian blur, using the deep learning framework and hence prove the efficacy of the proposed method over the traditional methods. This paper presents an investigative analysis of the satellite image deblurring problem and simultaneously tackles the problem of low-resolution satellite imagery. A deep convolutional neural network (CNN) architecture is proposed to remove the Gaussian blur artifacts from images and increase their resolution. Experimental evaluation is performed on a hyperspectral image dataset consisting of 8 different terrain types. Results obtained after deblurring the images are compared with state-of-the-art methods based on both subjective and objective image quality measures: peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). The image quality of deblurred images obtained by the proposed deep CNN method demonstrates improved performance over some of the existing CNN methods.

Most image enhancement methods focused on restoration of digitized textual documents are limited to cases where the text information is still preserved in the input image, which may often not be the case. In this work, we propose a novel generative document restoration method which allows conditioning the restoration on a guiding signal in the form of target text transcription and which does not need paired high- and low-quality images for training. We introduce a neural network architecture with an implicit text-to-image alignment module. We demonstrate good results on inpainting, debinarization and deblurring tasks, and we show that the trained models can be used to manually alter text in document images. A user study shows that that human observers confuse the outputs of the proposed enhancement method with reference high-quality images in as many as 30% of cases.

In recent years, the image deblurring task has attracted more and more researchers’ attention. Many researchers are devoted to eliminating motion blur by using a “coarse-to-fine” architecture. This architecture can effectively eliminate motion blur caused by a simple relative displacement. But when using the architecture directly for the face image deblurring task, there would exist some problems. For example, complex network structures make the model difficult to be trained, and a large number of parameters need to be calculated would result in expensive runtime. Since details of images can’t be completely restored, the quality of images will deteriorate, and deblurring visual effect is poor, so in order to solve these issues, combining the “coarse-to-fine” architecture with the “stacked” architecture, we propose a new network architecture-“Stack-based Scale-recurrent Network” for the face image deblurring task. The ConvLSTM network is employed to build a model. We achieves the purpose of improving the visual clarity and the quality of restored face images. And we use the improved encoder-decoder to enhance network performance. Compared with the other deblurring methods, our method can restore clear face images with higher-quality and clearer vision.

As a well-known ill-conditional problem in the image processing field, image deblurring has become a hot topic recently. The prior-based blind image deblurring methods have recently shown promising effectiveness. A lot of advanced algorithms such as dark channel prior, bright channel prior, and local maximum gradient prior are time-consuming since nonlinear operators are involved. Presented in this paper is a fast blind image deblurring algorithm which uses the simplified extreme channel prior (SECP) and gradient cepstrum. The inspiration for this work comes from the fact that the simplified bright channel prior (SBCP) of the clear image has fewer non-one elements than the blurred one. We propose a novel SECP based on the proposed SBCP and the simplified dark channel prior (SDCP). By enforcing the $$L_{0}$$ norm constraint to the terms involving SECP and incorporating them into the traditional deblurring framework, an effective optimization scheme is explored. Furthermore, gradient cepstrum is used to determine the size of the initial kernel and restrain excessive iterations in each scale. Experimental results illustrate that our algorithm outperforms the state-of-the-art deblurring algorithms in terms of computational efficiency and deblurring effect on both benchmark datasets and real-world blur scenes.

Blurred images are difficult to avoid in situations when minor Atmospheric turbulence or camera movement results in low-quality images. We propose a system that takes a blurred image as input and produces a deblurred image by utilizing various filtering techniques. Additionally, we utilize the Siamese Network to match local image segments. A Siamese Neural Network model is used that is trained to account for image matching in the spatial domain. The best-matched image returned by the model is then further used for Signal-to-Noise ratio and Point Spread Function estimation. The Wiener filter is then used to deblur the image. Finally, the results of the deblurring techniques with existing algorithms are compared and it is shown that the error in deblurring an image using the techniques presented in this paper is considerably lesser than other techniques.

Different image fusion systems have been developed to deal with the massive amounts of image data for different applications, such as remote sensing, computer vision, and environment monitoring. However, the generalizability and versatility of these fusion systems remain unknown. This article proposes an efficient regularization framework to achieve different kinds of fusion tasks accounting for the spatiospectral and spatiotemporal variabilities of the fusion process. A joint minimization functional is developed by taking an advantage of a composite regularizer for enforcing joint sparsity in the gradient domain and the frame domain. The proposed composite regularizer is composed of the Hessian Schatten-norm regularization and contourlet-based regularization terms. The resulting problems are solved by the alternating direction method of multipliers (ADMM). The effectiveness of the proposed method is validated in a variety of image fusion experiments: 1) hyperspectral (HS) and panchromatic image fusion; 2) HS and multispectral image fusion; 3) multitemporal image fusion (MIF); and 4) multi-image deblurring. Results show promising performance compared with state-of-the-art fusion methods.

The mean curvature-based image deblurring model is widely used to enhance the quality of the deblurred images. However, the discretization of the associated Euler-Lagrange equations produce a nonlinear ill-conditioned system which affect the convergence of the numerical algorithms like Krylov subspace methods. To overcome this difficulty, in this paper, we present two new symmetric positive definite (SPD) preconditioners. An efficient algorithm is presented for the mean curvature-based image deblurring problem which combines a fixed point iteration (FPI) with new preconditioned matrices to handle the nonlinearity and ill-conditioned nature of the large system. The eigenvalues analysis is also presented in the paper. Fast convergence has shown in the numerical results by using the proposed new preconditioners.

Video deblurring is a challenging low-level vision task due to variant blur artifacts caused by factors such as depth variations, high-speed movements and camera shakes. Although significant efforts have been devoted to addressing this task, two challenges of capturing temporal patterns and spatial topologies still remain. In this paper, an attention-based interframe compensation scheme is proposed to address the first challenge. The proposed scheme replaces frames in blurry sequences with newly restored frames, and estimates temporal patterns among the replaced sequence to restore the whole sequence. After each replacement, an attention block is employed to exploit dependencies among restored and blurry frames to capture stable temporal patterns. To tackle the second challenge, we propose an adaptive residual block that dynamically fuses multi-level features via learning location-specific weights. Comprehensive experimental results demonstrate that the proposed method achieves state-of-the-art performance in terms of accuracy, visual effect and model size.

The time-of-flight (TOF) information makes Orlov’s condition not necessary for positron emission tomography (PET) imaging. One-view imaging has never been attempted before. This article investigates whether one-view imaging is possible. We claim that it is possible to obtain an image with measurements from only one-view, provided the TOF time resolution is good enough and the photon counts are high enough. In fact, the image value at a point can be reconstructed by measurements along one line of response (LOR) that passes through the point of interest. The region of interest (ROI) can be reconstructed with ray-by-ray deblurring methods, which make TOF PET a local tomography system. One-view imaging is a severely ill-posed problem if the TOF resolution is not good enough. One indicator of an ill-posed problem is the ringing artifacts in the reconstruction. We show that ringing artifacts can be eliminated by using the nonnegativity constraint for the dot-like objects with a zero background.

Inverse synthetic aperture radar (ISAR) imaging for maneuvering targets suffers from a Doppler frequency time-varying problem, leading to the ISAR images blurred in the azimuth direction. Given that the traditional imaging methods have poor imaging performance or low efficiency, and the existing deep learning imaging methods cannot effectively reconstruct the deblurred ISAR images retaining rich details and textures, an unblurring ISAR imaging method based on an advanced Transformer structure for maneuvering targets is proposed. We first present a pseudo-measured data generation method based on the DeepLabv3+ network and Diamond-Square algorithm to acquire an ISAR dataset for training with good generalization to measured data. Next, with the locally-enhanced window Transformer block adopted to enhance the ability to capture local context as well as global dependencies, we construct a novel Uformer-based GAN (UFGAN) to restore the deblurred ISAR images with rich details and textures from blurred imaging results. The simulation and measured experiments show that the proposed method can achieve fast and high-quality imaging for maneuvering targets under the condition of a low signal-to-noise ratio (SNR) and sparse aperture.

The blurring of crop images acquired by agricultural Unmanned Aerial Vehicle (UAV) due to sudden inputs by operators, atmospheric disturbance, and many other factors will eventually affect the subsequent crop identification, information extraction, and yield estimation. Aiming at the above problems, the new proposed combined deblurring algorithm based on the re-weighted graph total variation (RGTV) and L0-regularized prior, and the other two representative deblurring algorithms were applied to restore blurry crop images acquired during UAV flight, respectively. The restoration performance was measured by subjective vision, and objective evaluation indexes. The crop shape-related and texture-related feature parameters were then extracted, the Support Vector Machine (SVM) classifier with four common kernel functions was implemented for crop classification to realize the purpose of crop information extraction. The deblurring results showed that the proposed algorithm performed better in suppressing the ringing effect and preserving the image fine details, and retained higher objective evaluation indexes than the other two deblurring algorithms. The comparative analysis of different classification kernel functions showed that the Polynomial kernel function with an average recognition rate of 94.83% was most suitable for crop classification and recognition. The research will help in further popularization of crop monitoring based on UAV low-altitude remote sensing.

For the success of video deblurring, it is essential to utilize information from neighboring frames. Most state-of-the-art video deblurring methods adopt motion compensation between video frames to aggregate information from multiple frames that can help deblur a target frame. However, the motion compensation methods adopted by previous deblurring methods are not blur-invariant, and consequently, their accuracy is limited for blurry frames with different blur amounts. To alleviate this problem, we propose two novel approaches to deblur videos by effectively aggregating information from multiple video frames. First, we present blur-invariant motion estimation learning to improve motion estimation accuracy between blurry frames. Second, for motion compensation, instead of aligning frames by warping with estimated motions, we use a pixel volume that contains candidate sharp pixels to resolve motion estimation errors. We combine these two processes to propose an effective recurrent video deblurring network that fully exploits deblurred previous frames. Experiments show that our method achieves the state-of-the-art performance both quantitatively and qualitatively compared to recent methods that use deep learning., 17 pages, Camera-ready version for ACM Transactions on Graphics (TOG) 2021

Out-of-focus blurring of the QR code is very common in mobile Internet systems, which often causes failure of authentication as a result of a misreading of the information hence adversely affects the operation of the system. To tackle this difficulty, this work firstly introduced an edge prior information, which is the average distance between the center point and the edge of the clear QR code images in the same batch. It is motivated by the theoretical analysis and the practical observation of the theory of CMOS image sensing, optics information, blur invariants, and the invariance of the center of the diffuse light spots. After obtaining the edge prior information, combining the iterative image and the center point of the binary image, the proposed method can accurately estimate the parameter of the out-of-focus blur kernel. Furthermore, we obtain the sharp image by Wiener filter, a non-blind image deblurring algorithm. By this, it avoids excessive redundant calculations. Experimental results validate that the proposed method has great practical utility in terms of deblurring quality, robustness, and computational efficiency, which is suitable for barcode application systems, e.g., warehouse, logistics, and automated production.

Active image sensors can only record scenes at low frame rates. The dynamic vision sensor (DVS) can capture high-speed scenes under challenging lighting conditions. However, DVS discards absolute light intensity information. The dynamic and active pixel vision image sensor (DAVIS) can output events and low-frame-rate grayscale images simultaneously. This paper proposes a high frame rate video reconstruction and deblurring algorithm based on events and low-speed blurred image sequences from DAVIS. First, we visualized events as a sequence of event images and performed image denoising process. Then, we used the extended 3D partial recursive search (E-3DPRS) method to generate high-performance motion vector field (MVF) between adjacent denoising event images with low computational complexity. MVF reflects the object motion trajectory with high time resolution. Furthermore, MVF is used as a priori information to restore potentially sharp images, which alleviates the highly ill-posed deblurring process. At last, we reconstructed a high frame rate sharp video based on a sharp image sequence and MVFs. Extensive experiments show that our proposed algorithm outperforms the existing state-of-the-art methods and achieves better average feature similarity and structural similarity.

Video deblurring is still a challenging low-level vision task since spatio-temporal characteristics across both the spatial and temporal domains are difficult to model. In this article, to model the temporal information, we develop a non-local block which estimates inter-frame similarity and inter-frame difference. Specially, for modeling the spatial characteristics and restoring sharp frame details, we propose a recursive block that iteratively refines feature maps generated at the last iteration. In addition, a novel temporal loss function is introduced to ensure the temporal consistency of generated frames. Experimental results on public datasets demonstrate that our method achieves state-of-the-art performance both quantitatively and qualitatively.

Hyperspectral image (HSI) restoration is to recover the clean image from degraded version, such as the noisy, blurred, or damaged. Recent low-rank tensor-based recovery methods have been widely explored in HSIs restoration. Most of previous methods, however, neglect an inconspicuous but important phenomenon that the physical meaning and dimension along the spatial, spectral, and nonlocal mode are markedly different. In this work, we discover the low-rank property discrepancy along spatial, spectral, and nonlocal self-similarity mode in the HSIs, and argue that the intrinsic low-rank correlations along each mode contribute different to the final restoration results. Consequently, we figure out that the combination of the spectral and nonlocal-induced low-rank is most beneficial for HSIs modeling, and propose an optimal low-rank tensor (OLRT) model for HSIs restoration. Furthermore, we not only explore the low-rank property in the image component, but also in the sparse error component (stripe noise in HSIs). Thus, we extend OLRT to the OLRT-robust principal component analysis (RPCA) with low-rank tensor priors for both the HSIs and sparse error. Besides, previous methods are usually designed for one specific HSI task, which is less robust to various tasks. We prove that the proposed optimal low-rank prior is very flexible for various HSI restoration problems including denoising, deblurring, inpainting, and destriping. The proposed methods have been extensively evaluated on several benchmarks and tasks, and greatly outperform state-of-the-art (STOA). We show the simple yet effective OLRT strategy is also beneficial to STOA.

Blind image deblurring is a long standing challenging problem in image processing and low-level vision. Recently, sophisticated priors such as dark channel prior, extreme channel prior, and local maximum gradient prior, have shown promising effectiveness. However, these methods are computationally expensive. Meanwhile, since these priors involved subproblems cannot be solved explicitly, approximate solution is commonly used, which limits the best exploitation of their capability. To address these problems, this work firstly proposes a simplified sparsity prior of local minimal pixels, namely patch-wise minimal pixels (PMP). The PMP of clear images is much more sparse than that of blurred ones, and hence is very effective in discriminating between clear and blurred images. Then, a novel algorithm is designed to efficiently exploit the sparsity of PMP in deblurring. The new algorithm flexibly imposes sparsity inducing on the PMP under the MAP framework rather than directly uses the half quadratic splitting algorithm. By this, it avoids non-rigorous approximation solution in existing algorithms, while being much more computationally efficient. Extensive experiments demonstrate that the proposed algorithm can achieve better practical stability compared with state-of-the-arts. In terms of deblurring quality, robustness and computational efficiency, the new algorithm is superior to state-of-the-arts. Code for reproducing the results of the new method is available at https://github.com/FWen/deblur-pmp.git., Comment: 14 pages, 16 figures

Image deblurring is a classic and important problem in industrial fields, such as aviation photo restoration, object recognition in robotics, and autonomous vehicles. Blurry images in real-world scenarios consist of mixed blurring types, such as a natural motion blurring owing to shaking of the camera. Fast deblurring does not deblur the entire image because it is not the best option. Considering the computational costs, it is also better to have an alternative kernel to deblur different objects at a high-semantic level. To achieve better image restoration quality, it is also beneficial to combine the blurring category location and important structural information in terms of specific artifacts and degree of blurring. The goal of blind image deblurring is to restore sharpness from the unknown blurring kernel of an image. Recent deblurring methods tend to reconstruct prior knowledge, neglecting the influence of blur estimation and visual fidelity on image details and structure. Generative adversarial networks(GANs) have recently been attracting considerable attention from both academia and industry because GAN can perfectly generate new data with the same statistics as the training set. Therefore, this study proposes a generative neural architecture and an edge attention algorithm developed to restore vivid multimedia patches. Joint edge generation and image restoration techniques are designed to solve the low-level multimedia retrieval. This multipath refinement fusion network (MRFNet) can not only perform deblurring of images directly but also individual the frames separately from videos. Ablation experiments validate that our generative adversarial network MRFNet performs better in joint training than in multimodel. Compared to other GAN methods, our two-phase method exhibited state-of-the-art performance in terms of speed and accuracy as well as has a significant visual improvement.

The dynamic non-uniform blur caused by Radio Frequency Identification (RFID) multi-label motion seriously affects the identification and location of labels. It is an ill-posed inverse problem for that the blur kernel and sharp image are unknown. The traditional method of removing the blur is very time-consuming. In this work, we propose Multi-scale Recursive Codec Network based on the Authority Parameter (MRCN-AP) to deblur RFID multi-label images in a vision-based RFID multi-label 3D measurement system. This network is composed of a stack of three encoder-decoder subnets of different scales, which restores the blurry image in an end-to-end manner, and extracts the detail edge on each scale effectively from coarse to fine. The proposed authority parameters reduce the parameters memory of redundant networks and improve the speed of the deblurring network. Also, we propose new large-scale RFID multi-label blur-sharp image pairs captured by the dual CCD camera. The proposed model is implemented on an extended dataset. We prove that our method improves the speed by at least 0.55 s, and also increases Peak Signal to Noise Ratio (PSNR) by 2.43dB. Besides, better visual effects are obtained by MRCN-AP deblurring network for RFID multi-label image, which is more conducive to subsequent positioning and optimization.

PURPOSE: The purpose of this study was to develop a spiral-based combined spin- and gradient-echo (spiral-SAGE) method for simultaneous dynamic contrast-enhanced (DCE-MRI) and dynamic susceptibility contrast MRI (DSC-MRI). METHODS: Using this sequence, we obtained gradient-echo TEs of 1.69 and 26 ms, a SE TE of 87.72 ms, with a TR of 1663 ms. Using an iterative SENSE reconstruction followed by deblurring, spiral-induced image artifacts were minimized. Healthy volunteer images are shown to demonstrate image quality using the optimized reconstruction, as well as for comparison with EPI-based SAGE. A bioreactor phantom was used to compare dynamic-contrast time courses with both spiral-SAGE and EPI-SAGE. A proof-of-concept cohort of patients with brain tumors shows the range of hemodynamic maps available using spiral-SAGE. RESULTS: Comparison of spiral-SAGE images with conventional EPI-SAGE images illustrates substantial reductions of image distortion and artifactual image intensity variations. Bioreactor phantom data show similar dynamic contrast time courses between standard EPI-SAGE and spiral-SAGE for the second and third echoes, whereas first-echo data show improvements in quantifying T(1) changes with shorter echo times. In a cohort of patients with brain tumors, spiral-SAGE-based perfusion and permeability maps are shown with comparison with the standard single-echo EPI perfusion map. CONCLUSION: Spiral-SAGE provides a substantial improvement for the assessment of perfusion and permeability by mitigating artifacts typically encountered with EPI and by providing a shorter echo time for improved characterization of permeability. Spiral-SAGE enables quantification of perfusion, permeability, and vessel architectural parameters, as demonstrated in brain tumors.

Image restoration (IR) has been extensively studied with lots of excellent strategies accumulated over the years. However, most existing methods still have large room for improvement. In this paper, we boost an unsupervised iterative feature refinement model (IFR) with the enhanced high-dimensional deep mean-shift prior (EDMSP), termed IFR-EDMSP. The proposed model inherits the fantastic noise suppression characteristic of embedded network and the fine detail preservation ability of IFR model. Moreover, based on the fact that multiple implementations of artificial noise in prior learning improve underlying representation capability, three-sigma rule is adopted in IFR-EDMSP for accurate and robust results. Extensive experiments demonstrated that IFR-EDMSP outperforms the typical methods in compressed sensing, image deblurring and super-resolution.

Image deblurring is an important problem encountered in many image restoration tasks. To remove the motion blur of images captured from dynamic scenes, various Convolutional Neural Networks (CNNs) based methods are developed to restore the latent sharp image via an end-to-end trainable. However, these CNNs-based methods cannot restore enough structure details as no significant information is provided by the blurry input only. In this work, we propose a reference-guided deep deblurring method by incorporating the high-quality reference image into the deep network for better deblurring effect. Concretely, the correlation between the blurry input and the reference image is computed in the high-level feature space, and further represented by the similarity maps. To pick up the most relevant similarity maps to the input, the selective attention module is designed, and the selected similarity maps are decoded to reconstruct the deblurred sharp image. Extensive experimental results on two public datasets demonstrate that the proposed method achieves superior performance to existing methods quantitatively and qualitatively. More ablation experiments also validate that the favourable deblurred results can still be obtained even if the reference image is not similar with the input image.