Search

Single image deblurring has achieved significant progress for natural daytime images. Saturation is a common phenomenon in blurry images, due to the low light conditions and long exposure times. However, conventional linear deblurring methods usually deal with natural blurry images well but result in severe ringing artifacts when recovering low-light saturated blurry images. To solve this problem, we formulate the saturation deblurring problem as a nonlinear model, in which all the saturated and unsaturated pixels are modeled adaptively. Specifically, we additionally introduce a nonlinear function to the convolution operator to accommodate the procedure of the saturation in the presence of the blurring. The proposed method has two advantages over previous methods. On the one hand, the proposed method achieves the same high quality of restoring the natural image as seen in conventional deblurring methods, while also reducing the estimation errors in saturated areas and suppressing ringing artifacts. On the other hand, compared with the recent saturated-based deblurring methods, the proposed method captures the formation of unsaturated and saturated degradations straightforwardly rather than with cumbersome and error-prone detection steps. Note that, this nonlinear degradation model can be naturally formulated into a maximum-a posterioriframework, and can be efficiently decoupled into several solvable sub-problems via the alternating direction method of multipliers (ADMM). Experimental results on both synthetic and real-world images demonstrate that the proposed deblurring algorithm outperforms the state-of-the-art low-light saturation-based deblurring methods. [ABSTRACT FROM AUTHOR]

On-orbit real-time image processing is of increasing demands due to requirements for quick-response missions. Image destriping is usually an important pre-processing step to improve image quality in practice. The unidirectional variational models have shown impressive destriping performance. However, they are not easy for real-time implementation for high-computation complexity and there are few attempts for hardware implementation. This article is the first hardware implementation of variational image destriping algorithm, which achieves high throughput for large-swath remote-sensing images. In this article, a fully pipelined hardware architecture is proposed. First, the involved iteration loop is unrolled and a coarse-grained parallelism is obtained. Second, for each deployed iteration computation blocks (ICBs), a dedicated timing arrangement is designed to alleviate the bottleneck caused by the data dependency within each ICB, obtaining a fine-grained parallelism. Moreover, to further optimize the critical path, an approximate simplification scheme is proposed, saving the resource usage and reducing computing delay. The proposed architecture is implemented and verified on a XILINX 6vcx240t field programmable gate array (FPGA); it achieves a maximum frame rate up to 41.9 frames/s with delay of only tens of row cycles for 8-bit ${{2048}} \times {{2048}}$ images. It performs all the processing on the pixels in raster scan order on- the-fly as they are being transmitted from camera payload, which significantly facilitates on- orbit real-time processing of large-swath remote-sensing images with high data rate. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) restoration is to recover the clean image from degraded version, such as the noisy, blurred, or damaged. Recent low-rank tensor-based recovery methods have been widely explored in HSIs restoration. Most of previous methods, however, neglect an inconspicuous but important phenomenon that the physical meaning and dimension along the spatial, spectral, and nonlocal mode are markedly different. In this work, we discover the low-rank property discrepancy along spatial, spectral, and nonlocal self-similarity mode in the HSIs, and argue that the intrinsic low-rank correlations along each mode contribute different to the final restoration results. Consequently, we figure out that the combination of the spectral and nonlocal-induced low-rank is most beneficial for HSIs modeling, and propose an optimal low-rank tensor (OLRT) model for HSIs restoration. Furthermore, we not only explore the low-rank property in the image component, but also in the sparse error component (stripe noise in HSIs). Thus, we extend OLRT to the OLRT-robust principal component analysis (RPCA) with low-rank tensor priors for both the HSIs and sparse error. Besides, previous methods are usually designed for one specific HSI task, which is less robust to various tasks. We prove that the proposed optimal low-rank prior is very flexible for various HSI restoration problems including denoising, deblurring, inpainting, and destriping. The proposed methods have been extensively evaluated on several benchmarks and tasks, and greatly outperform state-of-the-art (STOA). We show the simple yet effective OLRT strategy is also beneficial to STOA. [ABSTRACT FROM AUTHOR]

For many remote sensing image applications, orientation estimation is a crucial step for feature points extraction and matching, but it has attracted little attention. Due to the intensity differences between remote sensing image pairs, it is difficult to estimate the orientations of corresponding points accurately, resulting in performance degradation of feature matching. Thus, encountering the intensity differences, a robust spatial structure description for feature regions, and an effective calculation manner from description to orientation play a key role in accurate orientation estimation. To this end, in this letter, we first define a plausible orientation for feature points by the total gradient, offering an effective way to convert the gradient trend of the feature region to orientation. Therefore, we further propose a novel orientation estimation method, in which the rotation invariant gradient is introduced to improve the accuracy of gradient calculation and robustness of spatial structure description. Experimental results on multisensor remote sensing images demonstrate that our method increases the orientation estimation accuracy remarkably and outperforms other orientation estimation methods by a large margin, and effectively improves the performance of feature matching. [ABSTRACT FROM AUTHOR]

Facial landmark detection is one crucial step for face-based image/video analysis. Despite the fact that recently many facial landmark detection models have achieved remarkable performance, most state-of-the-art heatmap regression-based methods heavily rely on initialization of the face detector. However, there inevitably exists semantic gaps among different annotators or face detectors. An improper facial bounding box will tremendously drop off the performance of the facial landmark detection model. Facial landmark detection would be more practical if robust to face images cropped by diverse manners (see , the col.1 shows face images cropped by a proper bounding box, col.2 and col.3 show face images cropped by an oversize and a small bounding boxes respectively). To this end, we present a “Unconstrained Facial Landmark Detection(UFLD)” mechanism, that aims at enhancing the robustness of facial landmark detection, to deal with the inconsistent cropping manner issue. UFLD consists of two aspects: a Transformation-Invariant Landmark Detector(TILD) and an Availability-Guided Solver(AGS). TILD gives the ability to detect consistent landmarks for face images cropped by diverse manners. And AGS can alleviate the by-effect of “landmarks outside the image” caused by improper cropping results or TILD, and further promote the performance. The proposed mechanism achieved above 6.5% improvement in standard normalized landmarks mean error reduction on face images cropped by diverse manners compared to baselines. [ABSTRACT FROM AUTHOR]

Aircraft landmark detection (ALD) aims at detecting the keypoints of aircraft, which can serve as an important role for subsequent applications such as fine-grained aircraft recognition. In ALD, the physical size discrepancy between different kinds of aircraft may lead to inconsistent landmark structure, which significantly harms landmark detection results. In this letter, we take advantage of the category prior to alleviate the size discrepancy in ALD. The proposed category-aware landmark detection network (CALDN) possesses two streams: a classification stream for size categorization and a localization stream for landmark detection. Instance-level size category information captured by classification stream serves as the guidance in the localization stream for robust landmark detection. Moreover, a category attention module (CAM) is proposed for better-utilizing category information to guide ALD. Benefitting from the adaptive attention mechanism, CAM can automatically highlight category-specific features for ulteriorly reducing the influence of size discrepancy. Furthermore, to advance ALD research, we contribute the first perspective-variant aircraft landmark dataset. Solid experiments demonstrate the superiority of our method. [ABSTRACT FROM AUTHOR]

Hyperspectral imaging, providing abundant spatial and spectral information simultaneously, has attracted a lot of interest in recent years. Unfortunately, due to the hardware limitations, the hyperspectral image (HSI) is vulnerable to various degradations, such as noises (random noise), blurs (Gaussian and uniform blur), and downsampled (both spectral and spatial downsample), each corresponding to the HSI denoising, deblurring, and super-resolution tasks, respectively. Previous HSI restoration methods are designed for one specific task only. Besides, most of them start from the 1-D vector or 2-D matrix models and cannot fully exploit the structurally spectral–spatial correlation in 3-D HSI. To overcome these limitations, in this article, we propose a unified low-rank tensor recovery model for comprehensive HSI restoration tasks, in which nonlocal similarity within spectral–spatial cubic and spectral correlation are simultaneously captured by third-order tensors. Furthermore, to improve the capability and flexibility, we formulate it as a weighted low-rank tensor recovery (WLRTR) model by treating the singular values differently. We demonstrate the reweighed strategy, which has been extensively studied in the matrix, also greatly benefits the tensor modeling. We also consider the stripe noise in HSI as the sparse error by extending WLRTR to robust principal component analysis (WLRTR-RPCA). Extensive experiments demonstrate the proposed WLRTR models consistently outperform state-of-the-art methods in typical HSI low-level vision tasks, including denoising, destriping, deblurring, and super-resolution. [ABSTRACT FROM AUTHOR]

Most of the existing medical image denoising methods focus on estimating either the image or the residual noise. Moreover, they are usually designed for one specific noise with a strong assumption of the noise distribution. However, not only the random independent Gaussian or speckle noise but also the structurally correlated ring or stripe noise is ubiquitous in various medical imaging instruments. Explicitly modeling the distributions of these complex noises in the medical image is extremely hard. They cannot be accurately held by the Gaussian or mixture of Gaussian model. To overcome the two drawbacks, in this paper, we propose to treat the image and noise components equally and convert the image denoising task into an image decomposition problem naturally. More precisely, we present a two-stage deep convolutional neural network (CNN) to model both the noise and the medical image simultaneously. On the one hand, we utilize both the image and noise to separate them better. On the other hand, the noise subnetwork serves as a noise estimator which guides the image subnetwork with sufficient information about the noise, thus we could easily handle different noise distributions and noise levels. To better cope with the gradient vanishing problem in this very deep network, we introduce both the short-term and long-term connections in the network which could promote the information propagation between different layers efficiently. Extensive experiments have been performed on several kinds of medical noise images, such as the computed tomography and ultrasound images, and the proposed method has consistently outperformed state-of-the-art denoising methods. [ABSTRACT FROM AUTHOR]

Stripe noise from different remote sensing imaging systems varies considerably in terms of response, length, angle, and periodicity. Due to the complex distributions of different stripes, the destriping results of previous methods may be oversmoothed or contain residual stripe. To overcome this key problem, we provide a comprehensive analysis of existing destriping methods and propose a deep convolutional neural network (CNN) for handling various kinds of stripes. Moreover, previous methods individually model the stripe or the image priors, which may lose the relationship between them. In this article, a two-stream CNN is designed to simultaneously model the stripe and image, which better facilitates distinguishing them from each other. Moreover, we incorporate the wavelet into our CNN model for better directional feature representation. Therefore, the CNN learns the discriminative representation from the external data set, while the wavelet models the internal directionality of the stripe, in which both the internal and external priors are beneficial to the destriping task. In addition, the wavelet extracts the multiscale information with a larger receptive field for global contextual information modeling; thus, we can better distinguish the stripe from the similar image line pattern structures. The proposed method has been extensively evaluated on a number of data sets and outperforms the state-of-the-art methods by substantially a large margin in terms of quantitative and qualitative assessments, speed, and robustness. [ABSTRACT FROM AUTHOR]

In the infrared focal plane arrays imaging systems, the temperature-dependent nonuniformity effects severely degrade the image quality. In this letter, we propose a very deep convolutional neural network for unified infrared aerothermal nonuniform correction. Our network is built with the multiscale and residual training. The multiscale subnetworks utilize the multiscale property in the images, and the long–short-term residual learning contributes to the information propagation. Compared with the previous methods, the proposed method is more robust to various nonuniform artifacts and more efficient at processing time. Experimental results validate the superiority of our method for infrared nonuniform correction. [ABSTRACT FROM AUTHOR]

The spectral and the spatial information in hyperspectral images (HSIs) are the two sides of the same coin. How to jointly model them is the key issue for HSIs’ noise removal, including random noise, structural stripe noise, and dead pixels/lines. In this paper, we introduce the deep convolutional neural network (CNN) to achieve this goal. The learned filters can well extract the spatial information within their local receptive filed. Meanwhile, the spectral correlation can be depicted by the multiple channels of the learned 2-D filters, namely, the number of filters in each layer. The consequent advantages of our CNN-based HSI denoising method (HSI-DeNet) over previous methods are threefold. First, the proposed HSI-DeNet can be regarded as a tensor-based method by directly learning the filters in each layer without damaging the spectral-spatial structures. Second, the HSI-DeNet can simultaneously accommodate various kinds of noise in HSIs. Moreover, our method is flexible for both single image and multiple images by slightly modifying the channels of the filters in the first and last layers. Last but not least, our method is extremely fast in the testing phase, which makes it more practical for real application. The proposed HSI-DeNet is extensively evaluated on several HSIs, and outperforms the state-of-the-art HSI-DeNets in terms of both speed and performance. [ABSTRACT FROM AUTHOR]

Stripe noise removal (destriping) is a fundamental problem in remote sensing image processing that holds significant practical importance for subsequent applications. These variational destriping methods have obtained impressive results and attracted widely studied research interests. However, most of them are dedicated to estimate the clear image from the striped one, paying much attention to the image itself, while ignoring the structural characteristic of stripe, which would easily cause damages to the image structure and leave residual stripes in image recovery. In this paper, we treat the image and stripe components equally and convert the image destriping task as an image decomposition problem naturally. We first give a detailed analysis about the structural characteristic of stripes and the prior knowledge about the remote sensing images. Then, incorporating them, we propose a low-rank-based single-image decomposition model (LRSID) to separate the original image from the stripe component perfectly. This low-rank constraint for the stripe perfectly matches the fact that only parts of data vectors are corrupted but the others are not. Moreover, we further utilize the spectral information of the remote sensing images, and we extend our 2-D image decomposition method to the 3-D case. Extensive experiments on both simulated and real data have been carried out to validate the effectiveness and efficiency of the proposed algorithms. [ABSTRACT FROM PUBLISHER]

Remote sensing images destriping and denoising are both classical problems, which have attracted major research efforts separately. This letter shows that the two problems can be successfully solved together within a unified variational framework. To do this, we proposed a joint destriping and denoising method by integrating the unidirectional total variation and sparse representation regularizations. Experimental results on simulated and real data in terms of qualitative and quantitative assessments show significant improvements over conventional methods. [ABSTRACT FROM PUBLISHER]

Detecting and matching image features is a fundamental task in video analytics and computer vision systems. It establishes the correspondences between two images taken at different time instants or from different viewpoints. However, its large computational complexity has been a challenge to most embedded systems. This paper proposes a new FPGA-based embedded system architecture for feature detection and matching. It consists of scale-invariant feature transform (SIFT) feature detection, as well as binary robust independent elementary features (BRIEF) feature description and matching. It is able to establish accurate correspondences between consecutive frames for 720-p (1280x720) video. It optimizes the FPGA architecture for the SIFT feature detection to reduce the utilization of FPGA resources. Moreover, it implements the BRIEF feature description and matching on FPGA. Due to these contributions, the proposed system achieves feature detection and matching at 60 frame/s for 720-p video. Its processing speed can meet and even exceed the demand of most real-life real-time video analytics applications. Extensive experiments have demonstrated its efficiency and effectiveness. [ABSTRACT FROM PUBLISHER]

Passive millimeter-wave (PMMW) images often suffer common problems of noise and blurring. A new method is proposed to estimate the instrument response function (IRF) and desired image simultaneously. The proposed variational model integrates the adaptive weight data term, image smooth term, and IRF smooth term. The major novelty of this work is that Huber–Markov regularization is adopted for PMMW image restoration, which can preserve structural details as well as suppress noise effectively. The IRF is parametrically formulated as a Gaussian-shaped function based on experimental measurements through the utilized PMMW imaging system. The alternation minimization iterative method is applied to achieve the IRF width and desired image. Comparative experimental results with some real PMMW images reveal that the proposed approach can effectively suppress noise, reduce ringing artifacts, and improve the spatial resolution. [ABSTRACT FROM AUTHOR]

Raman spectral interpretation often suffers common problems of band overlapping and random noise. Spectral deconvolution and feature-parameter extraction are both classical problems, which are known to be difficult and have attracted major research efforts. This paper shows that the two problems are tightly coupled and can be successfully solved together. Mutual support of Raman spectral deconvolution and feature-extraction processes within a joint variational framework are theoretically motivated and validated by successful experimental results. The main idea is to recover latent spectrum and extract spectral feature parameters from slit-distorted Raman spectrum simultaneously. Moreover, a robust adaptive Tikhonov regularization function is suggested to distinguish the flat, noise, and points, which can suppress noise effectively as well as preserve details. To evaluate the performance of the proposed method, quantitative and qualitative analyses were carried out by visual inspection and quality indexes of the simulated and real Raman spectra. [ABSTRACT FROM PUBLISHER]

Deconvolution has become one of the most used methods for improving spectral resolution. Deconvolution is an ill-posed problem, especially when the point spread function (PSF) is unknown. Non-blind deconvolution methods use a predefined PSF, but in practice the PSF is not known exactly. Blind deconvolution methods estimate the PSF and spectrum simultaneously from the observed spectra, which become even more difficult in the presence of strong noise. In this paper, we present a semi-blind deconvolution method to improve the spectral resolution that does not assume a known PSF but models it as a parametric function in combination with the a priori knowledge about the characteristics of the instrumental response. First, we construct the energy functional, including Tikhonov regularization terms for both the spectrum and the parametric PSF. Moreover, an adaptive weighting term is devised in terms of the magnitude of the first derivative of spectral data to adjust the Tikhonov regularization for the spectrum. Then we minimize the energy functional to obtain the spectrum and the parameters of the PSF. We also discuss how to select the regularization parameters. Comparative results with other deconvolution methods on simulated degraded spectra, as well as on experimental infrared spectra, are presented. [ABSTRACT FROM AUTHOR]

Atmospheric turbulence affects imaging systems by virtue of wave propagation through a medium with a nonuniform index of refraction. It can lead to blurring in images acquired from a long distance away. In this letter, it is observed that blurring increases the second-order central moment (SOCM) of images, and we introduce a new parametric blur identification method by minimizing SOCM. The method applies to finite-support images, in which the scene consists of a finite-extent object against a uniformly black, gray, or white background. The SOCM method has been validated by direct comparisons with other methods on simulated and real degraded images. [ABSTRACT FROM PUBLISHER]

Imaging from a degenerated push broom scanner usually leads to an undesired stripe noise which seriously affected the image quality. The original unidirectional total variation ( UTV ) model produces a poor performance on stripe images which has a strong contrast between cleaning image area and striped area or different image area. In this research, a new destriping method which combines TV - Stokes and UTV model ( UTV-Stokes for short) has been developed to overcome the disadvantages of UTV . By distinguishing stripe region and no-stripe region in calculation, this method can avoid excessive smoothing or residual's appearing through less filtering. Comparative results on simulated and real striped images taken with MODIS and hyperspectral imaging systems demonstrated that the proposed method not only can handle various stripe images with different noise intensity but also can preserve the edge and detailed information. [ABSTRACT FROM AUTHOR]

Abstract: Multi-detectors imaging system often suffers from the problem of the stripe noise, which greatly degrades the quality of the resulting images. To better remove stripe noise and preserve the edge and texture information, a robust destriping algorithm with spatially adaptive unidirectional total variation (SAUTV) model is introduced. The spatial information of the striping noise is detected by using the stripe indicator called difference eigenvalue, and a weighted parameter determined by the difference eigenvalue information is added to constrain the regularization strength of the UTV regularization. The proposed algorithm can effectively remove the stripe noise and preserve the edge and detailed information. Moreover, it becomes more robust with the change of the regularization parameter. Split Bregman method is utilized to efficiently solve the resulting minimization problem. Comparative results on simulated and real striped images taken with two kinds of imaging systems are reported. [Copyright &y& Elsevier]

Abstract Pacific saury (Cololabis saira) are widely distributed in northwestern Pacific, migrating from their spawning ground in subtropical Kuroshio waters south of Japan to feeding grounds in Oyashio waters northeast of Japan. The abundance of Pacific saury is expected to be affected not only by environmental conditions in the fishing grounds of the Oyashio region, but also by recruitment processes influenced by environmental conditions in the Kuroshio region. In this study, we focus on the effects of environmental variations in the spawning ground on Pacific saury abundance, approximated by catch and catch per unit effort (CPUE) data. To examine interannual-decadal variability in the spawning ground, we developed a suite of spawning ground indices, including (1) average winter sea surface temperature (WSST), (2) the meridional position of 19 °C sea surface temperature (SST) isoline (MP19), and (3) SST-suitability-weighted size of spawning ground (WSSG), in the Kuroshio region. These spawning ground indices exhibited interannual-decadal variation patterns with regime shifts in 1962/63, 1976/77, 1987/88, 1997/98, and likely in the early 2010s, which corresponded well to data on catch and CPUE of Pacific saury. Large scale climatic indices such as Southern Oscillation Index (SOI) and Asian Monsoon Index (MOI) were negatively correlated with winter SST in most of the Kuroshio region, suggesting that large-scale climatic influences played important roles in the variability of SST within the Kuroshio region. Gradient forest analyses were used to quantify the importance of these spawning ground indices for explaining the variations of Pacific saury abundance and to identify shifts in catch and CPUE along the gradients of the spawning ground indices. MP19 with a 2-year lag (MP19_Lag2) was identified as the most important predictor of Pacific saury abundance in terms of CPUE, followed by WSST_Lag2, WSST, WSSG_Lag1, and WSSG. Spawning ground indices, particularly MP19_Lag2, were useful for rationalizing the dynamics of Pacific saury abundance, matching well the striking declines of catch both in the early 1960s and also in the most recent years. Highlights • Spawning ground indices were estimated to analyze the effects on Pacific saury. • Spawning ground indices showed synchronous regime shifts with Pacific saury. • MP19 and WSSG well explained the dramatic decline in catch of Pacific saury. • Climate-induced variations in spawning ground affect Pacific saury abundance. [ABSTRACT FROM AUTHOR]

Optical flow has made great progress in clean scenes, while suffers degradation under adverse weather due to the violation of the brightness constancy and gradient continuity assumptions of optical flow. Typically, existing methods mainly adopt domain adaptation to transfer motion knowledge from clean to degraded domain through one-stage adaptation. However, this direct adaptation is ineffective, since there exists a large gap due to adverse weather and scene style between clean and real degraded domains. Moreover, even within the degraded domain itself, static weather (e.g., fog) and dynamic weather (e.g., rain) have different impacts on optical flow. To address above issues, we explore synthetic degraded domain as an intermediate bridge between clean and real degraded domains, and propose a cumulative homogeneous-heterogeneous adaptation framework for real adverse weather optical flow. Specifically, for clean-degraded transfer, our key insight is that static weather possesses the depth-association homogeneous feature which does not change the intrinsic motion of the scene, while dynamic weather additionally introduces the heterogeneous feature which results in a significant boundary discrepancy in warp errors between clean and degraded domains. For synthetic-real transfer, we figure out that cost volume correlation shares a similar statistical histogram between synthetic and real degraded domains, benefiting to holistically aligning the homogeneous correlation distribution for synthetic-real knowledge distillation. Under this unified framework, the proposed method can progressively and explicitly transfer knowledge from clean scenes to real adverse weather. In addition, we further collect a real adverse weather dataset with manually annotated optical flow labels and perform extensive experiments to verify the superiority of the proposed method. Both the code and the dataset will be available at https://github.com/hyzhouboy/CH2DA-Flow.

Detecting oriented objects along with estimating their rotation information is one crucial step for image analysis, especially for remote sensing images. Despite that many methods proposed recently have achieved remarkable performance, most of them directly learn to predict object directions under the supervision of only one (e.g., the rotation angle) or a few (e.g., several coordinates) groundtruth (GT) values individually. Oriented object detection would be more accurate and robust if extra constraints, with respect to proposal and rotation information regression, are adopted for joint supervision during training. To this end, we propose a mechanism that simultaneously learns the regression of horizontal proposals, oriented proposals, and rotation angles of objects in a consistent manner, via naive geometric computing, as one additional steady constraint. An oriented center prior guided label assignment strategy is proposed for further enhancing the quality of proposals, yielding better performance. Extensive experiments on six datasets demonstrate the model equipped with our idea significantly outperforms the baseline by a large margin and several new state-of-the-art results are achieved without any extra computational burden during inference. Our proposed idea is simple and intuitive that can be readily implemented. Source codes are publicly available at: https://github.com/wangWilson/CGCDet.git.

Lossy image compression is a fundamental technology in media transmission and storage. Variable-rate approaches have recently gained much attention to avoid the usage of a set of different models for compressing images at different rates. During the media sharing, multiple re-encodings with different rates would be inevitably executed. However, existing Variational Autoencoder (VAE)-based approaches would be readily corrupted in such circumstances, resulting in the occurrence of strong artifacts and the destruction of image fidelity. Based on the theoretical findings of preserving image fidelity via invertible transformation, we aim to tackle the issue of high-fidelity fine variable-rate image compression and thus propose the Invertible Continuous Codec (I2C). We implement the I2C in a mathematical invertible manner with the core Invertible Activation Transformation (IAT) module. I2C is constructed upon a single-rate Invertible Neural Network (INN) based model and the quality level (QLevel) would be fed into the IAT to generate scaling and bias tensors. Extensive experiments demonstrate that the proposed I2C method outperforms state-of-the-art variable-rate image compression methods by a large margin, especially after multiple continuous re-encodings with different rates, while having the ability to obtain a very fine variable-rate control without any performance compromise.

Single-image rain streaks' removal has attracted great attention in recent years. However, due to the highly visual similarity between the rain streaks and the line pattern image edges, the over-smoothing of image edges or residual rain streaks' phenomenon may unexpectedly occur in the deraining results. To overcome this problem, we propose a direction and residual awareness network within the curriculum learning paradigm for the rain streaks' removal. Specifically, we present a statistical analysis of the rain streaks on large-scale real rainy images and figure out that rain streaks in local patches possess principal directionality. This motivates us to design a direction-aware network for rain streaks' modeling, in which the principal directionality property endows us with the discriminative representation ability of better differing rain streaks from image edges. On the other hand, for image modeling, we are motivated by the iterative regularization in classical image processing and unfold it into a novel residual-aware block (RAB) to explicitly model the relationship between the image and the residual. The RAB adaptively learns balance parameters to selectively emphasize informative image features and better suppress the rain streaks. Finally, we formulate the rain streaks' removal problem into the curriculum learning paradigm which progressively learns the directionality of the rain streaks, rain streaks' appearance, and the image layer in a coarse-to-fine, easy-to-hard guidance manner. Solid experiments on extensive simulated and real benchmarks demonstrate the visual and quantitative improvement of the proposed method over the state-of-the-art methods.

Most existing learning-based deraining methods are supervisedly trained on synthetic rainy-clean pairs. The domain gap between the synthetic and real rain makes them less generalized to complex real rainy scenes. Moreover, the existing methods mainly utilize the property of the image or rain layers independently, while few of them have considered their mutually exclusive relationship. To solve above dilemma, we explore the intrinsic intra-similarity within each layer and inter-exclusiveness between two layers and propose an unsupervised non-local contrastive learning (NLCL) deraining method. The non-local self-similarity image patches as the positives are tightly pulled together and rain patches as the negatives are remarkably pushed away, and vice versa. On one hand, the intrinsic self-similarity knowledge within positive/negative samples of each layer benefits us to discover more compact representation; on the other hand, the mutually exclusive property between the two layers enriches the discriminative decomposition. Thus, the internal self-similarity within each layer (similarity) and the external exclusive relationship of the two layers (dissimilarity) serving as a generic image prior jointly facilitate us to unsupervisedly differentiate the rain from clean image. We further discover that the intrinsic dimension of the non-local image patches is generally higher than that of the rain patches. This insight motivates us to design an asymmetric contrastive loss that precisely models the compactness discrepancy of the two layers, thereby improving the discriminative decomposition. In addition, recognizing the limited quality of existing real rain datasets, which are often small-scale or obtained from the internet, we collect a large-scale real dataset under various rainy weathers that contains high-resolution rainy images. Extensive experiments conducted on different real rainy datasets demonstrate that the proposed method obtains state-of-the-art performance in real deraining.

The optoelectronic properties of all-inorganic perovskite solar cells are greatly affected by the quality characteristics of films, such as the defect concentration, crystal growth orientation, crystallinity, and morphology. In this study, a PbI 2 -(DMSO) 2 complex is adopted to partially replace PbI 2 as the lead source in the preparation of perovskite precursor solutions. Due to the rapid dispersion of the PbI 2 -(DMSO) 2 complex in a solvent, raw materials can rapidly react to form perovskite colloids with a narrow size distribution. Such uniform colloidal particles are found to be beneficial for achieving films with improved quality and highly orientated growth along the [001] direction. The optimized film exhibits a clearly improved crystallinity and a decrease in defect concentration from 4.29 × 10 15 cm -3 to 3.20 × 10 15 cm -3 . The device based on the obtained all-inorganic CsPbI 2.8 Br 0.2 perovskite finally achieves an increase in photovoltaic power conversion efficiency from 10.5 to 14.15%. In addition, the environmental stability of the device also benefits from the improved film quality. After 480 h of storage in air, the device can still maintain nearly 80% of its initial performance.

The reconstructed slice quality of flat-detector computed tomography (CT) is often disturbed by concentric-ring artifacts. Since concentric rings in CT slices appear as straight stripes when transformed into polar coordinates, a variation-based model is proposed to suppress the stripes. The method is motivated by two observations about stripes in polar coordinates: (1) ring artifacts attenuate gradually along the radial direction, leading to a sparse distribution of stripes and (2) stripes greatly distort the image gradient across the stripes, while slightly affecting the image gradient along the stripes. Thus, a [Formula: see text]-norm-based data fidelity term and a [Formula: see text]-norm/[Formula: see text]-norm unidirectional variation-based regularization term are presented to characterize the stripes. The alternating direction method of multipliers is introduced to solve the resulting minimization problem. Moreover, we discuss the interpolation methods used in coordinate transformation and find that the nearest neighbor interpolation is optimal. Experimental results on simulated and real data demonstrate that our method can correct ring artifacts effectively compared with state-of-the-art coordinate transformation-based methods, as well as preserve the structures and details of slices.

Laser instruments often suffer from the problem of baseline drift and random noise, which greatly degrade spectral quality. In this article, we propose a variation model that combines baseline correction and denoising. First, to guide the baseline estimation, morphological operations are adopted to extract the characteristics of the degraded spectrum. Second, to suppress noise in both the spectrum and baseline, Tikhonov regularization is introduced. Moreover, we describe an efficient optimization scheme that alternates between the latent spectrum estimation and the baseline correction until convergence. The major novel aspect of the proposed algorithms is the estimation of a smooth spectrum and removal of the baseline simultaneously. Results of a comparison with state-of-the-art methods demonstrate that the proposed method outperforms them in both qualitative and quantitative assessments.

Multispectral remote sensing images often suffer from the common problem of stripe noise, which greatly degrades the imaging quality and limits the precision of the subsequent processing. The conventional destriping approaches usually remove stripe noise band by band, and show their limitations on different types of stripe noise. In this paper, we tentatively categorize the stripes in remote sensing images in a more comprehensive manner. We propose to treat the multispectral images as a spectral-spatial volume and pose an anisotropic spectral-spatial total variation regularization to enhance the smoothness of solution along both the spectral and spatial dimension. As a result, a more comprehensive stripes and random noise are perfectly removed, while the edges and detail information are well preserved. In addition, the split Bregman iteration method is employed to solve the resulting minimization problem, which highly reduces the computational load. We extensively validate our method under various stripe categories and show comparison with other approaches with respect to result quality, running time, and quantitative assessments.

Multidetector imaging systems often suffer from the problem of stripe noise and random noise, which greatly degrade the imaging quality. In this paper, we propose a variational destriping method that combines unidirectional total variation and framelet regularization. Total-variation-based regularizations are considered effective in removing different kinds of stripe noise, and framelet regularization can efficiently preserve the detail information. In essence, these two regularizations are complementary to each other. Moreover, the proposed method can also efficiently suppress random noise. The split Bregman iteration method is employed to solve the resulting minimization problem. Comparative results demonstrate that the proposed method significantly outperforms state-of-the-art destriping methods on both qualitative and quantitative assessments.