Search

Hyperspectral image (HSI) restoration is to recover the clean image from degraded version, such as the noisy, blurred, or damaged. Recent low-rank tensor-based recovery methods have been widely explored in HSIs restoration. Most of previous methods, however, neglect an inconspicuous but important phenomenon that the physical meaning and dimension along the spatial, spectral, and nonlocal mode are markedly different. In this work, we discover the low-rank property discrepancy along spatial, spectral, and nonlocal self-similarity mode in the HSIs, and argue that the intrinsic low-rank correlations along each mode contribute different to the final restoration results. Consequently, we figure out that the combination of the spectral and nonlocal-induced low-rank is most beneficial for HSIs modeling, and propose an optimal low-rank tensor (OLRT) model for HSIs restoration. Furthermore, we not only explore the low-rank property in the image component, but also in the sparse error component (stripe noise in HSIs). Thus, we extend OLRT to the OLRT-robust principal component analysis (RPCA) with low-rank tensor priors for both the HSIs and sparse error. Besides, previous methods are usually designed for one specific HSI task, which is less robust to various tasks. We prove that the proposed optimal low-rank prior is very flexible for various HSI restoration problems including denoising, deblurring, inpainting, and destriping. The proposed methods have been extensively evaluated on several benchmarks and tasks, and greatly outperform state-of-the-art (STOA). We show the simple yet effective OLRT strategy is also beneficial to STOA. [ABSTRACT FROM AUTHOR]

Most of the existing medical image denoising methods focus on estimating either the image or the residual noise. Moreover, they are usually designed for one specific noise with a strong assumption of the noise distribution. However, not only the random independent Gaussian or speckle noise but also the structurally correlated ring or stripe noise is ubiquitous in various medical imaging instruments. Explicitly modeling the distributions of these complex noises in the medical image is extremely hard. They cannot be accurately held by the Gaussian or mixture of Gaussian model. To overcome the two drawbacks, in this paper, we propose to treat the image and noise components equally and convert the image denoising task into an image decomposition problem naturally. More precisely, we present a two-stage deep convolutional neural network (CNN) to model both the noise and the medical image simultaneously. On the one hand, we utilize both the image and noise to separate them better. On the other hand, the noise subnetwork serves as a noise estimator which guides the image subnetwork with sufficient information about the noise, thus we could easily handle different noise distributions and noise levels. To better cope with the gradient vanishing problem in this very deep network, we introduce both the short-term and long-term connections in the network which could promote the information propagation between different layers efficiently. Extensive experiments have been performed on several kinds of medical noise images, such as the computed tomography and ultrasound images, and the proposed method has consistently outperformed state-of-the-art denoising methods. [ABSTRACT FROM AUTHOR]

The spectral and the spatial information in hyperspectral images (HSIs) are the two sides of the same coin. How to jointly model them is the key issue for HSIs’ noise removal, including random noise, structural stripe noise, and dead pixels/lines. In this paper, we introduce the deep convolutional neural network (CNN) to achieve this goal. The learned filters can well extract the spatial information within their local receptive filed. Meanwhile, the spectral correlation can be depicted by the multiple channels of the learned 2-D filters, namely, the number of filters in each layer. The consequent advantages of our CNN-based HSI denoising method (HSI-DeNet) over previous methods are threefold. First, the proposed HSI-DeNet can be regarded as a tensor-based method by directly learning the filters in each layer without damaging the spectral-spatial structures. Second, the HSI-DeNet can simultaneously accommodate various kinds of noise in HSIs. Moreover, our method is flexible for both single image and multiple images by slightly modifying the channels of the filters in the first and last layers. Last but not least, our method is extremely fast in the testing phase, which makes it more practical for real application. The proposed HSI-DeNet is extensively evaluated on several HSIs, and outperforms the state-of-the-art HSI-DeNets in terms of both speed and performance. [ABSTRACT FROM AUTHOR]

Spectral signals often suffer from the common problems of band overlap and random Gaussian noise. To address these problems, we propose a sparsity regularization-based model that deconvolutes the degraded spectral signals. Sparsity regularization is achieved by fitting the probability density function of the gradient of the signal, and then, the iteratively reweighted least-squares method is used to solve the minimization problem. Results from experiments using real spectral signals showed that this algorithm separates the overlapping peaks and effectively suppresses the noise. The deconvoluted spectral signals will promote the practical application of infrared spectral analysis in the fields of target recognition, material identification, and chemometrics analysis. [ABSTRACT FROM AUTHOR]

Stripe noise removal (destriping) is a fundamental problem in remote sensing image processing that holds significant practical importance for subsequent applications. These variational destriping methods have obtained impressive results and attracted widely studied research interests. However, most of them are dedicated to estimate the clear image from the striped one, paying much attention to the image itself, while ignoring the structural characteristic of stripe, which would easily cause damages to the image structure and leave residual stripes in image recovery. In this paper, we treat the image and stripe components equally and convert the image destriping task as an image decomposition problem naturally. We first give a detailed analysis about the structural characteristic of stripes and the prior knowledge about the remote sensing images. Then, incorporating them, we propose a low-rank-based single-image decomposition model (LRSID) to separate the original image from the stripe component perfectly. This low-rank constraint for the stripe perfectly matches the fact that only parts of data vectors are corrupted but the others are not. Moreover, we further utilize the spectral information of the remote sensing images, and we extend our 2-D image decomposition method to the 3-D case. Extensive experiments on both simulated and real data have been carried out to validate the effectiveness and efficiency of the proposed algorithms. [ABSTRACT FROM PUBLISHER]

Aeroheating-induced intensity nonuniformity effects severely influence the effective performance of an infrared (IR) imaging system in high-speed flight. In this paper, we propose a new approach to the correction of intensity nonuniformity in IR images. The basic assumption is that the low-frequency intensity bias is additive and smoothly varying so that it can be modeled as a bivariate polynomial and estimated by using an isotropic total variation (TV) model. A half quadratic penalty method is applied to the isotropic form of TV discretization. And an alternating minimization algorithm is adopted for solving the optimization model. The experimental results of simulated and real aerothermal images show that the proposed correction method can effectively improve IR image quality. [ABSTRACT FROM AUTHOR]

Abstract: A poor inherent resolution capability of the passive millimeter-wave (PMMW) imaging becomes a problem in many applications. This paper proposes a parametric blind deconvolution approach for improving the resolution of PMMW images with low signal-to-noise ratio. Image deconvolution is a challenging ill-posed inverse problem when only partial knowledge of the point spread function (PSF) is available, therefore, regularization techniques need to be used. To restore high quality PMMW image, framelet based regularization constraint is incorporated into parametric blind PMMW image deconvolution framework, which is modeled as an alternative optimization problem about the image and the PSF. The PSF is modeled as a parametric form to restrict the PSF solution space. Furthermore, the split Bregman iteration is used to solve the resulting minimization problem. Comparative experimental results on simulated and real PMMW images show that the proposed method can effectively suppress noise, reduce artifacts, and improve the spatial resolution. [Copyright &y& Elsevier]

Abstract: In the paper, a multiframe blind image deconvolution method based on total variation and framelet regularizer is proposed. An adapted version of the split Bregman method is proposed to efficiently solve the resulting minimization problems. In each iteration, four sub-problems need to be solved, one of which can be very efficiently and easily solved via fast Fourier transform implementation or closed form solution. Both simulated noisy and blurred frames and real degraded frames are used to verify the effectiveness of the proposed method. Comparative experimental results show that the proposed method can efficiently remove the blur and noises and restore high quality sharp image. [Copyright &y& Elsevier]

Passive millimeter-wave (PMMW) images often suffer common problems of noise and blurring. A new method is proposed to estimate the instrument response function (IRF) and desired image simultaneously. The proposed variational model integrates the adaptive weight data term, image smooth term, and IRF smooth term. The major novelty of this work is that Huber–Markov regularization is adopted for PMMW image restoration, which can preserve structural details as well as suppress noise effectively. The IRF is parametrically formulated as a Gaussian-shaped function based on experimental measurements through the utilized PMMW imaging system. The alternation minimization iterative method is applied to achieve the IRF width and desired image. Comparative experimental results with some real PMMW images reveal that the proposed approach can effectively suppress noise, reduce ringing artifacts, and improve the spatial resolution. [ABSTRACT FROM AUTHOR]

Raman spectral interpretation often suffers common problems of band overlapping and random noise. Spectral deconvolution and feature-parameter extraction are both classical problems, which are known to be difficult and have attracted major research efforts. This paper shows that the two problems are tightly coupled and can be successfully solved together. Mutual support of Raman spectral deconvolution and feature-extraction processes within a joint variational framework are theoretically motivated and validated by successful experimental results. The main idea is to recover latent spectrum and extract spectral feature parameters from slit-distorted Raman spectrum simultaneously. Moreover, a robust adaptive Tikhonov regularization function is suggested to distinguish the flat, noise, and points, which can suppress noise effectively as well as preserve details. To evaluate the performance of the proposed method, quantitative and qualitative analyses were carried out by visual inspection and quality indexes of the simulated and real Raman spectra. [ABSTRACT FROM PUBLISHER]

Deconvolution has become one of the most used methods for improving spectral resolution. Deconvolution is an ill-posed problem, especially when the point spread function (PSF) is unknown. Non-blind deconvolution methods use a predefined PSF, but in practice the PSF is not known exactly. Blind deconvolution methods estimate the PSF and spectrum simultaneously from the observed spectra, which become even more difficult in the presence of strong noise. In this paper, we present a semi-blind deconvolution method to improve the spectral resolution that does not assume a known PSF but models it as a parametric function in combination with the a priori knowledge about the characteristics of the instrumental response. First, we construct the energy functional, including Tikhonov regularization terms for both the spectrum and the parametric PSF. Moreover, an adaptive weighting term is devised in terms of the magnitude of the first derivative of spectral data to adjust the Tikhonov regularization for the spectrum. Then we minimize the energy functional to obtain the spectrum and the parameters of the PSF. We also discuss how to select the regularization parameters. Comparative results with other deconvolution methods on simulated degraded spectra, as well as on experimental infrared spectra, are presented. [ABSTRACT FROM AUTHOR]

On-orbit real-time image processing is of increasing demands due to requirements for quick-response missions. Image destriping is usually an important pre-processing step to improve image quality in practice. The unidirectional variational models have shown impressive destriping performance. However, they are not easy for real-time implementation for high-computation complexity and there are few attempts for hardware implementation. This article is the first hardware implementation of variational image destriping algorithm, which achieves high throughput for large-swath remote-sensing images. In this article, a fully pipelined hardware architecture is proposed. First, the involved iteration loop is unrolled and a coarse-grained parallelism is obtained. Second, for each deployed iteration computation blocks (ICBs), a dedicated timing arrangement is designed to alleviate the bottleneck caused by the data dependency within each ICB, obtaining a fine-grained parallelism. Moreover, to further optimize the critical path, an approximate simplification scheme is proposed, saving the resource usage and reducing computing delay. The proposed architecture is implemented and verified on a XILINX 6vcx240t field programmable gate array (FPGA); it achieves a maximum frame rate up to 41.9 frames/s with delay of only tens of row cycles for 8-bit ${{2048}} \times {{2048}}$ images. It performs all the processing on the pixels in raster scan order on- the-fly as they are being transmitted from camera payload, which significantly facilitates on- orbit real-time processing of large-swath remote-sensing images with high data rate. [ABSTRACT FROM AUTHOR]

Interrogation of subcellular biological dynamics occurring in a living cell often requires noninvasive imaging of the fragile cell with high spatiotemporal resolution across all three dimensions. It thereby poses big challenges to modern fluorescence microscopy implementations because the limited photon budget in a live-cell imaging task makes the achievable performance of conventional microscopy approaches compromise between their spatial resolution, volumetric imaging speed, and phototoxicity. Here, we incorporate a two-stage view-channel-depth (VCD) deep-learning reconstruction strategy with a Fourier light-field microscope based on diffractive optical element to realize fast 3D super-resolution reconstructions of intracellular dynamics from single diffraction-limited 2D light-filed measurements. This VCD-enabled Fourier light-filed imaging approach (F-VCD), achieves video-rate (50 volumes per second) 3D imaging of intracellular dynamics at a high spatiotemporal resolution of ~180 nm × 180 nm × 400 nm and strong noise-resistant capability, with which light field images with a signal-to-noise ratio (SNR) down to -1.62 dB could be well reconstructed. With this approach, we successfully demonstrate the 4D imaging of intracellular organelle dynamics, e.g., mitochondria fission and fusion, with ~5000 times of observation. A super-resolution light-field imaging method enables video-rate 3D imaging of intracellular dynamics beyond diffraction limit and with strong noise-resistant capability. [ABSTRACT FROM AUTHOR]

Stripe noise from different remote sensing imaging systems varies considerably in terms of response, length, angle, and periodicity. Due to the complex distributions of different stripes, the destriping results of previous methods may be oversmoothed or contain residual stripe. To overcome this key problem, we provide a comprehensive analysis of existing destriping methods and propose a deep convolutional neural network (CNN) for handling various kinds of stripes. Moreover, previous methods individually model the stripe or the image priors, which may lose the relationship between them. In this article, a two-stream CNN is designed to simultaneously model the stripe and image, which better facilitates distinguishing them from each other. Moreover, we incorporate the wavelet into our CNN model for better directional feature representation. Therefore, the CNN learns the discriminative representation from the external data set, while the wavelet models the internal directionality of the stripe, in which both the internal and external priors are beneficial to the destriping task. In addition, the wavelet extracts the multiscale information with a larger receptive field for global contextual information modeling; thus, we can better distinguish the stripe from the similar image line pattern structures. The proposed method has been extensively evaluated on a number of data sets and outperforms the state-of-the-art methods by substantially a large margin in terms of quantitative and qualitative assessments, speed, and robustness. [ABSTRACT FROM AUTHOR]

Taking the advantages of photonic technology, such as large bandwidth, good tunability, and immunity to electromagnetic interference, a two-step high-accuracy photonics-assisted microwave frequency measurement method based on frequency-to-time mapping is proposed. Through high-speed optical frequency sweeping and optical filtering, the method maps different signal frequencies to time-domain pulses at different time and can measure multiple frequencies simultaneously. With the assistance of ultra-high-speed frequency sweeping and frequency-to-time mapping, the fast coarse identification of multiple different signal frequencies in the wide bandwidth range of 12 GHz is realized, and the measurement accuracy is better than 100 MHz. After the signal frequency is coarsely determined, low-speed frequency sweeping and frequency-to-time mapping are employed to realize accurate identification of the signal frequency within a small bandwidth. When the sweep rate is 2 and 0.2 GHz/μs, the accuracy of accurate measurement is better than 30 and 20 MHz respectively. Through the two-step measurement, high-accuracy frequency measurement can be achieved in a large bandwidth range while greatly reducing the microwave frequency measurement time. [ABSTRACT FROM AUTHOR]

Highlights • Motion blur kernel size is a significant input parameter of deblurring algorithms. • A regression CNN is proposed to estimate the kernel size of a blurry image. • The kernel size for training is labeled in a relative way. • The kernel size to be estimated can be arbitrary. • The estimated kernel size helps existing deblurring algorithms yielding satisfactory results. Abstract Deblurring is to restore a latent clear image as well as to estimate an underlying blur kernel from a single blurry image. Motion blur kernel size is a significant input parameter of existing deblurring algorithms. Setting the size manually, which is a tedious trial-and-error process, can hardly obtain a satisfactory result directly and effectively. In this paper, we formulate the kernel size estimation as a regression problem and construct a convolutional neural network (CNN) to resolve it. After training the CNN by a large number of simulated blurry images with labels, which are produced by a relative way, we can estimate the kernel sizes of a given blurry image without size limitation. Both quantitative and qualitative results of experiments show that our method can not only estimate the motion blur kernel size accurately but also help to yield better pictures with estimated kernel sizes as input parameters. [ABSTRACT FROM AUTHOR]

Single image deblurring has achieved significant progress for natural daytime images. Saturation is a common phenomenon in blurry images, due to the low light conditions and long exposure times. However, conventional linear deblurring methods usually deal with natural blurry images well but result in severe ringing artifacts when recovering low-light saturated blurry images. To solve this problem, we formulate the saturation deblurring problem as a nonlinear model, in which all the saturated and unsaturated pixels are modeled adaptively. Specifically, we additionally introduce a nonlinear function to the convolution operator to accommodate the procedure of the saturation in the presence of the blurring. The proposed method has two advantages over previous methods. On the one hand, the proposed method achieves the same high quality of restoring the natural image as seen in conventional deblurring methods, while also reducing the estimation errors in saturated areas and suppressing ringing artifacts. On the other hand, compared with the recent saturated-based deblurring methods, the proposed method captures the formation of unsaturated and saturated degradations straightforwardly rather than with cumbersome and error-prone detection steps. Note that, this nonlinear degradation model can be naturally formulated into a maximum-a posterioriframework, and can be efficiently decoupled into several solvable sub-problems via the alternating direction method of multipliers (ADMM). Experimental results on both synthetic and real-world images demonstrate that the proposed deblurring algorithm outperforms the state-of-the-art low-light saturation-based deblurring methods. [ABSTRACT FROM AUTHOR]

Imaging from a degenerated push broom scanner usually leads to an undesired stripe noise which seriously affected the image quality. The original unidirectional total variation ( UTV ) model produces a poor performance on stripe images which has a strong contrast between cleaning image area and striped area or different image area. In this research, a new destriping method which combines TV - Stokes and UTV model ( UTV-Stokes for short) has been developed to overcome the disadvantages of UTV . By distinguishing stripe region and no-stripe region in calculation, this method can avoid excessive smoothing or residual's appearing through less filtering. Comparative results on simulated and real striped images taken with MODIS and hyperspectral imaging systems demonstrated that the proposed method not only can handle various stripe images with different noise intensity but also can preserve the edge and detailed information. [ABSTRACT FROM AUTHOR]

Abstract: Multi-detectors imaging system often suffers from the problem of the stripe noise, which greatly degrades the quality of the resulting images. To better remove stripe noise and preserve the edge and texture information, a robust destriping algorithm with spatially adaptive unidirectional total variation (SAUTV) model is introduced. The spatial information of the striping noise is detected by using the stripe indicator called difference eigenvalue, and a weighted parameter determined by the difference eigenvalue information is added to constrain the regularization strength of the UTV regularization. The proposed algorithm can effectively remove the stripe noise and preserve the edge and detailed information. Moreover, it becomes more robust with the change of the regularization parameter. Split Bregman method is utilized to efficiently solve the resulting minimization problem. Comparative results on simulated and real striped images taken with two kinds of imaging systems are reported. [Copyright &y& Elsevier]

Detecting and matching image features is a fundamental task in video analytics and computer vision systems. It establishes the correspondences between two images taken at different time instants or from different viewpoints. However, its large computational complexity has been a challenge to most embedded systems. This paper proposes a new FPGA-based embedded system architecture for feature detection and matching. It consists of scale-invariant feature transform (SIFT) feature detection, as well as binary robust independent elementary features (BRIEF) feature description and matching. It is able to establish accurate correspondences between consecutive frames for 720-p (1280x720) video. It optimizes the FPGA architecture for the SIFT feature detection to reduce the utilization of FPGA resources. Moreover, it implements the BRIEF feature description and matching on FPGA. Due to these contributions, the proposed system achieves feature detection and matching at 60 frame/s for 720-p video. Its processing speed can meet and even exceed the demand of most real-life real-time video analytics applications. Extensive experiments have demonstrated its efficiency and effectiveness. [ABSTRACT FROM PUBLISHER]

Abstract: SIFT has shown a great success in various computer vision applications. However, its large computational complexity has been a challenge to most embedded implementations. This paper presents a low-cost embedded system based on a new architecture that successfully integrates FPGA and DSP. It optimizes the FPGA architecture for the feature detection step of SIFT to reduce the resource utilization, and optimizes the implementation of the feature description step using a high-performance DSP. Due to this novel design, this system can detect SIFT feature and extract SIFT descriptor for detected features in real-time. Extensive experiments demonstrate its effectiveness and efficiency. [Copyright &y& Elsevier]

Abstract: The paper presents a versatile nonlinear diffusion method to visually enhance the angiogram images for improving the clinical diagnosis. Traditional nonlinear diffusion has been shown very effective in edge-preserved smoothing of images. However, the existing nonlinear diffusion models suffer several drawbacks: sensitivity to the choice of the conductance parameter, limited range of edge enhancement, and the sensitivity to the selection of evolution time. The new anisotropic diffusion we proposed is based on facet model which can solve the issues mentioned above adaptively according to the image content. This method uses facet model for fitting the image to reduce noise, and uses the sum square of eigenvalues of Hessian as the standard of the conductance parameter selection synchronously. The capability of dealing with noise and conductance parameter can also change adaptively in the whole diffusion process. Moreover, our method is not sensitive to the choice of evolution time. Experimental results show that our new method is more effective than the original anisotropic diffusion. [Copyright &y& Elsevier]

Facial landmark detection is one crucial step for face-based image/video analysis. Despite the fact that recently many facial landmark detection models have achieved remarkable performance, most state-of-the-art heatmap regression-based methods heavily rely on initialization of the face detector. However, there inevitably exists semantic gaps among different annotators or face detectors. An improper facial bounding box will tremendously drop off the performance of the facial landmark detection model. Facial landmark detection would be more practical if robust to face images cropped by diverse manners (see , the col.1 shows face images cropped by a proper bounding box, col.2 and col.3 show face images cropped by an oversize and a small bounding boxes respectively). To this end, we present a “Unconstrained Facial Landmark Detection(UFLD)” mechanism, that aims at enhancing the robustness of facial landmark detection, to deal with the inconsistent cropping manner issue. UFLD consists of two aspects: a Transformation-Invariant Landmark Detector(TILD) and an Availability-Guided Solver(AGS). TILD gives the ability to detect consistent landmarks for face images cropped by diverse manners. And AGS can alleviate the by-effect of “landmarks outside the image” caused by improper cropping results or TILD, and further promote the performance. The proposed mechanism achieved above 6.5% improvement in standard normalized landmarks mean error reduction on face images cropped by diverse manners compared to baselines. [ABSTRACT FROM AUTHOR]

Abstract Pacific saury (Cololabis saira) are widely distributed in northwestern Pacific, migrating from their spawning ground in subtropical Kuroshio waters south of Japan to feeding grounds in Oyashio waters northeast of Japan. The abundance of Pacific saury is expected to be affected not only by environmental conditions in the fishing grounds of the Oyashio region, but also by recruitment processes influenced by environmental conditions in the Kuroshio region. In this study, we focus on the effects of environmental variations in the spawning ground on Pacific saury abundance, approximated by catch and catch per unit effort (CPUE) data. To examine interannual-decadal variability in the spawning ground, we developed a suite of spawning ground indices, including (1) average winter sea surface temperature (WSST), (2) the meridional position of 19 °C sea surface temperature (SST) isoline (MP19), and (3) SST-suitability-weighted size of spawning ground (WSSG), in the Kuroshio region. These spawning ground indices exhibited interannual-decadal variation patterns with regime shifts in 1962/63, 1976/77, 1987/88, 1997/98, and likely in the early 2010s, which corresponded well to data on catch and CPUE of Pacific saury. Large scale climatic indices such as Southern Oscillation Index (SOI) and Asian Monsoon Index (MOI) were negatively correlated with winter SST in most of the Kuroshio region, suggesting that large-scale climatic influences played important roles in the variability of SST within the Kuroshio region. Gradient forest analyses were used to quantify the importance of these spawning ground indices for explaining the variations of Pacific saury abundance and to identify shifts in catch and CPUE along the gradients of the spawning ground indices. MP19 with a 2-year lag (MP19_Lag2) was identified as the most important predictor of Pacific saury abundance in terms of CPUE, followed by WSST_Lag2, WSST, WSSG_Lag1, and WSSG. Spawning ground indices, particularly MP19_Lag2, were useful for rationalizing the dynamics of Pacific saury abundance, matching well the striking declines of catch both in the early 1960s and also in the most recent years. Highlights • Spawning ground indices were estimated to analyze the effects on Pacific saury. • Spawning ground indices showed synchronous regime shifts with Pacific saury. • MP19 and WSSG well explained the dramatic decline in catch of Pacific saury. • Climate-induced variations in spawning ground affect Pacific saury abundance. [ABSTRACT FROM AUTHOR]

Estimating an accuracy spectrum from a linearly degraded and noisy data record is often desirable in infrared spectroscopy application. In this paper, we proposed a new framework which can estimate the desired spectrum and the instrument function simultaneously from the degraded data. Firstly, the instrument function is parametrically represented as a Lorentzian function for the first time. Then, we construct the cost functional with the DE regularization, and minimize the functional to obtain the desired spectrum and instrument function. It has been found that the program is very useful to determine accurate line widths and peak positions from degraded spectral spectrum. Comparative results including quantitative and qualitative analyses manifest that the proposed method outperforms the conventional methods on peak narrowing and noise suppression. [ABSTRACT FROM AUTHOR]

While Human-Object Interaction (HOI) Detection has achieved tremendous advances in recent, it still remains challenging due to complex interactions with multiple humans and objects occurring in images, which would inevitably lead to ambiguities. Most existing methods either generate all human-object pair candidates and infer their relationships by cropped local features successively in a two-stage manner, or directly predict interaction points in a one-stage procedure. However, the lack of spatial configurations or reasoning steps of two- or one- stage methods respectively limits their performance in such complex scenes. To avoid this ambiguity, we propose a novel actor-centric framework. The main ideas are that when inferring interactions: 1) the non-local features of the entire image guided by actor position are obtained to model the relationship between the actor and context, and then 2) we use an object branch to generate pixel-wise interaction area prediction, where the interaction area denotes the object central area. Moreover, we also use an actor branch to get interaction prediction of the actor and propose a novel composition strategy based on center-point indexing to generate the final HOI prediction. Thanks to the usage of the non-local features and the partly-coupled property of the human-objects composition strategy, our proposed framework can detect HOI more accurately especially for complex images. Extensive experimental results show that our method achieves the state-of-the-art on the challenging V-COCO and HICO-DET benchmarks and is more robust especially in multiple persons and/or objects scenes. [Display omitted] • We propose a novel actor-centric HOI detection framework to explore the relationships between one human and multiple objects. • We formulate the HOI detection task as a pixel-wise classification problem in a one-stage-like procedure. • We present an RGBM Generator to provide the actor-centric guidance and get the final prediction by scoremap indexing. • Both quantitative and qualitative results prove the superiority of our method, especially for complex images. [ABSTRACT FROM AUTHOR]

Moving Object Detection (MOD) aims at extracting foreground moving objects in videos from static cameras. While low-rank based approaches have achieved impressive success in the MOD task, their performance remains limited on dynamics background scenes. The main reason is that dynamic clutters, e.g. , swaying leaves and rippers , are easy to mix up with moving objects in the decomposition model which simply classify the sparse noise as foregrounds. In order to improve the generalization ability of low-rank based moving object detectors, we suggest adding an explicit dynamic clutter component in the decomposition framework with realistic dynamic background modeling. Then the dynamic clutter can be learned through object-free video data due to their self-similarity across time and space. Thus, the moving objects can be naturally separated by a tensor-based decomposition model which formulates the static background by a unidirectional low-rank tensor, learns the dynamic clutter by a two-stream neural network, and constrains moving objects with spatiotemporal continuity. To further provide a more accurate object detection result, an objectness prior is embedded into our model in an attention manner. Extensive experimental results on the challenging datasets of dynamic background clearly demonstrate the superior performance of our model over the state-of-the-art in terms of quantitative metrics and visual quality. [Display omitted] • A novel video decomposition framework, additionally modeling the dynamic clutter explicitly. • A unified tensor-based framework to capture the appearance and motion information simultaneously for three components. • A self-supervised strategy to learn the dynamic clutter well by considering the video sequence itself as a dataset. • Bridge the gap between discriminative prior and the low-rank MAP framework. [ABSTRACT FROM AUTHOR]

Image restoration is an ill-posed problem that requires regularization to solve. Many existing regularization terms in the literature are the convex function. However, nonconvex nonsmooth regularization has advantages over convex regularization for restoring images, but its practical interest used to be limited by the difficulty of the computational stage which requires a nonconvex nonsmooth minimization. In this paper, an adaptive nonconvex nonsmooth regularization is proposed for image restoration by using the spatial information indicator. Moreover, an efficient numerical algorithm for solving the resulting minimization problem is provided by applying the variable splitting and the penalty techniques. Finally, its advantages are shown in deblurring edges and restoring fines of image simultaneously in experiments. [ABSTRACT FROM AUTHOR]

Total variation (TV) has been used as a popular and effective image prior model in regularization-based image restoration, because of its ability to preserve edges. However, as the total variation model favors a piecewise constant solution, the processing results in the flat regions of the image are poor, and the amplitude of the edges will be underestimated; the underlying cause of the problem is that the model is based on derivation which only considers the local feature of the image. In this paper, we first propose an adaptive non-local total variation image blind restoration algorithm for deblurring a single image via a non-local total variation operator, which exploits the correlation in the image, and then an extended split Bregman iteration is proposed to address the joint minimization problem. Second, the maximum average absolute difference (MAAD) method is employed to estimate the blur support and initialize the blur kernel. Extensive experiments demonstrate that the proposed approach produces results superior to most methods in both visual image quality and quantitative measures. [ABSTRACT FROM AUTHOR]