Search

Hyperspectral unmixing is a crucial preprocessing step for material classification and recognition. In the last decade, nonnegative matrix factorization (NMF) and its extensions have been intensively studied to unmix hyperspectral imagery and recover the material end-members. As an important constraint for NMF, sparsity has been modeled making use of the L1 regularizer. Unfortunately, the L1 regularizer cannot enforce further sparsity when the full additivity constraint of material abundances is used, hence limiting the practical efficacy of NMF methods in hyperspectral unmixing. In this paper, we extend the NMF method by incorporating the L1/2 sparsity constraint, which we name L1/2-NMF. The L1/2 regularizer not only induces sparsity but is also a better choice among Lq(0 < q < 1) regularizers. We propose an iterative estimation algorithm for L1/2-NMF, which provides sparser and more accurate results than those delivered using the L1 norm. We illustrate the utility of our method on synthetic and real hyperspectral data and compare our results to those yielded by other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Object detection in remote sensing (RS) images is a challenging task due to the difficulties of small size, varied appearance, and complex background. Although a lot of methods have been developed to address this problem, many of them cannot fully exploit multilevel context information or handle cluttered background in RS images either. To this end, in this paper, we propose a feature fusion and filtration network ( F 3 -Net) to improve object detection in RS images, which has higher capacity of combining the context information at multiple scales while suppressing the interference from the background. Specifically, F 3 -Net leverages a feature adaptation block with a residual structure to adjust the backbone network in an end-to-end manner, better considering the characteristics of RS images. Afterward, the network learns the context information of the object at multiple scales by hierarchically fusing the feature maps from different layers. In order to suppress the interference from cluttered background, the fused feature is then projected into a low-dimensional subspace by an additional feature filtration module. As a result, more relevant and accurate context information is extracted for further detection. Extensive experiments on DOTA, NWPU VHR-10, and UCAS AOD datasets demonstrate that the proposed detector achieves very promising detection performance. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification has long been a hot research topic. Most previous researches concentrate on the classification task of a single HSI scene, called single-scene classification. This research focuses on two closely related HSI scenes (called source and target scenes, respectively), and the problem is named cross-scene classification. This paper aims to explore the shared feature sub-space between two HSI scenes. A transfer learning algorithm called cross-domain residual deep nonnegative matrix factorization (CDRDNMF) is proposed. CDRDNMF is a multi-layer architecture consisting of dual-dictionary nonnegative matrix factorization (DDNMF) layers. In each layer, DDNMF is performed on source and target features for domain-invariant feature extraction. Then a data recovery process is completed, and the residual components from the recovery are passed to the next layer after activation. With such a multi-layer architecture, CDRDNMF delivers knowledge transfer and multi-scale feature extraction tasks. The experimental results prove the excellent performance of CDRDNMF on cross-scene classification. [ABSTRACT FROM AUTHOR]

The bloom of Internet has made fast text categorization very essential. Generally, the popular methods have good classification accuracy but slow speed, and vice versa. This paper proposes a novel approach for fast text categorization, in which a collaborative work framework based on a linear classifier and an extreme learning machine (ELM) is constructed. The linear classifier, obtained by a modified non-negative matrix factorization algorithm, maps all documents from the original term space into the class space directly such that it performs classification very fast. The ELM, with good classification accuracy via some nonlinear and linear transformations, classifies a few of documents according to some given criteria to improve the classification quality of the total system. Experimental results show that the proposed method not only achieves good accuracy but also performs classification very fast, which improves the averaged speed by 180 % compared with its corresponding method. [ABSTRACT FROM AUTHOR]

Skeleton-based action recognition is widely used in varied areas, e.g., surveillance and human-machine interaction. Existing models are mainly learned in a supervised manner, thus heavily depending on large-scale labeled data, which could be infeasible when labels are prohibitively expensive. In this paper, we propose a novel Contrast-Reconstruction Representation Learning network (CRRL) that simultaneously captures postures and motion dynamics for unsupervised skeleton-based action recognition. It consists of three parts: Sequence Reconstructor (SER), Contrastive Motion Learner (CML), and Information Fuser (INF). SER learns representation from skeleton coordinate sequence via reconstruction. However the learned representation tends to focus on trivial postural coordinates and be hesitant in motion learning. To enhance the learning of motions, CML performs contrastive learning between the representation learned from coordinate sequences and additional velocity sequences, respectively. Finally, in the INF module, we explore varied strategies to combine SER and CML, and propose to couple postures and motions via a knowledge-distillation based fusion strategy which transfers the motion learning from CML to SER. Experimental results on several benchmarks, i.e., NTU RGB+D 60/120, PKU-MMD, CMU, and NW-UCLA, demonstrate the promise of the our method by outperforming state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Deep learning (DL) based hyperspectral images (HSIs) denoising approaches directly learn the nonlinear mapping between noisy and clean HSI pairs. They usually do not consider the physical characteristics of HSIs. This drawback makes the models lack interpretability that is key to understanding their denoising mechanism and limits their denoising ability. In this paper, we introduce a novel model-guided interpretable network for HSI denoising to tackle this problem. Fully considering the spatial redundancy, spectral low-rankness, and spectral-spatial correlations of HSIs, we first establish a subspace-based multidimensional sparse (SMDS) model under the umbrella of tensor notation. After that, the model is unfolded into an end-to-end network named SMDS-Net, whose fundamental modules are seamlessly connected with the denoising procedure and optimization of the SMDS model. This makes SMDS-Net convey clear physical meanings, i.e., learning the low-rankness and sparsity of HSIs. Finally, all key variables are obtained by discriminative training. Extensive experiments and comprehensive analysis on synthetic and real-world HSIs confirm the strong denoising ability, strong learning capability, promising generalization ability, and high interpretability of SMDS-Net against the state-of-the-art HSI denoising methods. The source code and data of this article will be made publicly available at https://github.com/bearshng/smds-net for reproducible research. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel method for boundary detection in close-range hyperspectral images. This method can effectively predict the boundaries of objects of similar colour but different materials. To effectively extract the material information in the image, the spatial distribution of the spectral responses of different materials or endmembers is first estimated by hyperspectral unmixing. The resulting abundance map represents the fraction of each endmember spectra at each pixel. The abundance map is used as a supportive feature such that the spectral signature and the abundance vector for each pixel are fused to form a new spectral feature vector. Then different spectral similarity measures are adopted to construct a sparse spectral-spatial affinity matrix that characterizes the similarity between the spectral feature vectors of neighbouring pixels within a local neighborhood. After that, a spectral clustering method is adopted to produce eigenimages. Finally, the boundary map is constructed from the most informative eigenimages. We created a new HSI dataset and use it to compare the proposed method with four alternative methods, one for hyperspectral image and three for RGB image. The results exhibit that our method outperforms the alternatives and can cope with several scenarios that methods based on colour images cannot handle. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification is a small-sample-size problem due to the expensive cost of labeling. As a novel approach to this problem, cross-scene HSI classification has become a hot research topic in recent years. In cross-scene HSI classification, the scene containing enough labeled samples (called source scene) is used to benefit the classification in another scene containing a small number of training samples (called target scene). Transfer learning is a typical solution for cross-scene classification. However, many transfer learning algorithms assume an identical feature space for source and target scenes, which violates the fact that source and target scenes often lie in different feature spaces with various dimensions due to different HSI sensors. Aiming at the different feature spaces between the two scenes, we propose an end-to-end heterogeneous deep transfer learning algorithm, namely, cross-domain variational autoencoder (CDVAE). This algorithm is mainly composed of two key parts: 1) the features of the two scenes are embedded into the shared feature subspace through the two-stream variational autoencoder (VAE) to ensure that the output feature dimensions of the two scenes are identical and 2) graph regularization is used to establish the manifold constraints between source and target scenes in the shared subspace, so as to align the feature spaces. Experiments on two different cross-scene HSI datasets have proved the superior performance of the proposed CDVAE algorithm. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) denoising is an ill-posed inverse problem. The underlying physical model is always important to tackle this problem, which is unfortunately ignored by most of the current deep learning (DL)-based methods, producing poor denoising performance. To address this issue, this article introduces an end-to-end model-aided nonlocal neural network (MAC-Net) which simultaneously takes the spectral low-rank model and spatial deep prior into account for HSI noise reduction. Specifically, motivated by the success of the spectral low-rank model in depicting the strong spectral correlations and the nonlocal similarity prior in capturing spatial long-range dependencies, we first build a spectral low-rank model and then integrate a nonlocal U-Net into the model. In this way, we obtain a hybrid model-based and DL-based HSI denoising method where the spatial local and nonlocal multi-scale and spectral low-rank structures are effectively exploited. After that, we cast the optimization and denoising procedure of the hybrid method as a forward process of a neural network and introduce a set of learnable modules to yield our MAC-Net. Compared with traditional model-based methods, our MAC-Net overcomes the difficulties of accurate modeling, thanks to the strong learning and representation ability of DL. Unlike most “black-box” DL-based methods, the spectral low-rank model is beneficial to increase the generalization ability of the network and decrease the requirement of training samples. Experimental results on the natural and remote-sensing HSIs show that MAC-Net achieves state-of-the-art performance over both model-based and DL-based methods. The source code and data of this article will be made publicly available at https://github.com/bearshng/mac-net for reproducible research. [ABSTRACT FROM AUTHOR]

Hyperspectral unmixing is recognized as an important tool to learn the constituent materials and corresponding distribution in a scene. The physical spectral mixture model is always important to tackle this problem because of its highly ill-posed nature. In this article, we introduce a linear spectral mixture model (LMM)-based end-to-end deep neural network named SNMF-Net for hyperspectral unmixing. SNMF-Net shares an alternating architecture and benefits from both model-based methods and learning-based methods. On the one hand, SNMF-Net is of high physical interpretability as it is built by unrolling $L_{p}$ sparsity constrained nonnegative matrix factorization ($L_{p}$ -NMF) model belonging to LMM families. On the other hand, all the parameters and submodules of SNMF-Net can be seamlessly linked with the alternating optimization algorithm of $L_{p}$ -NMF and unmixing problem. This enables us to reasonably integrate the prior knowledge on unmixing, the optimization algorithm, and the sparse representation theory into the network for robust learning, so as to improve unmixing. Experimental results on the synthetic and real-world data show the advantages of the proposed SNMF-Net over many state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• Stage-wise optimization and momentum have been widely employed to accelerate SGD. • Negative momentum provides acceleration and stabilization on stochastic first-order methods. • Negative momentum extends Nesterovs momentum to the stage-wise optimization. • Gradient correction avoids the oscillations and make stochastic gradient more effective and tolerant. Multi-stage optimization which invokes a stochastic algorithm restarting with the returned solution of previous stage, has been widely employed in stochastic optimization. Momentum acceleration technique is famously known for building gradient-based algorithms with fast convergence in large-scale optimization. In order to take the advantage of this acceleration in multi-stage stochastic optimization, we develop a multi-stage stochastic gradient descent with momentum acceleration method, named MAGNET, for first-order stochastic convex optimization. The main ingredient is the employment of a negative momentum, which extends the Nesterov's momentum to the multi-stage optimization. It can be incorporated in a stochastic gradient-based algorithm in multi-stage mechanism and provide acceleration. The proposed algorithm obtains an accelerated rate of convergence, and is adaptive and free from hyper-parameter tuning. The experimental results demonstrate that our algorithm is competitive with some state-of-the-art methods for solving several typical optimization problems in machine learning. [ABSTRACT FROM AUTHOR]

In hyperspectral image (HSI) classification, a big challenge is the limited sample size with a relatively high feature dimension. Therefore, effective feature extraction of data is essential, which is desired to remove the redundancy as well as improve the discrimination. A huge number of methods have been proposed for HSI feature extraction. In recent years, deep learning-based feature extraction algorithms have shown their superiorities in various classification problems. Within them, deep PCA (DPCA) is a simple but efficient algorithm, which runs fast due to the absence of back-propagation. However, DPCA fails to provide satisfactory classification accuracies on HSI datasets. In this paper, we try to combine DPCA with residual-based multi-scale feature extraction and propose a residual deep PCA (RDPCA) feature extraction algorithm for HSI classification. It is a hierarchical approach consisting of multiple layers. Within each layer, PCA is utilized for layer-wise feature extraction, and the reconstruction residual is fed into the next layer. When the feature is passed deeper into the RDPCA network, finer details are mined. The layer-wise features are concatenated to form the final output feature. Furthermore, to enhance the ability of nonlinear feature extraction, we add activation functions between adjacent layers. Experimental results on real-world HSI datasets have shown the superiority of the proposed RDPCA over DPCA and PCA. [ABSTRACT FROM AUTHOR]

Attributing to material identification ability powered by a large number of spectral bands, hyperspectral videos (HSVs) have great potential for object tracking. Most hyperspectral trackers employ manually designed features rather than deeply learned features to describe objects due to limited available HSVs for training, leaving a huge gap to improve the tracking performance. In this paper, we propose an end-to-end deep ensemble network (SEE-Net) to address this challenge. Specifically, we first establish a spectral self-expressive model to learn the band correlation, indicating the importance of a single band in forming hyperspectral data. We parameterize the optimization of the model with a spectral self-expressive module to learn the nonlinear mapping from input hyperspectral frames to band importance. In this way, the prior knowledge of bands is transformed into a learnable network architecture, which has high computational efficiency and can fast adapt to the changes of target appearance because of no iterative optimization. The band importance is further exploited from two aspects. On the one hand, according to the band importance, each frame of HSVs is divided into several three-channel false-color images which are then used for deep feature extraction and location. On the other hand, based on the band importance, the importance of each false-color image is computed, which is then used to assemble the tracking results from individual false-color images. In this way, the unreliable tracking caused by false-color images of low importance can be suppressed to a large extent. Extensive experimental results show that SEE-Net performs favorably against the state-of-the-art approaches. The source code will be available at https://github.com/hscv/SEE-Net.

Traditional color images only depict color intensities in red, green and blue channels, often making object trackers fail in challenging scenarios, e.g., background clutter and rapid changes of target appearance. Alternatively, material information of targets contained in large amount of bands of hyperspectral images (HSI) is more robust to these difficult conditions. In this paper, we conduct a comprehensive study on how material information can be utilized to boost object tracking from three aspects: dataset, material feature representation and material based tracking. In terms of dataset, we construct a dataset of fully-annotated videos, which contain both hyperspectral and color sequences of the same scene. Material information is represented by spectral-spatial histogram of multidimensional gradients, which describes the 3D local spectral-spatial structure in an HSI, and fractional abundances of constituted material components which encode the underlying material distribution. These two types of features are embedded into correlation filters, yielding material based tracking. Experimental results on the collected dataset show the potentials and advantages of material based object tracking.