Search

The hyperspectral feature extraction technique is one of the most popular topics in the remote sensing community. However, most hyperspectral feature extraction methods are based on region-based local information descriptors while neglecting the correlation and dependencies of different homogeneous regions. To alleviate this issue, this paper proposes a multi-view structural feature extraction method to furnish a complete characterization for spectral–spatial structures of different objects, which mainly is made up of the following key steps. First, the spectral number of the original image is reduced with the minimum noise fraction (MNF) method, and a relative total variation is exploited to extract the local structural feature from the dimension reduced data. Then, with the help of a superpixel segmentation technique, the nonlocal structural features from intra-view and inter-view are constructed by considering the intra- and inter-similarities of superpixels. Finally, the local and nonlocal structural features are merged together to form the final image features for classification. Experiments on several real hyperspectral datasets indicate that the proposed method outperforms other state-of-the-art classification methods in terms of visual performance and objective results, especially when the number of training set is limited. [ABSTRACT FROM AUTHOR]

Sun glint in hyperspectral images (HSIs) leads to undesirable spectral interference, which severely affects subsequent image interpretation, such as environmental monitoring of coastal areas. Sun glint removal methods aim to recover a high quality image without sun glint from the original image. Most methods depend on an assumption that the near infrared band is strongly absorbed by water. However, this assumption is not always reliable because the infrared radiation in shallow or turbid water can be reflected back by the seabed or sediment, rather than being fully absorbed. Therefore, the reflected infrared radiation still contains sun glint and these methods cannot sufficiently remove sun glint from HSIs. To address this problem, a texture-aware total variation (TATV)-based method is proposed to remove sun glint from HSIs. The original HSI first is formulated as a desired clean image and a sun glint image. Then, in order to remove the sun glint, we propose a variational model where the different spectral characteristics of sun glint and other surrounding materials are considered. Specifically, we propose a texture-aware total variation regularized method to heavily penalize the variation of the sun glint areas. Experiments performed on simulated and real data sets demonstrate that our method can greatly outperform other state-of-the-art methods in removing sun glint. [ABSTRACT FROM AUTHOR]

• A global and local information based visualization framework is proposed. • Edge-preserving filtering is first adopted to visualize hyperspectral image. • Experimental results show outstanding performance compared with other methods. In this paper, an edge-preserving filtering and principal component analysis (PCA)-based visualization method is proposed for hyperspectral images, in which both global and local image information of hyperspectral images (HSIs) are taken into account in the proposed visualization framework that consists of the following major steps. First, the band number of the original image is reduced with averaging-based image fusion (AIF). Then, the edge-preserving filtering is performed on the dimension reduced image so as to decompose it into two components, i.e., the base layers which contain the large-scale boundary information, and the detail layers which contain the mid- and small-scale edges and textures. Next, the base layers are fused with the principal component analysis method, and the detail layers are fused with the weighted sum method, in which the fusion weights are determined by the transform matrix of the base layers. Finally, the fused detail layers are enhanced with histogram equalization and combined with the fused base layers to visualize the hyperspectral image. Experimental results on four real hyperspectral data sets demonstrate that the proposed approach performs the best in improving image contrast and preserving the details. In addition, compared with seven state-of-the-art visualization methods, the quantitative metrics obtained by the proposed method on four data sets have been increased by 80.45%, 69.29%, 138.61%, and 23.21% on average in terms of separability of features (SF), standard deviation (SD), average gradient (AG), and entropy (EN). [ABSTRACT FROM AUTHOR]

Hyperspectral Image (HSI) visualization, which aims at displaying as much material information of original images as possible on a trichromatic monitor with natural color, plays an important role in image interpretation and analysis. However, most of the HSI visualization methods only focus on presenting the detail information of a scene without providing natural colors and distinguishing land covers with similar colors. In order to address this problem, this article proposes a multichannel pulse-coupled neural network (MPCNN)-based HSI visualization method, which consists of the following steps. First, the MPCNN is proposed and explored to fuse the original HSI so as to obtain a fused band with rich spatial details. Then, a color mapping scheme is proposed to determine the weights of red, green, and blue (RGB) channels. Finally, the weighted RGB channels are stacked together for visualization. Experiments performed on four hyperspectral data sets demonstrate that the proposed method not only displays the HSI with nature colors but also improves the details in the image. The effectiveness of the proposed method is demonstrated in terms of both visual effect and objective indexes. [ABSTRACT FROM AUTHOR]

In this article, a novel hyperspectral image (HSI) classification method based on fusing multiple edge-preserving operations (EPOs) is proposed, which consists of the following steps. First, the edge-preserving features are obtained by performing different types of EPOs, i.e., local edge-preserving filtering and global edge-preserving smoothing on the dimension-reduced HSI. Then, with the assistance of a superpixel segmentation method, the edge-preserving features are further improved by considering the inter and intra spectral properties of superpixels. Finally, the spectral and edge-preserving features are fused to form one composite kernel, which is fed into the support vector machine (SVM) followed by a majority voting fusion scheme. Experimental results on three data sets demonstrate the superiority of the proposed method over several state-of-the-art classification approaches, especially when the training sample size is limited. Furthermore, 21 well-known methods, including mathematical morphology-based approaches, sparse representation models, and deep learning-based classifiers, are adopted to be compared with the proposed method on Houston data set with standard sets of training and test samples released during 2013 Data Fusion Contest, which also shows the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, a novel method is introduced to detect and correct mislabeled training samples for hyperspectral image classification. First, domain transform recursive filtering-based feature extraction is used to improve the separability of the training samples. Then, constrained energy minimization-based object detection is performed on the training set with each training sample serving as the object spectrum. Finally, the label of each training sample is verified or corrected based on the averaged detection probabilities of different classes. Experiments performed on real hyperspectral data sets demonstrate the effectiveness of the proposed method in improving classification performance with respect to the classifier trained with the original training set that contains a number of mislabeled samples. [ABSTRACT FROM AUTHOR]

Image decolorization is known to be an effective way in transferring a color image into a gray one while well preserving the major information of all three bands. In this paper, a simple yet effective hyperspectral image visualization framework based on decolorization, named decolorization based hyperspectral visualization, is proposed, which enables us to fully exploit the benefits of decolorization technique. The proposed framework consists of the following two main steps. First, the hyperspectral image is partitioned into nine subsets of adjacent hyperspectral bands and the averaged band of each subset is calculated. Then, the dimension reduced image is further divided into three groups of adjacent bands, and the bands in each group are fused by using an image decolorization method. The main contribution of this paper is that the strong correlations in two different fields, i.e., image decolorization and hyperspectral image visualization, are first built. Experiments performed on several real hyperspectral data sets demonstrate that the proposed framework can obtain outstanding visualization performance in terms of both subjective and objective evaluations. [ABSTRACT FROM AUTHOR]

In this paper, a novel shift-invariant dual-tree complex shearlet transform (SIDCST) is constructed and applied to infrared and visible image fusion. Firstly, the mathematical morphology is used for the source images. Then, the images are decomposed by SIDCST to obtain the low frequency sub-band coefficients and high frequency sub-band coefficients. For the low frequency sub-band coefficients, a novel sparse representation (SR)-based fusion rule is presented. For the high frequency sub-band coefficients, a scheme based on the theory of adaptive dual-channel pulse coupled neural network (2APCNN) is presented, and the energy of edge is used for the external input of 2APCNN. Finally, the fused image is obtained by performing the inverse SIDCST. The experimental results show that the proposed approach can obtain state-of-the-art performance compared with conventional image fusion methods in terms of both objective evaluation criteria and visual quality. [ABSTRACT FROM AUTHOR]

Remote sensing image super-resolution (SR) aims to boost the image resolution while recovering rich high-frequency details. Currently, most of the SR methods are based on an assumption that the degradation kernel is a specific downsampler. However, the degradation kernel is unknown and sophisticated for real remote sensing scenes, leading to a severe performance drop. To alleviate this problem, we propose a multilayer degradation representation-guided blind SR method for remote sensing images, which mainly consists of three key steps. First, an unsupervised representation learning is exploited to learn the degradation representation from low-resolution images. Then, a degradation-guided deep residual module is designed to model high-order features across different scales from the original images. Finally, a multilayer degradation-aware feature fusion mechanism is proposed to restore the finer details. Experiments on synthetic and real datasets demonstrate that the proposed method can achieve promising performance with respect to other state-of-the-art SR approaches. [ABSTRACT FROM AUTHOR]

Multi-sensor data on the same area provide complementary information, which is helpful for improving the discrimination capability of classifiers. In this work, a novel multilevel structure extraction method is proposed to fuse multi-sensor data. This method is comprised of three steps: First, multilevel structure extraction is constructed by cascading morphological profiles and structure features, and is utilized to extract spatial information from multiple original images. Then, a low-rank model is adopted to integrate the extracted spatial information. Finally, a spectral classifier is employed to calculate class probabilities, and a maximum posteriori estimation model is used to decide the final labels. Experiments tested on three datasets including rural and urban scenes validate that the proposed approach can produce promising performance with regard to both subjective and objective qualities. [ABSTRACT FROM AUTHOR]

Combining both spectral and spatial information with enhanced resolution provides not only elaborated qualitative information on surfacing mineralogy but also mineral interactions of abundance, mixture, and structure. This enhancement in the resolutions helps geomineralogic features such as small intrusions and mineralization become detectable. In this paper, we investigate the potential of the resolution enhancement of hyperspectral images (HSIs) with the guidance of RGB images for mineral mapping. In more detail, a novel resolution enhancement method is proposed based on component decomposition. Inspired by the principle of the intrinsic image decomposition (IID) model, the HSI is viewed as the combination of a reflectance component and an illumination component. Based on this idea, the proposed method is comprised of several steps. First, the RGB image is transformed into the luminance component, blue-difference and red-difference chroma components (YCbCr), and the luminance channel is considered as the illumination component of the HSI with an ideal high spatial resolution. Then, the reflectance component of the ideal HSI is estimated with the downsampled HSI image and the downsampled luminance channel. Finally, the HSI with high resolution can be reconstructed by utilizing the obtained illumination and the reflectance components. Experimental results verify that the fused results can successfully achieve mineral mapping, producing better results qualitatively and quantitatively over single sensor data. [ABSTRACT FROM AUTHOR]

Semantic segmentation of remote sensing images aims to assign a specific label or class to each pixel in an image, which plays an extremely important role in scene understanding. Currently, many advanced deep learning-based semantic segmentation methods have been developed. However, these methods are always based on disjoint labels to identify ground objects while ignoring the correlation (e.g., semantic, shapes, materials, etc.) among different ground objects, which limits the segmentation performance of remote sensing images. To solve this issue, we propose a hierarchical class graph for semantic segmentation of high resolution remote sensing images, which can learn structured relation among different ground objects. Specifically, first, we construct hierarchical class graphs based on different attributes and layers. Then, a three-layer hierarchical segmentation framework is developed to learn the correlation among different ground objects. Finally, a decision fusion method is designed to fuse the benefits of different hierarchical attributes and layers. More importantly, the influence of different hierarchical class graphs on the segmentation performance is detailedly analyzed. Extensive experiments on the iSAID and Vaihingen datasets reveal that all studied segmentation methods with hierarchical class graph can obtain better segmentation performance compared to ones without hierarchical class graph. The limitation of the proposed method is that the training time of the segmentation model tends to increase a bit because of considering the correlation among different ground objects. • Construct a multi-layer hierarchical class graph for semantic segmentation. • Propose a three-layer hierarchical graph framework for semantic segmentation. • Explore the effectiveness of different attributes on semantic segmentation. • A fuzzy set-based decision fusion method is proposed. [ABSTRACT FROM AUTHOR]

Infrared and visible image fusion is an effective image processing technique to obtain more comprehensive information, which can help people better understand various scenarios. In this paper, a novel infrared and visible image fusion method is proposed which fully considers the attributes of objects in source images. Benefiting from the attribute and the edge-preserving filters, the prominent objects in the infrared source image are effectively extracted. Then, the weight-based Laplacian pyramid fusion strategy is adopted to get more natural fusion results. The experimental results on the public image fusion datasets and a new infrared–visible video fusion dataset show that the proposed method achieves state-of-the-art fusion performance in terms of both visual and objective evaluations. The proposed algorithm is also implemented in an infrared–visible dual sensor system, which demonstrated the practicability of our fusion method. • A novel attribute guided image fusion method is proposed. • A dual infrared and visible imaging system is constructed. • A high-quality infrared–color visible image and video dataset is established. [ABSTRACT FROM AUTHOR]

Most deep metric learning-based image characterization methods exploit supervised information to model the semantic relations among the remote sensing (RS) scenes. Nonetheless, the unprecedented availability of large-scale RS data makes the annotation of such images very challenging, requiring automated supportive processes. Whether the annotation is assisted by aggregation or crowd-sourcing, the RS large-variance problem, together with other important factors [e.g., geo-location/registration errors, land-cover changes, even low-quality Volunteered Geographic Information (VGI), etc.] often introduce the so-called label noise, i.e., semantic annotation errors. In this article, we first investigate the deep metric learning-based characterization of RS images with label noise and propose a novel loss formulation, named robust normalized softmax loss (RNSL), for robustly learning the metrics among RS scenes. Specifically, our RNSL improves the robustness of the normalized softmax loss (NSL), commonly utilized for deep metric learning, by replacing its logarithmic function with the negative Box–Cox transformation in order to down-weight the contributions from noisy images on the learning of the corresponding class prototypes. Moreover, by truncating the loss with a certain threshold, we also propose a truncated robust normalized softmax loss (t-RNSL) which can further enforce the learning of class prototypes based on the image features with high similarities between them, so that the intraclass features can be well grouped and interclass features can be well separated. Our experiments, conducted on two benchmark RS data sets, validate the effectiveness of the proposed approach with respect to different state-of-the-art methods in three different downstream applications (classification, clustering, and retrieval). The codes of this article will be publicly available from https://github.com/jiankang1991. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have achieved great success when characterizing remote sensing (RS) images. However, the lack of sufficient annotated data (together with the high complexity of the RS image domain) often makes supervised and transfer learning schemes limited from an operational perspective. Despite the fact that unsupervised methods can potentially relieve these limitations, they are frequently unable to effectively exploit relevant prior knowledge about the RS domain, which may eventually constrain their final performance. In order to address these challenges, this article presents a new unsupervised deep metric learning model, called spatially augmented momentum contrast (SauMoCo), which has been specially designed to characterize unlabeled RS scenes. Based on the first law of geography, the proposed approach defines spatial augmentation criteria to uncover semantic relationships among land cover tiles. Then, a queue of deep embeddings is constructed to enhance the semantic variety of RS tiles within the considered contrastive learning process, where an auxiliary CNN model serves as an updating mechanism. Our experimental comparison, including different state-of-the-art techniques and benchmark RS image archives, reveals that the proposed approach obtains remarkable performance gains when characterizing unlabeled scenes since it is able to substantially enhance the discrimination ability among complex land cover categories. The source codes of this article will be made available to the RS community for reproducible research. [ABSTRACT FROM AUTHOR]

In this article, a novel hyperspectral anomaly detection method with kernel Isolation Forest (iForest) is proposed. The method is based on an assumption that anomalies rather than background can be more susceptible to isolation in the kernel space. Based on this idea, the proposed method detects anomalies as follows. First, the hyperspectral data are mapped into the kernel space, and the first $K$ principal components are used. Then, the isolation samples in the image are detected with the iForest constructed using randomly selected samples in the principal components. Finally, the initial anomaly detection map is iteratively refined with locally constructed iForest in connected regions with large areas. Experimental results on several real hyperspectral data sets demonstrate that the proposed method outperforms other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Multi-modal image fusion is widely used in various fields, especially in military, medical, industrial detection and other fields. Image fusion can integrate redundant and complementary information of two or more multi-modal images into one image, so that the fused image contains more useful information. In this paper, we construct a novel dataset for Multi-modal Image Fusion Applications (MOFA), including four modals: visible, near-infrared (NIR), long-wavelength infrared (LWIR) and polarization. The MOFA dataset contains 1062 images of 118 groups, in which 450 are indoor and 612 are outdoor. The dataset is applied to different image fusion applications, including general multi-modal image fusion, fusion based image super-resolution and image restoration. Multiple image fusion methods are compared and analyzed on this dataset with a qualitative assessment of subjective and objective metrics. Based on the experiments, the advantages and disadvantages of different methods are discussed. Moreover, the challenging problems of image fusion are concluded. • Build a multi-modal imaging device. • Construct a multi-modal dataset, including visible, NIR, LWIR and polarization. • Explore the effectiveness of the dataset for different image processing tasks. • Analyze the advantages and disadvantages of different fusion methods. [ABSTRACT FROM AUTHOR]

Recently, a series of deep learning methods based on the convolutional neural networks (CNNs) have been introduced for classification of hyperspectral images (HSIs). However, in order to obtain the optimal parameters, a large number of training samples are required in the CNNs to avoid the overfitting problem. In this paper, a novel method is proposed to extend the training set for deep learning based hyperspectral image classification. First, given a small-sample-size training set, the principal component analysis based edge-preserving features (PCA-EPFs) and extended morphological attribute profiles (EMAPs) are used for HSI classification so as to generate classification probability maps. Second, a large number of pseudo training samples are obtained by the designed decision function which depends on the classification probabilities. Finally, a deep feature fusion network (DFFN) is applied to classify HSI with the training set consists of the original small-sample-size training set and the added pseudo training samples. Experiments performed on several hyperspectral data sets demonstrate the state-of-the-art performance of the proposed method in terms of classification accuracies. [ABSTRACT FROM AUTHOR]

Hyperspectral image classification has received a lot of attention in the remote sensing field. However, most classification methods require a large number of training samples to obtain satisfactory performance. In real applications, it is difficult for users to label sufficient samples. To overcome this problem, in this work, a novel multi-scale superpixel-guided structural profile method is proposed for the classification of hyperspectral images. First, the spectral number (of the original image) is reduced with an averaging fusion method. Then, multi-scale structural profiles are extracted with the help of the superpixel segmentation method. Finally, the extracted multi-scale structural profiles are fused with an unsupervised feature selection method followed by a spectral classifier to obtain classification results. Experiments on several hyperspectral datasets verify that the proposed method can produce outstanding classification effects in the case of limited samples compared to other advanced classification methods. The classification accuracies obtained by the proposed method on the Salinas dataset are increased by 43.25%, 31.34%, and 46.82% in terms of overall accuracy (OA), average accuracy (AA), and Kappa coefficient compared to recently proposed deep learning methods. [ABSTRACT FROM AUTHOR]

Coastal wetland monitoring plays an important role in the protection and restoration of ecosystems in this world. UAV-hyperspectral imaging, as an emerging technique for Earth observation and space exploration, provides the huge potential ability to identify different wetland species. In this work, a multilayer global spectral–spatial attention network (MGSSAN) is proposed for mapping coastal wetlands, which mainly consists of two major steps. First, a two-branch convolutional neural network (CNN) framework with residual connection is developed to obtain an initial classification probability map, in which one branch is used to capture the spectral information, the other branch is used to extract spatial information, and a global spectral–spatial attention module is designed to guide networks focusing on those features that are more discriminative. Second, an extended random walker method is utilized to optimize the initial classification probabilities, so as to yield the final map. Experiments performed on three wetland HSI datasets created by ourselves verify that the proposed method can obtain superior performance with respect to several state-of-the-art hyperspectral image classification methods. [ABSTRACT FROM AUTHOR]

This paper provides a comprehensive state-of-the-art investigation of the recent advances in data science in emerging economic applications. The analysis is performed on the novel data science methods in four individual classes of deep learning models, hybrid deep learning models, hybrid machine learning, and ensemble models. Application domains include a broad and diverse range of economics research from the stock market, marketing, and e-commerce to corporate banking and cryptocurrency. Prisma method, a systematic literature review methodology, is used to ensure the quality of the survey. The findings reveal that the trends follow the advancement of hybrid models, which outperform other learning algorithms. It is further expected that the trends will converge toward the evolution of sophisticated hybrid deep learning models. [ABSTRACT FROM AUTHOR]

In this letter, a novel edge-preserving filtering based approach is proposed for feature extraction of hyperspectral images, which consists of the following steps. First, the dimension of the hyperspectral image is reduced with an averaging based method. Then, the resulting features are obtained by performing edge-preserving filtering on the dimension reduced image, in which a learned edge detection map serves as one of the major cues in the filtering process. The advantage of the proposed method is that it makes full use of the learned edge information in the feature extraction process, and thus, able to improve the performance with respect to other traditional feature extraction methods. Experiments on two real hyperspectral data sets demonstrate the outstanding performance of the proposed method especially when the number of training samples is limited. [ABSTRACT FROM AUTHOR]

The popularity of deep reinforcement learning (DRL) applications in economics has increased exponentially. DRL, through a wide range of capabilities from reinforcement learning (RL) to deep learning (DL), offers vast opportunities for handling sophisticated dynamic economics systems. DRL is characterized by scalability with the potential to be applied to high-dimensional problems in conjunction with noisy and nonlinear patterns of economic data. In this paper, we initially consider a brief review of DL, RL, and deep RL methods in diverse applications in economics, providing an in-depth insight into the state-of-the-art. Furthermore, the architecture of DRL applied to economic applications is investigated in order to highlight the complexity, robustness, accuracy, performance, computational tasks, risk constraints, and profitability. The survey results indicate that DRL can provide better performance and higher efficiency as compared to the traditional algorithms while facing real economic problems in the presence of risk parameters and the ever-increasing uncertainties. [ABSTRACT FROM AUTHOR]