Search

The hyperspectral feature extraction technique is one of the most popular topics in the remote sensing community. However, most hyperspectral feature extraction methods are based on region-based local information descriptors while neglecting the correlation and dependencies of different homogeneous regions. To alleviate this issue, this paper proposes a multi-view structural feature extraction method to furnish a complete characterization for spectral–spatial structures of different objects, which mainly is made up of the following key steps. First, the spectral number of the original image is reduced with the minimum noise fraction (MNF) method, and a relative total variation is exploited to extract the local structural feature from the dimension reduced data. Then, with the help of a superpixel segmentation technique, the nonlocal structural features from intra-view and inter-view are constructed by considering the intra- and inter-similarities of superpixels. Finally, the local and nonlocal structural features are merged together to form the final image features for classification. Experiments on several real hyperspectral datasets indicate that the proposed method outperforms other state-of-the-art classification methods in terms of visual performance and objective results, especially when the number of training set is limited. [ABSTRACT FROM AUTHOR]

Remote sensing image super-resolution (SR) aims to boost the image resolution while recovering rich high-frequency details. Currently, most of the SR methods are based on an assumption that the degradation kernel is a specific downsampler. However, the degradation kernel is unknown and sophisticated for real remote sensing scenes, leading to a severe performance drop. To alleviate this problem, we propose a multilayer degradation representation-guided blind SR method for remote sensing images, which mainly consists of three key steps. First, an unsupervised representation learning is exploited to learn the degradation representation from low-resolution images. Then, a degradation-guided deep residual module is designed to model high-order features across different scales from the original images. Finally, a multilayer degradation-aware feature fusion mechanism is proposed to restore the finer details. Experiments on synthetic and real datasets demonstrate that the proposed method can achieve promising performance with respect to other state-of-the-art SR approaches. [ABSTRACT FROM AUTHOR]

Infrared and visible image fusion is an effective image processing technique to obtain more comprehensive information, which can help people better understand various scenarios. In this paper, a novel infrared and visible image fusion method is proposed which fully considers the attributes of objects in source images. Benefiting from the attribute and the edge-preserving filters, the prominent objects in the infrared source image are effectively extracted. Then, the weight-based Laplacian pyramid fusion strategy is adopted to get more natural fusion results. The experimental results on the public image fusion datasets and a new infrared–visible video fusion dataset show that the proposed method achieves state-of-the-art fusion performance in terms of both visual and objective evaluations. The proposed algorithm is also implemented in an infrared–visible dual sensor system, which demonstrated the practicability of our fusion method. • A novel attribute guided image fusion method is proposed. • A dual infrared and visible imaging system is constructed. • A high-quality infrared–color visible image and video dataset is established. [ABSTRACT FROM AUTHOR]

Most deep metric learning-based image characterization methods exploit supervised information to model the semantic relations among the remote sensing (RS) scenes. Nonetheless, the unprecedented availability of large-scale RS data makes the annotation of such images very challenging, requiring automated supportive processes. Whether the annotation is assisted by aggregation or crowd-sourcing, the RS large-variance problem, together with other important factors [e.g., geo-location/registration errors, land-cover changes, even low-quality Volunteered Geographic Information (VGI), etc.] often introduce the so-called label noise, i.e., semantic annotation errors. In this article, we first investigate the deep metric learning-based characterization of RS images with label noise and propose a novel loss formulation, named robust normalized softmax loss (RNSL), for robustly learning the metrics among RS scenes. Specifically, our RNSL improves the robustness of the normalized softmax loss (NSL), commonly utilized for deep metric learning, by replacing its logarithmic function with the negative Box–Cox transformation in order to down-weight the contributions from noisy images on the learning of the corresponding class prototypes. Moreover, by truncating the loss with a certain threshold, we also propose a truncated robust normalized softmax loss (t-RNSL) which can further enforce the learning of class prototypes based on the image features with high similarities between them, so that the intraclass features can be well grouped and interclass features can be well separated. Our experiments, conducted on two benchmark RS data sets, validate the effectiveness of the proposed approach with respect to different state-of-the-art methods in three different downstream applications (classification, clustering, and retrieval). The codes of this article will be publicly available from https://github.com/jiankang1991. [ABSTRACT FROM AUTHOR]

Multi-modal image fusion is widely used in various fields, especially in military, medical, industrial detection and other fields. Image fusion can integrate redundant and complementary information of two or more multi-modal images into one image, so that the fused image contains more useful information. In this paper, we construct a novel dataset for Multi-modal Image Fusion Applications (MOFA), including four modals: visible, near-infrared (NIR), long-wavelength infrared (LWIR) and polarization. The MOFA dataset contains 1062 images of 118 groups, in which 450 are indoor and 612 are outdoor. The dataset is applied to different image fusion applications, including general multi-modal image fusion, fusion based image super-resolution and image restoration. Multiple image fusion methods are compared and analyzed on this dataset with a qualitative assessment of subjective and objective metrics. Based on the experiments, the advantages and disadvantages of different methods are discussed. Moreover, the challenging problems of image fusion are concluded. • Build a multi-modal imaging device. • Construct a multi-modal dataset, including visible, NIR, LWIR and polarization. • Explore the effectiveness of the dataset for different image processing tasks. • Analyze the advantages and disadvantages of different fusion methods. [ABSTRACT FROM AUTHOR]

Hyperspectral image classification has received a lot of attention in the remote sensing field. However, most classification methods require a large number of training samples to obtain satisfactory performance. In real applications, it is difficult for users to label sufficient samples. To overcome this problem, in this work, a novel multi-scale superpixel-guided structural profile method is proposed for the classification of hyperspectral images. First, the spectral number (of the original image) is reduced with an averaging fusion method. Then, multi-scale structural profiles are extracted with the help of the superpixel segmentation method. Finally, the extracted multi-scale structural profiles are fused with an unsupervised feature selection method followed by a spectral classifier to obtain classification results. Experiments on several hyperspectral datasets verify that the proposed method can produce outstanding classification effects in the case of limited samples compared to other advanced classification methods. The classification accuracies obtained by the proposed method on the Salinas dataset are increased by 43.25%, 31.34%, and 46.82% in terms of overall accuracy (OA), average accuracy (AA), and Kappa coefficient compared to recently proposed deep learning methods. [ABSTRACT FROM AUTHOR]

Coastal wetland monitoring plays an important role in the protection and restoration of ecosystems in this world. UAV-hyperspectral imaging, as an emerging technique for Earth observation and space exploration, provides the huge potential ability to identify different wetland species. In this work, a multilayer global spectral–spatial attention network (MGSSAN) is proposed for mapping coastal wetlands, which mainly consists of two major steps. First, a two-branch convolutional neural network (CNN) framework with residual connection is developed to obtain an initial classification probability map, in which one branch is used to capture the spectral information, the other branch is used to extract spatial information, and a global spectral–spatial attention module is designed to guide networks focusing on those features that are more discriminative. Second, an extended random walker method is utilized to optimize the initial classification probabilities, so as to yield the final map. Experiments performed on three wetland HSI datasets created by ourselves verify that the proposed method can obtain superior performance with respect to several state-of-the-art hyperspectral image classification methods. [ABSTRACT FROM AUTHOR]