Search

In the existing unsupervised domain adaptation (UDA) methods for remote sensing images (RSIs) semantic segmentation, class symmetry is a widely followed ideal assumption, where the source and target RSIs have exactly the same class space. In practice, however, it is often very difficult to find a source RSI with exactly the same classes as the target RSI. More commonly, there are multiple source RSIs available. And there is always an intersection or inclusion relationship between the class spaces of each source–target pair, which can be referred to as class asymmetry. Nevertheless, the class asymmetry domain adaptation segmentation of RSIs with multiple sources has not yet been explored. To this end, a novel class asymmetry RSIs domain adaptation method is proposed for the first time in this article, which consists of four key components. First, a multibranch segmentation network is built to learn an expert for each source RSI. Second, a novel collaborative learning method with the cross-domain mixing strategy is proposed, to supplement the class information for each source while achieving the domain adaptation of each source–target pair. Third, a pseudolabel generation strategy is proposed to effectively combine the strengths of different experts, which can be flexibly applied to two cases where the source class union is equal to or includes the target class set. Fourth, a multiview-enhanced knowledge integration module is developed for high-level knowledge routing and transfer from multiple domains to target predictions. The experimental results of six different class settings on airborne and spaceborne RSIs show that the proposed method can effectively perform the multisource domain adaptation in the case of class asymmetry, and the obtained segmentation performance of target RSIs is significantly better than the existing relevant methods.

ABSTRACTRecently, deep-learning methods have significantly promoted the development of hyperspectral image classification. However, these methods are still limited by the lack of labelled samples. This study proposes a semi-supervised deep-learning method based on multi-view consistency to improve the classification accuracy of hyperspectral images using a few labelled samples. First, a classifier was built based on a residual network. Subsequently, an attention mechanism was introduced to enhance the classification performance of the residual network. Naturally, the class corresponding to different feature representations of the same sample should be consistent; this was used as an unsupervised constraint to assist model training. Therefore, the consistency of the model prediction of different views from the same unlabelled samples was calculated as the unsupervised loss. Finally, the sum of supervised and unsupervised losses was used as the final loss function to train the deep-learning model. The introduction of the unsupervised loss function enabled our model to fully use unlabelled samples to improve classification accuracy. Extensive classification experiments performed on two real hyperspectral image datasets demonstrated that the proposed method achieved higher classification accuracy than the existing state-of-the-art methods.

In unsupervised domain adaptation (UDA) of remote sensing images (RSIs), the huge interdomain discrepancies and intradomain variances lead to complicated class-level relations. Specifically, the instances of the same class differ greatly, while instances of different classes are similar, whether across different RSIs domains or within the same RSIs domain. However, existing methods cannot fully consider these problems, limiting the performance of UDA semantic segmentation of RSIs. To this end, this article proposes a novel cross-domain multiprototypes learning method, the core idea of which is to abstract the cross-domain and intradomain class-level relations into multiple prototypes. Specifically, the multiple prototypes belonging to different classes can detailedly describe complex interclass relations, and the multiple prototypes within the same class can better model rich intraclass relations. Furthermore, the source and target samples are jointly used for prototypes calculation, to fully fuse the feature information of different RSIs. In a nutshell, utilizing the samples from different RSIs domains to learn multiple prototypes for each class can achieve better domain alignment at the class level. In addition, considering that RSIs simultaneously contain large targets with wide coverage and important small targets, two masked consistency learning strategies are designed to better explore the contextual structure of target RSIs and improve the quality of pseudo labels for prototype updating. The global consistency strategy can strengthen the utilization of global context relations, while the local consistency strategy can further improve the learning of local context details. Therefore, the proposed method is actually a prototype and context-enhanced learning method for UDA semantic segmentation of RSIs. Extensive experiments demonstrate that the proposed method can achieve better performance than existing state-of-the-art UDA methods.

ABSTRACTRecent research on hyperspectral image (HSI) classification has primarily focused on deep learning methods. Although these methods can automatically mine HSI classification features, they typically require many labelled samples to ensure sufficient classification performance. When there are fewer labelled training samples, the manual feature extraction design rules are critical for HSI classification. Considering the small sample problem, a slow spatial–spectral feature extraction method is proposed for HSI classification in this study. The proposed method can achieve high classification accuracy by using slow spatial–spectral features. The extracted slow feature dimension is much lower than that of the original spectral feature. Thus, the feature dimension for the HSI classification task is reduced, improving the classification efficiency. The experimental results of two real HSI datasets demonstrate that the proposed method can significantly reduce the feature dimensions and improve HSI classification accuracy.

Image matching plays an important role in various computer vision tasks, such as image retrieval and loop closure detection in Simultaneous Localization and Mapping. The authors propose a discriminative patch‐based image matching method that converts the problem of whole image matching to that of local patch matching. To construct the patch representation, the Patch‐Level Vision Similarity Compare Network (PL‐VSCN) is proposed to produce the patch feature. In the image matching process, local patches that potentially contain objects within images are initially detected, and the discriminative feature of each patch is extracted based on the pre‐trained PL‐VSCN. Then, the similarities between the patch pairs are calculated to construct the similarity matrix, and the corresponding patch pairs are detected based on the mutual matching mechanism on the similarity matrix. Experimental results indicate that the proposed PL‐VSCN can generate the discriminative patch feature, which can accurately match the patch pairs with the corresponding content and distinguish those with non‐corresponding content. In addition, the comparison experiments demonstrate that the proposed image matching method outperforms existing approaches on most datasets and effectively completes the image matching task.

ABSTRACTHyperspectral image (HSI) classification has been a vibrant area of research in recent years. In this letter, a salient feature extraction method is designed to improve the HSI classification performance. First, three consecutive bands of HSI are decomposed into several elements in order to better extract salient feature. Second, two measures of contrast that rate the uniqueness and the spatial distribution of these elements are computed. Thirdly, two measures of contrast are combined to form the saliency value of an element. The final saliency of a pixel is formed by a weighted linear combination of its surrounding elements saliency. Finally, three band sliding window is used over the spectral dimension of HSI and salient features extracted from each three adjacent bands are stacked to form the final features. The important characteristic of the proposed approach is that both local spatial information and global spatial information are taken into account in the process of feature extraction. Experimental results on two real HSI datasets demonstrate that the designed feature extraction method could improve the classification performance.

ABSTRACTDeep neural networks have recently been successfully explored to extract deep features for hyperspectral image classification. Recurrent neural networks (RNNs) are an important branch of the deep learning family, which are widely used for sequence analysis. Indeed, RNNs have been used to model the dependencies between the different spectral bands of hyperspectral image, inspired by the observation that hyperspectral pixels can be considered as spectral sequences. A disadvantage of such methods is that they don’t consider the effect of neighborhood pixels on the final class label. In this letter, a RNN model is proposed for the spectral-spatial classification of hyperspectral image. Specifically, the hyperspectral image cube surrounding a central pixel is considered as a hyperspectral pixels sequence, and a RNN is used to model the dependencies between the different neighborhood pixels. The proposed RNN is conducted on two widely used hyperspectral image datasets. The experimental results demonstrate that the proposed approach provides a better performance than that of conventional methods.

ABSTRACTConvolutional neural network (CNN) for hyperspectral image classification can provide excellent performance when the number of labeled samples for training is sufficiently large. Unfortunately, a small number of labeled samples are available for training in hyperspectral images. In this letter, a novel semi-supervised convolutional neural network is proposed for the classification of hyperspectral image. The proposed network can automatically learn features from complex hyperspectral image data structures. Furthermore, skip connection parameters are added between the encoder layer and decoder layer in order to make the network suitable for semi-supervised learning. Semi-supervised method is adopted to solve the problem of limited labeled samples. Finally, the network is trained to simultaneously minimize the sum of supervised and unsupervised cost functions. The proposed network is conducted on a widely used hyperspectral image data. The experimental results demonstrate that the proposed approach provides competitive results to state-of-the-art methods.