Search

Labeled hyperspectral image (HSI) information is commonly difficult to acquire, so the lack of valid labeled data becomes a major puzzle for HSI classification. Semi-supervised methods can efficiently exploit unlabeled and labeled data for classification, which is highly valuable. Graph-based semi-supervised methods only focus on HSI local or global data and cannot fully utilize spatial–spectral information; this significantly limits the performance of classification models. To solve this problem, we propose an adaptive global–local feature fusion (AGLFF) method. First, the global high-order and local graphs are adaptively fused, and their weight parameters are automatically learned in an adaptive manner to extract the consistency features. The class probability structure is then used to express the relationship between the fused feature and the categories and to calculate their corresponding pseudo-labels. Finally, the fused features are imported into the broad learning system as weights, and the broad expansion of the fused features is performed with the weighted broad network to calculate the model output weights. Experimental results from three datasets demonstrate that AGLFF outperforms other methods.

Multimodal remote sensing data classification can enhance a model’s ability to distinguish land features through multimodal data fusion. In this context, how to help models understand the relationship between multimodal data and target tasks has become the focus of researchers. Inspired by the human feedback learning mechanism, causal reasoning mechanism, and knowledge induction mechanism, this paper integrates causal learning, reinforcement learning, and meta learning into a unified remote sensing data classification framework and proposes causal meta-reinforcement learning (CMRL). First, based on the feedback learning mechanism, we overcame the limitations of traditional implicit optimization of fusion features and customized a reinforcement learning environment for multimodal remote sensing data classification tasks. Through feedback interactive learning between agents and the environment, we helped the agents understand the complex relationships between multimodal data and labels, thereby achieving full mining of multimodal complementary information.Second, based on the causal inference mechanism, we designed causal distribution prediction actions, classification rewards, and causal intervention rewards, capturing pure causal factors in multimodal data and preventing false statistical associations between non-causal factors and class labels. Finally, based on the knowledge induction mechanism, we designed a bi-layer optimization mechanism based on meta-learning. By constructing a meta training task and meta validation task simulation model in the generalization scenario of unseen data, we helped the model induce cross-task shared knowledge, thereby improving its generalization ability for unseen multimodal data. The experimental results on multiple sets of multimodal datasets showed that the proposed method achieved state-of-the-art performance in multimodal remote sensing data classification tasks.

Hyperspectral image (HSI) classification has attracted much attention in the field of remote sensing. However, the lack of sufficient labeled training samples is a huge challenge for HSI classification. To face this challenge, we propose a semisupervised HSI classification method based on graph convolutional broad network (GCBN). First, to avoid the underfitting problem caused by the insufficient linear sparse feature representation ability of broad learning system (BLS), graph convolution operation is applied to extract nonlinear and discriminative spectral-spatial features from the original HSI to replace the linear mapping features in the traditional BLS. Second, to solve the problem of insufficient model classification ability caused by limited labeled samples, the combinatorial average method (CAM) is proposed to use valuable paired samples to generate sample expansion set for GCBN model training. Third, BLS is used to perform broad expansion on spectral-spatial features extracted by GCN and extended by CAM, which further enhances the feature representation ability. Finally, the output weights can be easily calculated by the ridge regression theory. Experimental results on three real HSI datasets demonstrate the effectiveness of our proposed GCBN.

With the development of science and technology, hyperspectral image (HSI) classification has been studied in depth by researchers as one of the important means of human cognition in living environments and the exploration of surface information. Nevertheless, the shortage of labeled samples is a major difficulty in HSI classification. To address this issue, we propose a novel HSI classification method called class-weighted domain adaptation network (CWDAN). First, the convolutional domain adaption network (ConDAN) is designed to align the marginal distributions and second-order statistics, respectively, of both domains via multi-kernel maximum mean discrepancy (MK-MMD) and CORAL loss. Then, the class-weighted MMD (CWMMD) is defined to simultaneously consider the conditional distribution discrepancy and changes of class prior distributions, and the CWMMD-based domain adaptation term is incorporated into the classical broad learning system (BLS) to construct the weighted conditional broad network (WCBN). The WCBN is applied to reduce the conditional distribution discrepancy and class weight bias across domains, while performing breadth expansion on domain-invariant features to further enhance representation ability. In comparison with several existing mainstream methods, CWDAN has excellent classification performance on eight real HSI data pairs when only using labeled source domain samples.

Hyperspectral images (HSI) are widely applied in numerous fields for their rich spatial and spectral information. However, in these applications, we always face the situation that the available labeled samples are limited or absent. Therefore, we propose an HSI classification method based on domain adaptation broad learning (DABL). First, according to the importance of the marginal and conditional distributions, the maximum mean discrepancy is used in mapped features to adapt these distributions between source and target domains. Meanwhile the manifold regularization is added to maintain the manifold structure of the input HSI data. Second, to further reduce the distribution difference and maintain manifold structure, the domain adaptation and manifold regularization are added to the output layer of DABL. Finally, the output weights can be easily calculated by the ridge regression theory. Experimental results on three real HSI datasets demonstrate the effectiveness of our proposed DABL.

Unsupervised Domain Adaptation (UDA) aims to leverage the knowledge from the labeled source domain to help the task of target domain with the unlabeled data. It is a key step for UDA to minimize the cross-domain distribution divergence. In this paper, we firstly propose a novel discrepancy metric, referred to as Cross Domain Mean Approximation (CDMA) discrepancy, to evaluate the distribution differences between source and target domains, which calculate the sum of the squares of the distances from the source and target domains to the mean of the other domain. Secondly, Joint Distribution Adaptation based on Cross Domain Mean Approximation (JDA-CDMA) is developed on the basis of CDMA to extract shared feature and simultaneously reduce the marginal and conditional distribution discrepancy between domains during the label refinement process. Thirdly, we construct a classifier utilizing CDMA metric and neighbor information. Finally, the proposed feature extraction approach and classifier are combined to realize transfer learning. Results from extensive experiments on five visual benchmarks including object, face, and digit images, show the proposed methods outperform the state-of-the-art unsupervised domain adaptation.

It is a vital task to design an integrated machine learning model to discover cancer subtypes and understand the heterogeneity of cancer based on multiple omics data. In recent years, some multi-view clustering algorithms have been proposed and applied to the prediction of cancer subtypes. Among them, the multi-view clustering methods based on graph learning are widely concerned. These multi-view approaches usually have one or more of the following problems. Many multi-view algorithms use the original omics data matrix to construct the similarity matrix and ignore the learning of the similarity matrix. They separate the data clustering process from the graph learning process, resulting in a highly dependent clustering performance on the predefined graph. In the process of graph fusion, these methods simply take the average value of the affinity graph of multiple views to represent the result of the fusion graph, and the rich heterogeneous information is not fully utilized. To solve the above problems, in this paper, a Multi-view Spectral Clustering Based on Multi-smooth Representation Fusion (MRF-MSC) method was proposed. Firstly, MRF-MSC constructs a smooth representation for each data type, which can be viewed as a sample (patient) similarity matrix. The smooth representation can explicitly enhance the grouping effect. Secondly, MRF-MSC integrates the smooth representation of multiple omics data to form a similarity matrix containing all biological data information through graph fusion. In addition, MRF-MSC adaptively gives weight factors to the smooth regularization representation of each omics data by using the self-weighting method. Finally, MRF-MSC imposes constrained Laplacian rank on the fusion similarity matrix to get a better cluster structure. The above problems can be transformed into spectral clustering for solving, and the clustering results can be obtained. MRF-MSC unifies the above process of graph construction, graph fusion and spectral clustering under one framework, which can learn better data representation and high-quality graphs, so as to achieve better clustering effect. In the experiment, MRF-MSC obtained good experimental results on the TCGA cancer data sets.

Estimation bias seriously affects the performance of reinforcement learning algorithms. The maximum operation may result in overestimation, while the double estimator operation often leads to underestimation. To eliminate the estimation bias, these two operations are combined together in our proposed algorithm named stochastic double deep Q-learning network (SDDQN), which is based on the idea of random selection. A tabular version of SDDQN is also given, named stochastic double Q-learning (SDQ). Both the SDDQN and SDQ are based on the double estimator framework. At each step, we choose to use either the maximum operation or the double estimator operation with a certain probability, which is determined by a random selection parameter. The theoretical analysis shows that there indeed exists a proper random selection parameter that makes SDDQN and SDQ unbiased. The experiments on Grid World and Atari 2600 games illustrate that our proposed algorithms can balance the estimation bias effectively and improve performance.

For the zero-shot image classification without intersection between training and testing sets, the high-quality representation of image attributes and features plays a key role to improve the classification performance. In order to overcome the limitations related to insufficient attribute and feature expression in zero-shot image classification, we propose a broad attribute prediction model with enhanced attribute and feature (EAF-BAP) based on broad learning and elastic net constraint. Firstly, the EAF-BAP enhances pre-defined attributes by elastic net constraint to obtain hybrid attributes, which effectively improves the finiteness of semantic attributes. Secondly, the enhanced features are constructed by broad learning to increase the discrimination ability of features in different classes. Meanwhile, the broad learning is employed to train multiple attribute classifiers synchronously, which is more efficient compared to traditional support vector machines. Finally, the similarity between predicted attributes and hybrid attributes in testing classes is calculated by Manhattan distance, which is further used to implement image classification. Experiments on both AwA and Shoes datasets show that the proposed EAF-BAP model is capable of improving the accuracy of zero-shot image classification efficiently.

Recently, graph embedding-based methods have been developed in dimensionality reduction (DR) and classification of hyperspectral image (HSI). The key step for graph embedding methods is the construction of graph. The commonly used method is to manually choose nearest neighbors and then, compute edge weights using the spectral feature. However, the adjacency graph is inappropriate due to the negligence of spatial information. What is more, the construction of graph only takes training samples or k nearest neighbors into account, which may lead to unsuitable graph representation. In this paper, we propose a novel incremental graph embedding (IGE) algorithm to construct a spatial-spectral neighbor graph for the HSI classification. The IGE can spread the discrimination information contained in training samples to their neighbors until each testing sample has a pseudo label. The spatial affinity weights between unlabeled data points and their labeled neighbors are calculated according to the construction strategies of the spatial-spectral neighbor graph. The pseudo label of the unlabeled data point is determined based on the maximum spatial affinity weights. Moreover, three weight strategies are designed for those samples nearby the decision boundary to improve the separability of different classes. In addition, the window size of spatial neighbors is able to be adjusted adaptively according to whether labeled data points in spatial neighbors exit. Experimental results on two datasets, Indian Pine and Pavia University have demonstrated that our algorithm is remarkably superior to other conventional DR algorithms on improving classification performance.

Deep convolutional network demonstrates that the classification accuracy can be remarkably improved by increasing the number of network layers, however, increases the accuracy by 1% of costs nearly doubling the number of layers. Meanwhile, gradient dispersion will occur in the training process, which leads to performance degradation. In order to solve the problem of training difficulty with the increased number of layers, we focus on network architecture and propose a deep convolutional network based on the pyramid structure. In the network architecture, as the number of layers increased, the feature map dimensions (i.e., the number of channels) are gradually increased at each layer to distribute the burden concentrated at locations of structural units affected by downsampling, such that all units are equally distributed. By exploring the sequence between the stacked elements inside the structural unit, we design a pyramidal building block, as its shape gradually widens from the top downwards, which is called the deep pyramid convolutional network (DPCNet). Experimental results on CIFAR-10 and CIFAR-100 datasets have shown that DPCNet has the superior generalization capability and can effectively improve the image classification accuracy.

Abstract It is urgent to diagnose colorectal cancer in the early stage. Some feature genes which are important to colorectal cancer development have been identified. However, for the early stage of colorectal cancer, less is known about the identity of specific cancer genes that are associated with advanced clinical stage. In this paper, we conducted a feature extraction method named Optimal Mean based Block Robust Feature Extraction method (OMBRFE) to identify feature genes associated with advanced colorectal cancer in clinical stage by using the integrated colorectal cancer data. Firstly, based on the optimal mean and L 2,1-norm, a novel feature extraction method called Optimal Mean based Robust Feature Extraction method (OMRFE) is proposed to identify feature genes. Then the OMBRFE method which introduces the block ideology into OMRFE method is put forward to process the colorectal cancer integrated data which includes multiple genomic data: copy number alterations, somatic mutations, methylation expression alteration, as well as gene expression changes. Experimental results demonstrate that the OMBRFE is more effective than previous methods in identifying the feature genes. Moreover, genes identified by OMBRFE are verified to be closely associated with advanced colorectal cancer in clinical stage.

By combining the broad learning and a convolutional neural network (CNN), a block-diagonal constrained multi-stage convolutional broad learning (MSCBL-BD) method is proposed for hyperspectral image (HSI) classification. Firstly, as the linear sparse feature extracted by the conventional broad learning method cannot fully characterize the complex spatial-spectral features of HSIs, we replace the linear sparse features in the mapped feature (MF) with the features extracted by the CNN to achieve more complex nonlinear mapping. Then, in the multi-layer mapping process of the CNN, information loss occurs to a certain degree. To this end, the multi-stage convolutional features (MSCFs) extracted by the CNN are expanded to obtain the multi-stage broad features (MSBFs). MSCFs and MSBFs are further spliced to obtain multi-stage convolutional broad features (MSCBFs). Additionally, in order to enhance the mutual independence between MSCBFs, a block diagonal constraint is introduced, and MSCBFs are mapped by a block diagonal matrix, so that each feature is represented linearly only by features of the same stage. Finally, the output layer weights of MSCBL-BD and the desired block-diagonal matrix are solved by the alternating direction method of multipliers. Experimental results on three popular HSI datasets demonstrate the superiority of MSCBL-BD.

The existing attribute‐based zero‐shot learning models at different levels ignore some necessary prior knowledge. It is essential to improve classification accuracy of zero‐shot learning that how to mine attribute‐related and class‐related prior knowledge further being incorporated into the attribute prediction models. For the mining of class‐related prior knowledge, measurement of the class–class correlation by using whitened cosine similarity is proposed. Likewise for the mining of attribute‐related prior knowledge, measurements of the attribute–class and attribute–attribute correlation are proposed by using sparse representation coefficient. Therefore, a novel indirect attribute prediction (IAP) model is presented by exploiting class‐related and attribute‐related prior knowledge (IAP_CAPK). Experimental results on animals with attributes and a‐Pascal/a‐Yahoo datasets show that, when compared with IAP and direct attribute prediction, the proposed IAP_CAPK not only yields more accurate attribute prediction and zero‐shot image classification, but also achieves much higher computational efficiency.

At present many researchers pay attention to a combination of spectral features and spatial features to enhance hyperspectral image (HSI) classification accuracy. However, the spatial features in some methods are utilized insufficiently. In order to further improve the performance of HSI classification, the spectral-spatial joint classification of HSI based on the broad learning system (BLS) (SSBLS) method was proposed in this paper; it consists of three parts. Firstly, the Gaussian filter is adopted to smooth each band of the original spectra based on the spatial information to remove the noise. Secondly, the test sample’s labels can be obtained using the optimal BLS classification model trained with the spectral features smoothed by the Gaussian filter. At last, the guided filter is performed to correct the BLS classification results based on the spatial contextual information for improving the classification accuracy. Experiment results on the three real HSI datasets demonstrate that the mean overall accuracies (OAs) of ten experiments are 99.83% on the Indian Pines dataset, 99.96% on the Salinas dataset, and 99.49% on the Pavia University dataset. Compared with other methods, the proposed method in the paper has the best performance.