Search

This paper begins with some brief historical comments to set the stage for a discussion of the long-term potential for land remote sensing technology. This is followed by comments about what is needed to accelerate the achievement of this potential. The discussion is concluded with what concomitant development is needed with regard to a hyperspectral data analysis system. Index Terms--Analysis methods, multispectral data, remote sensing uses.

In this paper, a new nonparametric feature extraction method is proposed for high-dimensional multiclass pattern recognition problems. It is based on a nonparametric extension of scatter matrices. There are at least two advantages to using the proposed nonparametric scatter matrices. First, they are generally of full rank. This provides the ability to specify the number of extracted features desired and to reduce the effect of the singularity problem. This is in contrast to parametric discriminant analysis, which usually only can extract L-1 (number of classes minus one) features. In a real situation, this may not be enough. Second, the nonparametric nature of scatter matrices reduces the effects of outliers and works well even for nonnormal datasets. The new method provides greater weight to samples near the expected decision boundary. This tends to provide for increased classification accuracy. Index Terms--Dimensionality reduction, discriminant analysis, nonparametric feature extraction.

In this paper, we propose a supervised classifier based on implementation of the Bayes rule with kernels. The proposed technique first proposes an implicit nonlinear transformation of the data into a feature space seeking to fit normal distributions having a common covariance matrix onto the mapped data. One requirement of this approach is the evaluation of posterior probabilities. We express the discriminant function in dot-product form, and then apply the kernel concept to efficiently evaluate the posterior probabilities. The proposed technique gives the flexibility required to model complex data structures that originate from a wide range of class-conditional distributions. Although we end up with piecewise linear decision boundaries in the feature space, these corresponds to powerful nonlinear boundaries in the original input space. For the data we considered, we have obtained some encouraging results. Index Terms--Bayes rule, kernels, supervised classification.

In this paper, we propose a cost-effective iterative semisupervised classifier based on a kernel concept. The proposed technique incorporates unlabeled data into the design of a binary classifier by introducing and optimizing a cost function in a feature space that maximizes the Rayleigh coefficient while minimizing the total cost associated with misclassified labeled samples. The cost assigned to misclassified labeled samples accounts for the number of misclassified labeled samples as well as the amount by which they are on the wrong side of the boundary, and this counterbalances any potential adverse effect of unlabeled data on the classifier performance. Several experiments performed with remotely sensed data demonstrate that using the proposed semisupervised classifier shows considerable improvements over the supervised-only counterpart. Index Terms--Fisher's discriminant, semisupervised classifier, unlabeled data.

It is well known that there is a strong relation between class definition precision and classification accuracy in pattern classification applications. In hyperspectral data analysis, usually classes of interest contain one or more components and may not be well represented by a single Gaussian density function. In this paper, a model-based mixture classifier, which uses mixture models to characterize class densities, is discussed. However, a key outstanding problem of this approach is how to choose the number of components and determine their parameters for such models in practice, and to do so in the face of limited training sets where estimation error becomes a significant factor. The proposed classifier estimates the number of subclasses and class statistics simultaneously by choosing the best model. The structure of class covariances is also addressed through a model-based covariance estimation technique introduced in this paper. Index Terms--Covariance estimator, expectation-maximization, Gaussian mixtures.

There are many factors to consider in carrying out a hyperspectral data classification. Perhaps chief among them are class training sample size, dimensionality, and distribution separability. The intent of this study is to design a classification procedure that is robust and maximally effective, but which provides the analyst with significant assists, thus simplifying the analyst's task. The result is a quadratic mixture classifier based on Mixed-LOOC2 regularized discriminant analysis and nonparametric weighted feature extraction. This procedure has the advantage of providing improved classification accuracy compared to typical previous methods but requires minimal need to consider the factors mentioned above. Experimental results demonstrating these properties are presented.

In this paper, an adaptive Bayesian contextual classification procedure that utilizes both spectral and spatial interpixel dependency contexts in estimation of statistics and classification is proposed. Essentially, this classifier is the constructive coupling of an adaptive classification procedure and a Bayesian contextual classification procedure. In this classifier, the joint prior probabilities of the classes of each pixel and its spatial neighbors are modeled by the Markov random field. The estimation of statistics and classification are performed in a recursive manner to allow the establishment of the positive-feedback process in a computationally efficient manner. Experiments with real hyperspectral data show that, starting with a small training sample set, this classifier can reach classification accuracies similar to that obtained by a pixelwise maximum likelihood pixel classifier with a very large training sample set. Additionally, classification maps are produced that have significantly less speckle error. Index Terms--Adaptive iterative classification procedure, Bayesian contextual classification procedure, hyperspectral data, iterative conditional mode (ICM), semilabeled samples.

In this paper, a family of adaptive covariance estimators is proposed to mitigate the problem of limited training samples for application to hyperspectral data analysis in quadratic maximum likelihood classification. These estimators are the combination of adaptive classification procedures and regularized covariance estimators. In these proposed estimators, the semi-labeled samples (whose labels are determined by a decision rule) are incorporated in the process of determining the optimal regularized parameters and estimating those supportive covariance matrices that formulate final regularized covariance estimators. In all experiments with simulated and real remote sensing data, these proposed combined covariance estimators achieved significant improvement on statistics estimation and classification accuracy over conventional regularized covariance estimators and an adaptive Maximum Likelihood classifier (MLC). The degree of improvement increases with dimensions, especially for ill-posed or very ill-posed problems where the total number of training samples is smaller than the number of dimensions. Index Terms--Adaptive iterative classification procedure, high-dimensional data, hyperspectral data, regularized covariance estimation, semi-labeled samples.

A key to successful classification of multivariate data is the defining of an accurate quantitative model of each class. This is especially the case when the dimensionality of the data is high, and the problem is exacerbated when the number of training samples is limited. For the commonly used quadratic maximum-likelihood classifier, the class mean vectors and covariance matrices are required and must be estimated from the available training samples. In high dimensional cases, it has been found that feature extraction methods are especially useful, so as to transform the problem to a lower dimensional space without loss of information, however, here too class statistics estimation error is significant. Finding a suitable regularized covariance estimator is a way to mitigate these estimation error effects. The main purpose of this work is to find an improved regularized covariance estimator of each class with the advantages of Leave-One-Out Covariance Estimator (LOOC) and Bayesian LOOC (BLOOC). Besides, using the proposed covariance estimator to improve the linear feature extraction methods when the multivariate data is singular or nearly so is demonstrated. This work is specifically directed at analysis methods for hyperspectral remote sensing data. Index Terms--Feature extraction, hyperspectral data classification, regularized covariance estimator.

Remote sensing data is collected and analyzed to enhance understanding of the terrestrial surface--in composition, in form, or in function. One approach for accomplishing this is to design the analysis process as an iterated composite of several analyst-directed modules. This paper proposes such a modular design for the data analysis. The proposed methodology was applied in a project to obtain the thematic map for a flightline over Washington, DC, with very satisfactory results--the qualification being in both the visual and the statistical sense. The project execution is presented as a case study in this paper. Index Terms--Digital elevation map, holistic solution, human--computer interaction, hyperspectral data classification, masking, methodology, segmentation.

In this paper, we propose a self-learning and self-improving adaptive classifier to mitigate the problem of small training sample size that can severely affect the recognition accuracy of classifiers when the dimensionality of the multispectral data is high. This proposed adaptive classifier utilizes classified samples (referred as semilabeled samples) in addition to original training samples iteratively. In order to control the influence of semilabeled samples, the proposed method gives full weight to the training samples and reduced weight to semilabeled samples. We show that by using additional semilabeled samples that are available without extra cost, the additional class label information may be extracted and utilized to enhance statistics estimation and hence improve the classifier performance, and therefore the Hughes phenomenon (peak phenomenon) may be mitigated. Experimental results show this proposed adaptive classifier can improve the classification accuracy as well as representation of estimated statistics significantly. Index Terms--Adaptive iterative classifier, high-dimensional data, labeled samples, limited training data set, semilabeled samples.

As the number of spectral bands of high-spectral resolution data increases, the ability to detect more detailed classes should also increase, and the classification accuracy should increase as well. Often the number of labeled samples used for supervised classification techniques is limited, thus limiting the precision with which class characteristics can be estimated. As the number of spectral bands becomes large, the limitation on performance imposed by the limited number of training samples can become severe. A number of techniques for case-specific feature extraction have been developed to reduce dimensionality without loss of class separability. Most of these techniques require the estimation of statistics at full dimensionality in order to extract relevant features for classification. If the number of training samples is not adequately large, the estimation of parameters in high-dimensional data will not be accurate enough. As a result, the estimated features may not be as effective as they could be. This suggests the need for reducing the dimensionality via a preprocessing method that takes into consideration high-dimensional feature-space properties. Such reduction should enable the estimation of feature-extraction parameters to be more accurate. Using a technique referred to as projection pursuit (PP), such an algorithm has been developed. This technique is able to bypass many of the problems of the limitation of small numbers of training samples by making the computations in a lower-dimensional space, and optimizing a function called the projection index. A current limitation of this method is that, as the number of dimensions increases, it is likely that a local maximum of the projection index will be found that does not enable one to fully exploit hyperspectral-data capabilities. A method to estimate an initial value that can lead to a maximum that increases the classification accuracy significantly will be presented. This method also leads to a high-dimensional version of a feature-selection algorithm, which requires significantly less computation than the normal procedure. Index Terms: Band subset selection, dimensionality reduction, feature extraction, hyperspectral data analysis, pattern recognition, projection pursuit, supervised classification.

It is well known that high-dimensional image data allows for the separation of classes that are spectrally very similar, i.e., possess nearly equal first-order statistics, provided that their second-order statistics differ significantly. The aim of this study is to contribute to a better understanding, from a more geometrically oriented point of view, of the role played by the second-order statistics in remote sensing digital image classification of natural scenes when the classes of interest are spectrally very similar and high dimensional multispectral image data is available. A number of the investigations that have been developed in this area deal with the fact that as the data dimensionality increases, so does the difficulty in obtaining a reasonably accurate estimate of the within-class covariance matrices from the number of available labeled samples, which is usually limited. Several approaches have been proposed to deal with this problem. This study aims toward a complementary goal. Assuming that reasonably accurate estimates for the within-class covariance matrices have been obtained, we seek to better understand what kind of geometrically-oriented interpretation can be given to them as the data dimensionality increases and also to understand how this knowledge can help the design of a classifier. In order to achieve this goal, the covariance matrix is decomposed into a number of parameters that are then analyzed separately with respect to their ability to separate the classes. Methods for image classification based on these parameters are investigated. Results of tests using data provided by the sensor system AVIRIS are presented and discussed. Index Terms: AVIRIS sensor, digital image classification, high-dimensional data, remote sensing, second-order statistics.

This paper describes a covariance estimator formulated under an empirical Bayesian setting to mitigate the problem of limited training samples in the Gaussian maximum likelihood (ML) classification for remote sensing. The most suitable covariance mixture is selected by maximizing the average leave-one-out log likelihood. Experimental results using AVIRIS data are presented. Index Terms - Covariance estimation, Gaussian maximum likelihood, leave-one-out log likelihood, regularization.

This paper addresses a classification problem in which class definition through training samples or otherwise is provided a priori only for a particular class of interest. Considerable time and effort may be required to label samples necessary for defining all the classes existent in a given data set by collecting ground truth or by other means. Thus, this problem is very important in practice, because one is often interested in identifying samples belonging to only one or a small number of classes. The problem is considered as an unsupervised clustering problem with initially one known cluster. The definition and statistics of the other classes are automatically developed through a weighted unsupervised clustering procedure that keeps the known cluster from losing its identity as the 'class of interest.' Once all the classes are developed, a conventional supervised classifier such as the maximum likelihood classifier is used in the classification. Experimental results with both simulated and real data verify the effectiveness of the proposed method. Index Terms - One-class classifier, partially supervised classifier, significance testing, single hypothesis testing, unsupervised clustering.

In this paper, we propose a new feature extraction method for feedforward neural networks. The method is based on the recently published decision boundary feature extraction algorithm which is based on the fact that all the necessary features for classification can be extracted from the decision boundary. The decision boundary feature extraction algorithm can take advantage of characteristics of neural networks which can solve complex problems with arbitrary decision boundaries without assuming underlying probability distribution functions of the data. To apply the decision boundary feature extraction method, we first give a specific definition for the decision boundary in a neural network. Then, we propose a procedure for extracting all the necessary features for classification from the decision boundary. Experiments show promising results. Index Terms - Neural networks, feature extraction, decision boundary, classification, decision boundary feature matrix, pattern recognition.

A new covariance matrix estimator useful for designing classifiers with limited training data is developed. In experiments, this estimator achieved higher classification accuracy than the sample covariance matrix and common covariance matrix estimates. In about half of the experiments, it achieved higher accuracy than regularized discriminant analysis, but required much less computation. Index Terms - Covariance matrix, estimation, leave-one-out method, cross validation, classification, high dimensional data.

In this paper, we study the use of unlabeled samples in reducing the problem of small training sample size that can severely affect the recognition rate of classifiers when the dimensionality of the multispectral data is high. We show that by using additional unlabeled samples that are available at no extra cost, the performance may be improved, and therefore the Hughes phenomenon can be mitigated. Furthermore, by experiments, we show that by using additional unlabeled samples more representative estimates can be obtained. We also propos a semiparametric method for incorporating the training (i.e., labeled) and unlabeled samples simultaneously into the parameter estimation process.

This correspondence proposes a fast Parzen density estimation algorithm that would be especially useful in the nonparametric discriminant analysis problems. By preclustering the data and applying a simple branch and bound procedure to the clusters, significant numbers of data samples that would contribute little to the density estimate can be excluded without detriment to actual evaluation via the kernel functions. This technique is especially helpful in the multivariant case, and does not require a uniform sampling grid. The proposed algorithm may also be used in conjunction with the data reduction technique of Fukunaga and Hayes to further reduce the computational load. Experimental results are presented to verify the effectiveness of this algorithm.

In this paper, through a series of specific examples, we illustrate some characteristics encountered in analyzing high-dimensional multispectral data. The increased importance of the second-order statistics in analyzing high-dimensional data is illustrated, as is the shortcoming of classifiers such as the minimum distance classifier which rely on first-order variations alone. We also illustrate how inaccurate estimation of first- and second-order statistics, e.g., from use of training sets which are too small, affects the performance of a classifier. Recognizing the importance of second-order statistics on the one hand, but the increased difficulty in perceiving and comprehending information present in statistics derived from high-dimensional data on the other, we propose a method to aid visualization of high-dimensional statistics using a color coding scheme.

In this paper, a novel approach to feature extraction for classification is proposed based directly on the decision boundaries. We note that feature extraction is equivalent to retaining informative features or eliminating redundant features; thus, the terms 'discriminantly information feature' and 'discriminantly redundant feature' are first defined relative to feature extraction for classification. Next, it is shown how discriminantly redundant features and discriminantly informative features are related to decision boundaries. A novel characteristic of the proposed method arises by noting that usually only a portion of the decision boundary is effective in discriminating between classes, and the concept of the effective decision boundary is therefore introduced. Next, a procedure to extract discriminantly informative features based on a decision boundary is proposed. The proposed feature extraction algorithm has several desirable properties: 1) It predicts the minimum number of features necessary to achieve the same classification accuracy as in the original space for a given pattern recognition problem; 2) it finds the necessary feature vectors. The proposed algorithm does not deteriorate under the circumstances of equal class means or equal class covariances as some previous algorithms do. Experiments show that the performance of the proposed algorithm compares favorably with those of previous algorithms.

Feature extraction has long been an important topic in pattern recognition. Although many authors have studied feature extraction for parametric classifiers, relatively few feature extraction algorithms are available for nonparametric classifiers. A new feature extraction algorithm based on decision boundaries for nonparametric classifiers is proposed. It is noted that feature extraction for pattern recognition is equivalent to retaining 'discriminantly informative features' and a discriminantly informative feature is related to the decision boundary. Since nonparametric classifiers do not define decision boundaries in analytic form, the decision boundary and normal vectors must be estimated numerically. A procedure to extract discriminantly informative features based on a decision boundary for non-parametric classification is proposed. Experiments show that the proposed algorithm finds effective features for the nonparametric classifier with Parzen density estimation.

A contextual classifier which can utilize both spatial and temporal interpixel dependency contexts is investigated. After spatial and temporal neighbors are defined, a general form of maximum a posterior spatio-temporal contextual classifier is derived. This contextual classifier is simplified under several assumptions. Joint prior probabilities of the classes of each pixel and its spatial neighbors are modeled by the Gibbs random filed. The classification is performed in a recursive manner to allow a computationally efficient contextual classification. Experimental results with bitemporal TM data show significant improvement of classification accuracy over noncontextual pixelwise classifier. This spatio-temporal contextual classifier will find its use in many real applications of remote sensing, especially when the classification accuracy is important.