Search

Bayesian Networks have emerged in recent years as a powerful data mining technique for handling uncertainty in complex domains. The Bayesian Network represents the joint probability distribution and domain (or expert) knowledge in a compact way and provides a comprehensive method of representing relationships and influences among nodes (variables) with a graphical diagram. Actually, however, in the classification domain it was not paid attention to by researchers until the simplest of form of Bayesian Networks, Naive Bayes Classifier, turned up. Naive Bayes Classifier is a simple and efficient probability classification method, and has shown surprising performance in some domains, which owes to the independence assumption that makes Naive Bayes Classifier fit the classification more easily. However, the independence assumption obviously does not hold in the real world. Therefore, in order to meet the "naive" (unreal) assumption, this paper proposes a new image texture classification method of aerial images, PCA-NBC, which combines the Principal Components Analysis (PCA) and Naive Bayes Classifier (NBC). The PCA transforms the highly correlated features into statistically independent and orthogonal "features", so it is suitable to solve that problem and can lay a solid theoretic foundation in the application. One hundred and thirteen aerial images are used to evaluate the classification performance in the experiment. The experimental results demonstrate that the proposed method can cut down the number of features and computational costs and improve the accuracy during classification. In one word, the new method, PCA-NBC, is an attractive and effective method, which outperforms the Naive Bayes Classifier.

Error estimation is a key aspect of statistical pattern recognition. The true classification error rate is usually unavailable since it depends on the unknown feature-label distribution. Hence, one needs to estimate the error rate from the available sample data. This paper presents a concise, mathematically rigorous review of the subject of error estimation in statistical pattern recognition, pointing to the pitfalls that arise in small-sample settings due to the use of "rules of thumb" and a neglect for proper mathematical understanding of the problem.

In ATR applications, each feature is a convolution of an image with a filter. It is important to use most discriminant features to produce compact representations. We propose two novel subspace methods for dimension reduction to address limitations associated with Fukunaga-Koontz Transform (FKT). The first method, Scatter-FKT, assumes that target is more homogeneous, while clutter can be anything other than target and anywhere. Thus, instead of estimating a clutter covariance matrix, Scatter-FKT computes a clutter scatter matrix that measures the spread of clutter from the target mean. We choose dimensions along which the difference in variation between target and clutter is most pronounced. When the target follows a Gaussian distribution, Scatter-FKT can be viewed as a generalization of FKT. The second method, Optimal Bayesian Subspace, is derived from the optimal Bayesian classifier. It selects dimensions such that the minimum Bayes error rate can be achieved. When both target and clutter follow Gaussian distributions, OBS computes optimal subspace representations. We compare our methods against FKT using character image as well as IR data.

Feature extraction and selection are important problems in statistical learning. We study the relationships between two previously proposed principles for their optimal solution: the minimization of Bayes error and the maximization of mutual information between features and class labels. It is shown that a quantity which provides insight on this relationship is the set of non-increasing probability mass functions (NIPMFs). We derive some basic properties of the members of this set, show that any classification problem defines an ensemble of NIPMFs, and that the probability distribution of this ensemble uniquely determines the associated Bayes error and mutual information. These results are then used to show that, when the classification problem is binary and some generic constraints hold, the optimal feature space is the same under the two formulations.

In this paper, a new neural network directed Bayes decision rule is developed for target classification exploiting the target's dynamic behavior. The system consists of a feature extractor, a neural network directed conditional probability generator and a sequential Bayes classifier. The velocity and curvature sequences extracted from each track are used as the primary features. Several hidden states are used to train the neural network, the output of which is the conditional probability of occurring the hidden states given the observations. These conditional probabilities are then used as the inputs to the Bayes classifier to make the classification. The classification results are updated recursively whenever a new scan of data is received. Simulation results on both clean tracks and heavily cluttered Infrared (IR) satellite images are presented to demonstrate the effectiveness of the proposed methods.

Error diffusion technique has been one of the most popular digital image halftoning methods. The quality of binary image resulting from the error diffusion technique is affected by the following three key factors; the values of error diffusion kernel, the locations of neighboring pixels for error propagation, and the quantization scheme. Among these factors, this paper is focused on the estimation of the values of error diffusion kernel. In previous efforts to propose modification to the original Floyd-Steinberg's algorithm, the values of error diffusion kernel have been determined by the trial and error method or by utilizing optimization techniques such as the least mean square estimation and neural network methods. This paper presents a new estimation method for the values of error diffusion kernel based on the genetic algorithm. Compared to the conventional optimization techniques, the genetic algorithm based approach lifts restrictions on the complexity of the error criterion for optimization. In this paper, two types of the error criteria are defined to improve image quality. They represent a measure of the reproduction of average brightness and an extent of undesirable artifacts appeared on the binary image for specific gray levels. The values of error diffusion kernel are estimated by simultaneously minimizing the defined error criteria using genetic algorithm. In the experiments, three types of error diffusion kernel are examined. The experimental results indicate that the binary images obtained based on the estimated error diffusion kernel exhibit less artifacts.

In this paper, a new method to predict the classification error in the Gaussian ML classifier is proposed. The Gaussian ML classifier is one of the most widely classifiers in pattern classification and remote sensing because of its speed and performance. Several methods have been proposed to estimate error of the Gaussian ML classifier. In particular, the Bhattacharyya distance gives theoretical upper and lower bounds of the classification error. However, in many cases, the bounds ar not tight enough to be useful.In this paper, we proposed a different approach to predict error of the Gaussian ML classifier using the Bhattacharyya distance. We generate two classes with normal distribution and calculate the Bhattacharyya distance and the classification accuracy. The class statistics used to generate data are obtained form real remotely sensed data. We repeat the experiment about 100 million times with different class statistics and try to find the relationship between the classification error and the Bhattacharyya distance empirically. The range of the dimension of the generated data is from 1 to 17. From the experiments, we are able to obtain a formula that gives a much better error estimation of the Gaussian ML classifier. Apparently, it is possible to predict the classification error within 1-2 percent margin.

Improved performance evaluation results for complex data sets, in the Bayes error estimation sense, are shown by first decomposing the data sets into approximately normal modes before input to the error estimator. More specifically, the utility of a particular nonparametric Bayes error estimator, the Parzen error estimator, is generalized to multimodal data sets by preprocessing the data through a mode seeker before input to the error estimator. The utility of the mode seeker and the Parzen error estimator for data analysis and performance evaluation is demonstrated on a field collected radar data set.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Empirical Bayes classification rules are derived for minefield detection. Laplace approximations forthe likelihood function and for the posterior density are used to construct approximate Bayes rules forclassifying each unit or region as belonging to one of the two possible types, indicating the presence or absence of mines. Approximate maximum likelihood estimation is proposed assuming that repeatedobservations are available. An application to a multivariate Poisson log normal model is discussed briefly. Some Key Words: Bayes and empirical Bayes classification rules; Minefield modeling and detection. 1. The Model Consider a situation where m sites in a given region are monitored by k image recording devicesset at different frequency bands. Since the sensor resolution in image recordings are not always arbi-trarily fine, it may be assumed that the available data points are in the form of pixels on a lattice. For simplicity, we suppose that each of the m sites belongs to one of the two types: type "0" indicatingmine-free and type "1" containing mines. The main goal of this paper is to develop a model for

A neural network approach to obtaining upper and lower bounds on the Bayes error rate for pattern recognition problems is presented. The approach is developed using the key concept of resubstitution and leave-one-out testing from conventional nonparametric error estimation techniques. The neural network approach is evaluated by applying it to several 8D, two-class `toy' problems, where the Bayes error rate is known. The neural network error estimate for a high-dimensional problem with an unknown Bayes error rate is also compared to error estimates obtained using conventional nonparametric estimation techniques. Using the neural network procedure, the upper bound of the Bayes error rate is reliably found for problems with complex decision boundary surfaces. Alternative testing approaches are suggested for reducing the difference between the bounds and the true Bayes rate.

In the Gaussian case, the Bayes classifiers and the minimum distance classifiers are compared. The comparison is based on the error probability and on the bias introduced by the estimation of the law parameters. It is shown, both theoretically and by simulations, that even a suboptimal use of the minimum distance classifiers may be justified when the finite design sample size is small with regard to dimensionality. An application to signal classification is studied where the best modelization of the signal is not the best representation for classification.

In this paper, the problem of designing a matched filter for image correlation will be treated as a statistical pattern recognition problem. It is shown that, by minimizing a suitable criterion, a matched filter can be estimated which approximates the optimum Bayes discriminant function in a least-squares sense. It is well known that the use of the Bayes discriminant function in target classification minimizes the Bayes risk, which in turn directly minimizes the probability of a false fix. A fast Fourier implementation of the minimum Bayes risk correlation procedure is described.