Search

• We propose discretization-aware architecture search (DA2S), and target at pushing the super-network towards the configuration of desired topol- ogy. DA2S is implemented with an entropy-based loss term, which can be regularized to differentiable architecture search in a plug-and-play fashion. • The regularization for architecture search is controlled by elaborated continuation functions, so that discretization is adaptive to the dynamic change of edges and operations. • Experiments on standard image classification benchmarks demonstrate the effectiveness of our approach, in particular, under imbalanced network configurations that were not studied before. The search cost of neural architecture search (NAS) has been largely reduced by differentiable architecture search and weight-sharing methods. Such methods optimize a super-network with all possible edges and operations, and determine the optimal sub-network by discretization, i.e. , pruning off operations/edges of small weights. However, the discretization process performed on either operations or edges incurs significant inaccuracy and thus the quality of the architecture is not guaranteed. In this paper, we propose discretization-aware architecture search (DA2S), and target at pushing the super-network towards the configuration of desired topology. DA2S is implemented with an entropy-based loss term, which can be regularized to differentiable architecture search in a plug-and-play fashion. The regularization is controlled by elaborated continuation functions, so that discretization is adaptive to the dynamic change of edges and operations. Experiments on standard image classification benchmarks demonstrate the effectiveness of our approach, in particular, under imbalanced network configurations that were not studied before. Code is available at github.com/sunsmarterjie/DAAS. [ABSTRACT FROM AUTHOR]

• Proposal of potential resemblance loss for measuring relative class distribution shape. • Proposal of unified over and undersampling framework based on potential resemblance. • Proposal of data difficulty index for evaluation of dataset complexity. • Experimental evaluation of the proposed approach. • Examination of factors influencing the relative performance of the proposed approach. Data imbalance remains one of the factors negatively affecting the performance of contemporary machine learning algorithms. One of the most common approaches to reducing the negative impact of data imbalance is preprocessing the original dataset with data-level strategies. In this paper we propose a unified framework for imbalanced data over- and undersampling. The proposed approach utilizes radial basis functions to preserve the original shape of the underlying class distributions during the resampling process. This is done by optimizing the positions of generated synthetic observations with respect to the proposed potential resemblance loss. The final Potential Anchoring algorithm combines over- and undersampling within the proposed framework. The results of the experiments conducted on 60 imbalanced datasets show outperformance of Potential Anchoring over state-of-the-art resampling algorithms, including previously proposed methods that utilize radial basis functions to model class potential. Furthermore, the results of the analysis based on the proposed data complexity index show that Potential Anchoring is particularly well suited for handling naturally complex (i.e. not affected by the presence of noise) datasets. [ABSTRACT FROM AUTHOR]

• We propose a novel multi-view DCNN-based approach which can take advantage of not only multi-view images but also the relationships between them. • The proposed networks learn the integrated features of multi-view fundus images for DR detection. • We introduce the attention mechanisms for mining the relationship between different views. • In order to boost the ability to capture the tiny lesion features in the retinal images, we combine the idea of attention mechanisms with the channel dimension. Diabetic retinopathy (DR) detection has attracted much attention recently, and the deep learning algorithms have gained traction in this area. At present, DR screening by deep learning algorithms is often based on single-view fundus images, which usually leads to an unsatisfactory accuracy of DR grading due to the incomplete lesion features. In this paper, we proposed a novel diabetic retinopathy detection convolutional network for automatic DR detection by integrating multi-view fundus images. Compared to existing single-view DCNN-based DR detection methods, the proposed method has the following advantages. First, our method fully utilizes the lesion features from the retina with a field-of-view around 120 ∘ − 150 ∘. Second, by introducing the attention mechanisms, more attention will be paid on the influential view and the performance can be improved. Besides, we also assign large weights to important channels in the network for effective feature extraction. Experiments are conducted on our collected multi-view DR dataset contained 15,468 images, in which each eye sample provides four-view images. The experimental results indicate that using multi-view images is suitable for automatic DR detection and our proposed method is superior to other benchmarking methods. [ABSTRACT FROM AUTHOR]

Abstract: In this paper we propose a new classifier called a dispersion matcher. Our proposal is especially well adapted to those scenarios where a large number of classes and a small number of samples per class are available for training. This is the situation of biometric systems where just three to five measures per person are acquired during enrollment. This is just the opposite situation of other pattern recognition applications where a small number of classes and a large amount of training samples are available, such as handwritten digit recognition (10 classes) for ZIP code identification. The dispersion matcher trains a quadratic discriminant classifier to solve the dichotomy “Do these two feature vectors belong to the same person?”. In this way, we solve an important set of topics: (a) we can classify an open world problem and we do not need to train the model again if a new user is added, (b) we find a natural solution for feature selection, (c) experimental results with a priori threshold provides good results. We evaluate the proposed system with hand-geometry and face recognition problems (identification and verification). In hand geometry, we get a minimum detection cost function (DCF) for verification of 0.21% and a maximum identification rate of 99.1%, which compares favorably with other state-of-the-art methods. In face verification we achieve 5.59% DCF and 92.77% identification rate, which also compares favorably with the literature. [Copyright &y& Elsevier]

Abstract: Novelty detection, also referred to as one-class classification, is the process of detecting ‘abnormal’ behavior in a system by learning the ‘normal’ behavior. Novelty detection has been of particular interest to researchers in domains where it is difficult or expensive to find examples of abnormal behavior (such as in medical/equipment diagnosis and IT network surveillance). Effective representation of normal data is of primary interest in pursuing one-class classification. While the literature offers several methods for one-class classification, very few methods can support representation of non-stationary classes without making stringent assumptions about the class distribution. This paper proposes a one-class classification method for non-stationary classes using a modified support vector machine and an efficient online version for reducing computational time. The presented method is applied to several simulated datasets and actual data from a drilling machine. In addition, we present comparison results with other methods that demonstrate its superior performance. [Copyright &y& Elsevier]

Abstract: In classifying high-dimensional patterns such as stellar spectra by a Gaussian classifier, the covariance matrix estimated with a small-number sample set becomes unstable, leading to degraded classification accuracy. In this paper, we investigate the covariance matrix estimation problem for small-number samples with high dimension setting based on minimum description length (MDL) principle. A new covariance matrix estimator is developed, and a formula for fast estimation of regularization parameters is derived. Experiments on spectrum pattern recognition are conducted to investigate the classification accuracy with the developed covariance matrix estimator. Higher classification accuracy results are obtained and demonstrated in our new approach. [Copyright &y& Elsevier]

Abstract: It is well known that the error probability, of the binary Gaussian classification problem with different class covariance matrices, cannot be generally evaluated exactly because of the lack of closed-form expression. This fact pointed out the need to find a tight upper bound for the error probability. This issue has been for more than 50 years ago and is still of interest. All derived upper-bounds are not free of flaws. They might be loose, computationally inefficient particularly in highly dimensional situations, or excessively time consuming if high degree of accuracy is desired. In this paper, a new technique is developed to estimate a tight upper bound for the error probability of the well-known binary Gaussian classification problem with different covariance matrices. The basic idea of the proposed technique is to replace the optimal Bayes decision boundary with suboptimal boundaries which provide an easy-to-calculate upper bound for the error probability. In particular, three types of decision boundaries are investigated: planes, elliptic cylinders, and cones. The new decision boundaries are selected in such a way as to provide the tightest possible upper bound. The proposed technique is found to provide an upper bound, tighter than many of the often used bounds such as the Chernoff bound and the Bayesian-distance bound. In addition, the computation time of the proposed bound is much less than that required by the Monte-Carlo simulation technique. When applied to real world classification problems, obtained from the UCI repository [H. Chernoff, A measure for asymptotic efficiency of a hypothesis based on a sum of observations, Ann. Math. Statist. 23 (1952) 493–507.], the proposed bound was found to provide a tight bound for the analytical error probability of the quadratic discriminant analysis (QDA) classifier and a good approximation to its empirical error probability. [Copyright &y& Elsevier]

Abstract: The primary goal of this paper is to improve the classification of multi-channel evoked potentials (EPs) by introducing a temporal domain artifact detection strategy and using this strategy to (a) evaluate how the performance of classifiers is affected by artifacts and (b) show how the performance can be improved by detecting and rejecting artifacts in offline and real-time classification experiments. Using a pattern recognition approach, an artifact is defined in this study as any signal that may lead to inaccurate classifier parameter estimation and inaccurate testing. The temporal domain artifact detection tests include: a within-channel standard deviation (STD) test that can detect signals with little or abnormal variations in each channel and also detect faulty channels, a within-channel clipping (CL) test to detect amplitude clipped EPs in each channel, and a multi-channel EP median distance (MC-MED) test to detect atypical signals not identified by the STD and CL tests. Because the MC-MED test is class-dependent, a novel “pre-testing” approach is developed to identify artifacts in real-time classification experiments. The performance of the artifact detection strategy is demonstrated on real single-trial EP ensembles and it is shown that the strategy is quite effective in identifying atypical EPs. In order to demonstrate the effects of artifacts on classifier performance, a series of classification experiments are designed using a multi-channel decision fusion classification algorithm. Specifically, the classification performance is evaluated on (a) real EP ensembles with artifact contaminations in the training and test sets and (b) ensembles that are free of artifacts in both the training and test sets. It is shown that the improvement in classification accuracy through the incorporation of the artifact detection strategy can be quite significant in real-time classification trials. Furthermore, the generalized formulation of the artifact rejection classification strategy makes it adaptable to various other problems involving the multi-class classification of multivariate signals of multiple sensors. [Copyright &y& Elsevier]

Abstract: The majority of the available classification systems focus on the minimization of the classification error rate. This is not always a suitable metric specially when dealing with two-class problems with skewed classes and cost distributions. In this case, an effective criterion to measure the quality of a decision rule is the area under the Receiver Operating Characteristic curve (AUC) that is also useful to measure the ranking quality of a classifier as required in many real applications. In this paper we propose a nonparametric linear classifier based on the maximization of AUC. The approach lies on the analysis of the Wilcoxon–Mann–Whitney statistic of each single feature and on an iterative pairwise coupling of the features for the optimization of the ranking of the combined feature. By the pairwise feature evaluation the proposed procedure is essentially different from other classifiers using AUC as a criterion. Experiments performed on synthetic and real data sets and comparisons with previous approaches confirm the effectiveness of the proposed method. [Copyright &y& Elsevier]

Abstract: There is a wish to be able to enter text into mobile computing devices at the speed of speech. Only handwritten shorthand schemes can achieve this data recording rate. A new, overall solution to the segmentation and recognition of phonetic features in Pitman shorthand is proposed in this paper. Approaches to the recognition of consonant outlines, vowel and diphthong symbols and shortforms, which are different components of Pitman shorthand, are presented. A new rule is introduced to solve the issue of smooth junctions in the consonant outlines which was normally the bottleneck for recognition. Experiments with a set of 1127 consonant outlines, 2039 vowels and diphthongs and 841 shortforms from three shorthand writers have demonstrated that the proposed solution is quite promising. The recognition accuracies for consonant outlines, vowels and diphthongs, and shortforms achieved 75.33%, 96.86% and 91.86%, respectively. From the evaluation of 461 outlines with smooth junction, the introduction of the new rule has a great positive effect on the performance of the solution. The recognition accuracy of smooth junction improves from 37.53% to 93.41% given a writing time increase of 14.42%. [Copyright &y& Elsevier]

Abstract: Head pose is an important indicator of a person''s focus of attention. Also, head pose estimation can be used as the front-end analysis for multi-view face analysis. For example, face recognition and identification algorithms are usually view dependent. Pose classification can help such face recognition systems to select the best view model. Subspace analysis has been widely used for head pose estimation. However, such techniques are usually sensitive to data alignment and background noise. In this paper a two-stage approach is proposed to address this issue by combining the subspace analysis together with the topography method. The first stage is based on the subspace analysis of Gabor wavelets responses. Different subspace techniques were compared for better exploring the underlying data structure. Nearest prototype matching with Euclidean distance was used to get the pose estimate. The single pose estimate was relaxed to a subset of poses around it to incorporate certain tolerance to data alignment and background noise. In the second stage, the pose estimate is refined by analyzing finer geometrical structure details captured by bunch graphs. This coarse-to-fine framework was evaluated with a large data set. We examined 86 poses, with the pan angle spanning from to and the tilt angle spanning from to . The experimental results indicate that the integrated approach has a remarkably better performance than using subspace analysis alone. [Copyright &y& Elsevier]

Abstract: We have developed an informative sample subspace (ISS) method that is suitable for projecting high-dimensional data onto a low-dimensional subspace for classification purposes. In this paper, we present an ISS algorithm that uses a maximal mutual information criterion to search a labelled training data set directly for the subspace''s projection base vectors. We evaluate the usefulness of the ISS method using synthetic data as well as real world problems. Experimental results demonstrate that the ISS algorithm is effective and can be used as a general method for representing high-dimensional data in a low-dimensional subspace for classification. [Copyright &y& Elsevier]

Abstract: Based on the geometric interpretation of support vector machines (SVMs), this paper presents a general technique that allows almost all the existing -norm penalty based geometric algorithms, including Gilbert''s algorithm, Schlesinger–Kozinec’s (SK) algorithm and Mitchell–Dem’yanov–Malozemov’s (MDM) algorithm, to be softened to achieve the corresponding learning -SVM classifiers. Intrinsically, the resulting soft algorithms are to find -optimal nearest points between two soft convex hulls. Theoretical analysis has indicated that our proposed soft algorithms are essentially generalizations of the corresponding existing hard algorithms, and consequently, they have the same properties of convergence and almost the identical cost of computation. As a specific example, the problem of solving -SVMs by the proposed soft MDM algorithm is investigated and the corresponding solution procedure is specified and analyzed. To validate the general soft technique, several real classification experiments are conducted with the proposed -norm based MDM algorithms and numerical results have demonstrated that their performance is competitive to that of the corresponding -norm based algorithms, such as SK and MDM algorithms. [Copyright &y& Elsevier]

Abstract: This paper describes an algorithm for the recognition of the type of grip being formed by a subject wearing an intelligent orthotic glove. The signal is a growing multivariate sequence to be correctly classified as soon as the data permit, so the task may be called one of sequential classification or sequential discrimination. The algorithm uses a variant of the ‘ nearest neighbours’ principle with the dissimilarity measure being an exponentially weighted moving average of distances of the Mahalanobis type. The approach taken seems suitable for other problems of this kind. [Copyright &y& Elsevier]

Abstract: Automated chromosome classification is an essential task in cytogenetics and has been an important pattern recognition problem. Numerous attempts were made in the past to characterize chromosomes for the purposes of clinical and cancer cytogenetics research. It is important to determine good features and develop feature extraction schemes for chromosome classification. In this paper we propose efficient approaches for medial axis determination and profile matching of human chromosomes without identifying centromeres. The medial axis determination is based on simple cross-section analysis. The features of the band profile obtained along the axis are then used to classify a chromosome based on a subsequence matching technique. Using a special indexing structure, we are able to perform fast similarity search and dynamic insertion and deletion over the established subsequence database of chromosome profiles. According to the experimental results, the developed adaptive system can automatically and efficiently determine the medial axis of a given chromosomes, and achieve satisfactory classification results. [Copyright &y& Elsevier]

Abstract: This paper presents a novel cascade ensemble classifier system for the recognition of handwritten digits. This new system aims at attaining a very high recognition rate and a very high reliability at the same time, in other words, achieving an excellent recognition performance of handwritten digits. The trade-offs among recognition, error, and rejection rates of the new recognition system are analyzed. Three solutions are proposed: (i) extracting more discriminative features to attain a high recognition rate, (ii) using ensemble classifiers to suppress the error rate and (iii) employing a novel cascade system to enhance the recognition rate and to reduce the rejection rate. Based on these strategies, seven sets of discriminative features and three sets of random hybrid features are extracted and used in the different layers of the cascade recognition system. The novel gating networks (GNs) are used to congregate the confidence values of three parallel artificial neural networks (ANNs) classifiers. The weights of the GNs are trained by the genetic algorithms (GAs) to achieve the overall optimal performance. Experiments conducted on the MNIST handwritten numeral database are shown with encouraging results: a high reliability of 99.96% with minimal rejection, or a 99.59% correct recognition rate without rejection in the last cascade layer. [Copyright &y& Elsevier]

Abstract: In machine learning, class noise occurs frequently and deteriorates the classifier derived from the noisy data set. This paper presents two promising classifiers for this problem based on a probabilistic model proposed by Lawrence and Schölkopf (2001). The proposed algorithms are able to tolerate class noise, and extend the earlier work of Lawrence and Schölkopf in two ways. First, we present a novel incorporation of their probabilistic noise model in the Kernel Fisher discriminant; second, the distribution assumption previously made is relaxed in our work. The methods were investigated on simulated noisy data sets and a real world comparative genomic hybridization (CGH) data set. The results show that the proposed approaches substantially improve standard classifiers in noisy data sets, and achieve larger performance gain in non-Gaussian data sets and small size data sets. [Copyright &y& Elsevier]

Abstract: In this article, we describe a new method of extracting information from signals, called functional dissipation, that proves to be very effective for enhancing classification of high resolution, texture-rich data. Our algorithm bypasses to some extent the need to have very specialized feature extraction techniques, and can potentially be used as an intermediate, feature enhancement step in any classification scheme. Functional dissipation is based on signal transforms, but uses the transforms recursively to uncover new features. We generate a variety of masking functions and ‘extract’ features with several generalized matching pursuit iterations. In each iteration, the recursive process modifies several coefficients of the transformed signal with the largest absolute values according to the specific masking function; in this way the greedy pursuit is turned into a slow, controlled, dissipation of the structure of the signal that for some masking functions, enhances separation among classes. Our case study in this paper is the classification of crystallization patterns of amino acids solutions affected by the addition of small quantities of proteins. [Copyright &y& Elsevier]

Abstract: Classification of data with imbalanced class distribution has posed a significant drawback of the performance attainable by most standard classifier learning algorithms, which assume a relatively balanced class distribution and equal misclassification costs. The significant difficulty and frequent occurrence of the class imbalance problem indicate the need for extra research efforts. The objective of this paper is to investigate meta-techniques applicable to most classifier learning algorithms, with the aim to advance the classification of imbalanced data. The AdaBoost algorithm is reported as a successful meta-technique for improving classification accuracy. The insight gained from a comprehensive analysis of the AdaBoost algorithm in terms of its advantages and shortcomings in tacking the class imbalance problem leads to the exploration of three cost-sensitive boosting algorithms, which are developed by introducing cost items into the learning framework of AdaBoost. Further analysis shows that one of the proposed algorithms tallies with the stagewise additive modelling in statistics to minimize the cost exponential loss. These boosting algorithms are also studied with respect to their weighting strategies towards different types of samples, and their effectiveness in identifying rare cases through experiments on several real world medical data sets, where the class imbalance problem prevails. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a new discriminant analysis using composite features for pattern classification. A composite feature consists of a number of primitive features, each of which corresponds to an input variable. The covariance of composite features is obtained from the inner product of composite features and can be considered as a generalized form of the covariance of primitive features. It contains information on statistical dependency among multiple primitive features. A discriminant analysis (C-LDA) using the covariance of composite features is a generalization of the linear discriminant analysis (LDA). Unlike LDA, the number of extracted features can be larger than the number of classes in C-LDA, which is a desirable property especially for binary classification problems. Experimental results on several data sets indicate that C-LDA provides better classification results than other methods based on primitive features. [Copyright &y& Elsevier]

Abstract: This paper proposes a method called layered genetic programming (LAGEP) to construct a classifier based on multi-population genetic programming (MGP). LAGEP employs layer architecture to arrange multiple populations. A layer is composed of a number of populations. The results of populations are discriminant functions. These functions transform the training set to construct a new training set. The successive layer uses the new training set to obtain better discriminant functions. Moreover, because the functions generated by each layer will be composed to a long discriminant function, which is the result of LAGEP, every layer can evolve with short individuals. For each population, we propose an adaptive mutation rate tuning method to increase the mutation rate based on fitness values and remaining generations. Several experiments are conducted with different settings of LAGEP and several real-world medical problems. Experiment results show that LAGEP achieves comparable accuracy to single population GP in much less time. [Copyright &y& Elsevier]

Abstract: We have proposed a constrained linear discriminant analysis (CLDA) approach for classifying the remotely sensed hyperspectral images. Its basic idea is to design an optimal linear transformation operator which can maximize the ratio of inter-class to intra-class distance while satisfying the constraint that the different class centers after transformation are aligned along different directions. Its major advantage over the traditional Fisher''s linear discriminant analysis is that the classification can be achieved simultaneously with the transformation. The CLDA is a supervised approach, i.e., the class spectral signatures need to be known a priori. But, in practice, these informations may be difficult or even impossible to obtain. So in this paper we will extend the CLDA algorithm into an unsupervised version, where the class spectral signatures are to be directly generated from an unknown image scene. Computer simulation is used to evaluate how well the algorithm performs in terms of finding the pure signatures. We will also discuss how to implement the unsupervised CLDA algorithm in real-time for resolving the critical situations when the immediate data analysis results are required. [Copyright &y& Elsevier]

Abstract: In this paper, a novel center-based nearest neighbor (CNN) classifier is proposed to deal with the pattern classification problems. Unlike nearest feature line (NFL) method, CNN considers the line passing through a sample point with known label and the center of the sample class. This line is called the center-based line (CL). These lines seem to have more capacity of representation for sample classes than the original samples and thus can capture more information. Similar to NFL, CNN is based on the nearest distance from an unknown sample point to a certain CL for classification. As a result, the computation time of CNN can be shortened dramatically with less accuracy decrease when compared with NFL. The performance of CNN is demonstrated in one simulation experiment from computational biology and high classification accuracy has been achieved in the leave-one-out test. The comparisons with nearest neighbor (NN) classifier and NFL classifier indicate that this novel classifier achieves competitive performance. [Copyright &y& Elsevier]

Abstract: This paper focuses on the problem of how data representation influences the generalization error of kernel based learning machines like support vector machines (SVM) for classification. Frame theory provides a well founded mathematical framework for representing data in many different ways. We analyze the effects of sparse and dense data representations on the generalization error of such learning machines measured by using leave-one-out error given a finite amount of training data. We show that, in the case of sparse data representations, the generalization error of an SVM trained by using polynomial or Gaussian kernel functions is equal to the one of a linear SVM. This is equivalent to saying that the capacity of separating points of functions belonging to hypothesis spaces induced by polynomial or Gaussian kernel functions reduces to the capacity of a separating hyperplane in the input space. Moreover, we show that, in general, sparse data representations increase or leave unchanged the generalization error of kernel based methods. Dense data representations, on the contrary, reduce the generalization error in the case of very large frames. We use two different schemes for representing data in overcomplete systems of Haar and Gabor functions, and measure SVM generalization error on benchmarked data sets. [Copyright &y& Elsevier]

Abstract: Representation and embedding are usually the two necessary phases in designing a classifier. Fisher discriminant analysis (FDA) is regarded as seeking a direction for which the projected samples are well separated. In this paper, we analyze FDA in terms of representation and embedding. The main contribution is that we prove that the general framework of FDA is based on the simplest and most intuitive FDA with zero within-class variance and therefore the mechanism of FDA is clearly illustrated. Based on our analysis, -insensitive SVM regression can be viewed as a soft FDA with -insensitive within-class variance and norm penalty. To verify this viewpoint, several real classification experiments are conducted to demonstrate that the performance of the regression-based classification technique is comparable to regular FDA and SVM. [Copyright &y& Elsevier]

Abstract: This paper presents a new fingerprint singular point detection method that is type-distinguishable and applicable to various fingerprint images regardless of their resolutions. The proposed method detects singular points by analyzing the shapes of the local directional fields of a fingerprint image. Using the predefined rules, all types of singular points (upper core, lower core, and delta points) can be extracted accurately and delineated in terms of the type of singular points. In case of arch-type fingerprints there exists no singular point, but reference points for arch-type fingerprints are required to be detected for registration. Therefore, we propose a new reference point detection method for arch-type fingerprints as well. The result of the experiments on the two public databases (FVC2000 2a, FVC2002 2a) with different resolutions demonstrates that the proposed method has high accuracy in locating each types of singular points and detecting the reference points of arch-type fingerprints without regard to their image resolutions. [Copyright &y& Elsevier]

Abstract: The aims of this paper are two-fold: to define Gaussian mixture models (GMMs) of colored texture on several feature spaces and to compare the performance of these models in various classification tasks, both with each other and with other models popular in the literature. We construct GMMs over a variety of different color and texture feature spaces, with a view to the retrieval of textured color images from databases. We compare supervised classification results for different choices of color and texture features using the Vistex database, and explore the best set of features and the best GMM configuration for this task. In addition we introduce several methods for combining the ‘color’ and ‘structure’ information in order to improve the classification performances. We then apply the resulting models to the classification of texture databases and to the classification of man-made and natural areas in aerial images. We compare the GMM model with other models in the literature, and show an overall improvement in performance. [Copyright &y& Elsevier]

Abstract: Acoustic events produced in controlled environments may carry information useful for perceptually aware interfaces. In this paper we focus on the problem of classifying 16 types of meeting-room acoustic events. First of all, we have defined the events and gathered a sound database. Then, several classifiers based on support vector machines (SVM) are developed using confusion matrix based clustering schemes to deal with the multi-class problem. Also, several sets of acoustic features are defined and used in the classification tests. In the experiments, the developed SVM-based classifiers are compared with an already reported binary tree scheme and with their correlative Gaussian mixture model (GMM) classifiers. The best results are obtained with a tree SVM-based classifier that may use a different feature set at each node. With it, a 31.5% relative average error reduction is obtained with respect to the best result from a conventional binary tree scheme. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a prototype classification method that employs a learning process to determine both the number and the location of prototypes. This learning process decides whether to stop adding prototypes according to a certain termination condition, and also adjusts the location of prototypes using either the K-means (KM) or the fuzzy c-means (FCM) clustering algorithms. When the prototype classification method is applied, the support vector machine (SVM) method can be used to post-process the top-rank candidates obtained during the prototype learning or matching process. We apply this hybrid solution to handwriting recognition and address the convergence behavior and runtime consumption of the prototype construction process, and discuss how to combine our prototype classifier with SVM classifiers to form an effective hybrid classifier. [Copyright &y& Elsevier]

Abstract: Kernel Matching Pursuit Classifier (KMPC), a novel classification machine in pattern recognition, has an excellent advantage in solving classification problems for the sparsity of the solution. Unfortunately, the performance of the KMPC is far from the theoretically expected level of it. Ensemble Methods are learning algorithms that construct a collection of individual classifiers which are independent and yet accurate, and then classify a new data point by taking vote of their predictions. In such a way, the performance of classifiers can be improved greatly. In this paper, on a thorough investigation into the principle of KMPC and Ensemble Method, we expatiate on the theory of KMPC ensemble and pointed out the ways to construct it. The experiments performed on the artificial data and UCI data show KMPC ensemble combines the advantages of KMPC with ensemble method, and improves classification performance remarkably. [Copyright &y& Elsevier]

Abstract: The polynomial classifier (PC) that takes the binomial terms of reduced subspace features as inputs has shown superior performance to multilayer neural networks in pattern classification. In this paper, we propose a class-specific feature polynomial classifier (CFPC) that extracts class-specific features from class-specific subspaces, unlike the ordinary PC that uses a class-independent subspace. The CFPC can be viewed as a hybrid of ordinary PC and projection distance method. The class-specific features better separate one class from the others, and the incorporation of class-specific projection distance further improves the separability. The connecting weights of CFPC are efficiently learned class-by-class to minimize the mean square error on training samples. To justify the promise of CFPC, we have conducted experiments of handwritten digit recognition and numeral string recognition on the NIST Special Database 19 (SD19). The digit recognition task was also benchmarked on two standard databases USPS and MNIST. The results show that the performance of CFPC is superior to that of ordinary PC, and is competitive with support vector classifiers (SVCs). [Copyright &y& Elsevier]

Abstract: This paper is about building an ensemble of classifiers each of which is trained based on a particular weighting over the training examples (a weighting is a set of weights associated with the examples). The task concerns search in a tremendous weighting space. In this view we propose to incorporate a genetic algorithm (GA). It performs a wide yet efficient search for appropriate weightings (chromosomes). The difference from a traditional GA is that all the weightings throughout evolution will be exploited to form the final ensemble, not just the best weighting. Our algorithm is tested on the UCI benchmark data sets and used to design a face detection system. Robust and consistently accurate classification is experienced. Comparative results with two other algorithms, i.e. AdaBoost and Bagging, are also given. [Copyright &y& Elsevier]

Abstract: Nearest neighbor (NN) classification assumes locally constant class conditional probabilities, and suffers from bias in high dimensions with a small sample set. In this paper, we propose a novel cam weighted distance to ameliorate the curse of dimensionality. Different from the existing neighborhood-based methods which only analyze a small space emanating from the query sample, the proposed nearest neighbor classification using the cam weighted distance (CamNN) optimizes the distance measure based on the analysis of inter-prototype relationship. Our motivation comes from the observation that the prototypes are not isolated. Prototypes with different surroundings should have different effects in the classification. The proposed cam weighted distance is orientation and scale adaptive to take advantage of the relevant information of inter-prototype relationship, so that a better classification performance can be achieved. Experiments show that CamNN significantly outperforms one nearest neighbor classification (1-NN) and -nearest neighbor classification (-NN) in most benchmarks, while its computational complexity is comparable with that of 1-NN classification. [Copyright &y& Elsevier]

Abstract: Given a large set of potential features, it is usually necessary to find a small subset with which to classify. The task of finding an optimal feature set is inherently combinatoric and therefore suboptimal algorithms are typically used to find feature sets. If feature selection is based directly on classification error, then a feature-selection algorithm must base its decision on error estimates. This paper addresses the impact of error estimation on feature selection using two performance measures: comparison of the true error of the optimal feature set with the true error of the feature set found by a feature-selection algorithm, and the number of features among the truly optimal feature set that appear in the feature set found by the algorithm. The study considers seven error estimators applied to three standard suboptimal feature-selection algorithms and exhaustive search, and it considers three different feature-label model distributions. It draws two conclusions for the cases considered: (1) depending on the sample size and the classification rule, feature-selection algorithms can produce feature sets whose corresponding classifiers possess errors far in excess of the classifier corresponding to the optimal feature set; and (2) for small samples, differences in performances among the feature-selection algorithms are less significant than performance differences among the error estimators used to implement the algorithms. Moreover, keeping in mind that results depend on the particular classifier-distribution pair, for the error estimators considered in this study, bootstrap and bolstered resubstitution usually outperform cross-validation, and bolstered resubstitution usually performs as well as or better than bootstrap. [Copyright &y& Elsevier]

Abstract: Hybrid wavelet–large margin classifiers have recently proven to solve difficult signal classification problems in cases where solely using a large margin classifier like, e.g., the Support Vector Machine may fail. In this paper, we evaluate several criteria rating feature sets obtained from various orthogonal filter banks for the classification by a Support Vector Machine. Appropriate criteria may then be used for adapting the wavelet filter with respect to the subsequent support vector classification. Our results show that criteria which are computationally more efficient than the radius-margin Support Vector Machine error bound are sufficient for our filter adaptation and, hence, feature selection. Further, we propose an adaptive search algorithm that, once the criterion is fixed, efficiently finds the optimal wavelet filter. As an interesting byproduct we prove a theorem which allows the computation of the radius of a set of vectors by a standard Support Vector Machine. [Copyright &y& Elsevier]

Abstract: This paper proposes a new technique for the classification of indoor and outdoor images based on edge analysis. Our technique is based on analysing edge straightness in images. We make an original proposal that indoor images have a greater proportion of edges that are straight compared to outdoor images, and use multi-resolution estimates on edge straightness to improve our results. We also consider this method''s possible applications in a real-time system. We compare our proposed technique with a number of other approaches that have been published on indoor/outdoor classification of images and convincingly show on a large database that our method generates much higher accuracy. [Copyright &y& Elsevier]

Abstract: Qualitatively, a filter is said to be “robust” if its performance degradation is acceptable for distributions close to the one for which it is optimal, that is, the one for which it has been designed. This paper adapts the signal-processing theory of optimal robust filters to classifiers. The distribution (class conditional distributions) to which the classifier is to be applied is parameterized by a state vector and the principle issue is to choose a design state that is optimal in comparison to all other states relative to some measure of robustness. A minimax robust classifier is one whose worst performance over all states is better than the worst performances of the other classifiers (defined at the other states). A Bayesian robust classifier is one whose expected performance is better than the expected performances of the other classifiers. The state corresponding to the Bayesian robust classifier is called the maximally robust state. Minimax robust classifiers tend to give too much weight to states for which classification is very difficult and therefore our effort is focused on Bayesian robust classifiers. Whereas the signal-processing theory of robust filtering concentrates on design with full distributional knowledge and a fixed number of observation variables (features), design via training from sample data and feature selection are so important for classification that robustness optimality must be considered from these perspectives—in particular, for small samples. In this context, for a given sample size, we will be concerned with the maximally robust state-feature pair. All definitions are independent of the classification rule; however, applications are only considered for linear and quadratic discriminant analysis, for which there are parametric forms for the optimal discriminants. [Copyright &y& Elsevier]

Abstract: Genetic programming (GP) is used to evolve secondary classifiers for disambiguating between pairs of handwritten digit images. The inherent property of feature selection accorded by GP is exploited to make sharper decision between conflicting classes. Classification can be done in several steps with an available feature set and a mixture of strategies. A two-step classification strategy is presented in this paper. After the first step of the classification using the full feature set, the high confidence recognition result will lead to an end of the recognition process. Otherwise a secondary classifier designed using a sub-set of the original feature set and the information available from the earlier classification step will help classify the input further. The feature selection mechanism employed by GP selects important features that provide maximum separability between classes under consideration. In this way, a sharper decision on fewer classes is obtained at the secondary classification stage. The full feature set is still available in both stages of classification to retain complete information. An intuitive motivation and detailed analysis using confusion matrices between digit classes is presented to describe how this strategy leads to improved recognition performance. In comparison with the existing methods, our method is aimed for increasing recognition accuracy and reliability. Results are reported for the BHA test-set and the NIST test-set of handwritten digits. [Copyright &y& Elsevier]

Abstract: In this paper, we investigate the practical implementation issues of the real-time constrained linear discriminant analysis (CLDA) approach for remotely sensed image classification. Specifically, two issues are to be resolved: (1) what is the best implementation scheme that yields lowest chip design complexity with comparable classification performance, and (2) how to extend CLDA algorithm for multispectral image classification. Two limitations about data dimensionality have to be relaxed. One is in real-time hyperspectral image classification, where the number of linearly independent pixels received for classification must be larger than the data dimensionality (i.e., the number of spectral bands) in order to generate a non-singular sample correlation matrix R for the classifier, and relaxing this limitation can help to resolve the aforementioned first issue. The other is in multispectral image classification, where the number of classes to be classified cannot be greater than the data dimensionality, and relaxing this limitation can help to resolve the aforementioned second issue. The former can be solved by introducing a pseudo inverse initiate of sample correlation matrix for adaptation, and the latter is taken care of by expanding the data dimensionality via the operation of band multiplication. Experiments on classification performance using these modifications are conducted to demonstrate their feasibility. All these investigations lead to a detailed ASIC chip design scheme for the real-time CLDA algorithm suitable to both hyperspectral and multispectral images. The proposed techniques to resolving these two dimensionality limitations are instructive to the real-time implementation of several popular detection and classification approaches in remote sensing image exploitation. [Copyright &y& Elsevier]

Hidden Markov models (HMM) are a widely used tool for sequence modelling. In the sequence classification case, the standard approach consists of training one HMM for each class and then using a standard Bayesian classification rule. In this paper, we introduce a novel classification scheme for sequences based on HMMs, which is obtained by extending the recently proposed similarity-based classification paradigm to HMM-based classification. In this approach, each object is described by the vector of its similarities with respect to a predetermined set of other objects, where these similarities are supported by HMMs. A central problem is the high dimensionality of resulting space, and, to deal with it, three alternatives are investigated. Synthetic and real experiments show that the similarity-based approach outperforms standard HMM classification schemes. [Copyright &y& Elsevier]

This paper presents a learning scheme for data classification based on genetic programming. The proposed learning approach consists of an adaptive incremental learning strategy and distance-based fitness functions for generating the discriminant functions using genetic programming. To classify data using the discriminant functions effectively, the mechanism called -value measure is developed. Based on the -value measure, we give two classification algorithms to resolve ambiguity among the discriminant functions. The experiments show that the proposed approach has less training time than previous GP learning methods. The learned classifiers also have high accuracy of classification in comparison with the previous classifiers. [Copyright &y& Elsevier]

A Support Vector Classifier (SVC) is formulated in terms of a kernel. The bandwidth of the kernel affects the generalization performance of the SVC. This paper presents a Leave One Support Vector Out Cross Validation (LOSVO-CV) algorithm for estimating the optimal bandwidth of the kernel for classification purpose. The proposed algorithm is based on the Leave One Out Cross Validation (LOO-CV) algorithm (Numer. Math. 31 (1979) 377) that was proposed to find the optimal bandwidth but difficult to be implemented due to its large amount of computation. The properties of LOSVO-CV are analyzed in comparison with the LOO-CV. The simulation study demonstrates that the LOSVO-CV is a fast algorithm and it has the same generalization performance optimized by a bootstrap method (Neural Process. Lett. 11 (2000) 51) which can find an optimal bandwidth of the kernel of the SVC. The LOSVO-CV algorithm is able to provide consistent results with different sizes of a benchmark data set which is obtained from the University of California (UCI) repository. [Copyright &y& Elsevier]

In this paper, we propose an Independent Component Analysis (ICA) based face recognition algorithm, which is robust to illumination and pose variation. Generally, it is well known that the first few eigenfaces represent illumination variation rather than identity. Most Principal Component Analysis (PCA) based methods have overcome illumination variation by discarding the projection to a few leading eigenfaces. The space spanned after removing a few leading eigenfaces is called the “residual face space”. We found that ICA in the residual face space provides more efficient encoding in terms of redundancy reduction and robustness to pose variation as well as illumination variation, owing to its ability to represent non-Gaussian statistics. Moreover, a face image is separated into several facial components, local spaces, and each local space is represented by the ICA bases (independent components) of its corresponding residual space. The statistical models of face images in local spaces are relatively simple and facilitate classification by a linear encoding. Various experimental results show that the accuracy of face recognition is significantly improved by the proposed method under large illumination and pose variations. [Copyright &y& Elsevier]

Prior research in scene classification has focused on mapping a set of classic low-level vision features to semantically meaningful categories using a classifier engine. In this paper, we propose improving the established paradigm by using a simplified low-level feature set to predict multiple semantic scene attributes that are integrated probabilistically to obtain a final indoor/outdoor scene classification. An initial indoor/outdoor prediction is obtained by classifying computationally efficient, low-dimensional color and wavelet texture features using support vector machines. Similar low-level features can also be used to explicitly predict the presence of semantic features including grass and sky. The semantic scene attributes are then integrated using a Bayesian network designed for improved indoor/outdoor scene classification. [Copyright &y& Elsevier]

When choosing a classification rule, it is important to take into account the amount of sample data available. This paper examines the performances of classifiers of differing complexities in relation to the complexity of feature-label distributions in the case of small samples. We define the distributional complexity of a feature-label distribution to be the minimal number of hyperplanes necessary to achieve the Bayes classifier if the Bayes classifier is achievable by a finite number of hyperplanes, and infinity otherwise. Our approach is to choose a model and compare classifier efficiencies for various sample sizes and distributional complexities. Simulation results are obtained by generating data based on the model and the distributional complexities. A linear support vector machine (SVM) is considered, along with several nonlinear classifiers. For the most part, we see that there is little improvement when one uses a complex classifier instead of a linear SVM. For higher levels of distributional complexity, the linear classifier degrades, but so do the more complex classifiers owing to insufficient training data. Hence, if one were to obtain a good result with a more complex classifier, it is most likely that the distributional complexity is low and there is no gain over using a linear classifier. Hence, under the model, it is generally impossible to claim that use of the nonlinear classifier is beneficial. In essence, the sample sizes are too small to take advantage of the added complexity. An exception to this observation is the behavior of the three-nearest-neighbor (3NN) classifier in the case of two variables (but not three) when there is very little overlap between the label distributions and the sample size is not too small. With a sample size of 60, the 3NN classifier performs close to the Bayes classifier, even for high levels of distributional complexity. Consequently, if one uses the 3NN classifier with two variables and obtains a low error, then the distributional complexity might be large and, if such is the case, there is a significant gain over using a linear classifier. [Copyright &y& Elsevier]

Current approaches to color texture analysis can be roughly divided into two categories: methods that process color and texture information separately, and those that consider color and texture a joint phenomenon. In this paper, both approaches are empirically evaluated with a large set of natural color textures. The classification performance of color indexing methods is compared to gray-scale and color texture methods, and to combined color and texture methods, in static and varying illumination conditions. Based on the results, we argue that color and texture are separate phenomena that can, or even should, be treated individually. [Copyright &y& Elsevier]

We propose a general method for error estimation that displays low variance and generally low bias as well. This method is based on “bolstering” the original empirical distribution of the data. It has a direct geometric interpretation and can be easily applied to any classification rule and any number of classes. This method can be used to improve the performance of any error-counting estimation method, such as resubstitution and all cross-validation estimators, particularly in small-sample settings. We point out some similarities shared by our method with a previously proposed technique, known as smoothed error estimation. In some important cases, such as a linear classification rule with a Gaussian bolstering kernel, the integrals in the bolstered error estimate can be computed exactly. In the general case, the bolstered error estimate may be computed by Monte-Carlo sampling; however, our experiments show that a very small number of Monte-Carlo samples is needed. This results in a fast error estimator, which is in contrast to other resampling techniques, such as the bootstrap. We provide an extensive simulation study comparing the proposed method with resubstitution, cross-validation, and bootstrap error estimation, for three popular classification rules (linear discriminant analysis, k-nearest-neighbor, and decision trees), using several sample sizes, from small to moderate. The results indicate the proposed method vastly improves on resubstitution and cross-validation, especially for small samples, in terms of bias and variance. In that respect, it is competitive with, and in many occasions superior to, bootstrap error estimation, while being tens to hundreds of times faster. We provide a companion web site, which contains: (1) the complete set of tables and plots regarding the simulation study, and (2) C source code used to implement the bolstered error estimators proposed in this paper, as part of a larger library for classification and error estimation, with full documentation and examples. The companion web site can be accessed at the URL http://ee.tamu.edu/~edward/bolster. [Copyright &y& Elsevier]

In this paper we present a multi-scale method based on mathematical morphology which can successfully be used in pattern classification tasks. A connected operator similar to the morphological hat-transform is defined, and two scale-space representations are built. The most important features are extracted from the scale spaces by unsupervised cluster analysis, and the resulting pattern vectors provide the input of a decision tree classifier. We report classification results obtained using contour features, texture features, and a combination of these. The method has been tested on two large sets, a database of diatom images and a set of images from the Brodatz texture database. For the diatom images, the method is applied twice, once on the curvature of the outline (contour), and once on the grey-scale image itself. [Copyright &y& Elsevier]

This paper outlines a philosophical and psycho-physiological basis for embodied perception, and develops a framework for conceptual embodiment of vision-guided robots. We argue that categorisation is important in all stages of robot vision. Further, that classical computer vision is unsuitable for this categorisation, however, through conceptual embodiment, active perception can be effective. We present a methodology for developing vision-guided robots that applies embodiment, explicitly and implicitly, in categorising visual data to facilitate efficient perception and action. Finally, we present systems developed using this methodology, and demonstrate that embodied categorisation can make algorithms more efficient and robust. [Copyright &y& Elsevier]

Face detection from cluttered images is challenging due to the wide variability of face appearances and the complexity of image backgrounds. This paper proposes a classification-based method for locating frontal faces in cluttered images. To improve the detection performance, we extract gradient direction features from local window images as the input of the underlying two-class classifier. The gradient direction representation provides better discrimination ability than the image intensity, and we show that the combination of gradient directionality and intensity outperforms the gradient feature alone. The underlying classifier is a polynomial neural network (PNN) on a reduced feature subspace learned by principal component analysis (PCA). The incorporation of the residual of subspace projection into the PNN was shown to improve the classification performance. The classifier is trained on samples of face and non-face images to discriminate between the two classes. The superior detection performance of the proposed method is justified in experiments on a large number of images. [Copyright &y& Elsevier]