Search

Texture analysis and classification remain as one of the biggest challenges for the field of computer vision and pattern recognition. This article presents a robust hybrid combination technique to build a combined classifier that is able to tackle the problem of classification of rotation-invariant 2D textures. Diversity in the components of the combined classifier is enforced through variation of the parameters related to both architecture design and training stages of a neural network classifier. The boosting algorithm is used to make perturbation of the training set using Multi-Layer Perceptron MLP as the base classifier. The final decision of the proposed combined classifier is based on the majority voting. Experiments’ results on a standard benchmark database of rotated textures show that the proposed hybrid combination method is very robust, and it presents an excellent texture discrimination for all considered classes, overcoming traditional texture modification methods.

This study proposes a new method for content-based image retrieval by finding an optimal classifier. The optimal classifier is achieved by a new active learning support vector machine (SVM) which combines the model selection with the active learning. The unlabelled samples close to the boundary of the SVM classifier are selected based on the feature similarity for the active learning, and the adaptive regularisation is used to select the optimal model. The combination of model selection with active learning accelerates the convergence of the classifier. The new method can improve the image retrieval accuracy and reduce the time consumption. The experimental results show that the proposed method has a better performance with fewer samples and less time consumption for image retrieval.

This paper proposes a simple yet effective novel classifier fusion strategy for multi-class texture classification. The resulting classification framework is named as Classification Confidence-based Multiple Classifier Approach (CCMCA). The proposed training based scheme fuses the decisions of two base classifiers (those constitute the classifier ensemble) using their classification confidence to enhance the final classification accuracy. 4-fold cross validation approach is followed to perform experiments on four different texture databases those vary in terms of orientation, number of texture classes and complexity. Apart from its simplicity, the proposed CCMCA method shows better and consistent performance with lowest standard deviation as compared to fixed rule and simple trainable fusion techniques irrespective of the feature set used across all the databases used in the experiment. The performance gain of the proposed CCMCA method over other competing methods is found to be statistically significant.

This paper attempts to combine Laplacian Twin Support Vector Machine (Lap-TWSVM) and Decision Tree Twin Support Vector Machine classifier to obtain an effective tree based classifier for semi-supervised multi-category classification. This classifier is termed as Tree Based Multi-Category Laplacian Twin Support Vector Machine (TB-Lap-TWSVM) classifier. TB-Lap-TWSVM is an improved One Against All (OAA) partition based tree classifier which takes advantage of both Lap-TWSVM and Decision Tree methodology. This paper concentrates on the application of TB-Lap-TWSVM to Content Based Image Retrieval problem. Further, extensive experiments on color images are carried out on different databases to establish the efficacy of the proposed model vis a vis TB-Lap-SVM and OAA-Lap-TWSVM.

Support Vector Machine (SVM) have been very popular as a large margin classifier due its robust mathematical theory. It has many practical applications in a number of fields such as in bioinformatics, in medical science for diagnosis of diseases, in various engineering applications for prediction of model, in finance for forecasting etc. It is widely used in medical science because of its powerful learning ability in classification. It can classify highly nonlinear data using kernel function. This paper proposes and analyses diagnostic model to classify the most common skin illnesses and also provide a useful insight into the SVM algorithm. In rural areas where people are generally treated by paramedical staff, skin patients are not subject to proper diagnosis resulting in mistreatment. We think SVM is a good tool for proper diagnosis. This paper uses various kernels for classification and achieving the best accuracy of 95.39%.

Imbalanced learning for classification problems is the active area of research in machine learning. Many classification systems like image retrieval and credit scoring systems have imbalanced distribution of training data sets which causes performance degradation of the classifier. Re-sampling of imbalanced data is commonly used to handle imbalanced distribution as it is independent of the classifier being used. But sometimes they can remove necessary data of the class or can cause over-fitting. Classifier Ensembles have recently achieved more attention as effective technique to handle skewed data. The focus of the work is to gain advantages of both data level and classifier ensemble approach in order to improve the classification performance. We present a novel approach that initially applies pre-processing to the imbalanced dataset in order to reduce the imbalance between the classes. The pre-processed data is provided as training dataset to the classifier ensemble that introduces diversity by using different training datasets as well as different classifier models. The experimentation conducted on the eight imbalanced datasets from KEEL repository helps to prove the significance of the proposed method. A comparative analysis shows the performance improvement in terms of Area under ROC Curve (AUC).

In this paper, we propose a combining classifier method based on the Bayesian inference framework. Specifically, the outputs of base classifiers (called Level1 data or meta-data) are utilized in a combiner to produce the final classification. In our ensemble system, each class in the training set induces a distribution on the Level1 data, which is modeled by a multivariate Gaussian distribution. Traditionally, the parameters of the Gaussian are estimated using a maximum likelihood approach. However, maximum likelihood estimation cannot be applied since the covariance matrix of Level1 data of each class is not full rank. Instead, we propose to estimate the multivariate Gaussian distribution of Level1 data of each class by using the Variational Inference method. Experiments conducted on eighteen UCI Machine Learning Repository datasets and a selected 10-class CLEF2009 medical imaging database demonstrated the advantage of our method compared with several well-known ensemble methods. We proposed a Variational Inference method for multivariate Gaussian distribution estimation.We proposed a novel combining classifier method by employing Variational Inference on base classifiers' outputs.The proposed method is highly competitive to many state-of-the-arts ensemble classifier methods.

Objects being recognized may arrive continuously to a classifier in the form of data stream, therefore contemporary classification systems have to make a decision not only on the basis of the static data, but on the data in motion as well. Additionally, we would like to start a classifier exploitation as soon as possible, then the models which can improve their models during exportation are very desirable. Basically, we may produce the model on the basis a few learning objects only and then we use and improve the classifier when new data comes. This concept is still vibrant and may be used in the plethora of practical cases. Nevertheless, constructing such a system we should realize, that we have the limited resources (as memory and computational power) at our disposal. Additionally, during the exploitation of a classifier system the chosen characteristic of the classifier model may change within a time. This phenomena is called concept drift and may lead the deep deterioration of the classification performance. This work deals with the data stream classification with the presence of concept drift . We propose a novel classifier training algorithm based on the sliding windows approach, which allows us to implement forgetting mechanism, i.e., that old objects come from outdated model will not be taken into consideration during the classifier updating and on the other hand we assume that only part of arriving examples can be labeled, because we assume that we have a limited budget for labeling. We will employ active learning paradigm to choose an “interesting” objects to be be labeled. The proposed approach has been evaluated on the basis of the computer experiments carried out on the data streams. Obtained results confirmed the usability of proposed method to the smoothly drifted data stream classification.

This paper introduces an improvement on the recently published Hybrid Extreme Rotation Forest (HERF), consisting in the anticipative determination of the the fraction of each classifier architecture included in the ensemble. We call it AHERF. Both HERF and AHERF are heterogeneous classifier ensembles, which aim to profit from the diverse problem domain specificities of each classifier architecture in order to achieve improved generalization over a larger spectrum of problem domains. In this paper AHERF are built from a pool of Decision Trees (DT), Extreme Learning Machines (ELM), Support Vector Machines (SVM), k-Nearest Neighbors (k-NN), Adaboost, Random Forests (RF), and Gaussian Naive Bayes (GNB) classifiers. Given a problem dataset, the process of anticipative determination of the ensemble composition is as follows: First, we estimate the performance of each classifier architecture by independent pilot cross-validation experiments on a small subsample of the data. Next, classifier architectures are ranked according to their accuracy results. A probability distribution of classifier architectures appearing in the ensemble is built from this ranking. Finally, the type of each individual classifier is decided by sampling this probability distribution. Computational experiments on a collection of benchmark classification problems shows improvement on the original HERF, and other state-of-the-art approaches.

The Gamma associative classifier is among the most used classifiers of the alpha-beta associative approach. It had been used successfully to solve many Pattern Recognition tasks, including environmental applications. However, as most classifiers, Gamma suffers with the presence of noisy or mislabeled instances in the training sets. This paper evaluates the impact of using instance selection techniques in the performance of Gamma classifier. The numerical experiments carried out over well-known repository datasets allows to conclude that instance selection may increase the testing accuracy of the Gamma classifier.

This paper proposes a Multiclass Least Squares Twin Support Vector Machine (MLSTSVM) classifier for multi-class classification problems. The formulation of MLSTSVM is obtained by extending the formulation of recently proposed binary Least Squares Twin Support Vector Machine (LSTSVM) classifier. For M-class classification problem, the proposed classifier seeks M-non parallel hyper-planes, one for each class, by solving M-linear equations. A regularization term is also added to improve the generalization ability. MLSTSVM works well for both linear and non-linear type of datasets. It is relatively simple and fast algorithm as compared to the other existing approaches. The performance of proposed approach has been evaluated on twelve benchmark datasets. The experimental result demonstrates the validity of proposed MLSTSVM classifier as compared to the typical multi-classifiers based on ‘Support Vector Machine’ and ‘Twin Support Vector Machine’. Statistical analysis of the proposed classifier with existing classifiers is also performed by using Friedman’s Test statistic and Nemenyi post hoc techniques.

We propose a real-time system that continuously recognizes emotions from body movements. The combined low-level 3D postural features and high-level kinematic and geometrical features are fed to a Random Forests classifier through summarization (statistical values) or aggregation (bag of features). In order to improve the generalization capability and the robustness of the system, a novel semisupervised adaptive algorithm is built on top of the conventional Random Forests classifier. The MoCap UCLIC affective gesture database (labeled with four emotions) was used to train the Random Forests classifier, which led to an overall recognition rate of 78% using a 10-fold cross-validation. Subsequently, the trained classifier was used in a stream-based semisupervised Adaptive Random Forests method for continuous unlabeled Kinect data classification. The very low update cost of our adaptive classifier makes it highly suitable for data stream applications. Tests performed on the publicly available emotion datasets (body gestures and facial expressions) indicate that our new classifier outperforms existing algorithms for data streams in terms of accuracy and computational costs.

In this paper, a new classifier based on both sparse representation classification (SRC) and simple-fast representation-based classifier (SFR), called L1-norm plus L2-norm sparse parameter (L1L2-SP) classifier, is proposed for image recognition. The SRC classifier only utilizes the distance between test image vector and the L1-norm sparse representation for classification. The SFR classifier uses the test image vector and the nearest image vector of class subspace to classify the test sample. However, the L1L2-SP classifier firstly computes the L1-norm sparse parameter and L1-norm sparse representation, which are used to compose a novel robust global space. Then the L1L2-SP utilizes the novel global space to get the L2-norm sparse parameter. In the last, L1L2-SP uses the sum of L1-norm sparse parameter and L2-norm sparse parameter for classification. A mass number of experiments on coil100 object database, eth80 object database, Oxford flower 17 database and FERET face database are used to assess the proposed classifier. The experimental results prove that the proposed approach achieves better recognition rate than the SRC classifier, SFR classifier, and several other classifiers.

In this paper, a new classifier based on coarse-to-fine face recognition (CFFR) method, called robust coarse-to-fine face recognition (RCFFR) classifier, is proposed for face recognition. The CCFR classifier first exploits the determined linear combination to coarsely determine candidate class labels of the test sample. Afterwards, the CCFR classifier utilizes the candidate classes to classify the test sample. However, the RCFFR classifier not only uses candidate classes, but also utilizes the original classes to gain the robust distance between the test sample and the class subspace for classification. A mass number of experiments on AR face database, Extended YaleB face database and PIE face database are used to assess the proposed classifier. The experimental results prove that the proposed approach achieves higher recognition rate than the CFFR classifier, and several state-of-the-art classifiers.

Support vector machine (SVM) is a machine learning method based on statistical learning theory. It has a lot of advantages, such as solid theoretical foundation, global optimization, the sparsity of the solution, nonlinear and generalization. The standard form of SVM only applies to supervised learning. Large amount of data generated in real life is unlabeled, and the standard form of SVM cannot make good use of these data to improve its learning ability. However, semi-supervised support vector machine (S3VM) is a good solution to this problem. This paper reviews the recent progress in semi-supervised support vector machine. First, the basic theory of S3VM is expounded and discussed in detail; then, the mainstream model of S3VM is presented, including transductive support vector machine, Laplacian support vector machine, S3VM training via the label mean, S3VM based on cluster kernel; finally, we give the conclusions and look ahead to the research on S3VM.

A classifier ensemble combines a set of individual classifier's predictions to produce more accurate results than that of any single classifier system. However, one classifier ensemble with too many classifiers may consume a large amount of computational time. This paper proposes a new ensemble subset evaluation method that integrates classifier diversity measures into a novel classifier ensemble reduction framework. The framework converts the ensemble reduction into an optimization problem and uses the harmony search algorithm to find the optimized classifier ensemble. Both pairwise and non-pairwise diversity measure algorithms are applied by the subset evaluation method. For the pairwise diversity measure, three conventional diversity algorithms and one new diversity measure method are used to calculate the diversity's merits. For the non-pairwise diversity measure, three classical algorithms are used. The proposed subset evaluation methods are demonstrated by the experimental data. In comparison with other classifier ensemble methods, the method implemented by the measurement of the interrater agreement exhibits a high accuracy prediction rate against the current ensembles' performance. In addition, the framework with the new diversity measure achieves relatively good performance with less computational time.

In recent years, classifier ensembles have received increasing attention in the machine learning and pattern recognition communities. However, constructing classifier ensembles for one-class classification problems has still remained as a challenging research topic. To pursue this line of research, we need to address issues on how to generate a set of diverse one-class classifiers that are individually accurate and how to combine the outputs of them in an effective way. In this paper, we present BeeOWA, a novel approach to construct highly accurate one-class classifier ensembles. It uses a novel binary artificial bee colony algorithm, called BeePruner, to prune an initial one-class classifier ensemble and find a near-optimal sub-ensemble of base classifiers in a reasonable computational time. To evaluate the fitness of an ensemble solution, BeePruner uses two different measures: an exponential consistency measure and a non-pairwise diversity measure based on the Kappa inter-rater agreement. After one-class classifier pruning, BeeOWA uses a novel exponential induced OWA (ordered weighted averaging) operator, called EIOWA, to combine the outputs of base classifiers in the sub-ensemble. The results of experiments carried out on a number of benchmark datasets show that BeeOWA can outperform several state-of-the-art approaches, both in terms of classification performance and statistical significance.

A new approach to generate oblique decision tree ensemble is proposed wherein each decision hyperplane in the internal node of tree classifier is not always orthogonal to a feature axis. All training samples in each internal node are grouped into two hyper-classes according to their geometric properties based on a randomly selected feature subset. Then multisurface proximal support vector machine is employed to obtain two clustering hyperplanes where each hyperplane is generated such that it is closest to one group of the data and as far as possible from the other group. Then, one of the bisectors of these two hyperplanes is regarded as the test hyperplane for this internal node. Several regularization methods have been applied to handle the small sample size problem as the tree grows. The effectiveness of the proposed method is demonstrated by 44 real-world benchmark classification data sets from various research fields. These classification results show the advantage of the proposed approach in both computation time and classification accuracy.

Maximum margin classifier with flexible convex hulls (MMC-FCH) is an adaptive pattern recognition method based on convex hull vector and shrinkage factor, which can effectively identify different fault states. However, MMC-FCH is a shallow learning algorithm that cannot effectively diagnose complex signals. Meanwhile, MMC-FCH is essentially a binary classifier. For multi-classification, MMC-FCH can only perform multiple binary classifications. To overcome the shortcomings of MMC-FCH, we propose a deep stacking center configuration convex hull (DSC3H), which combines the convex hull with the idea of stacking-based representation learning (SRL). In DSC3H, the output of all previous modules combine with the original signal as the input of the next module to fully learn the information in the original signal. At the same time, the concept of center configuration is used to construct the multi-classification objective function of center configuration convex hull (C3H). However, redundant information and noise information may still exist in the proposed DSC3H method. Therefore, we further propose a deep stacking l1-norm center configuration convex hull (DSl1C3H) method, which makes the prediction model more robust and sparser under the constraint of l1-norm distance. The experiments of rolling bearings show that the proposed DSl1C3H method has better classification performance.

Various applications in different fields, such as gene expression analysis or computer vision, suffer from data sets with high-dimensional low-sample-size (HDLSS), which has posed significant challenges for standard statistical and modern machine learning methods. In this paper, we propose a novel linear binary classifier, denoted by population-guided large margin classifier (PGLMC), which is applicable to any sorts of data, including HDLSS. PGLMC is conceived with a projecting direction w given by the comprehensive consideration of local structural information of the hyperplane and the statistics of the training samples. Our proposed model has several advantages compared to those widely used approaches. First, it is not sensitive to the intercept term b. Second, it operates well with imbalanced data. Third, it is relatively simple to be implemented based on Quadratic Programming. Fourth, it is robust to the model specification for various real applications. The theoretical properties of PGLMC are proven. We conduct a series of evaluations on two simulated and six real-world benchmark data sets, including DNA classification, digit recognition, medical image analysis, and face recognition. PGLMC outperforms the state-of-the-art classification methods in most cases, or at least obtains comparable results.

High dimensional data classification with very limited labeled training data is a challenging task in the area of data mining. In order to tackle this task, we first propose a feature selection-based semi-supervised classifier ensemble framework (FSCE) to perform high dimensional data classification. Then, we design an adaptive semi-supervised classifier ensemble framework (ASCE) to improve the performance of FSCE. When compared with FSCE, ASCE is characterized by an adaptive feature selection process, an adaptive weighting process (AWP), and an auxiliary training set generation process (ATSGP). The adaptive feature selection process generates a set of compact subspaces based on the selected attributes obtained by the feature selection algorithms, while the AWP associates each basic semi-supervised classifier in the ensemble with a weight value. The ATSGP enlarges the training set with unlabeled samples. In addition, a set of nonparametric tests are adopted to compare multiple semi-supervised classifier ensemble (SSCE)approaches over different datasets. The experiments on 20 high dimensional real-world datasets show that: 1) the two adaptive processes in ASCE are useful for improving the performance of the SSCE approach and 2) ASCE works well on high dimensional datasets with very limited labeled training data, and outperforms most state-of-the-art SSCE approaches.

Discrimination-aware classification is receiving an increasing attention in data science fields. The pre-process methods for constructing a discrimination-free classifier first remove discrimination from the training data, and then learn the classifier from the cleaned data. However, they lack a theoretical guarantee for the potential discrimination when the classifier is deployed for prediction. In this paper, we fill this gap by mathematically bounding the probability of the discrimination in prediction being within a given interval in terms of the training data and classifier. We adopt the causal model for modeling the data generation mechanism, and formally defining discrimination in population, in a dataset, and in prediction. We obtain two important theoretical results: (1) the discrimination in prediction can still exist even if the discrimination in the training data is completely removed; and (2) not all pre-process methods can ensure non-discrimination in prediction even though they can achieve non-discrimination in the modified training data. Based on the results, we develop a two-phase framework for constructing a discrimination-free classifier with a theoretical guarantee. The experiments demonstrate the theoretical results and show the effectiveness of our two-phase framework.

In many real-world applications, data is not collected as one batch, but sequentially over time, and often it is not possible or desirable to wait until the data is completely gathered before analyzing it. Thus, we propose a framework to sequentially update a maximum margin classifier by taking advantage of the Maximum Entropy Discrimination principle. Our maximum margin classifier allows for a kernel representation to represent large numbers of features and can also be regularized with respect to a smooth sub-manifold, allowing it to incorporate unlabeled observations. We compare the performance of our classifier to its non-sequential equivalents in both simulated and real datasets.

© 2013 IEEE. Missing data is a common issue in community health and epidemiological studies. Direct removal of samples with missing data can lead to reduced sample size and information bias, which deteriorates the significance of the results. While data imputation methods are available to deal with missing data, they are limited in performance and could introduce noises into the dataset. Instead of data imputation, a novel method based on additive least square support vector machine (LS-SVM) is proposed in this paper for predictive modeling when the input features of the model contain missing data. The method also determines simultaneously the influence of the features with missing values on the classification accuracy using the fast leave-one-out cross-validation strategy. The performance of the method is evaluated by applying it to predict the quality of life (QOL) of elderly people using health data collected in the community. The dataset involves demographics, socioeconomic status, health history, and the outcomes of health assessments of 444 community-dwelling elderly people, with 5% to 60% of data missing in some of the input features. The QOL is measured using a standard questionnaire of the World Health Organization. Results show that the proposed method outperforms four conventional methods for handling missing data - case deletion, feature deletion, mean imputation, and K-nearest neighbor imputation, with the average QOL prediction accuracy reaching 0.7418. It is potentially a promising technique for tackling missing data in community health research and other applications.

Dynamic ensemble selection systems work by estimating the level of competence of each classifier from a pool of classifiers. Only the most competent ones are selected to classify a given test sample. This is achieved by defining a criterion to measure the level of competence of a base classifier, such as, its accuracy in local regions of the feature space around the query instance. However, using only one criterion about the behavior of a base classifier is not sufficient to accurately estimate its level of competence. In this paper, we present a novel dynamic ensemble selection framework using meta-learning. We propose five distinct sets of meta-features, each one corresponding to a different criterion to measure the level of competence of a classifier for the classification of input samples. The meta-features are extracted from the training data and used to train a meta-classifier to predict whether or not a base classifier is competent enough to classify an input instance. During the generalization phase, the meta-features are extracted from the query instance and passed down as input to the meta-classifier. The meta-classifier estimates, whether a base classifier is competent enough to be added to the ensemble. Experiments are conducted over several small sample size classification problems, i.e., problems with a high degree of uncertainty due to the lack of training data. Experimental results show the proposed meta-learning framework greatly improves classification accuracy when compared against current state-of-the-art dynamic ensemble selection techniques., Comment: Article published on Pattern Recognition. arXiv admin note: text overlap with arXiv:1509.00825

Ranking is a strategy widely used for estimating relevance or importance of available characteristic features. Depending on the applied methodology, variables are assessed individually or as subsets, by some statistics referring to information theory, machine learning algorithms, or specialised procedures that execute systematic search through the feature space. The information about importance of attributes can be used in the pre-processing step of initial data preparation, to remove irrelevant or superfluous elements. It can also be employed in post-processing, for optimisation of already constructed classifiers. The chapter describes research on the latter approach, involving filtering inferred decision rules while exploiting ranking positions and scores of features. The optimised rule classifiers were applied in the domain of stylometric analysis of texts for the task of binary authorship attribution.

The traditional colony selection is mainly operated by manual mode, which takes on low efficiency and strong subjectivity. Therefore, it is important to develop an automatic monoclonal-picking instrument. The critical stage of the automatic monoclonal-picking and intelligent optimal selection is intelligent identification algorithm. An auto-screening algorithm based on Support Vector Machine (SVM) is proposed in this paper, which uses the supervised learning method, which combined with the colony morphological characteristics to classify the colony accurately. Furthermore, through the basic morphological features of the colony, system can figure out a series of morphological parameters step by step. Through the establishment of maximal margin classifier, and based on the analysis of the growth trend of the colony, the selection of the monoclonal colony was carried out. The experimental results showed that the auto-screening algorithm could screen out the regular colony from the other, which meets the requirement of various parameters.

In this paper, a fast, transparent, self-evolving, deep learning fuzzy rule-based (DLFRB) image classifier is proposed. This new classifier is a cascade of the recently introduced DLFRB classifier called MICE and an auxiliary SVM. The DLFRB classifier serves as the main engine and can identify a number of human interpretable fuzzy rules through a very short, transparent, highly parallelizable training process. The SVM based auxiliary plays the role of a conflict resolver when the DLFRB classifier produces two highly confident labels for a single image. Only the fundamental image transformation techniques (rotation, scaling and segmentation) and feature descriptors (GIST and HOG) are used for pre-processing and feature extraction, but the proposed approach significantly outperforms the state-of-art methods in terms of both time and precision. Numerical experiments based on a handwritten digits recognition problem are used to demonstrate the highly accurate and repeatable performance of the proposed approach.

The study focuses on the application of machine learning techniques for classifying the internal quality of 'Rocha' Pear (Pyrus communis L.), i.e., the total soluble solids (TSS), using the non-invasive technique of visible/near infra-red reflectance spectroscopy. Six representative classifiers were evaluated under realistic experimental conditions. The classifiers include representatives of classic parametric (logistic and multiple linear regression), non-parametric distance based methods (K-nearest neighbors), correlation-based (partial least squares), ensemble methods (random forests) and maximum margin classifiers (support vector machines). The classifiers were assessed against metrics such as accuracy, Cohen's Kappa, F-Measure, and the area under the precision recall curve (AUC) in a 10 x 10-fold cross-validation plan. For result analysis non-parametric statistical test of hypotheses were employed. A total of 4880 fruit samples from different origins, maturation states, and harvest years were considered. The main conclusion is that the maximum margin classifier outperforms all the others studied ones, including the commonly used partial least squares. The conclusion holds for both a reflectance spectrum with 1024 features and for a 128 subsample of these. An estimate of the out-of-sample performance for the best classifier is also provided. info:eu-repo/semantics/publishedVersion

Class binarization strategies decompose the original multi-class problem into several binary sub-problems. One versus One (OVO) is one of the most popular class binarization techniques, which considers every pair of classes as a different sub-problem. Usually, the same classifier is applied to every sub-problem and then all the outputs are combined by some voting scheme. In this paper we present a novel idea where for each test instance we try to assign the best classifier in each sub-problem of OVO. To do so, we have used two simple Dynamic Classifier Selection (DCS) strategies that have not been yet used in this context. The two DCS strategies use K-NN to obtain the local region of the test-instance, and the classifier that performs the best for those instances in the local region, is selected to classify the new test instance. The difference between the two DCS strategies remains in the weight of the instance. In this paper we have also proposed a novel approach in those DCS strategies. We propose to use the K-Nearest Neighbor Equality (K-NNE) method to obtain the local accuracy. K-NNE is an extension of K-NN in which all the classes are treated independently: the K nearest neighbors belonging to each class are selected. In this way all the classes take part in the final decision. We have carried out an empirical study over several UCI databases, which shows the robustness of our proposal.

Currently, most robust classifiers with parameters focus on the determination of the optimal or suboptimal parameters. There are no research studies or even discussions about robust classifiers on all of the possible parameters. This paper considers the robust rough classifier and finds that the robust rough classifier satisfies a nested topological structure; then, the nested classifier , which reflects the classifier on all of the possible parameters, is proposed. First, some notions, such as the robust discernibility vector, the robust value reduct, and the robust covering vector, are proposed; these notions can reflect the classical corresponding notions on all of the possible parameters. It is more important that these notions share a common characteristic: the nested structure. The nested structure of these notions makes nested classifier theoretically possible. Furthermore, some novel algorithms are designed to compute the robust value reduct, the robust covering degree, and the robust classifier. These algorithms make the nested classifier technologically possible. Finally, numerical experiments demonstrate that the nested classifier is effective and efficient for classification and predication.

This research article addresses the problem of land-cover classification from the multi-spectral remotely sensed images using a novel self-training based semi-supervised learning (SSL) technique. The proposed system, instead of using a single classifier, builds an ensemble of classifiers with the hope that the ensemble system will have a lesser generalization error than any of its members. Each component classifier is trained independently using the proposed self-training approach on different training sub-sets and their predictions on test samples are combined using a new instant run-off voting (IRV) based classifier combination method. Each training subset consists of a few labeled samples and a comparatively large number of unlabeled data items. The cluster-and-label method for self-training has been adopted here considering the support vector machines (SVM) as the supervised learner and minimum spanning tree based clustering. A normalized histogram intersection kernel has been proposed which has shown to outperform the state-of-the-art kernel functions in terms of the SVM generalization accuracy. A novel cluster validity index specifically for graph based clustering has also been introduced to access the quality of the clustering. The proposed training method has the advantage that it can find classes for which the labeled training data are unavailable initially. A number of multi-spectral images have been considered for the experimental evaluation and a comparison with some of the well-established techniques from the literature has confirmed the superiority of the proposed classifier system.

Goal: The aim of this study is to find a useful methodology to classify multiple distinct pulmonary conditions including the healthy condition and various pathological types, using pulmonary sounds data. Methods: Fourteen-channel pulmonary sounds data of 40 subjects (healthy and pathological, where the pathologies are of obstructive and restrictive types) are modeled using a second order 250-point vector autoregressive model. The estimated model parameters are fed to support vector machine and Gaussian mixture model (GMM) classifiers which are used in various configurations, resulting in eight different methodologies in total. Results: Among the eight methodologies, the hierarchical GMM classifier yields the best performance with a total correct classification rate of 85%, where the term hierarchical refers here to first classifying the data into healthy and pathological classes, then the pathological class into obstructive and restrictive types. Both the sensitivity and specificity for the healthy versus pathological classification at the first stage of hierarchy are 90%. Conclusion: The newly proposed methodologies provide improved results compared to the previous study. The hierarchical framework is suggested for diagnostic classification of pulmonary sounds, although the algorithms are still open for further improvements. Significance: This study proposes new methodologies for diagnostic classification of pulmonary sounds, and suggests using a hierarchical framework for the first time.

This paper proposes a novel approach to select the individual classifiers to take part in a Multiple-Classifier System. Individual classifier selection is a key step in the development of multi-classifiers. Several works have shown the benefits of fusing complementary classifiers. Nevertheless, the selection of the base classifiers to be used is still an open question, and different approaches have been proposed in the literature. This work is based on the selection of the appropriate single classifiers by means of an evolutionary algorithm. Different base classifiers, which have been chosen from different classifier families, are used as candidates in order to obtain variability in the classifications given. Experimental results carried out with 20 databases from the UCI Repository show how adequate the proposed approach is; Stacked Generalization multi-classifier has been selected to perform the experimental comparisons.

Least Squares Twin Support Vector Machine (LSTSVM) is a binary classifier and the extension of it to multiclass is still an ongoing research issue. In this paper, we extended the formulation of binary LSTSVM classifier to multi-class by using the concepts such as "One-versus-All", "One-versus-One", "All-versus-One" and Directed Acyclic Graph (DAG). This paper performs a comparative analysis of these multi-classifiers in terms of their advantages, disadvantages and computational complexity. The performance of all the four proposed classifiers has been validated on twelve benchmark datasets by using predictive accuracy and training-testing time. All the proposed multi-classifiers have shown better performance as compared to the typical multi-classifiers based on 'Support Vector Machine' and 'Twin Support Vector Machine'. Friedman's statistic and Nemenyi post hoc tests are also used to test significance of predictive accuracy differences between classifiers.

Inductive transfer learning is a major research area in transfer learning which aims at achieving a high performance in the target domain by inducing the useful knowledge from the source domain. By combining decisions from individual classifiers, ensemble learning can usually reduce variance and achieve higher accuracy than a single classifier. In this paper, we propose a novel Ensemble Inductive Transfer Learning (EITL) method. EITL builds a set of classifiers by recording the iterative process of knowledge transfer. In each iteration, it uses the classifier of the source domain, the base classifier of the target domain built on the initial labeled data, and the most recent classifier built on the updated labeled data, to classify unlabeled instances, and add some self-labeled instances to the labeled data, and then trains a new classifier. At the end, all the classifiers built in this process are used for classification. We conduct experiments on synthetic data sets and six UCI data sets, which show that EITL is an effective algorithm in terms of classification accuracy.

Facial expression is one of the most expressive ways to display human emotions. Facial expression analysis (FEA) has been broadly studied in the past decades. In our daily life, few of the facial expressions are exactly one of the predefined affective states but are blends of several basic expressions. Even though the concept of ‘blended emotions’ has been proposed years ago, most researchers did not deal with FEA as a multiple outputs problem yet. In this paper, multi-label learning algorithm for FEA is proposed to solve this problem. Firstly, to depict facial expressions more effectively, we model FEA as a multi-label problem, which depicts all facial expressions with multiple continuous values and labels of predefined affective states. Secondly, in order to model FEA jointly with multiple outputs, multi-label Group Lasso regularized maximum margin classifier (GLMM) and Group Lasso regularized regression (GLR) algorithms are proposed which can analyze all facial expressions at one time instead of modeling as a binary learning problem. Thirdly, to improve the effectiveness of our proposed model used in video sequences, GLR is further extended to be a Total Variation and Group Lasso based regression model (GLTV) which adds a prior term (Total Variation term) in the original model. JAFFE dataset and the extended Cohn Kanade (CK+) dataset have been used to verify the superior performance of our approaches with common used criterions in multi-label classification and regression realms.

This research proposes a novel bee colony optimized support vector classifier (BeeSVC) for predicting typhoon-induced slope collapses. The BeeSVC employs the support vector classifier (SVC) as a machine learning method to classify a slope into two classes: “stable slope” and “collapsed slope.” Furthermore, the artificial bee colony algorithm is used as a metaheuristic to determine the hyper-parameters of the SVC appropriately. The contribution of the proposed method to the body of knowledge is multifold. First, the combined framework of the BeeSVC allows the assessment process to be operated automatically. Second, since the number of the “collapsed” class occupied more than 70 % of the historical cases, a repeated random sub-sampling procedure with the Student’s t test is put forward to alleviate the class-imbalanced problem and reliably evaluate the model performance. Third, the mutual information between the input features and the slope performance is employed to reflect the contribution of each feature to the slope failure. Lastly, the superior performance has proved that the BeeSVC can be a very effective tool for decision-makers to forecast typhoon-induced slope collapses.

In this paper, we investigate the problem of classifying an image set of an object, and develop a novel image set representation and classification algorithm. We propose to represent an image set by a joint representation method using both an affine hull of its image samples and a combination of its reference images, and further classify it by a linear classification function from its representation. A unified objective function is formulated to learn both the representation and classifier parameters. Similar to support vector machine, the hinge losses and the squared l 2 norm of the image set classifier are minimized simultaneously in the objective. Moreover, the differences between the two different representations are also minimized. The objective function is optimized with respect to representation and classifier parameters alternately in an iterative algorithm. The proposed algorithm is named as support image set machine (SupISMac) because it takes advantage of support vector machine formulation to learn an image set classifier. The experiments on two different image set classification benchmark databases show that SupISMac not only outperforms the state-of-the-art image set classification methods, but also reduces the running time of test procedures significantly.

usc Refereed/Peer-reviewed In this paper, a novel evolving fuzzy-rule-based classifier, termed parsimonious classifier (pClass), is proposed. pClass can drive its learning engine from scratch with an empty rule base or initially trained fuzzy models. It adopts an open structure and plug and play concept where automatic knowledge building, rule-based simplification, knowledge recall mechanism, and soft feature reduction can be carried out on the fly with limited expert knowledge and without prior assumptions to underlying data distribution. In this paper, three state-of-the-art classifier architectures engaging multi-input-multi-output, multimodel, and round robin architectures are also critically analyzed. The efficacy of the pClass has been numerically validated by means of real-world and synthetic streaming data, possessing various concept drifts, noisy learning environments, and dynamic class attributes. In addition, comparative studies with prominent algorithms using comprehensive statistical tests have confirmed that the pClass delivers more superior performance in terms of classification rate, number of fuzzy rules, and number of rule-base parameters.

Sensor drift is the most challenging problem in gas sensing at present. We propose a novel two-dimensional classifier ensemble strategy to solve the gas discrimination problem, regardless of the gas concentration, with high accuracy over extended periods of time. This strategy is appropriate for multi-class classifiers that consist of combinations of pairwise classifiers, such as support vector machines. We compare the performance of the strategy with those of competing methods in an experiment based on a public dataset that was compiled over a period of three years. The experimental results demonstrate that the two-dimensional ensemble outperforms the other methods considered. Furthermore, we propose a pre-aging process inspired by that applied to the sensors to improve the stability of the classifier ensemble. The experimental results demonstrate that the weight of each multi-class classifier model in the ensemble remains fairly static before and after the addition of new classifier models to the ensemble, when a pre-aging procedure is applied.

Pattern recognition concerns assigning objects to classes. The objects are described by features (variables or measurements) organized as p-dimensional points in some feature space. A classifier is a formula, an algorithm or a technique that can assign a class label to any given point in the feature space. Pattern recognition comprises supervised learning (predefined class labels) and unsupervised learning (unknown class labels). Supervised learning includes choosing a classifier model, training and testing the classifier using a data set and selecting the relevant features. Unsupervised learning is usually approached by cluster analysis. Multiple classifier systems or classifier ensembles are a recent branch of pattern recognition whereby the outputs of several classifiers (regarded as experts) are combined for improved accuracy. Keywords: classification; discriminant analysis; cluster analysis; feature selection; classifier ensembles

the various biometric methods, Face Recognition has become one of the most burning topic tasks in the pattern recognition field during the past decades. In This Work a Face Recognition System has been developed By applying different multiple classifier selection schemes on the output of three different classification methods namely Artificial Neural Network, Genetic Algorithm And Euclidean distance measure based on Principal Component Analysis dimensionality reduction technique. Dynamic classifier selection technique using classifier of local accuracy measurement is to look up the individual correctness of individual classifier and select the best one from them. Here it is proposed a classifier local accuracy measurement technique which is used to dynamic classifier selection algorithm. From the result and performance analysis it can be said that numerous classifier selection schemes give better performance than single classifier and dynamic classifier selection algorithm using proposed classifier local accuracy measurement technique gives stable high and better performance.

While classical classification methods such as support vector machine and its extensions (SVMs) obtain strong generalization capability by maximizing the separation margin between binary classes, they are usually sensitive to affine (or scaling) transformation of data. The optimal solutions of SVMs may be misled by the spread of data and preferentially separate classes along large margin directions. To this end, in this paper, we propose a novel minimum class spread constrained support vector machine (MCSSVM) for pattern classification problems by simultaneously considering the maximization of inter-class separation margin and the minimization of within-class spread of data, which corresponds to a distribution-dependent regularization on the classification function by constraining the within-class spread of input data. The basic idea of MCSSVM is to find a class distribution constrained hyperplane such that one class (or normal class) can be enclosed in a minimum q-spread tube, while the other class (or abnormal class) is farthest from this tube. MCSSVM can simultaneously achieve both maximum inter-class margin and minimum within-class spread so as to enhance the generalization capability of the proposed classifier. Moreover, the proposed method only requires simple extensions to existing maximum margin formulations such as SVMs and still preserves their computational efficiency. Generalization bound for MCSSVM is theoretically derived and the validity of MCSSVM is examined by classification of toy and real-world classification problems, which demonstrate the superiority of our method in comparison to other related state-of-the-art algorithms.

Different from current foreground and background segmentation methods, we did not utilize the low level im- age representation method (such as boundaries and textures) to extract the feature of the videos, instead we proposed a spatio-temporal feature classifier to obtain the union region of object from natural videos as the interest points. We slide the temporal chunk along time axis to obtain samples from videos, and train the Support Vector Machine (SVM) with fea- ture vectors. Then we built a spatio-temporal feature classifier and tested our algorithm on the most popular benchmark dataset. Experiment results showed the effectiveness and robustness of the algorithm.

The VC dimension measures the complexity of a learning machine, and a low VC dimension leads to good generalization. While SVMs produce state-of-the-art learning performance, it is well known that the VC dimension of a SVM can be unbounded; despite good results in practice, there is no guarantee of good generalization. In this paper, we show how to learn a hyperplane classifier by minimizing an exact, or ? bound on its VC dimension. The proposed approach, termed as the Minimal Complexity Machine (MCM), involves solving a simple linear programming problem. Experimental results show, that on a number of benchmark datasets, the proposed approach learns classifiers with error rates much less than conventional SVMs, while often using fewer support vectors. On many benchmark datasets, the number of support vectors is less than one-tenth the number used by SVMs, indicating that the MCM does indeed learn simpler representations. HighlightsWe learn a hyperplane classifier by minimizing an exact bound on its VC dimension.A fractional programming problem is formulated and reduced to a LP problem.Linear and kernel versions of the approach are explored.The approach, called the Minimal Complexity Machine, generalizes better than SVMs.On numerous benchmark datasets, the MCM uses far fewer support vectors than SVMs.

An open question in ensemble-based active learning is how to choose one classifier type, or appropriate combinations of multiple classifier types, to construct ensembles for a given task. While existing approaches typically choose one classifier type, this paper presents a method that trains and adapts multiple instances of multiple classifier types toward an appropriate ensemble during active learning. The method is termed adaptive heterogeneous ensembles (henceforth referred to as AHE). Experimental evaluations show that AHE constructs heterogeneous ensembles that outperform homogeneous ensembles composed of any one of the classifier types, as well as bagging, boosting and the random subspace method with random sampling. We also show in this paper that the advantage of AHE over other methods is increased if (1) the overall size of the ensemble also adapts during learning; and (2) the target data set is composed of more than two class labels. Through analysis we show that the AHE outperforms other methods because it automatically discovers complementary classifiers: for each data instance in the data set, instances of the classifier type best suited for that data point vote together, while instances of the other, inappropriate classifier types disagree, thereby producing a correct overall majority vote.

The support vector machine (SVM) and other reproducing kernel Hilbert space (RKHS) based classifier systems are drawing much attention recently due to its robustness and generalization capability. General theme here is to construct classifiers based on the training data in a high dimensional space by using all available dimensions. The SVM achieves huge data compression by selecting only few observations which lie close to the boundary of the classifier function. However when the number of observations are not very large (small n) but the number of dimensions/features are large (large p), then it is not necessary that all available features are of equal importance in the classification context. Possible selection of an useful fraction of the available features may result in huge data compression. In this paper we propose an algorithmic approach by means of which such an optimal set of features could be selected. In short, we reverse the traditional sequential observation selection strategy of SVM to that of sequential feature selection. To achieve this we have modified the solution proposed by Zhu and Hastie (2005) in the context of import vector machine (IVM), to select an optimal sub-dimensional model to build the final classifier with sufficient accuracy.