Search

Plethora of ensemble techniques have been implemented and studied in order to achieve better classification results than base classifiers. In this paper an algorithm for integration of decision trees is proposed, which means that homogeneous base classifiers will be used. The novelty of the presented approach is the usage of the simultaneous distance of the object from the decision boundary and the center of mass of objects belonging to one class label in order to determine the score functions of base classifiers. This means that the score function assigned to the class label by each classifier depends on the distance of the classified object from the decision boundary and from the centroid. The algorithm was evaluated using an open-source benchmarking dataset. The results indicate an improvement in the classification quality in comparison to the referential method - majority voting method.

Although linear classifiers are one of the oldest methods in machine learning, they are still very popular in the machine learning community. This is due to their low computational complexity and robustness to overfitting. Consequently, linear classifiers are often used as base classifiers of multiple ensemble classification systems. This research is aimed at building a new fusion method dedicated to the ensemble of linear classifiers. The fusion scheme uses both measurement space and geometrical space. Namely, we proposed a probability-driven scoring function which shape depends on the orientation of the decision hyperplanes generated by the base classifiers. The proposed fusion method is compared with the reference method using multiple benchmark datasets taken from the KEEL repository. The comparison is done using multiple quality criteria. The statistical analysis of the obtained results is also performed. The experimental study shows that, under certain conditions, some improvement may be obtained.

One of the current challenges for the supervised classification methods is to obtain acceptable values of the performance measures for an imbalanced dataset. There is a significant disproportion in the number of objects from different class labels in datasets with a high imbalanced ratio. This article analyzes the clustering and weighted scoring algorithm based on estimating the number of clusters that consider the minimum number of objects from the minority class label in each cluster. This algorithm uses the distance metric when determining the value of the score function. Therefore, this article aims to analyze the impact of selecting the distance metric on the six classification performance measures’ value. The performed experiments show that the Euclidean distance allows obtaining the best classification results for imbalanced datasets.

Learning from imbalanced datasets is a challenging task for standard classification algorithms. In general, there are two main approaches to solve the problem of imbalanced data: algorithm-level and data-level solutions. This paper deals with the second approach. In particular, this paper shows a new proposition for calculating the weighted score function to use in the integration phase of the multiple classification system. The presented research includes experimental evaluation over multiple, open-source, highly imbalanced datasets, presenting the results of comparing the proposed algorithm with three other approaches in the context of six performance measures. Comprehensive experimental results show that the proposed algorithm has better performance measures than the other ensemble methods for highly imbalanced datasets.

In order to improve pattern recognition performance of an individual classifier an ensemble of classifiers is used. One of the phases of creating the multiple classifier system is the selection of base classifiers which are used as the original set of classifiers. In this paper we propose the algorithm of the dynamic ensemble selection that uses median and quartile of correctly classified objects. The resulting values are used to define the decision schemes, which are used in the selection of the base classifiers process. The proposed algorithm has been verified on a real dataset regarding the classification of cutting tools. The obtained results clearly indicate that the proposed algorithm improves the classification measure. The improvement concerns the comparison with the ensemble of classifiers method without the selection.

Ensembles of classifiers deserve attention because their stability and accuracy are usually superior compared to the single classifier. One of the aspects regarding the construction of multiple classifier systems is the fusion of each base model output. The state-of-the-art fusion of base classifiers approaches uses class labels, a rank array, or a score function to determine the classifier ensemble’s final decision. On the other hand, in this study, we use the base classifiers’ decision boundaries in the fusion process. Therefore the integration process occurs in a geometric space. In this paper, a new definition of the function that measures the central tendency has been proposed. This function allows integrating any number of linear base classifiers in the geometry space, removing the limit on the number of these classifiers in the ensemble. The limit on the number of base classifiers is noticeable in our earlier works. The proposal was compared with other fusion approaches to base classifiers outputs. The experiments on multiple binary datasets from UCI and KEEL datasets repositories demonstrate the effectiveness of our proposal of the fusion process in the geometric space. To discuss the results of our experiments, we use standard and imbalanced datasets separately.

Integration or fusion of the base classifiers is the final stage of creating multiple classifiers system. Known methods in this step use base classifier outputs, which are class labels or values of the confidence (predicted probabilities) for each class label. In this paper we propose an integration process which takes place in the geometric space. It means that the fusion of base classifiers is done using their decision boundaries. In order to obtain one decision boundary from boundaries defined by base classifiers the median or weighted average method will be used. In addition, the proposed algorithm uses the division of the entire feature space into disjoint regions of competence as well as the process of selection of base classifiers is carried out. The aim of the experiments was to compare the proposed algorithms with the majority voting method and assessment which of the analyzed approaches to integration of the base classifiers creates a more effective ensemble.

This paper presents a new extension of Gentle AdaBoost algorithm based on the distance of the object to the decision boundary, which is defined by the weak classifier used in boosting. In the proposed approach this distance is transformed by Gaussian function and defines the value of a score function. The assumed form of transforming functions means that the objects closest or farthest located from the decision boundary of the basic classifier have the lowest value of the scoring function. The described algorithm was tested on four data sets from UCI repository and compared with Gentle AdaBoost algorithm.

A vital aspect of the Multiple Classifier Systems construction process is the base model integration. For example, the Random Forest approach used the majority voting rule to fuse the base classifiers obtained by bagging the training dataset. In this paper we propose the algorithm that uses partitioning the feature space whose split is determined by the decision rules of each decision tree node which is the base classification model. After dividing the feature space, the centroid of each new subspace is determined. This centroids are used in order to determine the weights needed in the integration phase based on the weighted majority voting rule. The proposal was compared with other Multiple Classifier Systems approaches. The experiments regarding multiple open-source benchmarking datasets demonstrate the effectiveness of our method. To discuss the results of our experiments, we use micro and macro-average classification performance measures.

An ensemble of classifiers can improve the performance of a pattern recognition system. The task of constructing multiple classifier systems can be generally divided into three steps: generation, selection and integration. In this paper, we propose an integration process which takes place in the geometric space. It means that the fusion of base classifiers is done using decision boundaries. In our approach, we use the linear SVM model as a base classifier, the selection process is based on the accuracy and the final decision boundary is calculated by using the median of the decision boundary. The aim of the experiments was to compare the proposed algorithm and the majority voting method.

This paper presents a new extension of the AdaBoost algorithm. This extension concerns the weights used in this algorithm. In our approach the original weights are modified, we propose a linear modification of the weights. In our study we use the boosting by the reweighting method where each weak classifier is based on the linear classifier. The described algorithm was tested on Pima data set. The obtained results are compared with the original the AdaBoost algorithm.

Part 2: Biometrics and Pattern Recognition Applications; International audience; The classification process in the domain of brain computer interfaces (BCI) is usually carried out with simple linear classifiers, like LDA or SVM. Non-linear classifiers rarely provide a sufficient increase in the classification accuracy to use them in BCI. However, there is one more type of classifiers that could be taken into consideration when looking for a way to increase the accuracy - boosting classifiers. These classification algorithms are not common in BCI practice, but they proved to be very efficient in other applications.

Recently, several approaches have been proposed to deal with the concept drift detection. In this paper we propose the new concept drift detection algorithm based on the decision templates. The decision templates are obtained from the outputs of the base classifier that form an ensemble of classifiers. Experiments on several publicly available data sets verify the effectiveness of the proposed algorithm.

The article presents a new approach of calculating the weight of base classifiers from a committee of classifiers. The obtained weights are interpreted in the context of the interval-valued sets. The work proposes four different ways of calculating weights which consider both the correctness and incorrectness of the classification. The proposed weights have been used in the algorithms which combine the outputs of base classifiers. In this work we use both the outputs, represented by rank and measure level. Research experiments have involved several bases available in the UCI repository and two data sets that have generated distributions. The performed experiments compare algorithms which are based on calculating the weights according to the resubstitution and algorithms proposed in the work. The ensemble of classifiers has also been compared with the base classifiers entering the committee.

The paper presents the dynamic ensemble selection method. Proposed method uses information from so-called decision profiles which are formed from the outputs of the base classifiers. In order to verify these algorithms, a number of experiments have been carried out on several public available data sets. The proposed dynamic ensemble selection is experimentally compared against all base classifiers and the ensemble classifiers based on the sum and decision profile methods. As a base classifiers we used the pool of homogeneous classifiers.

Part 7: Decisions; International audience; In the classification task, the ensemble of classifiers have attracted more and more attention in pattern recognition communities. Generally, ensemble methods have the potential to significantly improve the prediction base classifier which are included in the team. In this paper, we propose the algorithm which modifies the confidence values. This values are obtained as an outputs of the base classifiers. The experiment results based on thirteen data sets show that the proposed method is a promising method for the development of multiple classifiers systems. We compared the proposed method with other known ensemble of classifiers and with all base classifiers.

The ensemble selection is one of the important problems in building multiple classifier systems (MCSs). This paper presents dynamic ensemble selection based on the analysis of discriminant functions. The idea of the selection is presented on the basis of binary classification tasks. The paper presents two approaches: one takes into account the normalization of the discrimination functions, and in the second approach, normalization is not performed. The reported results based on the data sets form the UCI repository show that the proposed ensemble selection is a promising method for the development of MCSs.

The paper considers the problem of classification error in pattern recognition. This model of classification is primarily based on the Bayes rule and secondarily on the notion of intuitionistic or interval-valued fuzzy sets. A probability of misclassifications is derived for a classifier under the assumption that the features are class-conditionally statistically independent, and we have intuitionistic or interval-valued fuzzy information on object features instead of exact information. A probability of the intuitionistic or interval-valued fuzzy event is represented by the real number. Additionally, the received results are compared with the bound on the probability of error based on information energy. Numerical example concludes the work.

The paper considers the problem of classification error in multistage pattern recognition. This model of classification is based primarily on the Bayes rule and secondarily on the notion of fuzzy numbers. In adopting a probability-fuzzy model two concepts of hierarchical rules are proposed. In the first approach the local criterion that denote the probabilities of misclassification for particular nodes of a tree is considered. In the second approach the global optimal strategy that minimises the mean probability of misclassification on the whole multistage recognition process is considered. A probability of misclassifications is derived for a multiclass hierarchical classifier under the assumption that the features at different nodes of the tree are class-conditionally statistically independent, and we have fuzzy information on object features instead of exact information. Numerical example of this difference concludes the work.

The choice of a pertinent objective function is one of the most crucial elements in static ensemble selection. In this study, a new approach of calculating the weight of base classifiers is developed. The values of these weights are the basis for the selection process of classifiers from the initial pool. The obtained weights are interpreted in the context of the interval logic. A number of experiments have been carried out on several datasets available in the UCI repository. The performed experiments compare the proposed algorithms with base classifiers, oracle, sum, product, and mean methods.

The selection of classifiers is one of the important problems in the creation of an ensemble of classifiers. The paper presents the static selection in which a new method of calculating the weights of individual classifiers is used. The obtained weights can be interpreted in the context of the interval logic. It means that the particular weights will not be provided precisely but their lower and upper values will be used. A number of experiments have been carried out on several data sets from the UCI repository.

This paper presents a new extension of AdaBoost algorithm based on interval-valued fuzzy sets. This extension is for the weights used in samples of the training sets. The original weights are the real number from the interval [0, 1]. In our approach the weights are represented by the interval-valued fuzzy set, that is any weight has a lower and upper membership function. The same value of lower and upper membership function has a weight of the appropriate weak classifier. In our study we use the boosting by the reweighting method where each weak classifier is based on the recursive partitioning method. The described algorithm was tested on two generation data sets and two sets from UCI repository. The obtained results are compared with the original AdaBoost algorithm.

The paper presents the problem of the error estimation in the Bayes classifier. The model of pattern recognition with fuzzy or exact observations of features and the zero-one loss function was assumed. For this model of pattern recognition difference of the probability of error for exact and fuzzy data was demonstrated. Received results were compared to the bound on the probability of error based on information energy for fuzzy events. The paper presents that the bound on probability of an error based on information energy is very inaccurate.

In the paper the problem of cost in hierarchical classifier is presented. Assuming that both the tree structure and the feature used at each non-terminal node have been specified, we present the expected total cost for two cases. The first one concerns the non fuzzy observation of object features, the second concerns the fuzzy observation. At the end of the work the difference between expected total cost of fuzzy and non fuzzy data is determined. Obtained results relate to the locally optimal strategy of Bayes multistage classifier.

The paper considers the problem of classification error in pattern recognition. This model of classification is primarily based on the Bayes rule and secondarily on the notion of fuzzy numbers. A probability of misclassifications is derived for a classifier under the assumption that the features are class-conditionally statistically independent, and we have fuzzy information on object features instead of exact information. Numerical example of this difference concludes the work.

In the work we consider the situation with exact classes and fuzzy information of object features. The classification error is presented for the two-class Bayes classifier. The results are received for the full probabilistic information. The new upper bound of the probability of an error is precise twice as much as the bound based on the information energy of fuzzy events.

The paper considers the partition problem of feature space in classification. The partition is based on information energy for fuzzy events. In this paper we use Bayes rule for classification with fuzzy observations and exact classes. Additionally a probability of misclassifications is derived for fuzzy information on object features. The results show deterioration the quality of classification when we use fuzzy information on object features instead of exact information and are compared with the partition of feature space. Numerical example concludes the work.

The paper considers the problem of classification error in pattern recognition. This model of classification is primarily based on the Bayes rule and secondarily on the notion of intuitionistic fuzzy sets. A probability of misclassifications is derived for a classifier under the assumption that the features are class-conditionally statistically independent, and we have intuitionistic fuzzy information on object features instead of exact information. Additionally, a probability of the intuitionistic fuzzy event is represented by the real number. Numerical example concludes the work.

The paper considers the problem of pattern recognition based on Bayes rule. In this model of classification, we use interval-valued fuzzy observations. The paper focuses on the probability of error on certain assumptions. A probability of misclassifications is derived for a classifier under the assumption that the features are class-conditionally statistically independent, and we have interval-valued fuzzy information on object features instead of exact information. Additionally, a probability of the interval-valued fuzzy event is represented by the real number as upper and lower probability. Numerical example concludes the work.

The paper considers the problem of classification error in pattern recognition. This model of classification is primarily based on the Bayes rule and secondarily on the notion of intuitionistic fuzzy sets. A probability of misclassifications is derived for a classifier under the assumption that the features are class-conditionally statistically independent, and we have intuitionistic fuzzy information on object features instead of exact information. Additionally, we consider the global optimal hierarchical classifier.

The paper deals with the multistage recognition task. In this problem of recognition the Bayesian statistic is applied. This model of classification is based on the notion of intuitionistic fuzzy sets. A probability of misclassifications is derived for a classifier under the assumption that the features are class-conditionally statistically independent, and we have intuitionistic fuzzy information on object features instead of exact information. The decision rules minimize the mean risk, that is the mean value of the zero-one loss function. Additionally, we consider the local optimal hierarchical classifier.

In this paper, a model to deal with Bayesian hierarchical classifier, in which consequences of decision are fuzzy-valued, is introduced. The model is based on the notion of fuzzy random variable and also on a subjective ranking method for fuzzy number defined by Campos and Gonzalez. The Bayesian hierarchical classifier is based on a decision-tree scheme for given tree skeleton and features to be used in each inertial nodes. The influence of selection of fuzzy-valued loss function on classification result is given. Finally, an example illustrating this case of Bayesian analysis is considered.

An application of a two-stage classifier to the prognosis of sacroileitis is presented in the paper. The method of classification is based on a decision tree scheme. A k-nearest neighbors is applied in pattern recognition task. In this model of classification a fuzzy loss function is used. The efficiency of this algorithm is compared with the algorithm based on zero-one loss function. In this paper also influence of choice of parameter λ in selected comparison fuzzy number method on classification results are presented.

The paper deals with the recognition task with a full probabilistic information. In this problem of recognition the Bayesian statistic is applied and multistage recognition is considered. The information on objects features is fuzzy-valued. The decision rules minimize the mean risk, that is the mean value of the zero-one loss function. The probability of misclassification for local optimal strategy and the difference between probability of misclassification for the fuzzy-valued and crisp data are presented. Simple example conclude the work and influence of the width of the fuzzy number on this difference is discussed.

The paper deals with the multistage recognition task. In this problem of recognition the Bayesian statistic is applied. The decision rules minimize the mean risk, that is the mean value of the zero-one loss function. The information on objects features is fuzzy or non-fuzzy. The probability of misclassification for local optimal strategy and the difference between probability of misclassification for the both information’s are presented. Simple example of this difference conclude the work.

The paper deals with the probability of misclassification in a multistage classifier. This classification problem is based on a decision-tree scheme. For given tree skeleton and features to be used, the Bayes decision rules at each non-terminal node are presented. Additionally the information on objects features is fuzzy or nonfuzzy. The upper bound of the difference between probability of misclassification for the both information’s is presented. In the paper we use the maximum likelihood estimator for fuzzy data.

The paper deals with the decision rules for a Bayesian hierarchical classifier based on a decision-tree scheme. For given tree skeleton and features to be used, the optimal (Bayes) decision rules (strategy) at each non-terminal node are presented. The case has been considered when a loss (utility) function is described using fuzzy numbers. The globally optimal Bayes strategy has been calculated for the case when the loss function depends on the node of the decision tree. The model is based on the notion of fuzzy random variable and also on crisp ranking method for fuzzy numbers. The obtained result is presented as a calculation example.