Search

Plethora of ensemble techniques have been implemented and studied in order to achieve better classification results than base classifiers. In this paper an algorithm for integration of decision trees is proposed, which means that homogeneous base classifiers will be used. The novelty of the presented approach is the usage of the simultaneous distance of the object from the decision boundary and the center of mass of objects belonging to one class label in order to determine the score functions of base classifiers. This means that the score function assigned to the class label by each classifier depends on the distance of the classified object from the decision boundary and from the centroid. The algorithm was evaluated using an open-source benchmarking dataset. The results indicate an improvement in the classification quality in comparison to the referential method - majority voting method.

Learning from imbalanced datasets is a challenging task for standard classification algorithms. In general, there are two main approaches to solve the problem of imbalanced data: algorithm-level and data-level solutions. This paper deals with the second approach. In particular, this paper shows a new proposition for calculating the weighted score function to use in the integration phase of the multiple classification system. The presented research includes experimental evaluation over multiple, open-source, highly imbalanced datasets, presenting the results of comparing the proposed algorithm with three other approaches in the context of six performance measures. Comprehensive experimental results show that the proposed algorithm has better performance measures than the other ensemble methods for highly imbalanced datasets.

A vital aspect of the Multiple Classifier Systems construction process is the base model integration. For example, the Random Forest approach used the majority voting rule to fuse the base classifiers obtained by bagging the training dataset. In this paper we propose the algorithm that uses partitioning the feature space whose split is determined by the decision rules of each decision tree node which is the base classification model. After dividing the feature space, the centroid of each new subspace is determined. This centroids are used in order to determine the weights needed in the integration phase based on the weighted majority voting rule. The proposal was compared with other Multiple Classifier Systems approaches. The experiments regarding multiple open-source benchmarking datasets demonstrate the effectiveness of our method. To discuss the results of our experiments, we use micro and macro-average classification performance measures.

The machine learning methods are often used in the development of the effective medical decision support systems. One of the latest trends in data mining is the ensemble selection. In this paper, we present the algorithm of the dynamic confidence values selection, which is dedicated to the binary classification task. In the experiment we use Support Vector Machine, k-Nearest-Neighbors, Neutral Network and Decision Trees models as base classifiers. Experiments on several publicly available medical diagnosis data sets verify the effectiveness of the proposed algorithm. The results demonstrate that the dynamic confidence values selection outperforms the ensemble classifier built with all base learning models.

Part 2: Biometrics and Pattern Recognition Applications; International audience; The classification process in the domain of brain computer interfaces (BCI) is usually carried out with simple linear classifiers, like LDA or SVM. Non-linear classifiers rarely provide a sufficient increase in the classification accuracy to use them in BCI. However, there is one more type of classifiers that could be taken into consideration when looking for a way to increase the accuracy - boosting classifiers. These classification algorithms are not common in BCI practice, but they proved to be very efficient in other applications.

Multiple classifier systems are used to improve the performance of base classifiers. One of the most important steps in the formation of multiple classifier systems is the integration process in which the base classifiers outputs are combined. The most commonly used classifiers outputs are class labels, the ranking list of possible classes or confidence levels. In this paper, we propose an integration process which takes place in the “geometry space”. It means that we use the decision boundary in the integration process. The results of the experiment based on several data sets show that the proposed integration algorithm is a promising method for the development of multiple classifiers systems.

The article presents a new approach of calculating the weight of base classifiers from a committee of classifiers. The obtained weights are interpreted in the context of the interval-valued sets. The work proposes four different ways of calculating weights which consider both the correctness and incorrectness of the classification. The proposed weights have been used in the algorithms which combine the outputs of base classifiers. In this work we use both the outputs, represented by rank and measure level. Research experiments have involved several bases available in the UCI repository and two data sets that have generated distributions. The performed experiments compare algorithms which are based on calculating the weights according to the resubstitution and algorithms proposed in the work. The ensemble of classifiers has also been compared with the base classifiers entering the committee.

The ensemble selection is one of the important problems in building multiple classifier systems (MCSs). This paper presents dynamic ensemble selection based on the analysis of discriminant functions. The idea of the selection is presented on the basis of binary classification tasks. The paper presents two approaches: one takes into account the normalization of the discrimination functions, and in the second approach, normalization is not performed. The reported results based on the data sets form the UCI repository show that the proposed ensemble selection is a promising method for the development of MCSs.

The paper presents a comparative study of selected recognition methods for the medical decision problem -acute abdominal pain diagnosis. We consider if it is worth using expert knowledge and learning set at the same time. The article shows two groups of decision tree approaches to the problem under consideration. The first does not use expert knowledge and generates classifier only on the basis of learning set. The second approach utilizes expert knowledge for specifying the decision tree structure and learning set for determining mode of decision making in each node based on Bayes decision theory. All classifiers are evaluated on the basis of computer experiments.

The paper considers the problem of classification error in multistage pattern recognition. This model of classification is based primarily on the Bayes rule and secondarily on the notion of fuzzy numbers. In adopting a probability-fuzzy model two concepts of hierarchical rules are proposed. In the first approach the local criterion that denote the probabilities of misclassification for particular nodes of a tree is considered. In the second approach the global optimal strategy that minimises the mean probability of misclassification on the whole multistage recognition process is considered. A probability of misclassifications is derived for a multiclass hierarchical classifier under the assumption that the features at different nodes of the tree are class-conditionally statistically independent, and we have fuzzy information on object features instead of exact information. Numerical example of this difference concludes the work.

The choice of a pertinent objective function is one of the most crucial elements in static ensemble selection. In this study, a new approach of calculating the weight of base classifiers is developed. The values of these weights are the basis for the selection process of classifiers from the initial pool. The obtained weights are interpreted in the context of the interval logic. A number of experiments have been carried out on several datasets available in the UCI repository. The performed experiments compare the proposed algorithms with base classifiers, oracle, sum, product, and mean methods.

The selection of classifiers is one of the important problems in the creation of an ensemble of classifiers. The paper presents the static selection in which a new method of calculating the weights of individual classifiers is used. The obtained weights can be interpreted in the context of the interval logic. It means that the particular weights will not be provided precisely but their lower and upper values will be used. A number of experiments have been carried out on several data sets from the UCI repository.

This paper presents the problem of building the sequential model of the classification task. In our approach the structure of the model is built in the learning phase of classification. In this paper a split criterion based on the MacArthur’s overlapping niches model is proposed. The MacArthur’s overlapping niches distribution is created for each row of the confusion matrix. The split criterion is associated with the analysis of the received distributions. The obtained results were verified on ten data sets. Nine data sets come from UCI repository and one is a real-life data set.

This paper presents the problem of sequential decision making in the pattern recognition task. This task can be presented using a decision tree. In this case, it is assumed that the structure of the decision tree is determined by experts. The classification process is made in each node of the tree. This paper proposes a way to change the structure of the decision tree to improve the quality of classification. The split criterion is based on the confusion matrix. The obtained results were verified on the basis of the example of the computer-aided medical diagnosis.

This paper presents the problem of building the sequential model of the classification task. In our approach the structure of the model is built in the learning phase of classification. In this paper a split criterion based on the broken stick model is proposed. The broken stick distribution is created for each column of the confusion matrix. The split criterion is associated with the analysis of the received distributions. The obtained results were verified on ten data sets. Nine data sets come from UCI repository and one is a real-life data set.

The paper considers the problem of pattern recognition when we have multiple data sources. We assume that for each data source there are estimated parameters of statistical distributions. The model of classification is primarily based on the Bayes rule and secondarily on the notion of interval-valued fuzzy sets. The set of possible class-conditional probability density functions is represented by an interval-valued fuzzy set. We consider the case where the uncertainty concerns the mean of Gaussian pdf. In the paper the bound on the optimal Bayess error is presented for a full probabilistic information.

Part 7: Algorithms; International audience; This paper presents the problem of building the decision scheme in the multistage pattern recognition task. This task can be presented using a decision tree. This decision tree is built in the learning phase of classification. This paper proposes a split criterion based on the analysis of the confusion matrix. Specifically, we propose the division associated with an incorrect classification. The obtained results were verified on the data sets form UCI Machine Learning Repository and one real-life data set of the computer-aided medical diagnosis.

This paper presents the recognition algorithm with random selection of features. In the proposed procedure of classification the choice of weights is one of the main problems. In this paper we propose the weighted majority vote rule in which weights are represented by interval-valued fuzzy set (IVFS). In our approach the weights have a lower and upper membership function. The described algorithm was tested on one data set from UCI repository. The obtained results are compared with the most popular majority vote and the weighted majority vote rule.

The article presents the concept of decomposition of the multidimensional classification task. The recognition procedure is divided into independent blocks. These blocks can be interpreted as lower classification problems. The structure of these blocks is presented as a decision tree. In this model the experts give the decision tree structure. The problem discussed in the work shows a selection of different classifiers (or their parameters) to the internal nodes of the decision tree. Experiments conducted for selected medical diagnosis problem show that the use of different classifiers can improve the quality of classification.

In the paper the problem of cost in hierarchical classifier is presented. Assuming that both the tree structure and the feature used at each non-terminal node have been specified, we present the expected total cost for two cases. The first one concerns the non fuzzy observation of object features, the second concerns the fuzzy observation. At the end of the work the difference between expected total cost of fuzzy and non fuzzy data is determined. Obtained results relate to the locally optimal strategy of Bayes multistage classifier.

The paper considers the problem of classification error in pattern recognition. This model of classification is primarily based on the Bayes rule and secondarily on the notion of fuzzy numbers. A probability of misclassifications is derived for a classifier under the assumption that the features are class-conditionally statistically independent, and we have fuzzy information on object features instead of exact information. Numerical example of this difference concludes the work.

The paper considers the partition problem of feature space in classification. The partition is based on information energy for fuzzy events. In this paper we use Bayes rule for classification with fuzzy observations and exact classes. Additionally a probability of misclassifications is derived for fuzzy information on object features. The results show deterioration the quality of classification when we use fuzzy information on object features instead of exact information and are compared with the partition of feature space. Numerical example concludes the work.

An application of a two-stage classifier to the prognosis of sacroileitis is presented in the paper. The method of classification is based on a decision tree scheme. A k-nearest neighbors is applied in pattern recognition task. In this model of classification a fuzzy loss function is used. The efficiency of this algorithm is compared with the algorithm based on zero-one loss function. In this paper also influence of choice of parameter λ in selected comparison fuzzy number method on classification results are presented.

The paper deals with the recognition task with a full probabilistic information. In this problem of recognition the Bayesian statistic is applied and multistage recognition is considered. The information on objects features is fuzzy-valued. The decision rules minimize the mean risk, that is the mean value of the zero-one loss function. The probability of misclassification for local optimal strategy and the difference between probability of misclassification for the fuzzy-valued and crisp data are presented. Simple example conclude the work and influence of the width of the fuzzy number on this difference is discussed.

The paper deals with the multistage recognition task. In this problem of recognition the Bayesian statistic is applied. The decision rules minimize the mean risk, that is the mean value of the zero-one loss function. The information on objects features is fuzzy or non-fuzzy. The probability of misclassification for local optimal strategy and the difference between probability of misclassification for the both information’s are presented. Simple example of this difference conclude the work.

The paper deals with the decision rules for a Bayesian hierarchical classifier based on a decision-tree scheme. For given tree skeleton and features to be used, the optimal (Bayes) decision rules (strategy) at each non-terminal node are presented. The case has been considered when a loss (utility) function is described using fuzzy numbers. The globally optimal Bayes strategy has been calculated for the case when the loss function depends on the node of the decision tree. The model is based on the notion of fuzzy random variable and also on crisp ranking method for fuzzy numbers. The obtained result is presented as a calculation example.

The medical decision problem – acute abdominal pain diagnosis is presented in the paper. We use two methods of classification, which are based on a decision tree scheme. The first of them generates classifier only based on learning set. It is boosted C4.5 algorithm. The second approach is based on Bayes decision theory. This decision algorithm utilizes expert knowledge for specifying decision tree structure and learning set for determining mode of decision making in each node. The experts-physicians gave the decision tree for performing Bayes hierarchical classifier.