Search

Plethora of ensemble techniques have been implemented and studied in order to achieve better classification results than base classifiers. In this paper an algorithm for integration of decision trees is proposed, which means that homogeneous base classifiers will be used. The novelty of the presented approach is the usage of the simultaneous distance of the object from the decision boundary and the center of mass of objects belonging to one class label in order to determine the score functions of base classifiers. This means that the score function assigned to the class label by each classifier depends on the distance of the classified object from the decision boundary and from the centroid. The algorithm was evaluated using an open-source benchmarking dataset. The results indicate an improvement in the classification quality in comparison to the referential method - majority voting method.

In order to improve the classification performance of a single classification model, Multiple Classifier Systems (MCS) are used. One of the most common techniques utilizing multiple decision trees is the random forest, where diversity between base classifiers is obtained by bagging the training dataset. In this paper, we propose the algorithm that uses horizontal partitioning the learning set and uses decision trees as base models to obtain decision regions. In the proposed approach feature space is divided into disjoint subspace. Additionally, the location of the subspace centroids, as well as the size and location of decision regions, are used in order to determine the weights needed in the last process of creating MCS, i.e. in the integration phase. The proposed algorithm was evaluated employing multiple open-source benchmarking datasets, compared using accuracy and Matthews correlation coefficient performance measures with two existing MCS methods - random forest and majority voting. The statistical analysis confirms an improvement in recognition compared to the random forest. In addition, we proved that for infinitely dense space division proposed algorithm is equivalent to majority voting.

Integration or fusion of the base classifiers is the final stage of creating multiple classifiers system. Known methods in this step use base classifier outputs, which are class labels or values of the confidence (predicted probabilities) for each class label. In this paper we propose an integration process which takes place in the geometric space. It means that the fusion of base classifiers is done using their decision boundaries. In order to obtain one decision boundary from boundaries defined by base classifiers the median or weighted average method will be used. In addition, the proposed algorithm uses the division of the entire feature space into disjoint regions of competence as well as the process of selection of base classifiers is carried out. The aim of the experiments was to compare the proposed algorithms with the majority voting method and assessment which of the analyzed approaches to integration of the base classifiers creates a more effective ensemble.

A vital aspect of the Multiple Classifier Systems construction process is the base model integration. For example, the Random Forest approach used the majority voting rule to fuse the base classifiers obtained by bagging the training dataset. In this paper we propose the algorithm that uses partitioning the feature space whose split is determined by the decision rules of each decision tree node which is the base classification model. After dividing the feature space, the centroid of each new subspace is determined. This centroids are used in order to determine the weights needed in the integration phase based on the weighted majority voting rule. The proposal was compared with other Multiple Classifier Systems approaches. The experiments regarding multiple open-source benchmarking datasets demonstrate the effectiveness of our method. To discuss the results of our experiments, we use micro and macro-average classification performance measures.