Search

Tree-based ensemble algorithms (TEAs) have had a transformative impact in various fields. However, when they are applied to real-time critical problems such as medical analysis, existing TEAs fall short of two-fold issues. The first is the trade-off among model interpretability, accuracy, and explainability. Second, traditional TEAs perform dramatically worse when the number of contributing features to model accuracy is small in comparison to the total number of features. We address the aforementioned issues in this paper and propose a novel "eXplainable Reasonably Randomised Forest" (XRRF) algorithm. The XRRF consists of four conclusive steps: learning performance, intrinsic interpretability, model accuracy, and explainability. The proposed forest algorithm is evaluated on three real-world problems (medical analysis, business analysis, and employee churn), a hybrid artificial dataset, and twenty multidisciplinary benchmark problems with varying characteristics. The experimental results demonstrate that the XRRF outperforms the six mainstream and two cutting-edge black-box and white-box algorithms, respectively. [ABSTRACT FROM AUTHOR]

Image ordinal estimation is to predict the ordinal label of a given image, which can be categorized as an ordinal regression (OR) problem. Recent methods formulate an OR problem as a series of binary classification problems. Such methods cannot ensure that the global ordinal relationship is preserved since the relationships among different binary classifiers are neglected. We propose a novel OR approach, termed convolutional OR forest (CORF), for image ordinal estimation, which can integrate OR and differentiable decision trees with a convolutional neural network for obtaining precise and stable global ordinal relationships. The advantages of the proposed CORF are twofold. First, instead of learning a series of binary classifiers independently, the proposed method aims at learning an ordinal distribution for OR by optimizing those binary classifiers simultaneously. Second, the differentiable decision trees in the proposed CORF can be trained together with the ordinal distribution in an end-to-end manner. The effectiveness of the proposed CORF is verified on two image ordinal estimation tasks, i.e., facial age estimation and image esthetic assessment, showing significant improvements and better stability over the state-of-the-art OR methods. [ABSTRACT FROM AUTHOR]

Difficult and expensive financing has always been a problem for domestic and foreign enterprises, and how to effectively improve financing efficiency and improve the financing environment is a key issue to be studied. LightGBM is an advanced machine learning algorithm, which uses histogram algorithm and Leaf-wise strategy with depth limitation to improve the accuracy of the model. However, there are almost no cases of applying this method to corporate financing risk prediction. Therefore, the paper establishes the LightGBM model to predict the financing risk profile of 186 enterprises. In order to compare the prediction performance of LightGBM for enterprise financing risk, the paper conducted comparison experiments using k-nearest-neighbors algorithm, decision tree algorithm, and random forest algorithm on the same data set. The experiments show that LightGBM has better prediction results than the other three algorithms for several metrics in corporate financing risk prediction. Therefore, we believe that the LightGBM algorithm can be used as an effective tool to predict the financing risk of enterprises. [ABSTRACT FROM AUTHOR]

Tree-based machine learning models, like random forests and decision trees, are low-complexity solutions that provide an efficient prediction for a wide range of applications. These models are particularly interesting for energy-constrained platforms since they can be implemented with simple logical operations. Tree-based accelerators are also intrinsically resilient to errors, and this can be leveraged to boost energy efficiency with approximate computing techniques. The key operations in these models are comparisons to constants, making comparators excellent candidates for approximation. This paper presents a technique to approximate comparisons to constants called C2PAx, which is capable of reducing the area and energy of tree-based accelerators. The method consists in finding alternative constants that reduce circuit area while keeping an efficient prediction performance. It is also shown that the selection of the constant parameters directly influences both hardware complexity and model performance, demanding cross-layer optimization. For that, we extend an existing framework that generates VLSI tree-based accelerators, inserting our approximation proposal that allows selecting the constant parameters that maximize energy efficiency at the cost of minor accuracy drops. Simulation results demonstrate that C2PAx outperforms the Don’t Care logic approximation technique when accuracy and energy are jointly considered. C2PAx trades accuracy for significant reductions in the VLSI area, power, delay, and energy consumption compared to precise models. [ABSTRACT FROM AUTHOR]

The random forest classifier is widely used in different fields due to its accuracy and robustness. Since its invention, the random forest algorithm is naturally developed for multi-dimensional vectorial data since features can be directly sampled during the decision tree construction procedure. In the context of discrete sequence classification, an explicit feature set is not readily available and we need to employ a feature extraction algorithm before building the random forest. However, such a predefined feature subset may limit the diversity of decision trees since the set of candidate features is composed of all subsequences. As a result, the predictive accuracy of constructed random forest classifier may be reduced. To address this, we propose a new algorithm that is able to directly build a random forest by choosing features from the set of all subsequences adaptively. To improve the running efficiency of our algorithm, the count-suffix tree is utilized to facilitate the fast frequency counting of subsequences so as to accelerate the generation of each randomized decision tree. The experimental results on 15 real datasets show that our method can outperform those state-of-the-art classification algorithms in terms of the predictive accuracy. The source code of our method can be found at: https://github.com/JiaqiWang-dlut/RSForest. [ABSTRACT FROM AUTHOR]

• It is possible to find partitions that outperform global and local approaches. • An exhaustive partitions search is not be feasible with a large number of outputs. • Genetic algorithms help accelerate the search but do not always outperform baselines. • Finding an optimal partitioning may lead to overfitting. In multi-target prediction, an instance has to be classified along multiple target variables at the same time, where each target represents a category or numerical value. There are several strategies to tackle multi-target prediction problems: the local strategy learns a separate model for each target variable independently, while the global strategy learns a single model for all target variables together. Previous studies suggested that the global strategy should be preferred because (1) learning is more efficient, (2) the learned models are more compact, and (3) it overfits much less than the local strategy, as it is harder to overfit on several targets at the same time than on one target. However, it is not clear whether the global strategy exploits correlations between the targets optimally. In this paper, we investigate whether better results can be obtained by learning multiple multi-target models on several partitions of the targets. To answer this question, we first determined alternative partitions using an exhaustive search strategy and a strategy based on a genetic algorithm, and then compared the results of the global and local strategies against these. We used decision trees and random forests as base models. The results show that it is possible to outperform global and local approaches, but finding a good partition without incurring in overfitting remains a challenging task. [ABSTRACT FROM AUTHOR]

The algorithm of C4.5 decision tree has the advantages of high classification accuracy, fast calculation speed and comprehensible classification rules, so it is widely used for medical data analysis. However, for imbalanced medical data, the classification accuracy of decision trees-based models is not ideal. Therefore, this paper proposes a cluster-based oversampling algorithm (KNSMOTE) combining Synthetic minority oversampling technique (SMOTE) and k -means algorithm. The sample classes clustered by k -means and the original sample classes are calculated to select the "safe samples" whose sample classes have not been changed. The "safe samples" are linearly interpolated to synthesize the new samples. The improved SMOTE sets the oversampling ratio according to the imbalance ratio of the original samples, which is used to synthesize the samples whose number is the same as that of the original samples. Compared with other oversampling algorithms on 8 UCI datasets, our algorithm has achieved significant advantages. Our algorithm was applied to the medical datasets, and the average values of the Sensitivity and Specificity indexes of the Random forest (RF) algorithm were 99.84% and 99.56%, respectively. [ABSTRACT FROM AUTHOR]

The foremost requirement for a decision forest to achieve better ensemble accuracy is building a set of accurate and diverse individual decision trees as base classifiers. Existing decision forest algorithms mainly differ from each other on how they induce diversity among the decision trees. At the same time, most of the drawbacks of existing algorithms originate from their induction processes of diversity. In this paper, we propose a new decision forest algorithm that is more balanced through effective synchronization between different sources of diversity. The proposed algorithm is balanced theoretically and empirically. We carried out experiments on 25 well-known data sets that are publicly available from the UCI Machine Learning Repository, to perform an extensive empirical evaluation. The experimental results indicate that the proposed algorithm has the best average ensemble accuracy rank of 1.8 compared to its closest competitor at 3.5. Using the Friedman and Bonferroni-Dunn tests, we also show that such an improvement is indeed statistically significant. In addition, the proposed algorithm is found to be competitive in terms of complexity and other relevant parameters. [ABSTRACT FROM AUTHOR]

The application of artificial intelligence models to raw multimedia data is susceptible to various data inference attacks, posing a significant risk in terms of sensitive input information leakage. Most of the existing studies on privacy-preserving multimedia applications based on artificial intelligence, focus on a single intelligent model and thus have various limitations. In this paper, we attempt to directly perturb the core component of many multimedia intelligent models in Bayesian networks and deep learning. That is, we apply conditional probability distribution estimation to guarantee the privacy of the models. At first, we present the formal problem formulation of private conditional probability distribution estimation and apply it to random forest for task classification in multimedia applications. Then, we design a simple perturbation approach called NaivePrivDistEst , to add noise to all the elements of probability estimation in random forest. Next, we present an improved approach called FastLRG , that utilizes the taxonomy tree to discretize the continuous attributes, thereby combining the attribute features to improve the prediction accuracy of random forest. Finally, we perform extensive experiments to evaluate the performance of random forest based on the proposed estimation algorithms. The experimental results indicate that the proposed models have better performance compared with existing private decision trees. • Conditional probability distribution estimation is applied to random forest for multimedia classification applications. • A naive private algorithm that directly perturbs the values of each attribute and label in a random forest is proposed. • An improved algorithm of probability distribution estimation with lower communication cost and better accuracy is proposed. • Experimental results show that the proposed private random forest has better performance than the existing decision models. [ABSTRACT FROM AUTHOR]

Random Forests and related tree-based methods are popular for supervised learning from table based data. Apart from their ease of parallelization, their classification performance is also superior. However, this performance, especially parallelizability, is offset by the loss of explainability. Statistical methods are often used to compensate for this disadvantage. Yet, their ability for local explanations, and in particular for global explanations, is limited. In the present work we propose an algebraic method, rooted in lattice theory, for the (global) explanation of tree ensembles. In detail, we introduce two novel conceptual views on tree ensemble classifiers and demonstrate their explanatory capabilities on Random Forests that were trained with standard parameters. [ABSTRACT FROM AUTHOR]