Search

Accurately predicting buffet frequency and aerodynamic noise level is crucial in transonic buffet noise reduction studies. In this study, the Random Forest (RF) algorithm is employed to predict the Power Spectral Density (PSD) and Overall Sound Pressure Level (OASPL) distribution over the supercritical airfoil RAE2822. The study indicates that the RF algorithm exhibits greater advantages over the Multi-Layer Perceptron (MLP). This algorithm does not suffer from the problems of high-frequency divergence or inflection distortion, maintaining a reduction in prediction errors for OASPL and PSD by approximately three to four orders of magnitude. Additionally, this paper proposes a priori criterion for evaluating the accuracy of PSD prediction. When there is a strong correlation between PSD at adjacent points, a data-driven modeling approach can achieve higher prediction accuracy. However, when the root mean square error of the cross-correlation functions, auto-correlation functions, and the PSD between adjacent points are both high, the generalization ability of pure data-driven modeling is insufficient, necessitating the additional monitoring points to ensure prediction accuracy. [ABSTRACT FROM AUTHOR]

• A PV/T device with sheet-and-needle finned serpentine tube collector is analyzed. • PV panels are cooled using the water-silver nanofluid. • Random forest machine learning approach is used to develop predictive models. • Thermal, electrical and overall efficiencies are utilized as objective functions. • Utilization of needle fins improve the energy and exergy performance of the system. The Photovoltaic thermal (PVT) collector performance is numerically investigated considering the effect of using needle fins in the serpentine channel with Nanofluid (NF). The influence of increasing the nanoparticle concentration (φ) and Reynolds number (Re) on the energy and exergy features of the PVT device is examined. A comparison is made between the hydrothermal characteristics of the PVT with the finned and plain serpentine channels. The utilization of needle fins improves the thermal efficiency (η th), electrical efficiency (η el), and overall efficiency (η el) by 8.56–10.22%, 0.13–0.24%, 5.12–5.67%, respectively, against the PVT with the plain serpentine channel. Moreover, thermal exergy efficiency (ξ th), electrical exergy efficiency (ξ el), and overall exergy efficiency (ξ ov) by 8.56–1.22%, 0.13–0.24%, and 2.61–2.79%, respectively, versus the PVT with the plain serpentine channel. Moreover, the Random Forest (RF) machine learning approach is used to develop a predictive model for η th , η el , η el , ξ th , ξ el and ξ ov in terms of Re and φ. The outcomes of modeling proved that all the results were in an acceptable level of accuracy and the overall efficiency in both energy and exergy yielded superior precision in comparison with the other targets. [ABSTRACT FROM AUTHOR]

The complex and opaque decision-making process of machine learning models restrains the interpretability of predictions and makes them cannot mine results outside of learning experiences. The causality between features and the target variable can be traced by injecting counterfactual explanations into the prediction model and generating counterfactual instances using adjusted features to reverse the prediction results. Existing algorithms, such as Diverse Counterfactual Explanations (DiCE) and Counterfactual Explanations Guided by Prototypes (Proto), can generate multiple/single counterfactual(s) for a data point by global optimization in the range of all-out features to gain on a local decision range. However, these methods cannot clearly identify which features are the key causes. Moreover, a Random Forest Optimal Counterfactual Set Extractor (RF-OCSE) extracts counterfactual sets from a random forest and needs to manipulate all the internal nodes of the tree, restricting it to only tree ensemble models. To address the above shortcomings, we proposed a Counterfactual Explanation Generation method with the Minimal Feature Boundary (MFB), named ( CEG MFB ). The proposed CEG MFB algorithm consists of two stages: 1) mining the MFB, which can reverse the prediction results to restrain the generation range of counterfactual instances, and 2) constructing a counterfactual generative method for generating counterfactual instances within the MFB to realize the minimum reversing cost. To evaluate its performance, we compared the proposed CEG MFB algorithm with six baseline algorithms on 16 datasets and conducted a case study in a real scenario. The results indicate that the proposed CEG MFB algorithm outperforms the compared algorithms. [ABSTRACT FROM AUTHOR]

One of the most important hydraulic structures that has been overlooked is culverts. Regular maintenance and inspections are required to ensure that these structures are used effectively and safely. Culvert inspection, on the other hand, takes a significant amount of time and resources, and agencies cannot afford to do it on a regular basis. To overcome physical inspection challenges, this study proposed employing machine learning algorithms to predict the condition of Utah's culverts based on historical data. This study assessed multiclass classification algorithms that had not been used before. The classification machine learning algorithms employed in this study are Random Forest, Decision Tree, Support Vector Machine, k-Nearest Neighbor, and Artificial Neural Network. Also, a combined dataset from four states was used to address Utah's limited culvert inventory data. The final dataset contained 2555 culvert records. Out of the developed prediction models, Random Forest had the highest accuracy of 82%. The findings demonstrated that Random Forests could be used to improve culvert network quality while reducing culvert maintenance expenses for UDOT. The proposed model may be applied to similarly existing culvert inventories located around the nation. [ABSTRACT FROM AUTHOR]

• This paper presents a soft and hard hybrid qualitative balanced clustering (SHHQBC). • Qualitative balancing produces clusters with the lowest cardinality and highest value which are very practical. • Creating value criterion with the SHHQBC method enhanced quantitative and qualitative clustering criteria. K-means is a popular clustering method that has consistently failed to produce a balanced cluster structure. While changes in cardinality, variance, and density have arisen due to the importance of balancing in different fields, balancing has never been viewed from a qualitative viewpoint. The current paper takes a new look at cluster balancing by presenting a soft (balance-driven) and hard (balance-constrained) hybrid qualitative balanced clustering (SHHQBC). It starts by identifying and prioritizing key features using a random forest algorithm and the mean decrease in the Gini coefficient criterion. It then uses a weighted linear combination of features with the importance of above 25%, 50%, and 75% to construct a feature called value criterion. The developed clustering approach is then implemented to establish clusters with the highest value with the least cardinality or a value similar to other clusters. By implementing the SHHQBC on 14 different datasets, first soft clustering is implemented for all three cases and balanced conditions are checked. Hard clustering is then performed to make balanced conditions. Finally, the best-balanced case with the least objective function is selected. Formulating the value criterion facilitates the interpretation and labeling of clusters, and quantitative clustering criteria are improved. [ABSTRACT FROM AUTHOR]

Tree-based ensemble algorithms (TEAs) have had a transformative impact in various fields. However, when they are applied to real-time critical problems such as medical analysis, existing TEAs fall short of two-fold issues. The first is the trade-off among model interpretability, accuracy, and explainability. Second, traditional TEAs perform dramatically worse when the number of contributing features to model accuracy is small in comparison to the total number of features. We address the aforementioned issues in this paper and propose a novel "eXplainable Reasonably Randomised Forest" (XRRF) algorithm. The XRRF consists of four conclusive steps: learning performance, intrinsic interpretability, model accuracy, and explainability. The proposed forest algorithm is evaluated on three real-world problems (medical analysis, business analysis, and employee churn), a hybrid artificial dataset, and twenty multidisciplinary benchmark problems with varying characteristics. The experimental results demonstrate that the XRRF outperforms the six mainstream and two cutting-edge black-box and white-box algorithms, respectively. [ABSTRACT FROM AUTHOR]

Recently, the estimation of facial age has attracted much attention. This letter extends and improves a recently developed method (Guehairia et al., 2020) for fusing multiple deep facial features for age estimation. This method was based on deep random forests. We propose a new pipeline that integrates tensor-based subspace learning before applying DRFs. Deep face features of a training set are represented as a 3D tensor. Multi-linear Whitened Principal Component (MWPCA) and Tensor Exponential Discriminant (TEDA) are used to extract the most discriminative information. The tensor subspace features are then fed into DRFs to predict age. Experiments conducted on five public face databases show that our method can compete with many state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• Aiming for multiphase batch processes with different sampling frequencies, two types of process variables are summarized and the multiphase-based cumulative quality model is developed. It provides useful insights for the quality prediction of industrial batch process. • Aiming for the complex association relationships between multiple process variables and multiple quality indicators, information theory with copula entropy is employed to measure the variable relations for critical variables selection. • A stacking multiway random forest algorithm is proposed to develop the prediction relationship between phase-specific critical process variables and multiple quality indicators. • Experiments on a real-world industrial dataset have demonstrated that the proposed method not only has better accuracy but also has stronger robustness than previous baseline methods. As one of the most important modes of industrial production, the batch process often involves complex and continuous physicochemical reactions, making it challenging to produce identical products between different batch runs even under the same working condition. Optimization and parameter adjustments depend mainly on a good quality prediction model. However, this industrial process has "3M" characteristics of multiple process variables, multiple production phases, and multiple quality indicators, which bring considerable challenges to the accuracy and robustness of the model. This study proposes a multiphase information fusion strategy for data-driven quality prediction of industrial batch processes. Firstly, aiming for real-world industrial datasets with different sampling frequencies, two types of state variables are summarized, and the multiphase-based cumulative quality model is developed. Secondly, information theory with copula entropy is employed to characterize the association relationships between each state variable and the set of multiple quality indicators; thus, phase-specific critical variables are selected by ranking copula entropy. Lastly, a stacking multiway random forest algorithm is proposed to develop the prediction relationship between phase-specific critical variables and multiple quality indicators. Experiments on a real-world industrial dataset have demonstrated that the proposed method has better accuracy and stronger robustness than previous baseline methods. [ABSTRACT FROM AUTHOR]

In this work, strategies to devise an optimal feedback control of probabilistic Boolean control networks (PBCNs) are discussed. Reinforcement learning (RL) based control is explored in order to minimize model design efforts and regulate PBCNs with high complexities. A Q -learning random forest (Q LRF) algorithm is proposed; by making use of the algorithm, state feedback controllers are designed to stabilize the PBCNs at a given equilibrium point. Further, by adopting Q LRF stabilized closed-loop PBCNs, a Lyapunov function is defined, and a method to construct the same is presented. By utilizing such Lyapunov functions, a novel self-triggered control (STC) strategy is proposed, whereby the controller is recomputed according to a triggering schedule, resulting in an optimal control strategy while retaining the closed-loop PBCN stability. Finally, the results are verified using computer simulations. [ABSTRACT FROM AUTHOR]

• A new feature set composed of 46 features for phishing URLs detection were proposed. • An updated feature selection algorithm to obtain the most representative features. • An optimum subset composed of 9 features for phishing URLs detection was proposed. • Random Forest obtains the best Accuracy results using the feature subset proposed. • The Accuracy results obtained outperforms those reported in the literature. Phishing is a cyber-attack that exploits victims' technical ignorance or naivety and commonly involves a Uniform Resources Locator (URL). Hence, it is advantageous to detect a phishing attack by analyzing URLs before accessing them. With the raising of the Internet of Things (IoT), phishing attacks are moving to this field due to the number of IoT devices and the amount of personal information they handle. Although several approaches were proposed for phishing attacks detection, the URL-based Machine Learning approaches obtain better performance results, but all of them are dependent on the feature set used. Contradictorily, only a few works on selecting the best-suited feature set for improving the phishing detection process have been published. The present research explores how to obtain a feature set that substantially enhances the phishing detection rate in IoT environments. Hence, a feature selection algorithm was adopted and extended for getting the most representative feature set. When Random Forest is used with the proposed data representation, the phishing URL attacks discovery rate is 99.57 %. [ABSTRACT FROM AUTHOR]

Difficult and expensive financing has always been a problem for domestic and foreign enterprises, and how to effectively improve financing efficiency and improve the financing environment is a key issue to be studied. LightGBM is an advanced machine learning algorithm, which uses histogram algorithm and Leaf-wise strategy with depth limitation to improve the accuracy of the model. However, there are almost no cases of applying this method to corporate financing risk prediction. Therefore, the paper establishes the LightGBM model to predict the financing risk profile of 186 enterprises. In order to compare the prediction performance of LightGBM for enterprise financing risk, the paper conducted comparison experiments using k-nearest-neighbors algorithm, decision tree algorithm, and random forest algorithm on the same data set. The experiments show that LightGBM has better prediction results than the other three algorithms for several metrics in corporate financing risk prediction. Therefore, we believe that the LightGBM algorithm can be used as an effective tool to predict the financing risk of enterprises. [ABSTRACT FROM AUTHOR]

The continuous emergence of new malware has been a severe threat to Industrial Internet of Things (IIoT), while identifying malware through detecting malicious traffic in encrypted, drift, and imbalanced traffic streams is a challenge. This paper proposes an approach based on adaptive online analysis to accurately determine the families of malware by analyzing traffic streams which are encrypted, drift, and imbalanced. This approach is based on Improved Adaptive Random Forests (IARF), to obtain the ability of adaptive update of parameters when processing new types of malware traffic in traffic streams and being sensitive to families of malware with few samples in imbalanced traffic. We build a prototype of this approach and evaluate the performance through experiments. The experiments are based on a mixed dataset composed of data from malware-traffic-analysis.net, Lastline Inc, MCFP dataset, and CTU-13 dataset. In addition, our approach is also compared with three state-of-the-art methods. The results of the experiments show that we have obtained a 99.66% F1-score in the classification of malware families, and our classifier also performs better than the other classifiers. [ABSTRACT FROM AUTHOR]

The classification of hazard is critical in industrial informatics, as it enhances early safety alerts, supports decision-making, and facilitates policy assessment. However, previous studies have generally neglected the temporal attributes of hazards, thereby constraining the effectiveness of models. This paper introduces a new model for hazard classification termed DLGM. DLGM represents a deep learning framework, with the structural parameters of grey models to encapsulate the hazard temporal attributes. To better accommodate the fluctuations of hazard series, a new grey model termed FSGM(1,1) equipped with Fourier series is proposed. Moreover, DLGM leverages a novel hierarchical feature fusion neural network (HFFNN) to optimize feature processing. Extensive experiments across three hazard themes involving 18 large-scale industrial processes have demonstrated the competitiveness of DLGM (e.g., it surpasses benchmark models such as Random Forest and BERT by approximately 2% in both accuracy and F1), the suitability of FSGM(1,1) (e.g., its mean absolute percentage error is less than 10% and mean squared error is below 0.02) and the effectiveness of HFFNN (e.g., it enhances the accuracy and F1 of DLGM by about 1%). [ABSTRACT FROM AUTHOR]

The random forest classifier is widely used in different fields due to its accuracy and robustness. Since its invention, the random forest algorithm is naturally developed for multi-dimensional vectorial data since features can be directly sampled during the decision tree construction procedure. In the context of discrete sequence classification, an explicit feature set is not readily available and we need to employ a feature extraction algorithm before building the random forest. However, such a predefined feature subset may limit the diversity of decision trees since the set of candidate features is composed of all subsequences. As a result, the predictive accuracy of constructed random forest classifier may be reduced. To address this, we propose a new algorithm that is able to directly build a random forest by choosing features from the set of all subsequences adaptively. To improve the running efficiency of our algorithm, the count-suffix tree is utilized to facilitate the fast frequency counting of subsequences so as to accelerate the generation of each randomized decision tree. The experimental results on 15 real datasets show that our method can outperform those state-of-the-art classification algorithms in terms of the predictive accuracy. The source code of our method can be found at: https://github.com/JiaqiWang-dlut/RSForest. [ABSTRACT FROM AUTHOR]

• The use of deep generative models to swap faces will reduce the detected feature points. • DeepFakes have fewer feature points than real faces, especially in detailed regions. • We propose FFR_FD as a vector to represent the facial feature description. • Random forest trained with FFR_FD can realize state-of-the-art DeepFake detection. • Our DeepFake detection method is effective, efficient, and generalizable. DeepFakes are widespread on social networks, and they result in severe information concerns. Although various detection methods have been proposed, there are still practical limitations. Previous specific artifact-based methods were insufficient to capture fine-grained features, which limited their effectiveness against advanced DeepFakes. Current DNN-based detectors tend to trade high costs for performance improvement, and are not efficient enough, given that DeepFakes can be created easily by mobile apps, and DNN-based models require expensive computational resources. Furthermore, most methods lack generalizability under the cross-dataset scenario. In this work, we instead mine the more subtle and generalized defects of DeepFakes and propose the fused facial region _ feature descriptor (FFR _ FD) , which is only a vector of the discriminative feature description, for effective and fast DeepFake detection. We show that DeepFake faces have fewer feature points than real ones, especially in facial regions. FFR_FD capitalizes on such key observations, and thus has strong generalizability. We train a random forest classifier with FFR_FD to achieve efficient detection. Extensive experiments on six large-scale DeepFake datasets demonstrate the effectiveness of our lightweight method. Our model generalizes well on the challenging Celeb-DF (v2) dataset, with 0.706 AUC, which is superior to most state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Missing values are a common phenomenon in modern medical research of complex diseases. The data often contains nominal or categorical variables, for example, single nucleotide polymorphisms (SNPs) in genetic studies. If the missing values are not handled properly, the downstream statistical analysis of incomplete data may be biased. While various imputation methods are available for metrically scaled variables, methods for categorical data are scarce. An imputation method that has been shown to work well for high dimensional metrically scaled variables is the imputation by nearest neighbor methods. In this paper, we propose a weighted nearest neighbors approach to impute missing values in categorical variables in high dimensional datasets. The proposed method explicitly uses the information on the association among attributes. Using different simulation settings, the performance is compared with available imputation methods. A variety of real data sets, containing heart, DNA, and lymphatic cancer, is also used to support the results obtained by simulations. The results show that the weighting of attributes yields smaller imputation errors than existing approaches like random forest and MICE. [ABSTRACT FROM AUTHOR]

Fare evasion behaviours in metro stations happen all the time with the continuous improvement of the metro network, leading to likely mass panic and unhealthy effect on the metro operation. This paper proposes an approach of detecting individual fare evasion behaviours. The method first estimates human pose by the algorithm of OpenPose to obtain human skeleton information and conducts multi-target tracking by the algorithm of Pose Flow to obtain human skeleton sequence. It then extracts features from the human pose, followed by the adoption of the relative distance, the angle and Random Forests to establish a fare evasion behaviour detection model. Since missing problems and false detections occur in detections in a single frame, time series are used to increase the accuracy. The output of the time series curve is processed by One-Hot Coding, Kalman Filtering and optimum threshold to obtain the final prediction of human status. Extensive experiments are conducted, and results show that our method can effectively recognize the human status and detect individual fare evasion behaviours, including jumps and squats. [ABSTRACT FROM AUTHOR]

The increased amount of high-dimensional imbalanced data in online social networks challenges existing feature selection methods. Although feature selection methods such as principal component analysis (PCA) are effective for solving high-dimensional imbalanced data problems, they can be computationally expensive. Hence, an effortless approach for identifying meaningful features that are indicative of anomalous behaviour between humans and malicious bots is presented herein. The most recent Twitter dataset that encompasses the behaviour of various types of malicious bots (including fake followers, retweet spam, fake advertisements, and traditional spambots) is used to understand the behavioural traits of such bots. The approach is based on Benford's law for predicting the frequency distribution of significant leading digits. This study demonstrates that features closely obey Benford's law on a human dataset, whereas the same features violate Benford's law on a malicious bot dataset. Finally, it is demonstrated that the features identified by Benford's law are consistent with those identified via PCA and the ensemble random forest method on the same datasets. This study contributes to the intelligent detection of malicious bots such that their malicious activities, such as the dissemination of spam, can be minimised. [ABSTRACT FROM AUTHOR]

A one-shot f ederated t ransfer learning method using r andom f orests (FTRF) is developed to improve the prediction accuracy at a target data site by leveraging information from auxiliary sites. Both theoretical and numerical results show that the proposed federated transfer learning approach is at least as accurate as the model trained on the target data alone regardless of possible data heterogeneity, which includes imbalanced and non-IID data distributions across sites and model mis-specification. FTRF has the ability to evaluate the similarity between the target and auxiliary sites, enabling the target site to autonomously select more similar site information to enhance its predictive performance. To ensure communication efficiency, FTRF adopts the model averaging idea that requires a single round of communication between the target and the auxiliary sites. Only fitted models from auxiliary sites are sent to the target site. Unlike traditional model averaging, FTRF incorporates predicted outcomes from other sites and the original variables when estimating model averaging weights, resulting in a variable-dependent weighting to better utilize models from auxiliary sites to improve prediction. Five real-world data examples show that FTRF reduces the prediction error by 2-40% compared to methods not utilizing auxiliary information. [ABSTRACT FROM AUTHOR]

Estimates of causal effects are needed to answer what-if questions about shifts in policy, such as new treatments in pharmacology or new pricing strategies for business owners. A new non-parametric approach is proposed to estimate the heterogeneous treatment effect based on random forests (HTERF). The potential outcome framework with unconfoundedness shows that the HTERF is pointwise almost surely consistent with the true treatment effect. Interpretability results are also presented. [ABSTRACT FROM AUTHOR]

This study develops an oversampling method designed for rebalancing datasets in impact damage classification of reinforced concrete walls. The proposed method addresses class imbalance and overlap issues using the Wasserstein Generative Adversarial Network with Gradient Penalty (WGANGP) approach, aiming to enhance classification accuracy. Initially, sample weights are calculated for overlapped and non-overlapped samples based on their proximity to the decision boundary via an initial classification process using the Support Vector Machine algorithm. Then, these weighted samples are combined with the original dataset to form a training dataset for the WGANGP model. To effectively learn from limited data samples and stabilize the training process, the Model Agnostic Meta-learning (MAML) method is incorporated into the WGANGP algorithm. New samples are generated to balance the impact damage dataset using the generator network of the WGANGP model. The proposed method is compared to advanced oversampling methods handling overlapping, noisy, near-borderline samples between the classes. Four well-known classifiers are trained on the impact damage dataset, including Multilayer Perceptron (MLP), Random Forest (RF), Extreme Gradient Boosting (XGB), and Support Vector Machine (SVM). Implementation results demonstrate the efficacy of the proposed oversampling technique on the impact damage dataset, outperforming others in terms of F1 score. [ABSTRACT FROM AUTHOR]

For hierarchical classification tasks, label relationships can be represented as a hierarchical structure ranging from coarse-grained to fine-grained. Existing hierarchical classifications typically employ a top-down classification approach, which leads to significant inter-level error propagation. Moreover, none of the existing approaches consider the impact of the path weights of different classes on the classification. In this paper, we propose a Hierarchical Classification method with Exponential Weighting of Multi-granularity Paths (HCEWMP), which combines path weights and hierarchical structure to propose a new hierarchical classification framework. Firstly, HCEWMP decomposes the datasets from coarse-grained to fine-grained based on the hierarchical structure and assigns weights to paths by the data distribution. Secondly, two different weighting strategies, probability weighting, and exponential weighting, are considered to calculate the probability of each class. Thirdly, the fine-grained top k classes are selected based on the probability descending order. Finally, HCEWMP obtains the best-predicted class using a random forest classifier. Compared with eight different algorithms on seven datasets, our experimental results demonstrate that the proposed method is effective in addressing the inter-level error propagation problem. The exponential weighting strategy has superior results among the two strategies, further indicating the significance of path weighting in hierarchical classification. [ABSTRACT FROM AUTHOR]

In big data era, some data, becoming meaningless or illegal over time and space, need to be deleted from historical knowledge. It is a challenging problem, called machine unlearning, to efficiently forget the information of those outdated data from historical models. Some unlearning techniques have been proposed in loss-well-defined classification models, such as SVM, Random Forest, and Federated learning model. Yet, it is under study to remove outdated data from learned feature selection in fuzzy rough philosophy. To narrow this gap, we propose a fuzzy rough unlearning model for feature selection. Specifically, the outdated information is first identified in a compact set, called unlearning set, that remarkably shrinks the search space of feature selection. Then, the unlearning mechanisms containing two main theorems are proposed by leveraging unlearning set, followed by a feature selection unlearning algorithm. Theoretical analyses verify that the proposed unlearning algorithm is equivalent to the traditional algorithm retraining on the remaining data with the beginning of historical results. Experimentally, extensive results on 20 datasets demonstrate that our proposed unlearning methods perform effectively with remarkably less time cost. For the source code of ARU, please refer to: https://github.com/yuxin370/ARU. [ABSTRACT FROM AUTHOR]

• It is possible to find partitions that outperform global and local approaches. • An exhaustive partitions search is not be feasible with a large number of outputs. • Genetic algorithms help accelerate the search but do not always outperform baselines. • Finding an optimal partitioning may lead to overfitting. In multi-target prediction, an instance has to be classified along multiple target variables at the same time, where each target represents a category or numerical value. There are several strategies to tackle multi-target prediction problems: the local strategy learns a separate model for each target variable independently, while the global strategy learns a single model for all target variables together. Previous studies suggested that the global strategy should be preferred because (1) learning is more efficient, (2) the learned models are more compact, and (3) it overfits much less than the local strategy, as it is harder to overfit on several targets at the same time than on one target. However, it is not clear whether the global strategy exploits correlations between the targets optimally. In this paper, we investigate whether better results can be obtained by learning multiple multi-target models on several partitions of the targets. To answer this question, we first determined alternative partitions using an exhaustive search strategy and a strategy based on a genetic algorithm, and then compared the results of the global and local strategies against these. We used decision trees and random forests as base models. The results show that it is possible to outperform global and local approaches, but finding a good partition without incurring in overfitting remains a challenging task. [ABSTRACT FROM AUTHOR]

• Highlight soccer detection is a hard but relevant task for match summarization. • An automatic system for highlight soccer identification is proposed • Highly discriminative low-level audio and video features are produced. • It selects a sparse subset of representative frames from long-term sliding windows. • It achieve s a match-time compression rate of 94%, identifying all highlights (100%). The massive growth of audiences eager for sport content has substantially increased workers' demand in this profitable segment. Highlight identification is vital for summarizing football matches. Decision support tools can significantly reduce the number of company employees required to tackle such a task, widely benefiting workforce resource allocation. This paper discusses the development of an automatic football highlight detector. The proposed system exploits discriminative low-level audio and video features extracted from a compact set of irregularly time–spaced frames that integrate a long-term sliding window. A new mixed wrapper-probabilistic algorithm leverages a cost-effective selection of the most significant frames submitted to a robust multi-frame consensus classification scheme. By considering a comprehensive database integrating 30 full matches, the proposed approach achieves a highlight identification rate of 100% (including all annotated goals), conjugated with a match-time compression rate of about 94%, when employing a Random Forest classifier. [ABSTRACT FROM AUTHOR]

The algorithm of C4.5 decision tree has the advantages of high classification accuracy, fast calculation speed and comprehensible classification rules, so it is widely used for medical data analysis. However, for imbalanced medical data, the classification accuracy of decision trees-based models is not ideal. Therefore, this paper proposes a cluster-based oversampling algorithm (KNSMOTE) combining Synthetic minority oversampling technique (SMOTE) and k -means algorithm. The sample classes clustered by k -means and the original sample classes are calculated to select the "safe samples" whose sample classes have not been changed. The "safe samples" are linearly interpolated to synthesize the new samples. The improved SMOTE sets the oversampling ratio according to the imbalance ratio of the original samples, which is used to synthesize the samples whose number is the same as that of the original samples. Compared with other oversampling algorithms on 8 UCI datasets, our algorithm has achieved significant advantages. Our algorithm was applied to the medical datasets, and the average values of the Sensitivity and Specificity indexes of the Random forest (RF) algorithm were 99.84% and 99.56%, respectively. [ABSTRACT FROM AUTHOR]

The foremost requirement for a decision forest to achieve better ensemble accuracy is building a set of accurate and diverse individual decision trees as base classifiers. Existing decision forest algorithms mainly differ from each other on how they induce diversity among the decision trees. At the same time, most of the drawbacks of existing algorithms originate from their induction processes of diversity. In this paper, we propose a new decision forest algorithm that is more balanced through effective synchronization between different sources of diversity. The proposed algorithm is balanced theoretically and empirically. We carried out experiments on 25 well-known data sets that are publicly available from the UCI Machine Learning Repository, to perform an extensive empirical evaluation. The experimental results indicate that the proposed algorithm has the best average ensemble accuracy rank of 1.8 compared to its closest competitor at 3.5. Using the Friedman and Bonferroni-Dunn tests, we also show that such an improvement is indeed statistically significant. In addition, the proposed algorithm is found to be competitive in terms of complexity and other relevant parameters. [ABSTRACT FROM AUTHOR]

• Adopt clustering to improve the performance of classification algorithms. • Propose a dynamic weighted probability combination strategy. • Apply an improved PSO to optimize multiple key sensitive parameters in CIEL. • Demonstrate CIEL's good performance experimentally on a large number of datasets. Classification is a vital task in machine learning. By learning patterns of samples of known categories, the model can develop the ability to distinguish the categories of samples of unknown categories. Noticing the advantages of the clustering method in cluster structure analysis, we combine the clustering and classification methods to develop the novel cluster-based intelligence ensemble learning (CIEL) method. We use the clustering method to analyze the inherent distribution of the data and divide all the samples into clusters according to the characteristics of the dataset. Then, for each specific cluster, we use different classification algorithms to establish the corresponding classification model. Finally, we integrate the prediction results of each base classifier to form the final prediction result. In view of the problem of parameter sensitivity, we use a swarm intelligence algorithm to optimize the key parameters involved in the clustering, classification, and ensemble stages in order to boost the classification performance. To assess the effectiveness of CIEL, we perform tenfold cross-validation experiments on the 24 benchmark datasets provided by UCI and KEEL. Designed to improve the performance of the classifiers, CIEL outperforms other popular machine learning methods such as naive Bayes, k -nearest neighbors, random forest, and support vector machine. [ABSTRACT FROM AUTHOR]

Random Forest remains one of Data Mining's most enduring ensemble algorithms, achieving well-documented levels of accuracy and processing speed, as well as regularly appearing in new research. However, with data mining now reaching the domain of hardware-constrained devices such as smartphones and Internet of Things (IoT) devices, there is continued need for further research into algorithm efficiency to deliver greater processing speed without sacrificing accuracy. Our proposed FastForest algorithm achieves this result through a combination of three optimising components - Subsample Aggregating ('Subbagging'), Logarithmic Split-Point Sampling and Dynamic Restricted Subspacing. Empirical testing shows FastForest delivers an average 24% increase in model-training speed compared with Random Forest whilst maintaining (and frequently exceeding) classification accuracy over tests involving 45 datasets on both PC and smartphone platforms. Further tests show FastForest achieves favourable results against a number of ensemble classifiers including implementations of Bagging and Random Subspace. With growing interest in machine-learning on mobile devices, FastForest provides an efficient ensemble classifier that can achieve faster results on hardware-constrained devices, such as smartphones. [ABSTRACT FROM AUTHOR]

• This research addresses a family of new kernels inspired by the Choquet integral. • The proposed kernel is able to capture interactions. • We embed this kernel into the kernel regression context. • We show that this kernel is more robust to over-fitting thread compared to other state-of-theart. • To verify the feasibility of this approach we conducted experiments on real dataset. Recently, we have presented a new family of kernels on the basis of the discrete Choquet integral. While a naïve computation of this kernel has an exponential complexity in the number of features, we have proposed an efficient approach with computational complexity of O (m 2 log (m)). 1 1 m is the number of features. This kernel family is able to recognize dependencies between features and moreover it can be regularized through a proper selection of q -additivity. In fact, to reduce the effect of over-fitting there is an opportunity to restrict the flexibility of kernel to a lower degree. A key feature of the Choquet integral in a data-driven way is its monotonicity, however, this representation does not consider any monotonicity constraint; hence it is versatile for other applications, too. This issue is highlighted in the experimental study. In this regard, we apply the Choquet kernel for regression task and compare the performance of the proposed kernel versus state-of-the-art support kernel-based regression methods as well as random forest. [ABSTRACT FROM AUTHOR]

In recent years, many different differential evolution (DE) variants have been proposed to solve real-world optimization problems. However, the performance of them is largely determined by the selection of the mutation strategy, an approach to choose favorable mutation strategy when solving various optimization problems has attracted increasing attention recently. In this paper, we propose a DE with an adaptive mutation operator based on fitness landscape (FLDE). The application of fitness landscape to DE requires three stages. First, we analyzed the fitness landscape features of each benchmark training function, a total of 45 benchmark functions are taken from CEC2014 and 2015. Then, the relationship between three mutation strategies and fitness landscape features is trained by random forest (RF) offline. Finally, the trained RF is used to predict which mutation strategy should be utilized to perform mutation operator for each problem during the evolutionary process. Besides, a historical memory parameter adaption mechanism and population size linear reduction are applied to the FLDE. The CEC2017 benchmark set is utilized to perform the experiments, and five well-known DE variant algorithms are compared with the FLDE algorithm. The experimental results attest that the proposed FLDE algorithm is highly competitive with the other five DE algorithms. [ABSTRACT FROM AUTHOR]

The extremely diffused architecture of the Internet enables the propagation of malware and presents a significant challenge for the development of defenses against such malware propagation. Although machine learning-based malware detection models can improve approaches in response to this problem, their detection rates vary according to their features and classification methods. Single machine learning approaches for malware detection can vary in effectiveness according to the suitability of their classifiers despite the use of an appropriate training dataset. Some classifiers result in high detection rates with a malicious training dataset but have low detection rates with a benign training dataset, and false positive rates are particularly dependent on the use of appropriate classifiers. In this paper, we propose a machine learning-based hybrid decision model that can achieve a high detection rate with a low false positive rate. This hybrid model combines a random forest and a deep learning model using 12 hidden layers to determine malware and benign files, respectively. This model also includes certain proposed voting rules to make final decisions. In an experiment involving 6,395 atypical samples, this hybrid decision model achieved a higher detection rate (85.1% and standard deviation of 0.006) than that of the prior model (65.5%) without voting rules. [ABSTRACT FROM AUTHOR]

Computational fluid dynamics is a cornerstone of the modern aerospace industry, providing important insights through aerodynamic analysis while reducing the need for expensive experiments and tests. A consistent spatial discretization on so-called grids approximates the analytical solution of the partial differential equation with increasing number of discrete points. But computing a rigorous amount of CFD simulations on fine grids can be a daunting task due to the high computational cost involved. Thus, it is of interest to find methods which generate accurate results while keeping computational costs low. This work proofs that machine learning as a post-processing tool is capable of improving the accuracy of coarse grid simulations. The results show that three machine learning models with varying complexity, namely random forest, neural network, and graph neural network, are capable of finding patterns in coarse grid simulations. These patterns are used for a vertex-wise prediction of the discretization error to approximate the field variables of interest of the corresponding fine grid simulation mapped onto the coarse grid. Initial training and testing is performed on the two dimensional RAE2822 airfoil leading to corrected flow fields, improved surface integrals and coefficients, even when shocks are present. Additional tests are performed on the RAE5212 airfoil, showing the generalization limits of the trained models. Finally, the training and prediction is showcased on a three dimensional geometry. The proposed method promises to reduce computational expenses while increasing the accuracy of the coarse grid results which works locally, e.g. it corrects the error for each vertex individually and is therefore not restricted by the number of grid points. The presented results obtained by the machine learning models during post-processing are a promising baseline for more integrated developments, where the models will interact in a dynamic fashion with the flow solver to further improve coarse grid simulations. • Point-wise approximation of fine grid CFD simulations on corresponding coarse grid. • Learning the discretization error on 2D and 3D turbulent transonic flows. • Improved accuracy on less expensive coarse grid computation achievable. • Graph neural networks account for information of neighboring data points. • Graph neural networks achieve higher accuracy than neural networks. [ABSTRACT FROM AUTHOR]

The challenges in detecting antineutrinos by segmented antineutrino detectors (AD) pushed us to get more exact results using new methods, including data mining and machine learning (ML). In this research, IRAND (a segmented plastic scintillator IRan ANtineutrino Detector; a 10 × 10 array of plastic scintillation) has been simulated using Geant4 Toolkit. Several features of energy deposition including behavior of time, energy, position have been collected and stored in detail from each event to provide the complete dataset with a minor loss for use in powerful ML packages. We utilized a variety of classifiers in ML in which Random Forest shows the best performance. Also, a Geant4-based dedicated code (IRAND-Sim) has been provided for IRAND. [ABSTRACT FROM AUTHOR]

• We propose a privacy-preserving and high-accurate outsourced disease predictor on random forest. • We design secure computation protocols over rational numbers to guarantee computation accuracy. • Our system can provide secure disease predictor over ciphertexts in multi-data source settings. • We prove that our system can implement privacy protection and high-accurate prediction. Training data distributed across multiple different institutions is ubiquitous in disease prediction applications. Data collection may involve multiple data sources who are willing to contribute their datasets to train a more precise classifier with a larger training set. Nevertheless, integrating multiple-source datasets will leak sensitive information to untrusted data sources. Hence, it is imperative to protect multiple-source data privacy during the predictor construction process. Besides, since disease diagnosis is strongly associated with health and life, it is vital to guarantee prediction accuracy. In this paper, we propose a privacy-preserving and high-accurate outsourced disease predictor on random forest, called PHPR. PHPR system can perform secure training with medical information which belongs to different data owners, and make accurate prediction. Besides, the original data and computed results in the rational field can be securely processed and stored in cloud without privacy leakage. Specifically, we first design privacy-preserving computation protocols over rational numbers to guarantee computation accuracy and handle outsourced operations on-the-fly. Then, we demonstrate that PHPR system achieves secure disease predictor. Finally, the experimental results using real-world datasets demonstrate that PHPR system not only provides secure disease predictor over ciphertexts, but also maintains the prediction accuracy as the original classifier. [ABSTRACT FROM AUTHOR]

Abstract It is shown how the problem of estimating conditional Kendall's tau can be rewritten as a classification task. Conditional Kendall's tau is a conditional dependence parameter that is a characteristic of a given pair of random variables. The goal is to predict whether the pair is concordant (value of 1) or discordant (value of − 1) conditionally on some covariates. The consistency and the asymptotic normality of a family of penalized approximate maximum likelihood estimators is proven, including the equivalent of the logit and probit regressions in our framework. Specific algorithms are detailed, adapting usual machine learning techniques, including nearest neighbors, decision trees, random forests and neural networks, to the setting of the estimation of conditional Kendall's tau. Finite sample properties of these estimators and their sensitivities to each component of the data-generating process are assessed in a simulation study. Finally, all these estimators are applied to a dataset of European stock indices. [ABSTRACT FROM AUTHOR]

The application of artificial intelligence models to raw multimedia data is susceptible to various data inference attacks, posing a significant risk in terms of sensitive input information leakage. Most of the existing studies on privacy-preserving multimedia applications based on artificial intelligence, focus on a single intelligent model and thus have various limitations. In this paper, we attempt to directly perturb the core component of many multimedia intelligent models in Bayesian networks and deep learning. That is, we apply conditional probability distribution estimation to guarantee the privacy of the models. At first, we present the formal problem formulation of private conditional probability distribution estimation and apply it to random forest for task classification in multimedia applications. Then, we design a simple perturbation approach called NaivePrivDistEst , to add noise to all the elements of probability estimation in random forest. Next, we present an improved approach called FastLRG , that utilizes the taxonomy tree to discretize the continuous attributes, thereby combining the attribute features to improve the prediction accuracy of random forest. Finally, we perform extensive experiments to evaluate the performance of random forest based on the proposed estimation algorithms. The experimental results indicate that the proposed models have better performance compared with existing private decision trees. • Conditional probability distribution estimation is applied to random forest for multimedia classification applications. • A naive private algorithm that directly perturbs the values of each attribute and label in a random forest is proposed. • An improved algorithm of probability distribution estimation with lower communication cost and better accuracy is proposed. • Experimental results show that the proposed private random forest has better performance than the existing decision models. [ABSTRACT FROM AUTHOR]

In personalised health, small datasets with missing data are quite common. Current Machine Learning methods are unable to process such datasets in a meaningful way due to the huge data volume requirement. To address this problem, we propose a new Small and iNcomplete Dataset Analyser (SaNDA) to process such datasets in a meaningful way. Due to the characteristics of these datasets and the criticality of the domain, an explainable method is mandatory to facilitate decision-making interpretation. Thus, SaNDA prioritises explainability over efficiency by design. We evaluated our proposal against Random Forest as a baseline for explainable methods, and against gcForest as state-of-the-art for small datasets. We observed that our proposal outperforms Random Forest when there is more missing data and/or lower number of entries in the dataset, obtaining less favourable results over larger, well-curated datasets. It is also preferable than gcForest due to its explainability and privacy protection capabilities. Given the difficulties in obtaining complete, reliable data in the healthcare field, we consider that our proposal could be useful for practitioners. • Personalised Health datasets are small and full of missing data. • Traditional methods require huge and complete datasets. • We propose an explainable method tweaked for those datasets. • We compare our method with Random Forest and gcForest. • Our method beats Random Forest when data is missing. [ABSTRACT FROM AUTHOR]

Random Forests and related tree-based methods are popular for supervised learning from table based data. Apart from their ease of parallelization, their classification performance is also superior. However, this performance, especially parallelizability, is offset by the loss of explainability. Statistical methods are often used to compensate for this disadvantage. Yet, their ability for local explanations, and in particular for global explanations, is limited. In the present work we propose an algebraic method, rooted in lattice theory, for the (global) explanation of tree ensembles. In detail, we introduce two novel conceptual views on tree ensemble classifiers and demonstrate their explanatory capabilities on Random Forests that were trained with standard parameters. [ABSTRACT FROM AUTHOR]

Current automatic data-driven vulnerability detection in smart contracts selects and processes features of interest under black box settings without empirical justification. In this paper, we propose a smart contract testing methodology that bestows developers with flexible, practical and customizable strategies to detect vulnerabilities. Our work enforces strong whitebox knowledge to a series of supervised multimodal tasks under static analysis. Each task encapsulates a vulnerability detection branch test and pipelines feature selection, dimension unification, feature fusion, model training and decision-making. We exploit multiple features made up of code and graph embeddings at the single modality level (intramodal settings) and across individual modalities (intermodal settings). We assign each task to either intramodal or intermodal settings, and show how to train state-of-the-art self-attentive bi-LSTM, textCNN, and random forest (RF) models to extract a joint multimodal feature representation per task. We evaluate our framework over 101,082 functions extracted from the SmartEmbed dataset, and rank each multimodal vulnerability mining strategy in terms of detection performance. Extensive experiments show that our work outperforms existing schemes, and the highest performance reaches 99.71%. [ABSTRACT FROM AUTHOR]

Highlights • Exploit patterns in filter measure values to auto-identify an optimal feature subset. • Reduce 79.2% of feature dimensionality without sacrificing detection accuracy. • Boost detection accuracy by using the recommended best performing classifier. • Robust and adaptable to different datasets with significant performance gain. • Prescribe a set of baseline features as the de facto standard for phishing detection. Abstract This paper proposes a new feature selection framework for machine learning-based phishing detection system, called the Hybrid Ensemble Feature Selection (HEFS). In the first phase of HEFS, a novel Cumulative Distribution Function gradient (CDF-g) algorithm is exploited to produce primary feature subsets, which are then fed into a data perturbation ensemble to yield secondary feature subsets. The second phase derives a set of baseline features from the secondary feature subsets by using a function perturbation ensemble. The overall experimental results suggest that HEFS performs best when it is integrated with Random Forest classifier, where the baseline features correctly distinguish 94.6% of phishing and legitimate websites using only 20.8% of the original features. In another experiment, the baseline features (10 in total) utilised on Random Forest outperforms the set of all features (48 in total) used on SVM, Naive Bayes, C4.5, JRip, and PART classifiers. HEFS also shows promising results when benchmarked using another well-known phishing dataset from the University of California Irvine (UCI) repository. Hence, the HEFS is a highly desirable and practical feature selection technique for machine learning-based phishing detection systems. [ABSTRACT FROM AUTHOR]

Abstract Finite mixture regression (FMR) is widely used for modeling data that originate from heterogeneous populations. In these settings, FMR can offer increased predictive power compared to more traditional one-class models. However, existing FMR methods rely heavily on mixtures of linear models, where the linear predictor must be given as an input. A flexible FMR model is presented using a combination of the random forest learner and a penalized linear FMR. The performance of the new method is assessed by predictive log-likelihood in extensive simulation studies. The method is shown to achieve equal performance with the existing FMR methods when the true regression functions are in fact linear and superior performance in cases where at least one of the regression functions is nonlinear. The method can handle a large number of covariates, and its predictive ability is not greatly affected by surplus variables. [ABSTRACT FROM AUTHOR]

Highlights • A novel binary classification scheme for prediction of drug side effect was built. • A prediction model for the binary classification was proposed. • The prediction model integrated several heterogeneous information of drugs. • Several comparisons were performed to indicate the effectiveness of the model. • Drug similarity in fingerprint is most related to drug side effects. Abstract Drugs can produce intended therapeutic effects to treat different diseases. However, they may also cause side effects at the same time. For an approved drug, it is best to detect all side effects it can produce. Otherwise, it may bring great risks for pharmaceuticals companies as well as be harmful to human body. It is urgent to design quick and reliable identification methods to detect the side effects for a given drug. In this study, a binary classification model was proposed to predict drug side effects. Different from most previous methods, our model termed the pair of drug and side effect as a sample and convert the original problem to a binary classification problem. Based on the similarity idea, each pair was represented by five features, each of which was derived from a type of drug property. The strong machine learning algorithm, random forest, was adopted as the prediction engine. The ten-fold cross-validation on five datasets with different negative samples indicated that the proposed model yielded a good performance of Matthews correlation coefficient around 0.550 and AUC around 0.8492. In addition, we also analyzed the contribution of each drug property for construction of the model. The results indicated that drug similarity in fingerprint was most related to the prediction of drug side effects and all drug properties gave less or more contributions. [ABSTRACT FROM AUTHOR]

Signature verification is widely used in the fields of finance, access control and security. However, one of the biggest problems associated with signature verification is its instability. This instability may concern specific fragments of a signature or its entirety. The location of the unstable fragments in a signature depends on individual writing styles, which makes the signature verification more difficult, resulting in a negative impact on recognition effectiveness. In this paper the authors propose a new method of signature verification. The key stage of this approach is the determination of reference signature stability. The proposed signature stability measure, based on fuzzy set theory, is a biometric strategy that has never been used before. The fragments of one reference signature that differ from the corresponding fragments of the remaining reference signatures of the same person are treated as unstable fragments and will not be taken into consideration when comparing the reference sample with the signature being verified. The proposed method employs fuzzy sets to extract a signature’s stable fragments. To test the proposed method’s performance, seven different classifiers were used: PSO oriented, k -Nearest-Neighbor, Naive Bayes, Random Forest, RIDOR, Support Vector Machine and J48. Experiments conducted on the two independent datasets demonstrated that the method proposed here returns highly satisfactory results, outperforming the other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Active learning has gained attention as a method to expedite the learning curve of classifiers. To this end, uncertainty sampling is a widely adopted strategy that selects instances closer to the decision boundary. However, uncertainty sampling alone may not be sufficient in batch active learning due to the redundancy of instances and its susceptibility to outliers. In this study, we utilize query-by-committee (QBC) for uncertainty and demonstrate that its performance can be improved by introducing diversity and density in instance utility. Test results show that uncertainty sampling by QBC can be significantly improved with diversity and density incorporated in instance selection. Furthermore, we investigate several distance measures for use in diversity and density and show that random forest dissimilarity can be an effective distance measure in batch active learning. The effects of the characteristics of the data on the results are also analyzed. [ABSTRACT FROM AUTHOR]

Hypothesis testing of variable importance measures (VIMPs) is still the subject of ongoing research. This particularly applies to random forests (RF), for which VIMPs are a popular feature. Among recent developments, heuristic approaches to parametric testing have been proposed whose distributional assumptions are based on empirical evidence. Other formal tests under regularity conditions were derived analytically. But these approaches can be computationally expensive or even practically infeasible. This problem also occurs with non-parametric permutation tests, which are, however, distribution-free and can generically be applied to any kind of prediction model and VIMP. Embracing this advantage, it is proposed to use sequential permutation tests and sequential p-value estimation to reduce the computational costs associated with conventional permutation tests. These costs can be particularly high in case of complex prediction models. Therefore, RF's popular and widely used permutation VIMP (pVIMP) serves as a practical and relevant application example. The results of simulation studies confirm the theoretical properties of the sequential tests, that is, the type-I error probability is controlled at a nominal level and a high power is maintained with considerably fewer permutations needed compared to conventional permutation testing. The numerical stability of the methods is investigated in two additional application studies. In summary, theoretically sound sequential permutation testing of VIMP is possible at greatly reduced computational costs. Recommendations for application are given. A respective implementation for RF's pVIMP is provided through the accompanying R package rfvimptest. [ABSTRACT FROM AUTHOR]

Learning a Bayesian networks with bounded treewidth is important for reducing the complexity of the inferences. We present a novel anytime algorithm (k-MAX) method for this task, which scales up to thousands of variables. Through extensive experiments we show that it consistently yields higher-scoring structures than its competitors on complete data sets. We then consider the problem of structure learning from incomplete data sets. This can be addressed by structural EM, which however is computationally very demanding. We thus adopt the novel k-MAX algorithm in the maximization step of structural EM, obtaining an efficient computation of the expected sufficient statistics. We test the resulting structural EM method on the task of imputing missing data, comparing it against the state-of-the-art approach based on random forests. Our approach achieves the same imputation accuracy of the competitors, but in about one tenth of the time. Furthermore we show that it has worst-case complexity linear in the input size, and that it is easily parallelizable. [ABSTRACT FROM AUTHOR]

In the context of the Semantic Web, assigning individuals to their respective classes is a fundamental reasoning service. It has been shown that, when purely deductive reasoning falls short, this problem can be solved as a prediction task to be accomplished through inductive classification models built upon the statistical evidence elicited from ontological knowledge bases. However also these data-driven alternative classification models may turn out to be inadequate when instances are unevenly distributed over the various targeted classes To cope with this issue, a framework based on logic decision trees and ensemble learning is proposed. The new models integrate the Dempster–Shafer theory with learning methods for terminological decision trees and forests . These enhanced classification models allow to explicitly take into account the underlying uncertainty due to the variety of branches to be followed up to classification leaves (in the context of a single tree) and/or to the different trees within the ensemble model (the forest). In this extended paper, we propose revised versions of the algorithms for learning Evidential Terminological Decision Trees and Random Forests considering alternative heuristics and additional evidence combination rules with respect to our former preliminary works. A comprehensive and comparative empirical evaluation proves the effectiveness and stability of the classification models, especially in the form of ensembles. [ABSTRACT FROM AUTHOR]

Recent studies in Machine Learning indicates that the classifiers most likely to be the bests are the random forests. As an ensemble classifier, random forest combines multiple decision trees to significant decrease the overall variances. Conventional random forest employs orthogonal decision tree which selects one “optimal” feature to split the data instances within a non-leaf node according to impurity criteria such as Gini impurity, information gain and so on. However, orthogonal decision tree may fail to capture the geometrical structure of the data samples. Motivated by this, we make the first attempt to study the oblique random forest in the context of time series forecasting. In each node of the decision tree, instead of the single “optimal” feature based orthogonal classification algorithms used by standard random forest, a least square classifier is employed to perform partition. The proposed method is advantageous with respect to both efficiency and accuracy. We empirically evaluate the proposed method on eight generic time series datasets and five electricity load demand time series datasets from the Australian Energy Market Operator and compare with several other benchmark methods. [ABSTRACT FROM AUTHOR]

Recently, researchers have proposed a variety of new methods for employing exploratory data mining algorithms to address missing data. Two promising classes of missing data methods take advantage of classification and regression trees and random forests. A first method uses the predicted probabilities of response (vs. non-response) generated by a CART analysis to create inverse probability weights. This method has been shown to perform well in prior simulations when nonresponse was generated by tree-based structures, even under low sample sizes. A second method uses the values falling in terminal nodes of CART trees to generate multiple imputations. In prior studies, these methods performed well at estimating main effects and interactions in regression models when sample sizes were large ( N = 1000 ), but their performance was not evaluated under small sample conditions. In the present research, we assess the performance of CART-based weights and CART-based imputations under low sample sizes ( N = 125 or 250) and nonnormality when missing data are generated by smooth functions (linear, quadratic, cubic, interactive). Results suggest that random forest weights excel under low sample sizes, regardless of nonnormality, whereas CART multiple imputation is more efficient with larger samples ( N = 500 or 1000). [ABSTRACT FROM AUTHOR]