Search

Accurately predicting buffet frequency and aerodynamic noise level is crucial in transonic buffet noise reduction studies. In this study, the Random Forest (RF) algorithm is employed to predict the Power Spectral Density (PSD) and Overall Sound Pressure Level (OASPL) distribution over the supercritical airfoil RAE2822. The study indicates that the RF algorithm exhibits greater advantages over the Multi-Layer Perceptron (MLP). This algorithm does not suffer from the problems of high-frequency divergence or inflection distortion, maintaining a reduction in prediction errors for OASPL and PSD by approximately three to four orders of magnitude. Additionally, this paper proposes a priori criterion for evaluating the accuracy of PSD prediction. When there is a strong correlation between PSD at adjacent points, a data-driven modeling approach can achieve higher prediction accuracy. However, when the root mean square error of the cross-correlation functions, auto-correlation functions, and the PSD between adjacent points are both high, the generalization ability of pure data-driven modeling is insufficient, necessitating the additional monitoring points to ensure prediction accuracy. [ABSTRACT FROM AUTHOR]

• A PV/T device with sheet-and-needle finned serpentine tube collector is analyzed. • PV panels are cooled using the water-silver nanofluid. • Random forest machine learning approach is used to develop predictive models. • Thermal, electrical and overall efficiencies are utilized as objective functions. • Utilization of needle fins improve the energy and exergy performance of the system. The Photovoltaic thermal (PVT) collector performance is numerically investigated considering the effect of using needle fins in the serpentine channel with Nanofluid (NF). The influence of increasing the nanoparticle concentration (φ) and Reynolds number (Re) on the energy and exergy features of the PVT device is examined. A comparison is made between the hydrothermal characteristics of the PVT with the finned and plain serpentine channels. The utilization of needle fins improves the thermal efficiency (η th), electrical efficiency (η el), and overall efficiency (η el) by 8.56–10.22%, 0.13–0.24%, 5.12–5.67%, respectively, against the PVT with the plain serpentine channel. Moreover, thermal exergy efficiency (ξ th), electrical exergy efficiency (ξ el), and overall exergy efficiency (ξ ov) by 8.56–1.22%, 0.13–0.24%, and 2.61–2.79%, respectively, versus the PVT with the plain serpentine channel. Moreover, the Random Forest (RF) machine learning approach is used to develop a predictive model for η th , η el , η el , ξ th , ξ el and ξ ov in terms of Re and φ. The outcomes of modeling proved that all the results were in an acceptable level of accuracy and the overall efficiency in both energy and exergy yielded superior precision in comparison with the other targets. [ABSTRACT FROM AUTHOR]

The complex and opaque decision-making process of machine learning models restrains the interpretability of predictions and makes them cannot mine results outside of learning experiences. The causality between features and the target variable can be traced by injecting counterfactual explanations into the prediction model and generating counterfactual instances using adjusted features to reverse the prediction results. Existing algorithms, such as Diverse Counterfactual Explanations (DiCE) and Counterfactual Explanations Guided by Prototypes (Proto), can generate multiple/single counterfactual(s) for a data point by global optimization in the range of all-out features to gain on a local decision range. However, these methods cannot clearly identify which features are the key causes. Moreover, a Random Forest Optimal Counterfactual Set Extractor (RF-OCSE) extracts counterfactual sets from a random forest and needs to manipulate all the internal nodes of the tree, restricting it to only tree ensemble models. To address the above shortcomings, we proposed a Counterfactual Explanation Generation method with the Minimal Feature Boundary (MFB), named ( CEG MFB ). The proposed CEG MFB algorithm consists of two stages: 1) mining the MFB, which can reverse the prediction results to restrain the generation range of counterfactual instances, and 2) constructing a counterfactual generative method for generating counterfactual instances within the MFB to realize the minimum reversing cost. To evaluate its performance, we compared the proposed CEG MFB algorithm with six baseline algorithms on 16 datasets and conducted a case study in a real scenario. The results indicate that the proposed CEG MFB algorithm outperforms the compared algorithms. [ABSTRACT FROM AUTHOR]

One of the most important hydraulic structures that has been overlooked is culverts. Regular maintenance and inspections are required to ensure that these structures are used effectively and safely. Culvert inspection, on the other hand, takes a significant amount of time and resources, and agencies cannot afford to do it on a regular basis. To overcome physical inspection challenges, this study proposed employing machine learning algorithms to predict the condition of Utah's culverts based on historical data. This study assessed multiclass classification algorithms that had not been used before. The classification machine learning algorithms employed in this study are Random Forest, Decision Tree, Support Vector Machine, k-Nearest Neighbor, and Artificial Neural Network. Also, a combined dataset from four states was used to address Utah's limited culvert inventory data. The final dataset contained 2555 culvert records. Out of the developed prediction models, Random Forest had the highest accuracy of 82%. The findings demonstrated that Random Forests could be used to improve culvert network quality while reducing culvert maintenance expenses for UDOT. The proposed model may be applied to similarly existing culvert inventories located around the nation. [ABSTRACT FROM AUTHOR]

• This paper presents a soft and hard hybrid qualitative balanced clustering (SHHQBC). • Qualitative balancing produces clusters with the lowest cardinality and highest value which are very practical. • Creating value criterion with the SHHQBC method enhanced quantitative and qualitative clustering criteria. K-means is a popular clustering method that has consistently failed to produce a balanced cluster structure. While changes in cardinality, variance, and density have arisen due to the importance of balancing in different fields, balancing has never been viewed from a qualitative viewpoint. The current paper takes a new look at cluster balancing by presenting a soft (balance-driven) and hard (balance-constrained) hybrid qualitative balanced clustering (SHHQBC). It starts by identifying and prioritizing key features using a random forest algorithm and the mean decrease in the Gini coefficient criterion. It then uses a weighted linear combination of features with the importance of above 25%, 50%, and 75% to construct a feature called value criterion. The developed clustering approach is then implemented to establish clusters with the highest value with the least cardinality or a value similar to other clusters. By implementing the SHHQBC on 14 different datasets, first soft clustering is implemented for all three cases and balanced conditions are checked. Hard clustering is then performed to make balanced conditions. Finally, the best-balanced case with the least objective function is selected. Formulating the value criterion facilitates the interpretation and labeling of clusters, and quantitative clustering criteria are improved. [ABSTRACT FROM AUTHOR]

Tree-based ensemble algorithms (TEAs) have had a transformative impact in various fields. However, when they are applied to real-time critical problems such as medical analysis, existing TEAs fall short of two-fold issues. The first is the trade-off among model interpretability, accuracy, and explainability. Second, traditional TEAs perform dramatically worse when the number of contributing features to model accuracy is small in comparison to the total number of features. We address the aforementioned issues in this paper and propose a novel "eXplainable Reasonably Randomised Forest" (XRRF) algorithm. The XRRF consists of four conclusive steps: learning performance, intrinsic interpretability, model accuracy, and explainability. The proposed forest algorithm is evaluated on three real-world problems (medical analysis, business analysis, and employee churn), a hybrid artificial dataset, and twenty multidisciplinary benchmark problems with varying characteristics. The experimental results demonstrate that the XRRF outperforms the six mainstream and two cutting-edge black-box and white-box algorithms, respectively. [ABSTRACT FROM AUTHOR]

Recently, the estimation of facial age has attracted much attention. This letter extends and improves a recently developed method (Guehairia et al., 2020) for fusing multiple deep facial features for age estimation. This method was based on deep random forests. We propose a new pipeline that integrates tensor-based subspace learning before applying DRFs. Deep face features of a training set are represented as a 3D tensor. Multi-linear Whitened Principal Component (MWPCA) and Tensor Exponential Discriminant (TEDA) are used to extract the most discriminative information. The tensor subspace features are then fed into DRFs to predict age. Experiments conducted on five public face databases show that our method can compete with many state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• Aiming for multiphase batch processes with different sampling frequencies, two types of process variables are summarized and the multiphase-based cumulative quality model is developed. It provides useful insights for the quality prediction of industrial batch process. • Aiming for the complex association relationships between multiple process variables and multiple quality indicators, information theory with copula entropy is employed to measure the variable relations for critical variables selection. • A stacking multiway random forest algorithm is proposed to develop the prediction relationship between phase-specific critical process variables and multiple quality indicators. • Experiments on a real-world industrial dataset have demonstrated that the proposed method not only has better accuracy but also has stronger robustness than previous baseline methods. As one of the most important modes of industrial production, the batch process often involves complex and continuous physicochemical reactions, making it challenging to produce identical products between different batch runs even under the same working condition. Optimization and parameter adjustments depend mainly on a good quality prediction model. However, this industrial process has "3M" characteristics of multiple process variables, multiple production phases, and multiple quality indicators, which bring considerable challenges to the accuracy and robustness of the model. This study proposes a multiphase information fusion strategy for data-driven quality prediction of industrial batch processes. Firstly, aiming for real-world industrial datasets with different sampling frequencies, two types of state variables are summarized, and the multiphase-based cumulative quality model is developed. Secondly, information theory with copula entropy is employed to characterize the association relationships between each state variable and the set of multiple quality indicators; thus, phase-specific critical variables are selected by ranking copula entropy. Lastly, a stacking multiway random forest algorithm is proposed to develop the prediction relationship between phase-specific critical variables and multiple quality indicators. Experiments on a real-world industrial dataset have demonstrated that the proposed method has better accuracy and stronger robustness than previous baseline methods. [ABSTRACT FROM AUTHOR]

In this work, strategies to devise an optimal feedback control of probabilistic Boolean control networks (PBCNs) are discussed. Reinforcement learning (RL) based control is explored in order to minimize model design efforts and regulate PBCNs with high complexities. A Q -learning random forest (Q LRF) algorithm is proposed; by making use of the algorithm, state feedback controllers are designed to stabilize the PBCNs at a given equilibrium point. Further, by adopting Q LRF stabilized closed-loop PBCNs, a Lyapunov function is defined, and a method to construct the same is presented. By utilizing such Lyapunov functions, a novel self-triggered control (STC) strategy is proposed, whereby the controller is recomputed according to a triggering schedule, resulting in an optimal control strategy while retaining the closed-loop PBCN stability. Finally, the results are verified using computer simulations. [ABSTRACT FROM AUTHOR]

• A new feature set composed of 46 features for phishing URLs detection were proposed. • An updated feature selection algorithm to obtain the most representative features. • An optimum subset composed of 9 features for phishing URLs detection was proposed. • Random Forest obtains the best Accuracy results using the feature subset proposed. • The Accuracy results obtained outperforms those reported in the literature. Phishing is a cyber-attack that exploits victims' technical ignorance or naivety and commonly involves a Uniform Resources Locator (URL). Hence, it is advantageous to detect a phishing attack by analyzing URLs before accessing them. With the raising of the Internet of Things (IoT), phishing attacks are moving to this field due to the number of IoT devices and the amount of personal information they handle. Although several approaches were proposed for phishing attacks detection, the URL-based Machine Learning approaches obtain better performance results, but all of them are dependent on the feature set used. Contradictorily, only a few works on selecting the best-suited feature set for improving the phishing detection process have been published. The present research explores how to obtain a feature set that substantially enhances the phishing detection rate in IoT environments. Hence, a feature selection algorithm was adopted and extended for getting the most representative feature set. When Random Forest is used with the proposed data representation, the phishing URL attacks discovery rate is 99.57 %. [ABSTRACT FROM AUTHOR]

Difficult and expensive financing has always been a problem for domestic and foreign enterprises, and how to effectively improve financing efficiency and improve the financing environment is a key issue to be studied. LightGBM is an advanced machine learning algorithm, which uses histogram algorithm and Leaf-wise strategy with depth limitation to improve the accuracy of the model. However, there are almost no cases of applying this method to corporate financing risk prediction. Therefore, the paper establishes the LightGBM model to predict the financing risk profile of 186 enterprises. In order to compare the prediction performance of LightGBM for enterprise financing risk, the paper conducted comparison experiments using k-nearest-neighbors algorithm, decision tree algorithm, and random forest algorithm on the same data set. The experiments show that LightGBM has better prediction results than the other three algorithms for several metrics in corporate financing risk prediction. Therefore, we believe that the LightGBM algorithm can be used as an effective tool to predict the financing risk of enterprises. [ABSTRACT FROM AUTHOR]

The continuous emergence of new malware has been a severe threat to Industrial Internet of Things (IIoT), while identifying malware through detecting malicious traffic in encrypted, drift, and imbalanced traffic streams is a challenge. This paper proposes an approach based on adaptive online analysis to accurately determine the families of malware by analyzing traffic streams which are encrypted, drift, and imbalanced. This approach is based on Improved Adaptive Random Forests (IARF), to obtain the ability of adaptive update of parameters when processing new types of malware traffic in traffic streams and being sensitive to families of malware with few samples in imbalanced traffic. We build a prototype of this approach and evaluate the performance through experiments. The experiments are based on a mixed dataset composed of data from malware-traffic-analysis.net, Lastline Inc, MCFP dataset, and CTU-13 dataset. In addition, our approach is also compared with three state-of-the-art methods. The results of the experiments show that we have obtained a 99.66% F1-score in the classification of malware families, and our classifier also performs better than the other classifiers. [ABSTRACT FROM AUTHOR]

The classification of hazard is critical in industrial informatics, as it enhances early safety alerts, supports decision-making, and facilitates policy assessment. However, previous studies have generally neglected the temporal attributes of hazards, thereby constraining the effectiveness of models. This paper introduces a new model for hazard classification termed DLGM. DLGM represents a deep learning framework, with the structural parameters of grey models to encapsulate the hazard temporal attributes. To better accommodate the fluctuations of hazard series, a new grey model termed FSGM(1,1) equipped with Fourier series is proposed. Moreover, DLGM leverages a novel hierarchical feature fusion neural network (HFFNN) to optimize feature processing. Extensive experiments across three hazard themes involving 18 large-scale industrial processes have demonstrated the competitiveness of DLGM (e.g., it surpasses benchmark models such as Random Forest and BERT by approximately 2% in both accuracy and F1), the suitability of FSGM(1,1) (e.g., its mean absolute percentage error is less than 10% and mean squared error is below 0.02) and the effectiveness of HFFNN (e.g., it enhances the accuracy and F1 of DLGM by about 1%). [ABSTRACT FROM AUTHOR]

The random forest classifier is widely used in different fields due to its accuracy and robustness. Since its invention, the random forest algorithm is naturally developed for multi-dimensional vectorial data since features can be directly sampled during the decision tree construction procedure. In the context of discrete sequence classification, an explicit feature set is not readily available and we need to employ a feature extraction algorithm before building the random forest. However, such a predefined feature subset may limit the diversity of decision trees since the set of candidate features is composed of all subsequences. As a result, the predictive accuracy of constructed random forest classifier may be reduced. To address this, we propose a new algorithm that is able to directly build a random forest by choosing features from the set of all subsequences adaptively. To improve the running efficiency of our algorithm, the count-suffix tree is utilized to facilitate the fast frequency counting of subsequences so as to accelerate the generation of each randomized decision tree. The experimental results on 15 real datasets show that our method can outperform those state-of-the-art classification algorithms in terms of the predictive accuracy. The source code of our method can be found at: https://github.com/JiaqiWang-dlut/RSForest. [ABSTRACT FROM AUTHOR]

• The use of deep generative models to swap faces will reduce the detected feature points. • DeepFakes have fewer feature points than real faces, especially in detailed regions. • We propose FFR_FD as a vector to represent the facial feature description. • Random forest trained with FFR_FD can realize state-of-the-art DeepFake detection. • Our DeepFake detection method is effective, efficient, and generalizable. DeepFakes are widespread on social networks, and they result in severe information concerns. Although various detection methods have been proposed, there are still practical limitations. Previous specific artifact-based methods were insufficient to capture fine-grained features, which limited their effectiveness against advanced DeepFakes. Current DNN-based detectors tend to trade high costs for performance improvement, and are not efficient enough, given that DeepFakes can be created easily by mobile apps, and DNN-based models require expensive computational resources. Furthermore, most methods lack generalizability under the cross-dataset scenario. In this work, we instead mine the more subtle and generalized defects of DeepFakes and propose the fused facial region _ feature descriptor (FFR _ FD) , which is only a vector of the discriminative feature description, for effective and fast DeepFake detection. We show that DeepFake faces have fewer feature points than real ones, especially in facial regions. FFR_FD capitalizes on such key observations, and thus has strong generalizability. We train a random forest classifier with FFR_FD to achieve efficient detection. Extensive experiments on six large-scale DeepFake datasets demonstrate the effectiveness of our lightweight method. Our model generalizes well on the challenging Celeb-DF (v2) dataset, with 0.706 AUC, which is superior to most state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Missing values are a common phenomenon in modern medical research of complex diseases. The data often contains nominal or categorical variables, for example, single nucleotide polymorphisms (SNPs) in genetic studies. If the missing values are not handled properly, the downstream statistical analysis of incomplete data may be biased. While various imputation methods are available for metrically scaled variables, methods for categorical data are scarce. An imputation method that has been shown to work well for high dimensional metrically scaled variables is the imputation by nearest neighbor methods. In this paper, we propose a weighted nearest neighbors approach to impute missing values in categorical variables in high dimensional datasets. The proposed method explicitly uses the information on the association among attributes. Using different simulation settings, the performance is compared with available imputation methods. A variety of real data sets, containing heart, DNA, and lymphatic cancer, is also used to support the results obtained by simulations. The results show that the weighting of attributes yields smaller imputation errors than existing approaches like random forest and MICE. [ABSTRACT FROM AUTHOR]

Fare evasion behaviours in metro stations happen all the time with the continuous improvement of the metro network, leading to likely mass panic and unhealthy effect on the metro operation. This paper proposes an approach of detecting individual fare evasion behaviours. The method first estimates human pose by the algorithm of OpenPose to obtain human skeleton information and conducts multi-target tracking by the algorithm of Pose Flow to obtain human skeleton sequence. It then extracts features from the human pose, followed by the adoption of the relative distance, the angle and Random Forests to establish a fare evasion behaviour detection model. Since missing problems and false detections occur in detections in a single frame, time series are used to increase the accuracy. The output of the time series curve is processed by One-Hot Coding, Kalman Filtering and optimum threshold to obtain the final prediction of human status. Extensive experiments are conducted, and results show that our method can effectively recognize the human status and detect individual fare evasion behaviours, including jumps and squats. [ABSTRACT FROM AUTHOR]

The increased amount of high-dimensional imbalanced data in online social networks challenges existing feature selection methods. Although feature selection methods such as principal component analysis (PCA) are effective for solving high-dimensional imbalanced data problems, they can be computationally expensive. Hence, an effortless approach for identifying meaningful features that are indicative of anomalous behaviour between humans and malicious bots is presented herein. The most recent Twitter dataset that encompasses the behaviour of various types of malicious bots (including fake followers, retweet spam, fake advertisements, and traditional spambots) is used to understand the behavioural traits of such bots. The approach is based on Benford's law for predicting the frequency distribution of significant leading digits. This study demonstrates that features closely obey Benford's law on a human dataset, whereas the same features violate Benford's law on a malicious bot dataset. Finally, it is demonstrated that the features identified by Benford's law are consistent with those identified via PCA and the ensemble random forest method on the same datasets. This study contributes to the intelligent detection of malicious bots such that their malicious activities, such as the dissemination of spam, can be minimised. [ABSTRACT FROM AUTHOR]

In big data era, some data, becoming meaningless or illegal over time and space, need to be deleted from historical knowledge. It is a challenging problem, called machine unlearning, to efficiently forget the information of those outdated data from historical models. Some unlearning techniques have been proposed in loss-well-defined classification models, such as SVM, Random Forest, and Federated learning model. Yet, it is under study to remove outdated data from learned feature selection in fuzzy rough philosophy. To narrow this gap, we propose a fuzzy rough unlearning model for feature selection. Specifically, the outdated information is first identified in a compact set, called unlearning set, that remarkably shrinks the search space of feature selection. Then, the unlearning mechanisms containing two main theorems are proposed by leveraging unlearning set, followed by a feature selection unlearning algorithm. Theoretical analyses verify that the proposed unlearning algorithm is equivalent to the traditional algorithm retraining on the remaining data with the beginning of historical results. Experimentally, extensive results on 20 datasets demonstrate that our proposed unlearning methods perform effectively with remarkably less time cost. For the source code of ARU, please refer to: https://github.com/yuxin370/ARU. [ABSTRACT FROM AUTHOR]

• It is possible to find partitions that outperform global and local approaches. • An exhaustive partitions search is not be feasible with a large number of outputs. • Genetic algorithms help accelerate the search but do not always outperform baselines. • Finding an optimal partitioning may lead to overfitting. In multi-target prediction, an instance has to be classified along multiple target variables at the same time, where each target represents a category or numerical value. There are several strategies to tackle multi-target prediction problems: the local strategy learns a separate model for each target variable independently, while the global strategy learns a single model for all target variables together. Previous studies suggested that the global strategy should be preferred because (1) learning is more efficient, (2) the learned models are more compact, and (3) it overfits much less than the local strategy, as it is harder to overfit on several targets at the same time than on one target. However, it is not clear whether the global strategy exploits correlations between the targets optimally. In this paper, we investigate whether better results can be obtained by learning multiple multi-target models on several partitions of the targets. To answer this question, we first determined alternative partitions using an exhaustive search strategy and a strategy based on a genetic algorithm, and then compared the results of the global and local strategies against these. We used decision trees and random forests as base models. The results show that it is possible to outperform global and local approaches, but finding a good partition without incurring in overfitting remains a challenging task. [ABSTRACT FROM AUTHOR]

• Highlight soccer detection is a hard but relevant task for match summarization. • An automatic system for highlight soccer identification is proposed • Highly discriminative low-level audio and video features are produced. • It selects a sparse subset of representative frames from long-term sliding windows. • It achieve s a match-time compression rate of 94%, identifying all highlights (100%). The massive growth of audiences eager for sport content has substantially increased workers' demand in this profitable segment. Highlight identification is vital for summarizing football matches. Decision support tools can significantly reduce the number of company employees required to tackle such a task, widely benefiting workforce resource allocation. This paper discusses the development of an automatic football highlight detector. The proposed system exploits discriminative low-level audio and video features extracted from a compact set of irregularly time–spaced frames that integrate a long-term sliding window. A new mixed wrapper-probabilistic algorithm leverages a cost-effective selection of the most significant frames submitted to a robust multi-frame consensus classification scheme. By considering a comprehensive database integrating 30 full matches, the proposed approach achieves a highlight identification rate of 100% (including all annotated goals), conjugated with a match-time compression rate of about 94%, when employing a Random Forest classifier. [ABSTRACT FROM AUTHOR]

The algorithm of C4.5 decision tree has the advantages of high classification accuracy, fast calculation speed and comprehensible classification rules, so it is widely used for medical data analysis. However, for imbalanced medical data, the classification accuracy of decision trees-based models is not ideal. Therefore, this paper proposes a cluster-based oversampling algorithm (KNSMOTE) combining Synthetic minority oversampling technique (SMOTE) and k -means algorithm. The sample classes clustered by k -means and the original sample classes are calculated to select the "safe samples" whose sample classes have not been changed. The "safe samples" are linearly interpolated to synthesize the new samples. The improved SMOTE sets the oversampling ratio according to the imbalance ratio of the original samples, which is used to synthesize the samples whose number is the same as that of the original samples. Compared with other oversampling algorithms on 8 UCI datasets, our algorithm has achieved significant advantages. Our algorithm was applied to the medical datasets, and the average values of the Sensitivity and Specificity indexes of the Random forest (RF) algorithm were 99.84% and 99.56%, respectively. [ABSTRACT FROM AUTHOR]

The foremost requirement for a decision forest to achieve better ensemble accuracy is building a set of accurate and diverse individual decision trees as base classifiers. Existing decision forest algorithms mainly differ from each other on how they induce diversity among the decision trees. At the same time, most of the drawbacks of existing algorithms originate from their induction processes of diversity. In this paper, we propose a new decision forest algorithm that is more balanced through effective synchronization between different sources of diversity. The proposed algorithm is balanced theoretically and empirically. We carried out experiments on 25 well-known data sets that are publicly available from the UCI Machine Learning Repository, to perform an extensive empirical evaluation. The experimental results indicate that the proposed algorithm has the best average ensemble accuracy rank of 1.8 compared to its closest competitor at 3.5. Using the Friedman and Bonferroni-Dunn tests, we also show that such an improvement is indeed statistically significant. In addition, the proposed algorithm is found to be competitive in terms of complexity and other relevant parameters. [ABSTRACT FROM AUTHOR]

• Adopt clustering to improve the performance of classification algorithms. • Propose a dynamic weighted probability combination strategy. • Apply an improved PSO to optimize multiple key sensitive parameters in CIEL. • Demonstrate CIEL's good performance experimentally on a large number of datasets. Classification is a vital task in machine learning. By learning patterns of samples of known categories, the model can develop the ability to distinguish the categories of samples of unknown categories. Noticing the advantages of the clustering method in cluster structure analysis, we combine the clustering and classification methods to develop the novel cluster-based intelligence ensemble learning (CIEL) method. We use the clustering method to analyze the inherent distribution of the data and divide all the samples into clusters according to the characteristics of the dataset. Then, for each specific cluster, we use different classification algorithms to establish the corresponding classification model. Finally, we integrate the prediction results of each base classifier to form the final prediction result. In view of the problem of parameter sensitivity, we use a swarm intelligence algorithm to optimize the key parameters involved in the clustering, classification, and ensemble stages in order to boost the classification performance. To assess the effectiveness of CIEL, we perform tenfold cross-validation experiments on the 24 benchmark datasets provided by UCI and KEEL. Designed to improve the performance of the classifiers, CIEL outperforms other popular machine learning methods such as naive Bayes, k -nearest neighbors, random forest, and support vector machine. [ABSTRACT FROM AUTHOR]

Computational fluid dynamics is a cornerstone of the modern aerospace industry, providing important insights through aerodynamic analysis while reducing the need for expensive experiments and tests. A consistent spatial discretization on so-called grids approximates the analytical solution of the partial differential equation with increasing number of discrete points. But computing a rigorous amount of CFD simulations on fine grids can be a daunting task due to the high computational cost involved. Thus, it is of interest to find methods which generate accurate results while keeping computational costs low. This work proofs that machine learning as a post-processing tool is capable of improving the accuracy of coarse grid simulations. The results show that three machine learning models with varying complexity, namely random forest, neural network, and graph neural network, are capable of finding patterns in coarse grid simulations. These patterns are used for a vertex-wise prediction of the discretization error to approximate the field variables of interest of the corresponding fine grid simulation mapped onto the coarse grid. Initial training and testing is performed on the two dimensional RAE2822 airfoil leading to corrected flow fields, improved surface integrals and coefficients, even when shocks are present. Additional tests are performed on the RAE5212 airfoil, showing the generalization limits of the trained models. Finally, the training and prediction is showcased on a three dimensional geometry. The proposed method promises to reduce computational expenses while increasing the accuracy of the coarse grid results which works locally, e.g. it corrects the error for each vertex individually and is therefore not restricted by the number of grid points. The presented results obtained by the machine learning models during post-processing are a promising baseline for more integrated developments, where the models will interact in a dynamic fashion with the flow solver to further improve coarse grid simulations. • Point-wise approximation of fine grid CFD simulations on corresponding coarse grid. • Learning the discretization error on 2D and 3D turbulent transonic flows. • Improved accuracy on less expensive coarse grid computation achievable. • Graph neural networks account for information of neighboring data points. • Graph neural networks achieve higher accuracy than neural networks. [ABSTRACT FROM AUTHOR]

The challenges in detecting antineutrinos by segmented antineutrino detectors (AD) pushed us to get more exact results using new methods, including data mining and machine learning (ML). In this research, IRAND (a segmented plastic scintillator IRan ANtineutrino Detector; a 10 × 10 array of plastic scintillation) has been simulated using Geant4 Toolkit. Several features of energy deposition including behavior of time, energy, position have been collected and stored in detail from each event to provide the complete dataset with a minor loss for use in powerful ML packages. We utilized a variety of classifiers in ML in which Random Forest shows the best performance. Also, a Geant4-based dedicated code (IRAND-Sim) has been provided for IRAND. [ABSTRACT FROM AUTHOR]

The application of artificial intelligence models to raw multimedia data is susceptible to various data inference attacks, posing a significant risk in terms of sensitive input information leakage. Most of the existing studies on privacy-preserving multimedia applications based on artificial intelligence, focus on a single intelligent model and thus have various limitations. In this paper, we attempt to directly perturb the core component of many multimedia intelligent models in Bayesian networks and deep learning. That is, we apply conditional probability distribution estimation to guarantee the privacy of the models. At first, we present the formal problem formulation of private conditional probability distribution estimation and apply it to random forest for task classification in multimedia applications. Then, we design a simple perturbation approach called NaivePrivDistEst , to add noise to all the elements of probability estimation in random forest. Next, we present an improved approach called FastLRG , that utilizes the taxonomy tree to discretize the continuous attributes, thereby combining the attribute features to improve the prediction accuracy of random forest. Finally, we perform extensive experiments to evaluate the performance of random forest based on the proposed estimation algorithms. The experimental results indicate that the proposed models have better performance compared with existing private decision trees. • Conditional probability distribution estimation is applied to random forest for multimedia classification applications. • A naive private algorithm that directly perturbs the values of each attribute and label in a random forest is proposed. • An improved algorithm of probability distribution estimation with lower communication cost and better accuracy is proposed. • Experimental results show that the proposed private random forest has better performance than the existing decision models. [ABSTRACT FROM AUTHOR]

In personalised health, small datasets with missing data are quite common. Current Machine Learning methods are unable to process such datasets in a meaningful way due to the huge data volume requirement. To address this problem, we propose a new Small and iNcomplete Dataset Analyser (SaNDA) to process such datasets in a meaningful way. Due to the characteristics of these datasets and the criticality of the domain, an explainable method is mandatory to facilitate decision-making interpretation. Thus, SaNDA prioritises explainability over efficiency by design. We evaluated our proposal against Random Forest as a baseline for explainable methods, and against gcForest as state-of-the-art for small datasets. We observed that our proposal outperforms Random Forest when there is more missing data and/or lower number of entries in the dataset, obtaining less favourable results over larger, well-curated datasets. It is also preferable than gcForest due to its explainability and privacy protection capabilities. Given the difficulties in obtaining complete, reliable data in the healthcare field, we consider that our proposal could be useful for practitioners. • Personalised Health datasets are small and full of missing data. • Traditional methods require huge and complete datasets. • We propose an explainable method tweaked for those datasets. • We compare our method with Random Forest and gcForest. • Our method beats Random Forest when data is missing. [ABSTRACT FROM AUTHOR]

Random Forests and related tree-based methods are popular for supervised learning from table based data. Apart from their ease of parallelization, their classification performance is also superior. However, this performance, especially parallelizability, is offset by the loss of explainability. Statistical methods are often used to compensate for this disadvantage. Yet, their ability for local explanations, and in particular for global explanations, is limited. In the present work we propose an algebraic method, rooted in lattice theory, for the (global) explanation of tree ensembles. In detail, we introduce two novel conceptual views on tree ensemble classifiers and demonstrate their explanatory capabilities on Random Forests that were trained with standard parameters. [ABSTRACT FROM AUTHOR]

Current automatic data-driven vulnerability detection in smart contracts selects and processes features of interest under black box settings without empirical justification. In this paper, we propose a smart contract testing methodology that bestows developers with flexible, practical and customizable strategies to detect vulnerabilities. Our work enforces strong whitebox knowledge to a series of supervised multimodal tasks under static analysis. Each task encapsulates a vulnerability detection branch test and pipelines feature selection, dimension unification, feature fusion, model training and decision-making. We exploit multiple features made up of code and graph embeddings at the single modality level (intramodal settings) and across individual modalities (intermodal settings). We assign each task to either intramodal or intermodal settings, and show how to train state-of-the-art self-attentive bi-LSTM, textCNN, and random forest (RF) models to extract a joint multimodal feature representation per task. We evaluate our framework over 101,082 functions extracted from the SmartEmbed dataset, and rank each multimodal vulnerability mining strategy in terms of detection performance. Extensive experiments show that our work outperforms existing schemes, and the highest performance reaches 99.71%. [ABSTRACT FROM AUTHOR]

Hypothesis testing of variable importance measures (VIMPs) is still the subject of ongoing research. This particularly applies to random forests (RF), for which VIMPs are a popular feature. Among recent developments, heuristic approaches to parametric testing have been proposed whose distributional assumptions are based on empirical evidence. Other formal tests under regularity conditions were derived analytically. But these approaches can be computationally expensive or even practically infeasible. This problem also occurs with non-parametric permutation tests, which are, however, distribution-free and can generically be applied to any kind of prediction model and VIMP. Embracing this advantage, it is proposed to use sequential permutation tests and sequential p-value estimation to reduce the computational costs associated with conventional permutation tests. These costs can be particularly high in case of complex prediction models. Therefore, RF's popular and widely used permutation VIMP (pVIMP) serves as a practical and relevant application example. The results of simulation studies confirm the theoretical properties of the sequential tests, that is, the type-I error probability is controlled at a nominal level and a high power is maintained with considerably fewer permutations needed compared to conventional permutation testing. The numerical stability of the methods is investigated in two additional application studies. In summary, theoretically sound sequential permutation testing of VIMP is possible at greatly reduced computational costs. Recommendations for application are given. A respective implementation for RF's pVIMP is provided through the accompanying R package rfvimptest. [ABSTRACT FROM AUTHOR]

Tree-based ensembles such as the Random Forest are modern classics among statistical learning methods. In particular, they are used for predicting univariate responses. In case of multiple outputs the question arises whether it is better to separately fit univariate models or directly follow a multivariate approach. For the latter, several possibilities exist that are, e.g. based on modified splitting or stopping rules for multi-output regression. These methods are compared in extensive simulations and a real data example to help in answering the primary question when to use multivariate ensemble techniques instead of univariate ones. [ABSTRACT FROM AUTHOR]

'LatticeKrig' (LK) is a spatial model that is often used for modeling multiresolution spatial data with flexible covariance structures. An extension to LK under a Bayesian framework is proposed that uses integrated nested Laplace approximations (INLA). The extension enables the spatial analysis of non-Gaussian responses in latent Gaussian models, joint spatial modeling with structured and unstructured random effects, and native support for multithreaded parallel likelihood computation. The proposed extended LatticeKrig (ELK) model uses a reparameterization of LK so that the parameters and prior selection are intuitive and interpretable. Priors can be used to make inference robust by penalizing more complex models, and integration over model parameters allows for posterior uncertainty estimates that account for uncertainty in covariance parameters. ELK's ability to reliably resolve multiresolution spatial structure for pointwise and areal predictions is demonstrated in both simulation study and two applications with non-Gaussian observations: a set of 188,717 LiDAR forest canopy height observations in Bonanza Creek Experimental Forest in Alaska, and a set of 1,612 clusters containing counts of secondary education completion from the 2014 Kenya demographic health survey. ELK has improved central predictions as well as uncertainty characterization according to the considered scoring rules when compared against a number of other models, particularly in the forest canopy height application, and performed faster than LK in our tests in part due to its support for and use of parallelization. [ABSTRACT FROM AUTHOR]

Although interaction effects can be exploited to improve predictions and allow for valuable insights into covariate interplay, they are given limited attention in analysis. Interaction forests are a variant of random forests for categorical, continuous, and survival outcomes that explicitly models quantitative and qualitative interaction effects in bivariable splits performed by the trees constituting the forests. The new effect importance measure (EIM) associated with interaction forests allows for ranking of covariate pairs with respect to their interaction effects' importance to prediction. Using EIM, separate importance value lists for univariable effects, quantitative interaction effects, and qualitative interaction effects are obtained. In the spirit of interpretable machine learning, the bivariable split types of interaction forests target easily interpretable and communicable interaction effects. To learn about the nature of the interplay between covariates identified as interacting it is convenient to visualise their estimated bivariable influence. Functions that perform this task are provided in the R package diversityForest , which implements interaction forests. In a large-scale empirical study using 220 data sets, interaction forests tended to deliver better predictions than conventional random forests and competing random forest variants that use multivariable splitting. In a simulation study, EIM delivered considerably better rankings for the relevant quantitative and qualitative interaction effects than competing approaches. These results indicate that interaction forests are suitable tools for the challenging task of identifying and making use of easily interpretable and communicable interaction effects in predictive modelling. [ABSTRACT FROM AUTHOR]

• A general nonparametric feature selection algorithm is proposed. • The theoretical properties of the proposed algorithm are established under regularity conditions. • The proposed algorithm outperforms its alternatives in terms of detecting useful features and avoiding useless ones. • The proposed algorithm can be widely applied in other machine learning methods. Random forests are a widely used machine learning algorithm, but their computational efficiency is undermined when applied to large-scale datasets with numerous instances and useless features. Herein, we propose a nonparametric feature selection algorithm that incorporates random forests and deep neural networks, and its theoretical properties are also investigated under regularity conditions. Using different synthetic models and a real-world example, we demonstrate the advantage of the proposed algorithm over other alternatives in terms of identifying useful features, avoiding useless ones, and the computation efficiency. Although the algorithm is proposed using standard random forests, it can be widely adapted to other machine learning algorithms, as long as features can be sorted accordingly. [ABSTRACT FROM AUTHOR]

Following the line of classification-based two-sample testing, tests based on the Random Forest classifier are proposed. The developed tests are easy to use, require almost no tuning, and are applicable for any distribution on R d. Furthermore, the built-in variable importance measure of the Random Forest gives potential insights into which variables make out the difference in distribution. An asymptotic power analysis for the proposed tests is conducted. Finally, two real-world applications illustrate the usefulness of the introduced methodology. To simplify the use of the method, the R-package "hypoRF" is provided. [ABSTRACT FROM AUTHOR]

A new method is developed for quantifying the uncertainties of the estimates and predictions produced by honest random forests. This new method is based on the generalized fiducial methodology, and provides a fiducial density function that measures how likely each single honest tree is the true model. With such a density function, estimates and predictions, as well as their confidence/prediction intervals, can be obtained. The promising empirical properties of the proposed method are demonstrated by numerical comparisons with several state-of-the-art methods, and by applications to a few real data sets. Lastly, the proposed method is theoretically backed up by an asymptotic guarantee. [ABSTRACT FROM AUTHOR]

Due to delay in reporting, the daily national and statewide COVID-19 incidence counts are often unreliable and need to be estimated from recent data. This process is known in economics as nowcasting. We describe in this paper a simple random forest statistical model for nowcasting the COVID-19 daily new infection counts based on historic data along with a set of simple covariates, such as the currently reported infection counts, day of the week, and time since first reporting. We apply the model to adjust the daily infection counts in Ohio, and show that the predictions from this simple data-driven method compare favorably both in quality and computational burden to those obtained from the state-of-the-art hierarchical Bayesian model employing a complex statistical algorithm. The interactive notebook for performing nowcasting is available online at https://tinyurl.com/simpleMLnowcasting. • Delays in reporting often make daily statewide COVID-19 incidence counts unreliable. • We developed a simple model for nowcasting the daily counts from historic data. • To calibrate the model we use historic rates of backfilling COVID-19 cases in Ohio. • On the Ohio COVID-19 dataset our method outperforms the standard model. • The algorithm is implemented in jupyter notebook environment for public use. [ABSTRACT FROM AUTHOR]

• The systemic impact (loss caused) is mainly driven by topological features. • For banks, this importance increases with the level of the initial shock. • For credit unions, this importance decreases with the level of the initial shock. • The systemic vulnerability (loss suffered) is mainly driven by financial features. • This importance increases with the initial shock for both banks and credit unions. The purpose of this paper is to assess the role of financial variables and network topology as determinants of systemic risk (SR). The SR, for different levels of the initial shock, is computed for institutions in the Brazilian interbank market by applying the differential DebtRank methodology. The financial institution(FI)-specific determinants of SR are evaluated through two machine learning techniques: XGBoost and random forest. Shapley values analysis provided a better interpretability for our results. Furthermore, we performed this analysis separately for banks and credit unions. We have found the importance of a given feature in driving SR varies with i) the level of the initial shock, ii) the type of FI, and iii) the dimension of the risk which is being assessed – i.e., potential loss caused by (systemic impact) or imputed to (systemic vulnerability) the FI. Systemic impact is mainly driven by topological features for both types of FIs. However, while the importance of topological features to the prediction of systemic impact of banks increases with the level of the initial shock, it decreases for credit unions. Concerning systemic vulnerability, this is mainly determined by financial features, whose importance increases with the initial shock level for both types of FIs. [ABSTRACT FROM AUTHOR]

• Impact of covariates on COVID-19 response were tested via Granger-causality tests. • Climatic impact on COVID-19 cases-to-mortality ratios were studied in 36 countries. • Relationship equations were established using regression analysis. • Temperature data were factored-in in training forecasting models. • The trained models were used for forecasting COVID-19 cases-to-mortality ratios. There are several recent publications criticizing the failure of COVID-19 forecasting models, with swinging over predictions and underpredictions, which have made it difficult for decision and policy making. Observing the failures of several COVID-19 forecasting models and the alarming spread of the virus, we seek to use some stable response for forecasting COVID-19, viz., ratios of COVID-19 cases to mortalities, rather than COVID-19 cases or fatalities. A trend of low COVID-19 cases-to-mortality ratios calls for urgent attention: the need for vaccines, for instance. Studies have shown that there are influences of weather parameters on COVID-19; and COVID-19 may have come to stay and could manifest a seasonal outbreak profile similar to other infectious respiratory diseases. In this paper, the influences of some weather, geographical, economic and demographic covariates were evaluated on COVID-19 response based on a series of Granger-causality tests. The effect of four weather parameters, viz., temperature, rainfall, solar irradiation and relative humidity, on daily COVID-19 cases-to-mortality ratios of 36 countries from 5 continents of the world were determined through regression analysis. Regression studies show that these four weather factors impact ratios of COVID-19 cases-to-mortality differently. The most impactful factor is temperature which is positively correlated with COVID-19 cases-to-mortality responses in 24 out of 36 countries. Temperature minimally affects COVID-19 cases-to-mortality ratios in the tropical countries. The most influential weather factor – temperature – was incorporated in training random forest and deep learning models for forecasting the cases-to-mortality rate of COVID-19 in clusters of countries in the world with similar weather conditions. Evaluation of trained forecasting models incorporating temperature features show better performance compared to a similar set of models trained without temperature features. This implies that COVID-19 forecasting models will predict more accurately if temperature features are factored in, especially for temperate countries. [ABSTRACT FROM AUTHOR]