Search

The complex and opaque decision-making process of machine learning models restrains the interpretability of predictions and makes them cannot mine results outside of learning experiences. The causality between features and the target variable can be traced by injecting counterfactual explanations into the prediction model and generating counterfactual instances using adjusted features to reverse the prediction results. Existing algorithms, such as Diverse Counterfactual Explanations (DiCE) and Counterfactual Explanations Guided by Prototypes (Proto), can generate multiple/single counterfactual(s) for a data point by global optimization in the range of all-out features to gain on a local decision range. However, these methods cannot clearly identify which features are the key causes. Moreover, a Random Forest Optimal Counterfactual Set Extractor (RF-OCSE) extracts counterfactual sets from a random forest and needs to manipulate all the internal nodes of the tree, restricting it to only tree ensemble models. To address the above shortcomings, we proposed a Counterfactual Explanation Generation method with the Minimal Feature Boundary (MFB), named ( CEG MFB ). The proposed CEG MFB algorithm consists of two stages: 1) mining the MFB, which can reverse the prediction results to restrain the generation range of counterfactual instances, and 2) constructing a counterfactual generative method for generating counterfactual instances within the MFB to realize the minimum reversing cost. To evaluate its performance, we compared the proposed CEG MFB algorithm with six baseline algorithms on 16 datasets and conducted a case study in a real scenario. The results indicate that the proposed CEG MFB algorithm outperforms the compared algorithms. [ABSTRACT FROM AUTHOR]

• This paper presents a soft and hard hybrid qualitative balanced clustering (SHHQBC). • Qualitative balancing produces clusters with the lowest cardinality and highest value which are very practical. • Creating value criterion with the SHHQBC method enhanced quantitative and qualitative clustering criteria. K-means is a popular clustering method that has consistently failed to produce a balanced cluster structure. While changes in cardinality, variance, and density have arisen due to the importance of balancing in different fields, balancing has never been viewed from a qualitative viewpoint. The current paper takes a new look at cluster balancing by presenting a soft (balance-driven) and hard (balance-constrained) hybrid qualitative balanced clustering (SHHQBC). It starts by identifying and prioritizing key features using a random forest algorithm and the mean decrease in the Gini coefficient criterion. It then uses a weighted linear combination of features with the importance of above 25%, 50%, and 75% to construct a feature called value criterion. The developed clustering approach is then implemented to establish clusters with the highest value with the least cardinality or a value similar to other clusters. By implementing the SHHQBC on 14 different datasets, first soft clustering is implemented for all three cases and balanced conditions are checked. Hard clustering is then performed to make balanced conditions. Finally, the best-balanced case with the least objective function is selected. Formulating the value criterion facilitates the interpretation and labeling of clusters, and quantitative clustering criteria are improved. [ABSTRACT FROM AUTHOR]

Tree-based ensemble algorithms (TEAs) have had a transformative impact in various fields. However, when they are applied to real-time critical problems such as medical analysis, existing TEAs fall short of two-fold issues. The first is the trade-off among model interpretability, accuracy, and explainability. Second, traditional TEAs perform dramatically worse when the number of contributing features to model accuracy is small in comparison to the total number of features. We address the aforementioned issues in this paper and propose a novel "eXplainable Reasonably Randomised Forest" (XRRF) algorithm. The XRRF consists of four conclusive steps: learning performance, intrinsic interpretability, model accuracy, and explainability. The proposed forest algorithm is evaluated on three real-world problems (medical analysis, business analysis, and employee churn), a hybrid artificial dataset, and twenty multidisciplinary benchmark problems with varying characteristics. The experimental results demonstrate that the XRRF outperforms the six mainstream and two cutting-edge black-box and white-box algorithms, respectively. [ABSTRACT FROM AUTHOR]

Recently, the estimation of facial age has attracted much attention. This letter extends and improves a recently developed method (Guehairia et al., 2020) for fusing multiple deep facial features for age estimation. This method was based on deep random forests. We propose a new pipeline that integrates tensor-based subspace learning before applying DRFs. Deep face features of a training set are represented as a 3D tensor. Multi-linear Whitened Principal Component (MWPCA) and Tensor Exponential Discriminant (TEDA) are used to extract the most discriminative information. The tensor subspace features are then fed into DRFs to predict age. Experiments conducted on five public face databases show that our method can compete with many state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• Aiming for multiphase batch processes with different sampling frequencies, two types of process variables are summarized and the multiphase-based cumulative quality model is developed. It provides useful insights for the quality prediction of industrial batch process. • Aiming for the complex association relationships between multiple process variables and multiple quality indicators, information theory with copula entropy is employed to measure the variable relations for critical variables selection. • A stacking multiway random forest algorithm is proposed to develop the prediction relationship between phase-specific critical process variables and multiple quality indicators. • Experiments on a real-world industrial dataset have demonstrated that the proposed method not only has better accuracy but also has stronger robustness than previous baseline methods. As one of the most important modes of industrial production, the batch process often involves complex and continuous physicochemical reactions, making it challenging to produce identical products between different batch runs even under the same working condition. Optimization and parameter adjustments depend mainly on a good quality prediction model. However, this industrial process has "3M" characteristics of multiple process variables, multiple production phases, and multiple quality indicators, which bring considerable challenges to the accuracy and robustness of the model. This study proposes a multiphase information fusion strategy for data-driven quality prediction of industrial batch processes. Firstly, aiming for real-world industrial datasets with different sampling frequencies, two types of state variables are summarized, and the multiphase-based cumulative quality model is developed. Secondly, information theory with copula entropy is employed to characterize the association relationships between each state variable and the set of multiple quality indicators; thus, phase-specific critical variables are selected by ranking copula entropy. Lastly, a stacking multiway random forest algorithm is proposed to develop the prediction relationship between phase-specific critical variables and multiple quality indicators. Experiments on a real-world industrial dataset have demonstrated that the proposed method has better accuracy and stronger robustness than previous baseline methods. [ABSTRACT FROM AUTHOR]

• A new feature set composed of 46 features for phishing URLs detection were proposed. • An updated feature selection algorithm to obtain the most representative features. • An optimum subset composed of 9 features for phishing URLs detection was proposed. • Random Forest obtains the best Accuracy results using the feature subset proposed. • The Accuracy results obtained outperforms those reported in the literature. Phishing is a cyber-attack that exploits victims' technical ignorance or naivety and commonly involves a Uniform Resources Locator (URL). Hence, it is advantageous to detect a phishing attack by analyzing URLs before accessing them. With the raising of the Internet of Things (IoT), phishing attacks are moving to this field due to the number of IoT devices and the amount of personal information they handle. Although several approaches were proposed for phishing attacks detection, the URL-based Machine Learning approaches obtain better performance results, but all of them are dependent on the feature set used. Contradictorily, only a few works on selecting the best-suited feature set for improving the phishing detection process have been published. The present research explores how to obtain a feature set that substantially enhances the phishing detection rate in IoT environments. Hence, a feature selection algorithm was adopted and extended for getting the most representative feature set. When Random Forest is used with the proposed data representation, the phishing URL attacks discovery rate is 99.57 %. [ABSTRACT FROM AUTHOR]

Difficult and expensive financing has always been a problem for domestic and foreign enterprises, and how to effectively improve financing efficiency and improve the financing environment is a key issue to be studied. LightGBM is an advanced machine learning algorithm, which uses histogram algorithm and Leaf-wise strategy with depth limitation to improve the accuracy of the model. However, there are almost no cases of applying this method to corporate financing risk prediction. Therefore, the paper establishes the LightGBM model to predict the financing risk profile of 186 enterprises. In order to compare the prediction performance of LightGBM for enterprise financing risk, the paper conducted comparison experiments using k-nearest-neighbors algorithm, decision tree algorithm, and random forest algorithm on the same data set. The experiments show that LightGBM has better prediction results than the other three algorithms for several metrics in corporate financing risk prediction. Therefore, we believe that the LightGBM algorithm can be used as an effective tool to predict the financing risk of enterprises. [ABSTRACT FROM AUTHOR]

The continuous emergence of new malware has been a severe threat to Industrial Internet of Things (IIoT), while identifying malware through detecting malicious traffic in encrypted, drift, and imbalanced traffic streams is a challenge. This paper proposes an approach based on adaptive online analysis to accurately determine the families of malware by analyzing traffic streams which are encrypted, drift, and imbalanced. This approach is based on Improved Adaptive Random Forests (IARF), to obtain the ability of adaptive update of parameters when processing new types of malware traffic in traffic streams and being sensitive to families of malware with few samples in imbalanced traffic. We build a prototype of this approach and evaluate the performance through experiments. The experiments are based on a mixed dataset composed of data from malware-traffic-analysis.net, Lastline Inc, MCFP dataset, and CTU-13 dataset. In addition, our approach is also compared with three state-of-the-art methods. The results of the experiments show that we have obtained a 99.66% F1-score in the classification of malware families, and our classifier also performs better than the other classifiers. [ABSTRACT FROM AUTHOR]

The classification of hazard is critical in industrial informatics, as it enhances early safety alerts, supports decision-making, and facilitates policy assessment. However, previous studies have generally neglected the temporal attributes of hazards, thereby constraining the effectiveness of models. This paper introduces a new model for hazard classification termed DLGM. DLGM represents a deep learning framework, with the structural parameters of grey models to encapsulate the hazard temporal attributes. To better accommodate the fluctuations of hazard series, a new grey model termed FSGM(1,1) equipped with Fourier series is proposed. Moreover, DLGM leverages a novel hierarchical feature fusion neural network (HFFNN) to optimize feature processing. Extensive experiments across three hazard themes involving 18 large-scale industrial processes have demonstrated the competitiveness of DLGM (e.g., it surpasses benchmark models such as Random Forest and BERT by approximately 2% in both accuracy and F1), the suitability of FSGM(1,1) (e.g., its mean absolute percentage error is less than 10% and mean squared error is below 0.02) and the effectiveness of HFFNN (e.g., it enhances the accuracy and F1 of DLGM by about 1%). [ABSTRACT FROM AUTHOR]

The random forest classifier is widely used in different fields due to its accuracy and robustness. Since its invention, the random forest algorithm is naturally developed for multi-dimensional vectorial data since features can be directly sampled during the decision tree construction procedure. In the context of discrete sequence classification, an explicit feature set is not readily available and we need to employ a feature extraction algorithm before building the random forest. However, such a predefined feature subset may limit the diversity of decision trees since the set of candidate features is composed of all subsequences. As a result, the predictive accuracy of constructed random forest classifier may be reduced. To address this, we propose a new algorithm that is able to directly build a random forest by choosing features from the set of all subsequences adaptively. To improve the running efficiency of our algorithm, the count-suffix tree is utilized to facilitate the fast frequency counting of subsequences so as to accelerate the generation of each randomized decision tree. The experimental results on 15 real datasets show that our method can outperform those state-of-the-art classification algorithms in terms of the predictive accuracy. The source code of our method can be found at: https://github.com/JiaqiWang-dlut/RSForest. [ABSTRACT FROM AUTHOR]

• The use of deep generative models to swap faces will reduce the detected feature points. • DeepFakes have fewer feature points than real faces, especially in detailed regions. • We propose FFR_FD as a vector to represent the facial feature description. • Random forest trained with FFR_FD can realize state-of-the-art DeepFake detection. • Our DeepFake detection method is effective, efficient, and generalizable. DeepFakes are widespread on social networks, and they result in severe information concerns. Although various detection methods have been proposed, there are still practical limitations. Previous specific artifact-based methods were insufficient to capture fine-grained features, which limited their effectiveness against advanced DeepFakes. Current DNN-based detectors tend to trade high costs for performance improvement, and are not efficient enough, given that DeepFakes can be created easily by mobile apps, and DNN-based models require expensive computational resources. Furthermore, most methods lack generalizability under the cross-dataset scenario. In this work, we instead mine the more subtle and generalized defects of DeepFakes and propose the fused facial region _ feature descriptor (FFR _ FD) , which is only a vector of the discriminative feature description, for effective and fast DeepFake detection. We show that DeepFake faces have fewer feature points than real ones, especially in facial regions. FFR_FD capitalizes on such key observations, and thus has strong generalizability. We train a random forest classifier with FFR_FD to achieve efficient detection. Extensive experiments on six large-scale DeepFake datasets demonstrate the effectiveness of our lightweight method. Our model generalizes well on the challenging Celeb-DF (v2) dataset, with 0.706 AUC, which is superior to most state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Missing values are a common phenomenon in modern medical research of complex diseases. The data often contains nominal or categorical variables, for example, single nucleotide polymorphisms (SNPs) in genetic studies. If the missing values are not handled properly, the downstream statistical analysis of incomplete data may be biased. While various imputation methods are available for metrically scaled variables, methods for categorical data are scarce. An imputation method that has been shown to work well for high dimensional metrically scaled variables is the imputation by nearest neighbor methods. In this paper, we propose a weighted nearest neighbors approach to impute missing values in categorical variables in high dimensional datasets. The proposed method explicitly uses the information on the association among attributes. Using different simulation settings, the performance is compared with available imputation methods. A variety of real data sets, containing heart, DNA, and lymphatic cancer, is also used to support the results obtained by simulations. The results show that the weighting of attributes yields smaller imputation errors than existing approaches like random forest and MICE. [ABSTRACT FROM AUTHOR]

Fare evasion behaviours in metro stations happen all the time with the continuous improvement of the metro network, leading to likely mass panic and unhealthy effect on the metro operation. This paper proposes an approach of detecting individual fare evasion behaviours. The method first estimates human pose by the algorithm of OpenPose to obtain human skeleton information and conducts multi-target tracking by the algorithm of Pose Flow to obtain human skeleton sequence. It then extracts features from the human pose, followed by the adoption of the relative distance, the angle and Random Forests to establish a fare evasion behaviour detection model. Since missing problems and false detections occur in detections in a single frame, time series are used to increase the accuracy. The output of the time series curve is processed by One-Hot Coding, Kalman Filtering and optimum threshold to obtain the final prediction of human status. Extensive experiments are conducted, and results show that our method can effectively recognize the human status and detect individual fare evasion behaviours, including jumps and squats. [ABSTRACT FROM AUTHOR]

The increased amount of high-dimensional imbalanced data in online social networks challenges existing feature selection methods. Although feature selection methods such as principal component analysis (PCA) are effective for solving high-dimensional imbalanced data problems, they can be computationally expensive. Hence, an effortless approach for identifying meaningful features that are indicative of anomalous behaviour between humans and malicious bots is presented herein. The most recent Twitter dataset that encompasses the behaviour of various types of malicious bots (including fake followers, retweet spam, fake advertisements, and traditional spambots) is used to understand the behavioural traits of such bots. The approach is based on Benford's law for predicting the frequency distribution of significant leading digits. This study demonstrates that features closely obey Benford's law on a human dataset, whereas the same features violate Benford's law on a malicious bot dataset. Finally, it is demonstrated that the features identified by Benford's law are consistent with those identified via PCA and the ensemble random forest method on the same datasets. This study contributes to the intelligent detection of malicious bots such that their malicious activities, such as the dissemination of spam, can be minimised. [ABSTRACT FROM AUTHOR]

• It is possible to find partitions that outperform global and local approaches. • An exhaustive partitions search is not be feasible with a large number of outputs. • Genetic algorithms help accelerate the search but do not always outperform baselines. • Finding an optimal partitioning may lead to overfitting. In multi-target prediction, an instance has to be classified along multiple target variables at the same time, where each target represents a category or numerical value. There are several strategies to tackle multi-target prediction problems: the local strategy learns a separate model for each target variable independently, while the global strategy learns a single model for all target variables together. Previous studies suggested that the global strategy should be preferred because (1) learning is more efficient, (2) the learned models are more compact, and (3) it overfits much less than the local strategy, as it is harder to overfit on several targets at the same time than on one target. However, it is not clear whether the global strategy exploits correlations between the targets optimally. In this paper, we investigate whether better results can be obtained by learning multiple multi-target models on several partitions of the targets. To answer this question, we first determined alternative partitions using an exhaustive search strategy and a strategy based on a genetic algorithm, and then compared the results of the global and local strategies against these. We used decision trees and random forests as base models. The results show that it is possible to outperform global and local approaches, but finding a good partition without incurring in overfitting remains a challenging task. [ABSTRACT FROM AUTHOR]

• Highlight soccer detection is a hard but relevant task for match summarization. • An automatic system for highlight soccer identification is proposed • Highly discriminative low-level audio and video features are produced. • It selects a sparse subset of representative frames from long-term sliding windows. • It achieve s a match-time compression rate of 94%, identifying all highlights (100%). The massive growth of audiences eager for sport content has substantially increased workers' demand in this profitable segment. Highlight identification is vital for summarizing football matches. Decision support tools can significantly reduce the number of company employees required to tackle such a task, widely benefiting workforce resource allocation. This paper discusses the development of an automatic football highlight detector. The proposed system exploits discriminative low-level audio and video features extracted from a compact set of irregularly time–spaced frames that integrate a long-term sliding window. A new mixed wrapper-probabilistic algorithm leverages a cost-effective selection of the most significant frames submitted to a robust multi-frame consensus classification scheme. By considering a comprehensive database integrating 30 full matches, the proposed approach achieves a highlight identification rate of 100% (including all annotated goals), conjugated with a match-time compression rate of about 94%, when employing a Random Forest classifier. [ABSTRACT FROM AUTHOR]

The algorithm of C4.5 decision tree has the advantages of high classification accuracy, fast calculation speed and comprehensible classification rules, so it is widely used for medical data analysis. However, for imbalanced medical data, the classification accuracy of decision trees-based models is not ideal. Therefore, this paper proposes a cluster-based oversampling algorithm (KNSMOTE) combining Synthetic minority oversampling technique (SMOTE) and k -means algorithm. The sample classes clustered by k -means and the original sample classes are calculated to select the "safe samples" whose sample classes have not been changed. The "safe samples" are linearly interpolated to synthesize the new samples. The improved SMOTE sets the oversampling ratio according to the imbalance ratio of the original samples, which is used to synthesize the samples whose number is the same as that of the original samples. Compared with other oversampling algorithms on 8 UCI datasets, our algorithm has achieved significant advantages. Our algorithm was applied to the medical datasets, and the average values of the Sensitivity and Specificity indexes of the Random forest (RF) algorithm were 99.84% and 99.56%, respectively. [ABSTRACT FROM AUTHOR]

The foremost requirement for a decision forest to achieve better ensemble accuracy is building a set of accurate and diverse individual decision trees as base classifiers. Existing decision forest algorithms mainly differ from each other on how they induce diversity among the decision trees. At the same time, most of the drawbacks of existing algorithms originate from their induction processes of diversity. In this paper, we propose a new decision forest algorithm that is more balanced through effective synchronization between different sources of diversity. The proposed algorithm is balanced theoretically and empirically. We carried out experiments on 25 well-known data sets that are publicly available from the UCI Machine Learning Repository, to perform an extensive empirical evaluation. The experimental results indicate that the proposed algorithm has the best average ensemble accuracy rank of 1.8 compared to its closest competitor at 3.5. Using the Friedman and Bonferroni-Dunn tests, we also show that such an improvement is indeed statistically significant. In addition, the proposed algorithm is found to be competitive in terms of complexity and other relevant parameters. [ABSTRACT FROM AUTHOR]

• Adopt clustering to improve the performance of classification algorithms. • Propose a dynamic weighted probability combination strategy. • Apply an improved PSO to optimize multiple key sensitive parameters in CIEL. • Demonstrate CIEL's good performance experimentally on a large number of datasets. Classification is a vital task in machine learning. By learning patterns of samples of known categories, the model can develop the ability to distinguish the categories of samples of unknown categories. Noticing the advantages of the clustering method in cluster structure analysis, we combine the clustering and classification methods to develop the novel cluster-based intelligence ensemble learning (CIEL) method. We use the clustering method to analyze the inherent distribution of the data and divide all the samples into clusters according to the characteristics of the dataset. Then, for each specific cluster, we use different classification algorithms to establish the corresponding classification model. Finally, we integrate the prediction results of each base classifier to form the final prediction result. In view of the problem of parameter sensitivity, we use a swarm intelligence algorithm to optimize the key parameters involved in the clustering, classification, and ensemble stages in order to boost the classification performance. To assess the effectiveness of CIEL, we perform tenfold cross-validation experiments on the 24 benchmark datasets provided by UCI and KEEL. Designed to improve the performance of the classifiers, CIEL outperforms other popular machine learning methods such as naive Bayes, k -nearest neighbors, random forest, and support vector machine. [ABSTRACT FROM AUTHOR]

The application of artificial intelligence models to raw multimedia data is susceptible to various data inference attacks, posing a significant risk in terms of sensitive input information leakage. Most of the existing studies on privacy-preserving multimedia applications based on artificial intelligence, focus on a single intelligent model and thus have various limitations. In this paper, we attempt to directly perturb the core component of many multimedia intelligent models in Bayesian networks and deep learning. That is, we apply conditional probability distribution estimation to guarantee the privacy of the models. At first, we present the formal problem formulation of private conditional probability distribution estimation and apply it to random forest for task classification in multimedia applications. Then, we design a simple perturbation approach called NaivePrivDistEst , to add noise to all the elements of probability estimation in random forest. Next, we present an improved approach called FastLRG , that utilizes the taxonomy tree to discretize the continuous attributes, thereby combining the attribute features to improve the prediction accuracy of random forest. Finally, we perform extensive experiments to evaluate the performance of random forest based on the proposed estimation algorithms. The experimental results indicate that the proposed models have better performance compared with existing private decision trees. • Conditional probability distribution estimation is applied to random forest for multimedia classification applications. • A naive private algorithm that directly perturbs the values of each attribute and label in a random forest is proposed. • An improved algorithm of probability distribution estimation with lower communication cost and better accuracy is proposed. • Experimental results show that the proposed private random forest has better performance than the existing decision models. [ABSTRACT FROM AUTHOR]

In personalised health, small datasets with missing data are quite common. Current Machine Learning methods are unable to process such datasets in a meaningful way due to the huge data volume requirement. To address this problem, we propose a new Small and iNcomplete Dataset Analyser (SaNDA) to process such datasets in a meaningful way. Due to the characteristics of these datasets and the criticality of the domain, an explainable method is mandatory to facilitate decision-making interpretation. Thus, SaNDA prioritises explainability over efficiency by design. We evaluated our proposal against Random Forest as a baseline for explainable methods, and against gcForest as state-of-the-art for small datasets. We observed that our proposal outperforms Random Forest when there is more missing data and/or lower number of entries in the dataset, obtaining less favourable results over larger, well-curated datasets. It is also preferable than gcForest due to its explainability and privacy protection capabilities. Given the difficulties in obtaining complete, reliable data in the healthcare field, we consider that our proposal could be useful for practitioners. • Personalised Health datasets are small and full of missing data. • Traditional methods require huge and complete datasets. • We propose an explainable method tweaked for those datasets. • We compare our method with Random Forest and gcForest. • Our method beats Random Forest when data is missing. [ABSTRACT FROM AUTHOR]

Current automatic data-driven vulnerability detection in smart contracts selects and processes features of interest under black box settings without empirical justification. In this paper, we propose a smart contract testing methodology that bestows developers with flexible, practical and customizable strategies to detect vulnerabilities. Our work enforces strong whitebox knowledge to a series of supervised multimodal tasks under static analysis. Each task encapsulates a vulnerability detection branch test and pipelines feature selection, dimension unification, feature fusion, model training and decision-making. We exploit multiple features made up of code and graph embeddings at the single modality level (intramodal settings) and across individual modalities (intermodal settings). We assign each task to either intramodal or intermodal settings, and show how to train state-of-the-art self-attentive bi-LSTM, textCNN, and random forest (RF) models to extract a joint multimodal feature representation per task. We evaluate our framework over 101,082 functions extracted from the SmartEmbed dataset, and rank each multimodal vulnerability mining strategy in terms of detection performance. Extensive experiments show that our work outperforms existing schemes, and the highest performance reaches 99.71%. [ABSTRACT FROM AUTHOR]