Search

Edge computing is an emerging computing paradigm, which offers a great opportunity to implement data mining-based services and applications for a large number of devices and sensors in Internet of Things. However, the new paradigm is faced with security and privacy challenges due to the diversity and the limited capability of edge components. In particular, data privacy is one of the most concerned problems for all the participants. In this paper, we propose a framework of privacy-preserving data mining based on private random decision trees in edge computing, which not only gives the strong privacy guarantee, but also provides a certain amount of data utility. Firstly, we design a preservation framework to implement private random decision trees satisfying local differential privacy. Secondly, we present the concrete implementations of algorithms and the corresponding task that each participant needs to undertake. Thirdly, we analyze the key factors to influence privacy and utility, including the allocation of data and privacy budget. Fourthly, we give the improved algorithms to further increase the utility with strong privacy preservation. Finally, extensive experiments demonstrate the good performance of our designed framework. [ABSTRACT FROM AUTHOR]

Emission control is vital for environmental and health protection. Traditional methods to assess toxic substance emissions from combustion engines are generally expensive and time-consuming. This paper proposes a new approach using indirect methods that predict emissions based on current engine operational parameters. The focus is on analyzing the engine's vibrational signal. Extensive research was conducted in a laboratory setting, using an diesel engine, fueled by various types of fuel. The study linked the spectral uncertainty of engine vibrations with the emission levels of toxic substances. A emission prediction model was developed, utilizing a decision tree system. This model's hyperparameters were finely tuned using Bayesian optimization techniques. The model for a single predictor in the form of spectral uncertainty already showed satisfactory accuracy in predicting the emission of several harmful substances (10%). However, supplementing it with a second predictor in the form of engine torque allows for even smaller errors (8%). [Display omitted] • Examined engine vibrations' spectral uncertainty and exhaust toxin emissions. • Developed a decision-tree-based toxic emission prediction model. • Used spectral uncertainty average and engine torque as predictors. • Found fuel type irrelevant for prediction accuracy, enhancing universality. • Achieved satisfactory forecast accuracy — less than 8%. [ABSTRACT FROM AUTHOR]

How to effectively detect geochemical anomalies associated with mineralization is a challenging task due to the extreme class-imbalance of geochemical exploration data. To address this challenge, various machine learning techniques have been employed to detect geochemical anomalies associated with mineralization. However, almost all of these machine learning techniques have their limitations when it comes to modeling class-imbalanced geochemical exploration data. To establish efficient robust high-performance models for detecting geochemical anomalies associated mineralization, a case study was carried out in the Helong area, Jilin Province, China. Decision tree (C4.5) classifiers were used as the base or weak classifiers, four imbalanced learning ensemble models, including self-paced ensemble model, under-bagging ensemble model, synthetic minority oversampling technique (SMOTE)-boost ensemble model and random under sampling (RUS)-boost ensemble model were established and compared in the detection of polymetallic mineralization anomalies from the 1:50,000 stream sediment survey data. The output of the base classifier is a vector consisting of the probability that the sample belongs to a mineralization anomaly and the probability that the sample belongs to the background. The performance of each ensemble model was evaluated using the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC). The ROC curves of the four models are close to the upper left corner of the ROC space. The AUC values of the four models are not lower than 0.9589. The polymetallic mineralization anomalies detected by the four models account for less than 7 % of the entire study area, but contain not less than 93 % of the polymetallic deposits found in the study area. In addition, the polymetallic mineralization anomalies detected by the four models spatially coincide with the regional controlling factors of polymetallic mineralization in the study area. To summarize, the four models have very high performance in detecting polymetallic mineralization anomalies, and the polymetallic mineralization anomaly detection results are consistent with the regional geological and metallogenic characteristics in the study area. Therefore, the imbalanced learning ensemble techniques are powerful tools for the establishment of high-performance ensemble classification models for detecting mineralization anomalies from geochemical exploration data. • Imbalanced learning techniques are adopted to detect mineralization anomalies. • Four imbalanced ensemble models are established on stream sediment survey data. • A case study shows the robustness and high-performance of the four ensemble models. [ABSTRACT FROM AUTHOR]

As a popular machine learning model, decision trees classify and generalize well, but face challenges in engineering applications: 1) Sensitivity to perturbations and lack of interpretability due to correlation reliance. 2) Manual setting of stopping criterion which is unrelated to correlation strength and easily leads to over-partitioning. To address these two challenges, we first theoretically analyze what leads to sub-optimal decision trees. By incorporating causal discovery, this limitation can be attributed to the fact that trees grown with spurious correlations often fall into sub-optimal that lead to overfitting and unfair behaviors. Neglecting causality motivates us to develop a 'better' tree with low Kolmogorov complexity and high generalization capability. Then we propose a causality decision tree framework, CausalDT, based on our theoretical expectation, where Hilbert-Schmidt independence criterion serves as a baseline. Unlike previous approaches that prioritize relevance, our framework determines branch nodes based on causation between features, with the significance level determining whether the tree should be expanded further. Experimental results demonstrate that our model maintains performance while reducing average tree depth by 35% on various datasets. Furthermore, our model enhances decision fairness and interpretability. • Analyzing spurious correlation's impact on decision trees. • A novel causality-guided framework has been designed for tree construction. • Consideration of significance levels during tree growth. • Achieving comparable performance with up to 35% reduction in tree depth. [ABSTRACT FROM AUTHOR]

• Propose an improved random forest based on the improvement of decision trees. • Improve the evaluation mechanism for the classification effect of decision trees. • Propose a method for quantifying the diversity between decision trees. • Multiple tests verify the superiority of the proposed improved random forest. Random forest is one of the most widely used machine learning algorithms. Decision trees used to construct the random forest may have low classification accuracies or high correlations, which affects the comprehensive performance of the random forest. Aiming at these problems, the authors proposed an improved random forest based on the classification accuracy and correlation measurement of decision trees in this paper. Its main idea includes two parts, one is retaining the classification and regression trees (CARTs) with better classification effects, the other is reducing the correlations between the CARTs. Specifically, in the classification effect evaluation part, each CART was applied to make predictions on three reserved data sets, then the average classification accuracies were achieved, respectively. Thus, all the CARTs were sorted in descending order according to their achieved average classification accuracies. In the correlation measurement part, the improved dot product method was proposed to calculate the cosine similarity, i.e., the correlation, between CARTs in the feature space. By using the achieved average classification accuracy as reference, the grid search method was used to find the inner product threshold. On this basis, the CARTs with low average classification accuracy among CART pairs whose inner product values are higher than the inner product threshold were marked as deletable. The achieved average classification accuracies and correlations of CARTs were comprehensively considered, those with high correlation and weak classification effect were deleted, and those with better quality were retained to construct the random forest. Multiple experiments show that, the proposed improved random forest achieved higher average classification accuracy than the five random forests used for comparison, and the lead was stable. The G-means and out-of-bag data (OBD) score obtained by the proposed improved random forest were also higher than the five random forests, and the lead was more obvious. In addition, the test results of three non-parametric tests show that, there were significant diversities between the proposed improved random forest and the other five random forests. This effectively proves the superiority and practicability of the proposed improved random forest. [ABSTRACT FROM AUTHOR]

Machine learning models are promising as surrogates in optimization when replacing difficult to solve equations or black-box type models. This work demonstrates the viability of linear model decision trees as piecewise-linear surrogates in decision-making problems. Linear model decision trees can be represented exactly in mixed-integer linear programming (MILP) and mixed-integer quadratic constrained programming (MIQCP) formulations. Furthermore, they can represent discontinuous functions, bringing advantages over neural networks in some cases. We present several formulations using transformations from Generalized Disjunctive Programming (GDP) formulations and modifications of MILP formulations for gradient boosted decision trees (GBDT). We then compare the computational performance of these different MILP and MIQCP representations in an optimization problem and illustrate their use on engineering applications. We observe faster solution times for optimization problems with linear model decision tree surrogates when compared with GBDT surrogates using the Optimization and Machine Learning Toolkit (OMLT). • Linear model decision trees are viable as piecewise-linear surrogates. • Linear trees are representable in mixed-integer programming problems. • Linear model trees can reduce optimization time compared to gradient boosted trees. • Linear model decision trees can represent discontinuous functions. [ABSTRACT FROM AUTHOR]

• We propose a tree representation based on constrained matrices that leads to two new generic linear programming formulations of the optimal decision tree problem allowing to easily integrate domain-knowledge constraints. • Both formulations give the possibility for users to easily control complexity through the constraints on the number of leaf nodes and the maximum depth of the decision tree. • We theoretically show that elements of these matrices give the ability to easily formalise domain knowledge as constraints to improve trustworthiness. When applied to critical domains, machine learning models usually need to comply with prior knowledge and domain-specific requirements. For example, one may require that a learned decision tree model should be of limited size and fair, so as to be easily interpretable, trusted, and adopted. However, most state-of-the-art models, even on decision trees, only aim to maximising expected accuracy. In this paper, we propose a framework in which a diverse family of prior and domain knowledge can be formalised and imposed as constraints on decision trees. This framework is built upon a newly introduced tree representation that leads to two generic linear programming formulations of the optimal decision tree problem. The first one targets binary features, while the second one handles continuous features without the need for discretisation. We theoretically show how a diverse family of constraints can be formalised in our framework. We validate the framework with constraints on several applications and perform extensive experiments, demonstrating empirical evidence of comparable performance w.r.t. state-of-the-art tree learners. [ABSTRACT FROM AUTHOR]

In this paper, we study arbitrary infinite binary information systems each of which consists of an infinite set of elements and an infinite set of two-valued non-constant functions (attributes) defined on the set of elements. We consider the notion of a problem over information system, which is described by a finite number of attributes: for a given element, we should determine values of these attributes. As algorithms for problem solving, we study decision trees that use arbitrary attributes from the considered infinite set of attributes and solve the problem based on 1-consequences. In such a tree, we take into account consequences each of which follows from one equation of the kind "attribute = value" obtained during the decision tree work and ignore consequences that can be derived only from at least two equations. As time complexity, we study the depth of decision trees. We prove that in the worst case, with the growth of the number of attributes in the problem description, the minimum depth of decision trees based on 1-consequences grows either as a logarithm or linearly. [ABSTRACT FROM AUTHOR]

This paper introduces new flexible loss functions for binary classification in Gradient-Boosted Decision Trees (GBDT) that combine Dice-based and cross-entropy-based losses and offer link functions from either a generalized extreme value (GEV) or exponentiated exponential logistic (EEL) distribution. Testing 27 different GBDT models using XGBoost on a Freddie Mac mortgage loan database showed that the choice of the loss function is useful. Specifically, when the class imbalance ratio (IR) is less than 99, using a skewed GEV distribution-based link function in XGBoost enhances discriminatory power and classification accuracy while retaining a simple model structure, which is particularly important in credit scoring applications. In cases where class imbalances are severe, typically between IRs of 99 and 200, we found that an advanced loss function, which is composed of a symmetric hybrid loss function and a link derived from a positively skewed EEL distribution, outperforms other XGBoost variants. Based on our findings, the accuracy improvements of these proposed extensions result in lower misclassification costs, which are especially evident when IR is below 99, which results in higher profitability for the business. Furthermore, the study highlights the transparency associated with GBDT, which is also an integral component of financial applications. Researchers and practitioners can use these insights to create more accurate and discriminative machine learning models, with possible extensions to other GBDT implementations and machine learning techniques that take into account loss functions. The source code for the proposed approach is publicly available at https://github.com/jm-ml/flexible-losses-for-binary-classification-with-GBDT. [ABSTRACT FROM AUTHOR]

This study addresses the limitations of current parametric equations and artificial neural networks (ANNs) in accurately predicting the stress concentration factor (SCF) of fillet welded joints stemming from the simplification of their real weld profiles. To improve the accuracy, this study introduces bagged trees for estimating local stresses. The dataset used as the foundation for training the bagged trees is extracted from the actual weld geometry of T-shaped joints. It is created via a digitalization process involving the extraction of actual geometric parameters from the joints, which are transformed into finite element models (FEMs). These models are then employed to determine the ratio between the simulated sectional stress and the nominal stress (σ sec / ∆ σ nom) under an axial loading condition. A comprehensive comparison is carried out among existing parametric equations, ANNs, and the proposed bagged trees. The results emphasize the inadequacy of idealized geometry models in accurately determining local stresses for real weld profiles. In contrast, bagged trees are a promising method for accurately computing sectional weld stresses (σ sec) within real weld geometry. • Considering the actual weld geometry to assess local stresses. • Using artificial intelligence to predict local stresses in welded joints. • Examining the applicability of developed methods for predicting the stress concentration factor considering the actual weld geometry. [ABSTRACT FROM AUTHOR]

Energy losses, premature failure, and environmental degradation are the adverse impacts of the friction at sliding interfaces. Two-dimensional (2D) materials have recently attracted immense interest as solid lubricants. Here we develop a novel ternary nanocomposite overcoat comprising multi-layer graphene (MGr), MoS 2 and multi-walled carbon nanotubes (CNT) which entangle each other, generate ensembles of trio, alter the sliding interfaces and thereby exceptionally reduce the friction (80%–85 %) and wear of diverse surfaces including metals (stainless steel (SS) 304, SS316L, mild steel (MS), Al-composite), ceramics (glass and Al 2 O 3 +TiC composite), and semiconductor (silicon). Extensive spectroscopic and microscopic characterizations are performed to elucidate the fundamental mechanisms for friction and wear reduction. The comprehensive investigation clearly confirms that the tribological efficacy of our ternary nanocomposite overcoat surpasses the monolithic and binary nanocomposite overcoats made of same materials. This discovery opens new avenues to control sliding interfaces with nanocomposites of 1D/2D materials, and can transform many moving mechanical technologies. [Display omitted] • We developed a novel ternary nanocomposite overcoat comprising multi-layer graphene (MGr), MoS 2 and multi-walled carbon nanotubes (CNT). • The ternary MGr+ MoS 2 + CNT nanocomposite overcoat reduces the friction of diverse surfaces (metals, ceramics, and semiconductor) by 80%- 85%. • The ternary nanocomposite overcoat reduces the wear of diverse surfaces up to 85%. • This work establishes a pathway for designing high-performance tribological composite coatings applicable to a wide range of industrial applications. [ABSTRACT FROM AUTHOR]

The interpretability of ML models is important, but it is not clear what it amounts to. So far, most philosophers have discussed the lack of interpretability of black-box models such as neural networks, and methods such as explainable AI that aim to make these models more transparent. The goal of this paper is to clarify the nature of interpretability by focussing on the other end of the "interpretability spectrum". The reasons why some models, linear models and decision trees, are highly interpretable will be examined, and also how more general models, MARS and GAM, retain some degree of interpretability. It is found that while there is heterogeneity in how we gain interpretability, what interpretability is in particular cases can be explicated in a clear manner. [ABSTRACT FROM AUTHOR]

Highlights • A combination algorithm for ensemble of decision trees is proposed. • RuleCOSI algorithm transforms boosted decision trees into a single ruleset. • The combined rules are simpler than original rules of boosting ensemble. • The performance of the simple production rules was acceptable. Abstract In the field of machine learning, the problem of imbalanced classification arises when the class percentage on the data is unevenly distributed. Different strategies using boosting ensemble algorithms have shown improved results over the imbalanced classification problem by combining weak learners to produce a single strong learner. In particular, decision trees are often used as base learners in ensemble learning for classification or regression. However, boosting ensemble algorithms sometimes generate a large number of decision trees that could grow too large to be understandable and interpretable. Additionally, the use of weights adds more complexity to the final result. For this reason, in this paper, we present RuleCOSI, a novel method for combining and simplifying the output of an ensemble of binary decision trees into a single set of production rules. The proposed method takes into account the weight of each decision tree and using a combination matrix generates a single set of simplified production rules with performance comparable to that of the original boosting ensemble. In order to measure the performance and prove the applicability of the proposed method, we carried out an empirical validation using three different boosting algorithms over several well-known machine learning datasets as well as real-life data collected from a manufacturing company. The results of the algorithm are acceptable in most of the experiments reducing the complexity of the boosting ensemble output while maintaining a similar performance. [ABSTRACT FROM AUTHOR]

In recent years, reinforcement learning (RL) techniques have achieved great success in many different applications. However, their heavy reliance on complex deep neural networks makes most RL models uninterpretable, limiting their application in domains where trust and security are important. To address this challenge, we propose MENS-DT-RL, an algorithm capable of constructing interpretable models for RL via the evolution of decision tree (DT) models. MENS-DT-RL uses a multi-method ensemble algorithm to evolve univariate DTs, guiding the process with a fitness metric that prioritizes interpretability and consistent high performance. Three different initializations for the MENS-DT-RL are proposed, including the use of Imitation Learning (IL) techniques, and a novel pruning approach that reduces solution size without compromising performance. To evaluate the proposed approach, we compare it with other models from the literature on three benchmark tasks from the OpenAI Gym library, as well as on a fertilization problem inspired by real-world crop management. To the best of our knowledge, the proposed scheme is the first to solve the Lunar Lander benchmark with both interpretability and a high confidence rate (90% of episodes are successful), as well as the first to solve the Mountain Car environment with a tree of only 7 nodes. On the real-world task, the proposed MENS-DT-RL is able to produce solutions with the same quality as deep RL policies, with the added bonus of interpretability. We also analyze the best solutions found by the algorithm and show that they are not only interpretable but also diverse in their behavior, empowering the end user with the choice of which model to apply. Overall, the findings show that the proposed approach is capable of producing high-quality transparent models for RL, achieving interpretability without losing performance. • A multi-method ensemble evolutionary algorithm is proposed for Interpretable RL. • Interpretability is achieved by using Decision Trees as RL agents. • Imitation Learning is leveraged to jumpstart the evolutionary algorithm. • A novel method is proposed to prune DTs in RL environments. • State-of-the-art results are obtained for several RL benchmarks. [ABSTRACT FROM AUTHOR]

Decision forests are known to excel in tabular data once their hyperparameters are well-tuned. In addition to being accurate and robust classifiers, these models can be easily converted into a collection of if-else rules that clearly describe the model's decision-making process. However, in practice, decision trees may reach enormous depth and size, and as a result, the collection of rules is vast and complicated. Furthermore, the model may consume a significant amount of memory space on the machine. By creating compact models, it is possible to establish a modest set of rules that may be easier to understand, require less memory, and by nature, may increase the decision-making ability and avoid overfitting. Previous studies attempted to reduce the size of the trees by altering their structure, but this affects both their advantages and simplicity because their structure is much more complicated. (e.g. oblique trees). In this research, we present FACET , a novel algorithm for retaining the compactness of trees while referring to the model as a black box. FACET addresses this by utilizing automated feature engineering methods, which generate a new feature set from the data set by manipulating the current feature set, resulting in a drastic reduction in the size of the decision trees while preserving and even improving the model's accuracy. Our algorithm, FACET, has been extensively tested on multiple datasets, models, and operators to demonstrate its effectiveness. On average, FACET achieves a 24% reduction in the size criteria of the tree-based model without sacrificing accuracy. This reduction in size leads to an average memory reduction of 44% on the dataset required for learning. These statistically significant results demonstrate the potential of FACET to enable more efficient and interpretable tree-based models, without compromising their accuracy, in practical applications. • New algorithm retains decision tree compactness while treating model as black box. • First literature to reduce tree size by modifying user features. • New approach formally defined in paper. • Automated feature engineering methods proposed for problem handling. • Algorithms assessed on diverse data sets. [ABSTRACT FROM AUTHOR]

Visualizations are powerful tools that are commonly used by data scientists to get more insights about their high dimensional data. One can for example cite t -SNE, which is probably one of the most famous and widely-used visualization techniques. However, t -SNE is a nonlinear and non-parametric technique that makes it suffer from a lack of interpretability. In this paper, we present a new technique inspired by t -SNE's objective function that combines its ability to build nice visualizations with the interpretability of decision trees. This new visualization technique, called DT-SNE, can be seen as a discrete visualization technique where groups of instances are provided, as well as a ranking between them. The decision rules of the decision tree provide clear insights to interpret these different groups. [ABSTRACT FROM AUTHOR]

3D printing has the potential to revolutionize industrial manufacturing through efficient and sustainable techniques. Fused Deposition Modeling (FDM) is a broadly deployed technique among various 3D printing methods. However, the surface quality of FDM is greatly influenced by multiple factors, making it challenging to unravel the relationship between printing quality and parameter settings. To break through this bottleneck, this study proposes an intelligent approach that combines Transfer Learning (TL)-based Feature Extractor (FE) and Gradient-Boosting Decision Trees (GBDT) to investigate the effects of FDM printing parameters on surface quality. Experiments are conducted in the laboratory to validate the effectiveness of the FE-GBDT, which is then compared with the exemplary Machine Learning (ML) algorithms. The results show that our proposed TL model can achieve high precision and accuracy over 0.9900, demonstrating the efficacy of FE-GBDT in deciphering the impact of FDM printing parameters on surface quality. The contribution of each parameter is evaluated and indicates that layer height could dramatically affect the surface quality with an importance score of 0.626. The results provide valuable insights for the 3D printing community, proving that the FE-GBDT approach offers improved generalization, faster training, enhanced feature extraction, addressing data scarcity, and the ability to leverage the strengths of both approaches for superior performance across various tasks. [ABSTRACT FROM AUTHOR]

Highlights • A novel modeling strategy for Decision Trees is proposed. • Interactive basis functions induce oblique and non-linear decision boundaries. • Benchmarking shows potential gains in accuracy and/or parsimony of models. • Empirical examples illustrate the application of the modeling strategy. • Decision charts are introduced to improve the interpretability of models. Abstract Decision Trees (DTs) are a machine learning technique widely used for regression and classification purposes. Conventionally, the decision boundaries of Decision Trees are orthogonal to the features under consideration. A well-known limitation of this is that the algorithm may fail to find optimal partitions, or in some cases any partitions at all, depending on the underlying distribution of the data. To remedy this limitation, several modifications have been proposed that allow for oblique decision boundaries. The objective of this paper is to propose a new strategy for generating flexible decision boundaries by means of interactive basis functions (IBFs). We show how oblique decision boundaries can be obtained as a particular case of IBFs, and in addition how non-linear decision boundaries can be induced. One attractive aspect of the strategy proposed in this paper is that training Decision Trees with IBFs does not require custom software, since the functions can be precalculated for use in any existing implementation of the algorithm. Since the underlying mechanisms remain unchanged there is no substantial computational overhead compared to conventional trees. Furthermore, this also means that IBFs can be used in any extensions of the Decision Tree algorithm, such as evolutionary trees, boosting, and bagging. We conduct a benchmarking exercise to understand under which conditions the use of IBFs can improve model the performance. In addition, we present three empirical applications that illustrate the approach in classification and regression. As part of discussing the empirical applications, we introduce a device called decision charts to facilitate the interpretation of DTs with IBFs. Finally, we conclude the paper by outlining some directions for future research. [ABSTRACT FROM AUTHOR]

Due to the high rate of geriatric patient visits, scoring systems are needed to predict increasing mortality rates. In this study, we aimed to investigate the in-hospital mortality prediction power of the National Early Warning Score 2 (NEWS2) and the Laboratory Data Decision Tree Early Warning Score (LDT-EWS), which consists of frequently performed laboratory parameters. We retrospectively analyzed 651 geriatric patients who visited the emergency department (ED), were not discharged on the same day from ED, and were hospitalized. The patients were categorized according to their in-hospital mortality status. The NEWS2 and LDT-EWS values of these patients were calculated and compared on the basis of deceased and living patients. Median (interquartile range [IQR]) NEWS2 and LDT-EWS values of the 127 patients who died were found to be statistically significantly higher than those of the patients who survived (NEWS2: 5 [3–8] vs. 3 [1–5]; p < 0.001; LDT-EWS: 8 [7–10] vs. 6 [5–8]; p < 0.001). In the receiver operating characteristic curve analysis, the NEWS2, LDT-EWS, and NEWS2+LDT-EWS—formed by the sum of the two scoring systems—resulted in 0.717, 0.705, and 0.775 area under curve values, respectively. The NEWS2 and LDT-EWS were found to be valuable for predicting in-hospital mortality in geriatric patients. The power of the NEWS2 to predict in-hospital mortality increased when used with the LDT-EWS. [ABSTRACT FROM AUTHOR]

• Constructing the decision tree according to new criteria for dividing. • Enhancing information gain computation with ontology-based knowledge. • Reducing the bias issue that exists in the conventional information gain. • Reducing the likelihood of the overfitting issue. • Acquiring superior decision criteria that resemble human logic. The decision tree (DT) is a well-established tool used to classify information. Its efficiency in accessing data and the accuracy of its outcomes is largely dependent on the characteristics of the input data and the efficacy of the construction processes and algorithms. In this study, we are introducing an innovative approach called the Semantic Decision Tree (SDT). The SDT builds upon the concept of the Iterative Dichotomiser 3 (ID3) algorithm. Our proposed SDT's first application involves utilizing a knowledge base, intentionally designed to support the DT construction process. The primary goal here is to effectively address the multi-value bias selection issue, a recurring problem found in a number of DT algorithms. Building on this, we adopt an ontology summarization technique to fine-tune the splitting condition of the DT. This adjustment is influenced by the attribute's importance value, extracted directly from the ontology. This process leads to the creation of nodes that align more precisely with the dataset than those generated by traditional DT methods. To validate the robustness and precision of our proposed SDT, we evaluated it across four distinct datasets. In addition, we benchmarked the classification accuracy of our SDT against several leading baseline algorithms; the traditional ID3, C4.5, CART, and the Mutual Information Decision Tree (MIDT). The findings demonstrate that our proposed SDT outperforms these industry-standard algorithms in terms of accuracy. Further in-depth qualitative analysis showed that the modifications in information gain resulted in DT structures that align more naturally with human decision-making logic. [ABSTRACT FROM AUTHOR]

Ensemble diversity is an important characteristic of Multiple Classifier Systems (MCS), which aim at improving the overall performance of a classification system by combining the response of several models. While diversity may be introduced through various manipulations at the data level and the model level, some MCSs incorporate local information in order to increase it and/or take advantage of it, based on the idea that the different classifiers in the ensemble may have expertise in distinct areas of the feature space. 1 1 Code available at: https://github.com/marianaasouza/olp%5fplusplus. Following a similar reasoning, we introduced in a previous work an ensemble method which produces in test time a few experts in the local region where each given query sample is located. These local experts, which are generated with slightly differing views of the target area, are then used to label the corresponding unknown instance. While the framework was shown to perform well especially over imbalanced problems, the locality definition in the method is based on the nearest neighbors rule and Euclidian distance, as is the case of various local-based ensembles, which may suffer from the effects of the curse of dimensionality over high dimensional problems. Thus, in this work, we propose a local ensemble method in which we leverage the data partitions given by decision trees for locality definition. More specifically, the partitions defined at different levels of the decision path that a given query instance traverses in the tree(s) are used as the regions over which the local experts are produced. By using different node levels from the path, each classifier in the local pool has a moderately distinct view of the target region without resorting to a dissimilarity metric, which might be susceptible to high dimensional spaces. Experimental results over 39 high dimensional problems showed that the proposed approach was significantly superior to our previous, distance-based framework in balanced accuracy rate. Compared to other six local-based ensemble methods, including dynamic selection and weighting schemes, the proposed method achieved competitive results, outperforming the random forest baseline and two state-of-the-art dynamic ensemble selection techniques. • Local-based ensemble methods produce and/or exploit models specialized in each area. • Locality is often achieved via distance metrics which degrade in high dimensionality. • We propose a method that produces local experts on the fly for high dimensional data. • The models are fit over multiple partitions recursively defined by decision trees. • The proposed method outperformed several distance-based local ensemble methods. [ABSTRACT FROM AUTHOR]

• First explainable clustering algorithm for k-means sensitive to the tree's depth. • Superior results to similar algorithms in experiments with 16 datasets. • Parameter calibration allows exploration of trade-off between cost and explainability. A number of recent works have employed decision trees for the construction of explainable partitions that aim to minimize the k -means cost function. These works, however, largely ignore metrics related to the depths of the leaves in the resulting tree, which is perhaps surprising considering how the explainability of a decision tree depends on these depths. To fill this gap in the literature, we propose an efficient algorithm with a penalty term in its loss function to favor the construction of shallow decision trees – i.e., trees whose leaves are not very deep, which translate to clusters that are defined by a small number of attributes and are therefore easier to explain. In experiments on 16 datasets, our algorithm yields better results than decision-tree clustering algorithms recently presented in the literature, typically achieving lower or equivalent costs with considerably shallower trees. [ABSTRACT FROM AUTHOR]

The ability to predict new chemical entity performance using in vivo animal models has been under investigation for more than two decades. Pharmaceutical companies use their own strategies to make decisions on the most appropriate formulation starting early in development. In this paper the biopharmaceutical decision trees available in four EFPIA partners (Bayer, Boehringer Ingelheim, Bristol Meyers Squibb and Janssen) were discussed by 7 companies of which 4 had no decision tree currently defined. The strengths, weaknesses and opportunities for improvement are discussed for each decision tree. Both pharmacokineticists and preformulation scientists at the drug discovery & development interface responsible for lead optimization and candidate selection contributed to an overall picture of how formulation decisions are progressed. A small data set containing compound information from the database designed for the IMI funded OrBiTo project is examined for interrelationships between measured physicochemical, dissolution and relative bioavailability parameters. In vivo behavior of the drug substance and its formulation in First in human (FIH) studies cannot always be well predicted from in vitro and/or in silico tools alone at the time of selection of a new chemical entity (NCE). Early identification of the risks, challenges and strategies to prepare for formulations that provide sufficient preclinical exposure in animal toxicology studies and in FIH clinical trials is needed and represents an essential part of the IMI funded OrBiTo project. This article offers a perspective on the use of in vivo models and biopharmaceutical decision trees in the development of new oral drug products. [ABSTRACT FROM AUTHOR]

• Sugar beet nitrogen fertilization was optimized using leaf nutrient status samples. • Root yield and yield quality parameters were used as training and test data. • Ensemble decision trees and artificial neural network regression were used. • Na had high variable importance for root yield and sucrose content. • The top accuracy had a median R2 of 0.927 and ranged from 0.842 to 0.998. Nitrogen fertilization has a crucial role in sugar beet production, especially concerning root yield and quality. This study employed a machine learning approach to predict root yield and quality parameters based on the nutrient status of sugar beet leaves in relation to nitrogen fertilization. The field experiment included the following N fertilization treatments for sugar beet production: control (N0), presowing (N1 = 45 kg N ha−1) and presowing with top-dressing (N2 = 99 and 154.5 kg ha−1). Leaf samples were collected during the vegetation period in six intervals (May-Sept) to determine the levels of N, K and Na in the leaf dry matter. The machine learning regression based on ensemble decision trees and artificial neural network was used to determine the relationship of leaf samples based on varying N fertilization with yield parameters. Among the leaf elements analyzed, Na exhibited the highest average relative variable importance for root yield, sucrose content, and other quality parameters during the season with greater precipitation. In the season with less precipitation, N content at the beginning of July showed higher importance on root yield (74.6). The evaluated machine learning methods consistently achieved high accuracy across various combinations of input data and yield parameters, with a median R2 of 0.927 and a range from 0.842 to 0.998. [ABSTRACT FROM AUTHOR]

• Uncertainty bounds obtained by decision trees (DTs) may be used as limits-of-acceptability. • Uncertainty bounds obtained by decision trees (DTs) may account for measurement uncertainty. • A heuristic mathematical proof supports the above assertion for hypothetical large sample case. • Results from some real-world rainfall runoff data are also in agreement. • DTs can be particularly useful for model inference in baseflow dominated and ungauged catchments. A hydrological model incurs three types of uncertainties: measurement, structural and parametric uncertainty. For instance, in rainfall-runoff models, measurement uncertainty exists due to errors in measurements of rainfall and streamflow data. Structural uncertainty exists due to errors in mathematical representation of hydrological processes. Parametric uncertainty is a consequence of our inability to measure effective model parameters, limited data available to calibrate model parameters, and measurement and structural uncertainties. The existence of these predominantly epistemic uncertainties makes the model inference difficult. Limits-of-acceptability (LOA) framework has been proposed in the literature for model inference under a rejectionist framework. LOAs can be useful in model inference if they reflect the effect of errors in rainfall and streamflow measurements. In this study, the usefulness of quantile random forest (QRF) algorithm has been explored for constructing LOAs. LOAs obtained by QRF were compared to the uncertainty bounds obtained by rating-curve analysis and the LOAs obtained by runoff ratio method. Rating curve analysis yields uncertainty in streamflow measurements only and the runoff ratio method is expected to reflect uncertainty in rainfall and streamflow volume measurements. LOAs obtained by using QRF were found to envelop the uncertainty bounds due to streamflow measurement errors. LOAs obtained by QRF and runoff ratio methods were similar. Further, QRF LOAs were scrutinized in terms of their ability to reflect the effect of rainfall uncertainty, both qualitatively and quantitatively. Results indicate that QRF LOAs reflect the effect of rainfall uncertainty: increase in standard deviation with increase in mean streamflow values and decrease in coefficient of variation with increase in mean streamflow values. A mathematical analysis of the LOAs obtained by the QRF method is presented to provide a theoretical foundation. [ABSTRACT FROM AUTHOR]

Decision trees and rule-based expert systems (RBES) are standard diagnostic tools. We propose a mixed technique that starts with a probabilistic decision tree where information is obtained from a real world data base. The decision tree is automatically translated into a set of probabilistic rules. Meanwhile a panel of experts proposes their own set of probabilistic rules, according with their experience on the subject. Both sets of rules are combined, generating a mixed RBES with probabilistic rules. The expected probabilities of the rules translating the knowledge in the decision tree are discretized by considering a mapping from intervals of expected probabilities into a set of five values. This way, knowledge coming from real data is completed with the experience of the panel of experts in order to provide a more accurate prediction of suffering from type 2 diabetes mellitus (T2DM) before seven and a half years in the future. The proposed technique is illustrated with a real case using a diabetes diagnosis probabilistic decision tree built using 1350 out of 1800 real cases and the rules provided by a panel of experts in diabetes. The final result takes into account both the probabilities of the rules and the number of times that each possible consequent is reached, giving a probabilistic result among seven possibilities. For modeling the decision tree, 75% of the individuals in the database (randomly selected) have been used and the rest (25%) have been used to test the results. The results of the Mixed RBES have been compared with the results of the Tree RBES (the RBES built using only the rules from the decision tree) and the results of the Experts' RBES (the RBES built using only the rules from the panel of experts). The accuracy of the predictions of the Mixed RBES is much better. [ABSTRACT FROM AUTHOR]

Gradient boosting models like XGBoost are among the most popular models for tabular classification problems. Unfortunately, the greediness of gradient boosting algorithms can cause them to rely too heavily on some features, thereby starving the other features. We propose Iterative Feature eXclusion (IFX) to alleviate this problem by iteratively removing the most influential feature from the training data and continuing training. By forcing the model to learn from weaker features, we increase the diversity of the gradient boosting model and improve the predictive performance. Our experiments show that in most cases, IFX improves XGBoost predictive performance, sometimes by a large margin. All of the code and results from our experiments are freely available online. Iterative Feature eXclusion can be used as a drag-and-drop replacement for XGBoost, thereby easing the adoption of our work by machine learning researchers and practitioners. • Strategically removing features during training improves predictive performance. • Features can be removed during training, without changing modeling code. • Iterative Feature eXclusion yields an XGBoost model with improved performance. [ABSTRACT FROM AUTHOR]

Identifying predictors of opioid overdose following release from prison is critical for opioid overdose prevention. We leveraged an individually linked, state-wide database from 2015–2020 to predict the risk of opioid overdose within 90 days of release from Massachusetts state prisons. We developed two decision tree modeling schemes: a model fit on all individuals with a single weight for those that experienced an opioid overdose and models stratified by race/ethnicity. We compared the performance of each model using several performance measures and identified factors that were most predictive of opioid overdose within racial/ethnic groups and across models. We found that out of 44,246 prison releases in Massachusetts between 2015–2020, 2237 (5.1%) resulted in opioid overdose in the 90 days following release. The performance of the two predictive models varied. The single weight model had high sensitivity (79%) and low specificity (56%) for predicting opioid overdose and was more sensitive for White non-Hispanic individuals (sensitivity = 84%) than for racial/ethnic minority individuals. Stratified models had better balanced performance metrics for both White non-Hispanic and racial/ethnic minority groups and identified different predictors of overdose between racial/ethnic groups. Across racial/ethnic groups and models, involuntary commitment (involuntary treatment for alcohol/substance use disorder) was an important predictor of opioid overdose. [ABSTRACT FROM AUTHOR]

• Scalable gradient boosting systems, XGBoost, LightGBM and CatBoost compared for formation lithology classification. • Hyperparameter tuning, training and model testing done using well log data obtained from Ordos Basin, China. • LightGBM possesses the highest weighted and macro average values of precision, recall and F1. • LightGBM and CatBoost suggested as first-choice algorithms for lithology classification using well log data. The classification of underground formation lithology is an important task in petroleum exploration and engineering since it forms the basis of geological research studies and reservoir parameter calculations. Hence, there have recently been increased efforts to automate lithology classification by incorporating various data science tools and principles. In this regard, efforts were made recently to evaluate machine learning methods to classify formation lithology by using data from the Daniudui gas field (DGF) and Hangjinqi gas field (HGF), both located in China. Although the boosted ensemble learners utilized in the studies performed well, there is still scope for improvement with respect to the prediction metrics. Additionally, the issue of scalability of some of these algorithms is also of concern. Hence, building upon the success of these algorithms in the previous studies, we tap into the state of the art of scalable ensemble decision tree algorithms, in our study. Specifically, we applied recently developed gradient boosted decision tree (GBDT) systems, namely, XGBoost, LightGBM and CatBoost, after combining well log data obtained from DGF and HGF. We compare their performance with random forests (RFs), AdaBoost and gradient boosting machines (GBMs) which serve as a baseline. We evaluated the algorithms using metrics such as the micro average, macro average and weighted average of precision (Pr), recall (Re) and F1-score (F1) on the test set after hyperparameter tuning. In our analysis, among the applied algorithms, we found that LightGBM possessed the highest metrics. Our work identifies LightGBM and CatBoost as good first-choice algorithms for the supervised classification of lithology when utilizing well log data. [ABSTRACT FROM AUTHOR]

• Development of non-linear QSPR model to predict diels alder reaction rate coefficient. • Connectivity indices coupled with multivariate statistical methods used in model development. • Multi-gene genetic programming (MGGP) was employed. • Modifications of GPTIPS 2.0 toolbox was suggested for model generation with lowest predictive error possible. Developing a QSPR model, which not only captures the influence of reactant structures but also the solvent effect on reaction rate, is of significance. Such QSPR models will serve as a prerequisite for the simultaneous computer-aided molecular design (CAMD) of reactants, products and solvents. They will also be useful in predicting the rate constant without entirely relying on experiments. To develop such a QSPR, recently, Datta et al. (2017) used the Diels-Alder reaction as a case study. Their model displayed great promise, but there is scope for improvement in the model's prediction metrics. In our work, we improve upon their model by introducing non-linearity. This is achieved using multi-gene genetic programming (MGGP). In our methodology, a combination of genetic algorithm (GA) and directed trees was used to develop a branched version of chromosomes, allowing increased possibility of generation of models with high prediction metrics. In our work, prior to model development through MGGP, principal component analysis (PCA) was conducted. Lastly, models were evaluated based on metrics such as R2, Q2, and RMSE. [ABSTRACT FROM AUTHOR]

This paper presents a tracking algorithm for arbitrary objects in challenging video sequences. Targets are modelled at three different levels of granularity (pixel, parts and bounding box levels), which are cross-constrained to enable robust model relearning. The main contribution is an adaptive clustered decision tree method which dynamically selects the minimum combination of features necessary to sufficiently represent each target part at each frame, thereby providing robustness with computational efficiency. The adaptive clustered decision tree is used in two separate ways: firstly for parts level matching between successive frames; secondly to select the best candidate image regions for learning new parts of the target. We test the tracker using two different tracking benchmarks (VOT2013-2014 and CVPR2013 tracking challenges), based on two different test methodologies, and show it to be more robust than the state-of-the-art methods from both of those tracking challenges, while also offering competitive tracking precision. Additionally, we evaluate the contribution of each key component of the tracker to overall performance; test the sensitivity of the tracker under different initialization conditions; investigate the effect of using features in different orders within the decision trees; illustrate the flexibility of the method for handling arbitrary kinds of features, by showing how it easily extends to handle RGB-D data. [ABSTRACT FROM AUTHOR]

Decision tree learning algorithms are known to be unstable, because small changes in the training data can result in highly different decision trees. An important issue is how to quantify decision tree stability. Two types of stability are defined in the literature: structural and semantic stability. However, existing structural stability measures are meaningless when applied to apparently different decision trees, and semantic stability only focuses on prediction accuracy without considering structural information. This paper proposes a region compatibility based structural stability measure for decision trees that considers the structural distribution of leaves from the view of basic probability assignments in evidence theory. To the best of our knowledge, we are the first to use basic probability assignments to quantify decision tree stability. We prove convergence for region compatibility, and show that apparently different decision trees have some inherent similarity from the view of region compatibility. We also clarify the meaning of region compatibility for measuring decision tree stability, and derive a method to select a relatively stable learning algorithm for a given dataset. Experimental results validate that region compatibility is effective to quantify the stability of decision tree learning algorithms. [ABSTRACT FROM AUTHOR]

The history of the evolutionarily significant unit has deep roots, back to our appreciation that what we defined as species were often themselves divided into separate units with some degree of isolation. There are many approaches for what is generally recognized as a common objective, the identification of intraspecific groupings that require separate designation for effective conservation. Some definitions of the evolutionarily significant unit have received greater recognition than others, and interpretation can be varied leading to varied outcomes. I propose a process that first identifies conservation units broadly, and then recognizes those that are divergent beyond defined thresholds based on quantitative assessment. The process can be facilitated in a four-step decision tree. The designation of units for conservation has been a necessary but challenging objective since conservation efforts began. Most species are divided, typically by environment, into populations with independent evolutionary trajectories. There are practical conservation objectives for defining these boundaries. Separate genetic clusters provide future evolutionary potential as environments change, and individuals in isolated populations may lose fitness when population size is reduced. The history of the effort to define units is briefly reviewed here, but I focus on finding a process that may facilitate uniform and effective application at a time when conservation urgency is great. I propose a refinement of the designated unit concept, distinguishing between conservation units (CUs) and evolutionarily sustaining conservation units (ESCUs). [ABSTRACT FROM AUTHOR]

This study aims at presenting innovative ensemble models for mapping potential inundation areas due to riverine floods in the Najafabad basin in Iran. Altogether, a total of 16 flood causative factors were derived and used as independent variables based on Variance Inflation Factor (VIF) multi-collinearity test. Flood inventory consisted of 154 riverine flood locations, which were used in training and testing phases of modelling procedures. At the first stage, we developed a standalone credal decision tree (CDT) model and then integrated with five algorithms: bagging (BA), dagging (DA), multiboost (MB), decorate (DE), and random subspace (RSS). The validation results in case of the training data confirmed very high accuracy of all six models used, i.e. CDT, BA-CD, DA-CDT, DE-CDT, MB-CDT, and RSS-CDT models recorded the AUC-ROC values of 0.910, 0.944, 0.950, 0.931, 0.992, and 0.956, respectively. With regard to validation data, five ensemble models (BA-CD, DA-CDT, DE-CDT, MB-CDT, and RSS-CDT) recorded AUC-ROC values higher than 0.9 while only the standalone CDT model recorded the AUC-ROC value of 0.844. The best accuracy was recorded by the ensemble MB-CDT model with the AUC-ROC values of 0.992 and 0.941 in case of the training phase and testing phase, respectively. The results presented in this study are useful for flood risk management and, especially during the preliminary phase, where the most susceptible areas that can be inundated by riverine flooding are determined. [ABSTRACT FROM AUTHOR]

• Stratification strategy based on the decision tree model for attributing vegetation change drivers. • Vegetation photosynthesis exhibited an improvement trend in 81.5% of areas in China. • Changes in vegetation photosynthesis were mainly caused by climate change. Vegetation photosynthesis plays a key role in maintaining carbon balance in terrestrial ecosystems. Solar-Induced chlorophyll Fluorescence (SIF) was utilized as a proxy for vegetation photosynthesis in this study, and the Theil-Sen and Mann-Kendall methods were used to examine trends of vegetation change in China. The impacts of climate conditions on vegetation photosynthesis were examined with the correlation analysis and recursive feature elimination (RFE) method. To distinguish the contributions of climate change and human activities on vegetation photosynthesis, land use data and SIF change trends were employed as inputs to a decision tree model. The results revealed that 81.5% of the areas in China exhibited an improving trend of vegetation photosynthesis. Vegetation degradation areas were mainly located in Northern arid and semi-arid region (NASR), Qinghai Tibet Plateau (QTP) and Huang-Huai-Hai Plain (HHHP). Precipitation dominated the effect of vegetation photosynthesis in these regions. The increase of vegetation photosynthesis in the southern regions was positively impacted by solar radiation. The growth of agricultural technology obviously benefited the improvement of vegetation photosynthesis in the Middle-lower Yangtze Plain (MLYP) and Northeast China Plain (NCP). Furthermore, the establishment of ecological conservation programs has helped to restore vegetation in arid and semi-arid areas. The detrimental effects of human activities included overgrazing disturbance and coal mining activities, which resulted in vegetation deterioration in areas of QTP and HHHP. The methodologies and findings of this study can help to comprehend changes in vegetation dynamics and give a theoretical foundation for evaluating regional ecological conservation initiatives. [ABSTRACT FROM AUTHOR]

The International Conference of Harmonization (ICH) Q6A document provides guidance on setting specifications for new drug substances and drug products. 1 In this paper we focus on decision trees 4 (#1) to (#3) in the guidance related to solid-state form transformation. Form transformation could occur from use of high energy forms to overcome solubility challenges or stresses from manufacturing processes. The decision trees provide guidance on when and how polymorphic form changes should be monitored and controlled. However, guidance is high level and does not capture aspects related to assessments needed to understand if there is a risk of transformation or tools that can be integrated to understand the severity of bioavailability impact at different stages of development. The objective of this paper is therefore to provide comprehensive chemistry manufacturing and controls (CMC) and regulatory strategies to manage the risk of form transformation. This includes practical workflows for form transformation risk assessment, analytical tools to detect and quantify the transformation including their shortcomings, biopharmaceutical tools to understand the severity of transformation risk and if needed justify the limits based on clinical relevance. Finally, a few case studies are discussed that capture how the workflow can be used to manage transformation risk. [ABSTRACT FROM AUTHOR]

Decision trees have long been recognized as models of choice in sensitive applications where interpretability is of paramount importance. In this paper, we examine the computational ability of Boolean decision trees for the explanation purpose. We focus on both abductive explanations (suited to explaining why a given instance has been classified as such by the decision tree at hand) and on contrastive explanations (suited to explaining why a given instance has not been classified by the decision tree as it was expected). More precisely, we are interested in deriving, minimizing, and counting abductive explanations and contrastive explanations. We prove that the set of all irredundant abductive explanations (also known as PI-explanations or sufficient reasons) for an instance given a decision tree can be exponentially larger than the size of the input (the instance and the decision tree). Therefore, generating the full set of sufficient reasons for an instance can be out of reach. In addition, deriving a single sufficient reason, though computationally easy when dealing with decision trees, does not prove enough in general; indeed, two sufficient reasons for the same instance may differ on many features. To deal with this issue and generate synthetic views of the set of all sufficient reasons, we define notions of relevant features and of necessary features that characterize the (possibly negated) features appearing in at least one or in every sufficient reason for an instance, and we show that they can be computed in polynomial time. We also introduce the notion of explanatory importance, that indicates how frequent each (possibly negated) feature is in the set of all sufficient reasons. We show how the explanatory importance of a (possibly negated) feature and the number of sufficient reasons for an instance can be obtained via a model counting operation, which turns out to be practical in many cases. We also explain how to enumerate minimum-size sufficient reasons. We finally show that, unlike sufficient reasons, the set of all contrastive explanations for an instance given a decision tree can be derived, minimized and counted in polynomial time. [ABSTRACT FROM AUTHOR]

Decision Trees (DT) describe a type of machine learning method that has been widely used in the geosciences to automatically extract patterns from complex and high dimensional data. However, like any data-based method, the application of DT is hindered by data limitations, such as significant biases, leading to potentially physically unrealistic results. We develop interactive DT (iDT) that put humans in the loop to integrate the power of experts' scientific knowledge with the power of the algorithms to automatically learn patterns from large datasets. We created an open-source Python toolbox that implements the iDT framework. Users can interactively create new composite variables, change the variable and threshold to split, prune and group variables based on their physical meaning. We demonstrate with three case studies how iDT overcomes problems with current DT thus achieving higher interpretability and robustness of the result. • We propose a framework for building Decision Trees that put humans in the loop. • The framework compensates for dataset issues encountered in standard Decision Trees. • Interactive Decision Trees enhance interpretability and physical consistency. • We developed an open-source toolbox for constructing Interactive Decision Trees. [ABSTRACT FROM AUTHOR]

When collecting panel data, we need to acknowledge that responses do not represent independent measurements. The known apparatus in transportation research offers several opportunities to estimate panel effects for well-known and widely applied models such as hazard and dynamic logit models. However, the transportation research community is not endowed with a rich set of methods to account for panel effects in dynamic probabilistic decision trees, which have been used as a formalism for the representation of decision heuristics. Building on scarce prior work in statistics, we elaborate an approach to estimate panel effects in dynamic probabilistic decision trees with multinomial action states. Given that panel data naturally have a hierarchical structure with repeated measures nested within individuals, we implement a mixed-effects model that simultaneously accounts for population-level effects (fixed effects), between-individual variances (random effects), and within-individual variances (autocorrelations). The approach uses an iterative estimation procedure between CHAID-based probabilistic tree induction and Bayesian generalized linear mixture modeling (GLMM). When extracting the dynamic probabilistic decision trees, it is assumed that the random effects are known, while it is assumed that the fixed effects are known when estimating the Bayesian GLMM. This iterative process continues until convergence is reached. A Monte Carlo technique is used to navigate between aggregate choice probabilities and individual level multinomial choices. We also test the significance of temporal autocorrelation within individuals. The suggested approach is illustrated using charging station choice of users of Plug-in Electric Vehicles (PEV). Results support the potential value of the suggested approach. [ABSTRACT FROM AUTHOR]

• Travelling waves to fault location in radial distribution systems is proposed. • A structure is pointed for locating faults precisely in distribution systems. • A new method for mitigation of the multiple estimation problem is presented. • Machine learning algorithms are used for mitigation of multiple estimation. • Tests show accurate results for the fault location and mitigation tasks. Electrical systems have been facing transformations, such as distributed generation insertion, system expansion and regulatory standards in order to increase reliability and quality of the power supply. Thus, fault location methods must be updated to ensure accuracy in estimating the location of electrical faults. The delay in restoring the system causes damage to utilities and consumers. Considering this, the current work presents an approach capable of locating faults accurately in radial distribution systems. At first, the distance is estimated using the travelling wave theory with data acquired from two terminals. Next, due to the radial characteristic of the system, the proposal aims to mitigate the problem of multiple estimation of faults. Thus, features are extracted from the voltage and current signals, which are used as inputs of decision trees to identify the fault region. The proposed approach was validated in a medium voltage distribution system, in which the results presented an average error of 0.79% (with a standard deviation of 0.4%) in estimating the fault distances and an average accuracy above 88.7% in identifying the region under fault. Thus, it was demonstrated that the proposed methodology is efficient to locate faults, mitigating the problem of multiple estimation. [ABSTRACT FROM AUTHOR]

The article describes the results of a naturalistic driving study done in Kuwait performed by 34 participants wearing a mobile eye tracker to monitor the effect of roadside advertisements on user attention. Eye-tracking (fixations) are the main dependent variable and examined as a function of driving/roadside characteristics, particularly billboards, speed, and so forth. The results obtained were analyzed using traditional statistics (ANOVA test and multiple linear regression) and machine learning (decision tree estimation methods). From the results, it was found that road advertisements negatively affect driver attention and thus road safety. How the level of safety varies with the type and size of advertisement was also investigated. As a consequence, all of the estimates revealed that different aspects of advertising have a detrimental impact on drivers' behavior, and that the duration of fixation and the rate of acceleration before viewing an ad are impacted by advertising type and size, respectively. [ABSTRACT FROM AUTHOR]

Highlights • Tree-CAT, a decision tree-based technique for building CATs is presented. • It builds an adaptive test with minimum mean square error. • It reduces the item-selection time to milliseconds. • It is well suited for simultaneous administration and for on-line testing. • Opens new research lines combining psychology, statistics and operational research. Abstract In the last few years, several articles have proposed decision trees (DTs) as an alternative to computerized adapted tests (CATs). These works have focused on showing the differences between the two methods with the aim of identifying the advantages of each of them and thus determining when it is preferable to use one method or another. In this article, Tree-CAT, a new technique for building CATs is presented. Unlike the existing work, Tree-CAT exploits the similarities between CATs and DTs. This technique allows the creation of CATs that minimise the mean square error in the estimation of the examinee's ability level, and controls the item's exposure rate. The decision tree is sequentially built by means of an innovative algorithmic procedure that selects the items associated with each of the tree branches by solving a linear program. In addition, our work presents further advantages over alternative item selection techniques with exposure control, such as instant item selection or simultaneous administration of the test to an unlimited number of participants. These advantages allow accurate on-line CATs to be implemented even when the item selection method is computationally costly. [ABSTRACT FROM AUTHOR]

Although many studies have explored the relationship between the built environment and travel behavior, the literature offers limited evidence about the collective influence of built environment attributes, and their non-linear effects on travel. This study innovatively adopts gradient boosting decision trees to fill the gaps. Using data from Oslo, we apply this method to the data on both weekdays and weekends to illustrate the differential effects of built environment characteristics on driving distance. We found that they have a stronger effect on weekdays than on weekends. On weekdays, their collective influence is larger than that of demographics. Furthermore, they show salient non-linear effects on driving distance in both models, challenging the linearity assumption commonly adopted in the literature. This study also identifies effective ranges of distance to different centers and population density, and highlights the important role of sub-centers in driving reduction. [ABSTRACT FROM AUTHOR]

This paper focuses on the practical implementation of online transient stability assessment (TSA) tools that employ, in conjunction with high-speed synchronized phasor measurements obtained from phasor measurement units (PMUs), classification and regression trees (CART) and multivariate adaptive regression splines (MARS) models. To build CART and MARS models that are amenable to real-time applications, pertinent transient stability-related system characteristics are identified; these include voltage and current phasors, deviations from the centre-of-inertia angle and speed, and potential- and kinetic-energy related quantities. These characteristic quantities are evaluated using PMU measurements and then leveraged to train CART and MARS models for the full Western Electricity Coordinating Council (WECC) system. The resultant models are tested and validated with the full WECC system using credible contingency scenarios in the BC Hydro subsystem. High prediction accuracy rates are observed for both CART and MARS methods, making them attractive options for real-time TSA. [ABSTRACT FROM AUTHOR]