Search

Ozone pollution in chemical industrial parks is severe and complicated and is significantly influenced by pollutant emissions and meteorological parameters. In this study, we innovatively investigated the formation rules of ozone by using observation-based analyses and a gradient-boosting decision tree (GBDT) model, focusing on a typical chemical industrial park located in the Yangtze River Delta of China. The results revealed that ozone concentration was positively correlated with temperature while negatively correlated with NO2 concentration and relative humidity (RH). Ozone pollution was predominantly observed from April to October (M4–10). The optimized GBDT model was subsequently utilized to establish a specific and quantifiable relationship between each single dominant impact factor (RH, NO2, temperature, and PM2.5) and ozone within a complex and uncertain multi-factor context during M4–10. Detailed discussions were conducted on the reaction rate of ozone-related to different levels of RH and temperature. The accumulation of ozone was favored by high temperature and low RH, with the maximum ozone concentration observed at the RH of 50% and the temperature of 35 °C. The NO2-O3 change curve exhibited distinct phases, including a period of stability, gradual increase, rapid increase, and equilibrium. During the second and third periods, the ratio of ozone production to NO2 consumption was 0.10 and 2.73, respectively. Furthermore, there was a non-monotonic relationship between variations in ozone concentration and PM2.5 concentration. Hence, it is imperative to implement fine control strategies in the park, such as adopting seasonal production strategies, implementing targeted measures for controlling NOx and active VOCs, and employing special control methods during periods of high temperature. This study provides aid in achieving effective management of localized ozone pollution and ensuring compliance with air quality standards. [ABSTRACT FROM AUTHOR]

Animal psychophysics can generate rich behavioral datasets, often comprised of many 1000s of trials for an individual subject. Gradient-boosted models are a promising machine learning approach for analyzing such data, partly due to the tools that allow users to gain insight into how the model makes predictions. We trained ferrets to report a target word's presence, timing, and lateralization within a stream of consecutively presented non-target words. To assess the animals' ability to generalize across pitch, we manipulated the fundamental frequency (F0) of the speech stimuli across trials, and to assess the contribution of pitch to streaming, we roved the F0 from word token-to-token. We then implemented gradient-boosted regression and decision trees on the trial outcome and reaction time data to understand the behavioral factors behind the ferrets' decision-making. We visualized model contributions by implementing SHAPs feature importance and partial dependency plots. While ferrets could accurately perform the task across all pitch-shifted conditions, our models reveal subtle effects of shifting F0 on performance, with within-trial pitch shifting elevating false alarms and extending reaction times. Our models identified a subset of non-target words that animals commonly false alarmed to. Follow-up analysis demonstrated that the spectrotemporal similarity of target and non-target words rather than similarity in duration or amplitude waveform was the strongest predictor of the likelihood of false alarming. Finally, we compared the results with those obtained with traditional mixed effects models, revealing equivalent or better performance for the gradient-boosted models over these approaches. Author summary: The sorts of listening challenges faced by real-world listeners are rarely captured by most laboratory-based auditory paradigms, particularly those testing animal models. However, many labs are attempting to utilize more realistic experiments, and more complicated behavioral paradigms require more sophisticated approaches to analyzing the resulting data. Here, we used a new behavioral paradigm to test the ability of ferret listeners to identify target speech sounds and assess their ability to generalize across changes in pitch. To make sense of the resulting dataset, we used machine learning to understand how trained ferrets perform this task. Gradient-boosted regression and decision trees are well-established machine learning methods that do not require users to predetermine interaction effects and are accompanied by visualization tools that allow insights into how multiple factors ultimately shape behavior. We compare the use of gradient-boosted models to more standard regression approaches and that this machine learning approach is ideal for analyzing behavioral data in animal models. [ABSTRACT FROM AUTHOR]

As the field of zero energy building design and research continues to progress, the use of data analysis methods is on the rise. These methods are applied to create assessment criteria, compare performance, and aid in design decision making. Decision trees, as a data-driven approach, offer interpretability and predictability, assisting designers in summarizing their design experience and serving as a foundation for design references. However, the current application of decision tree methods in the zero energy house sector primarily focuses on HVAC systems, lacking a comprehensive exploration from an architectural design perspective. Therefore, this study presents an empirical method for building and applying models based on decision trees, using zero energy house cases in severely cold regions of China as samples. Through an analysis of the interactions among various passive design parameters and the use of EnergyPlus for performance simulations, a decision tree model is established. This model aids in determining the recommended combinations of passive design parameters that meet the criteria of low energy consumption. Moreover, feature weighting highlights the most influential passive design parameters on building energy consumption, including the length of the architectural gestalt plane, the roof shape, and the ground thermal resistance. This research provides valuable methods and guidance for the design and construction of zero energy houses in severely cold regions of China. [ABSTRACT FROM AUTHOR]

Scientists and engineers often spend days choosing a problem and years solving it. This imbalance limits impact. Here, we offer a framework for problem choice: prompts for ideation, guidelines for evaluating impact and likelihood of success, the importance of fixing one parameter at a time, and opportunities afforded by failure. [ABSTRACT FROM AUTHOR]

Promoting the safety of commercial trucks by identifying countermeasures that eliminate/reduce the effect of factors that increase the severity of truck-related crashes is crucial. Crash causal factors for rural interstate roads, located within the mountain plains, are inherently unique compared to urban interstate roads. This is due to the presence of challenging road geometry coupled with severe weather conditions and high truck traffic volumes. This study investigated Interstate 80 in Wyoming using decision trees, as a data mining approach, and structural equation model (SEM) as a latent factor modeling approach. SEM was employed to clarify the direct and indirect relationships between endogenous and exogenous variables while accounting for the variation and covariation within and between the constructed measurement models. Crash severity data were processed to account for factors affecting single vehicles and multivehicle trucks. The results showed that the interaction with surrounding traffic was the most significant latent variable affecting the crash severity of multivehicle truck crashes, while adverse weather conditions were the most significant latent variable affecting the crash severity of single-truck crashes. The results of this study highlighted the importance of increasing the situational awareness of commercial truck drivers with upcoming hazardous events. This could be performed by communicating information using variable message signs, the 511 application, the commercial vehicle operator portal (CVOP), or the connected vehicle (CV) technologies. [ABSTRACT FROM AUTHOR]

In this paper, we consider classes of conventional decision tables closed relative to the removal of attributes (columns) and changing decisions assigned to rows. For tables from an arbitrary closed class, we study the dependence of the minimum complexity of deterministic and nondeterministic decision trees on the complexity of the set of attributes attached to columns. We also study the dependence of the minimum complexity of deterministic decision trees on the minimum complexity of nondeterministic decision trees. Note that a nondeterministic decision tree can be interpreted as a set of true decision rules that covers all rows of the table. [ABSTRACT FROM AUTHOR]

This article shows that decision trees constructed with Classification and Regression Trees (CART) and C4.5 methodology are consistent for regression and classification tasks, even when the number of predictor variables grows sub-exponentially with the sample size, under natural 0-norm and 1-norm sparsity constraints. The theory applies to a wide range of models, including (ordinary or logistic) additive regression models with component functions that are continuous, of bounded variation, or, more generally, Borel measurable. Consistency holds for arbitrary joint distributions of the predictor variables, thereby accommodating continuous, discrete, and/or dependent data. Finally, we show that these qualitative properties of individual trees are inherited by Breiman's random forests. A key step in the analysis is the establishment of an oracle inequality, which allows for a precise characterization of the goodness of fit and complexity tradeoff for a mis-specified model. for this article are available online. [ABSTRACT FROM AUTHOR]

The structure of climate finance flows from donors (multilateral sources) to recipients (sub-Saharan African countries) was studied. This is the first study to provide a comprehensive network structure of the climate finance flows into Africa, based on the global public climate finance governance system. Network theory and decision tree techniques were employed. The results obtained generally fit the multilateral funding units (MFUs) into two categories: central funding units (CFUs), which simultaneously attend to the climate-related projects of many African countries, and the boundary funding units (BFUs), which cater to only a few countries at once. An isolated region with no BFUs was identified. African countries within this group could be more exposed to climate financial risk as they rely on only the CFUs. In general, with the exception of mitigation REDD (reducing emissions from deforestation and forest degradation) climate finance, a disproportionate distribution of climate themes, with particular reference to adaptation finance, was observed across sub-Saharan Africa. This has real implications for equitable resource allocation of climate funds. The need for African-bred region-wide MFUs is recommended. [ABSTRACT FROM AUTHOR]

Differentiating multiple system atrophy (MSA) from related neurodegenerative movement disorders (NMD) is challenging. MRI is widely available and automated decision-tree analysis is simple, transparent, and resistant to overfitting. Using a retrospective cohort of heterogeneous clinical MRIs broadly sourced from a tertiary hospital system, we aimed to develop readily translatable and fully automated volumetric diagnostic decision-trees to facilitate early and accurate differential diagnosis of NMDs. 3DT1 MRI from 171 NMD patients (72 MSA, 49 PSP, 50 PD) and 171 matched healthy subjects were automatically segmented using Freesurfer6.0 with brainstem module. Decision trees employing substructure volumes and a novel volumetric pons-to-midbrain ratio (3D-PMR) were produced and tenfold cross-validation performed. The optimal tree separating NMD from healthy subjects selected cerebellar white matter, thalamus, putamen, striatum, and midbrain volumes as nodes. Its sensitivity was 84%, specificity 94%, accuracy 84%, and kappa 0.69 in cross-validation. The optimal tree restricted to NMD patients selected 3D-PMR, thalamus, superior cerebellar peduncle (SCP), midbrain, pons, and putamen as nodes. It yielded sensitivities/specificities of 94/84% for MSA, 72/96% for PSP, and 73/92% PD, with 79% accuracy and 0.62 kappa. There was correct classification of 16/17 MSA, 5/8 PSP, 6/8 PD autopsy-confirmed patients, and 6/8 MRIs that preceded motor symptom onset. Fully automated decision trees utilizing volumetric MRI data distinguished NMD patients from healthy subjects and MSA from other NMDs with promising accuracy, including autopsy-confirmed and pre-symptomatic subsets. Our open-source methodology is well-suited for widespread clinical translation. Assessment in even more heterogeneous retrospective and prospective cohorts is indicated. [ABSTRACT FROM AUTHOR]

Background: Depression is a highly common and recurrent condition. Predicting who is at most risk of relapse or recurrence can inform clinical practice. Applying machine-learning methods to Individual Participant Data (IPD) can be promising to improve the accuracy of risk predictions. Methods: Individual data of four Randomized Controlled Trials (RCTs) evaluating antidepressant treatment compared to psychological interventions with tapering ( N = 714 ) were used to identify predictors of relapse and/or recurrence. Ten baseline predictors were assessed. Decision trees with and without gradient boosting were applied. To study the robustness of decision-tree classifications, we also performed a complementary logistic regression analysis. Results: The combination of age, age of onset of depression, and depression severity significantly enhances the prediction of relapse risk when compared to classifiers solely based on depression severity. The studied decision trees can (i) identify relapse patients at intake with an accuracy, specificity, and sensitivity of about 55% (without gradient boosting) and 58% (with gradient boosting), and (ii) slightly outperform classifiers that are based on logistic regression. Conclusions: Decision tree classifiers based on multiple–rather than single–risk indicators may be useful for developing treatment stratification strategies. These classification models have the potential to contribute to the development of methods aimed at effectively prioritizing treatment for those individuals who require it the most. Our results also underline the existing gaps in understanding how to accurately predict depressive relapse. [ABSTRACT FROM AUTHOR]

In this study, we presented a novel hybrid artificial intelligence method for landslide susceptibility mapping (LSM) in the Kalaleh Watershed, Golestan Province, Iran. This method uses rotation forests (RF) as a Meta classifier based on Java decision trees (J48), as a base classifier named RF–J48. The created model was compared to some benchmark models, such as the J48, the open-source naive Bayes tree, and the reduced error pruning tree (REPTree). Locations of 260 different landslides were found and mapped. Additionally, it was split into training data (70%) and testing data (30%) for spatial modeling and validation analysis. Twelve landslide effective factors were chosen for the landslide susceptibility mapping process based on a literature review and the results of the multicollinearity evaluation (LSM). The approximate percentage of high LS over the Kalaleh Watershed catchment for four models is RF–J48 = 19.92%, J48 = 15.87%, NBTree = 20.13% and REPTree = 16.76%, respectively. The LS model is evaluated through the methods of TSS, efficiency, and kappa index methods on the training and validation datasets, shown in Fig. 8. Efficiency values computed using validation data/training data for each model is NBTree = 0.855/0.908, J48 = 0.868/0.911, REPTree = 0.862/0.877, and RF–J48 = 0.875/0.992, respectively. The Kappa index for each model has shown significant accuracy. The index values are maximum for the ensemble model RF–J48 of 0.537 and 0.88 for validation and training data. [ABSTRACT FROM AUTHOR]

The mixture cure model is widely used to analyze survival data in the presence of a cured subgroup. Standard logistic regression‐based approaches to model the incidence may lead to poor predictive accuracy of cure, specifically when the covariate effect is non‐linear. Supervised machine learning techniques can be used as a better classifier than the logistic regression due to their ability to capture non‐linear patterns in the data. However, the problem of interpret‐ability hangs in the balance due to the trade‐off between interpret‐ability and predictive accuracy. We propose a new mixture cure model where the incidence part is modeled using a decision tree‐based classifier and the proportional hazards structure for the latency part is preserved. The proposed model is very easy to interpret, closely mimics the human decision‐making process, and provides flexibility to gauge both linear and non‐linear covariate effects. For the estimation of model parameters, we develop an expectation maximization algorithm. A detailed simulation study shows that the proposed model outperforms the logistic regression‐based and spline regression‐based mixture cure models, both in terms of model fitting and evaluating predictive accuracy. An illustrative example with data from a leukemia study is presented to further support our conclusion. [ABSTRACT FROM AUTHOR]

We develop a new method that combines a decision tree with a wavelet transform to forecast time series data with spatial spillover effects. The method can not only improve prediction but also give good interpretability of the time series mechanism. As a feature exploration method, the wavelet transform represents information at different resolution levels, which may improve the performance of decision trees. The method is applied to simulated data, air pollution and COVID time series data sets. In the simulation, Haar, LA8, D4 and D6 wavelets are compared, with the Haar wavelet having the best performance. In the air pollution application, by using wavelet transform-based decision trees, the temporal effect of air quality index including autoregressive and seasonal effects can be described as well as the spatial correlation effect. To describe the spillover spatial effect in contiguous regions, a spatial weight is constructed to improve the modeling performance. The results show that air quality index has autoregressive, seasonal and spatial spillover effects. The wavelet transformed variables have a better forecasting performance and enhanced interpretability than the original variables. For the COVID time series of cumulative cases, spatial weighted variables are not selected which shows the lock-down policies are truly effective. [ABSTRACT FROM AUTHOR]

Korea is showing the fastest trend in the world in population aging; there is a high interest in the elderly population nationwide. Among the common chronic diseases, the elderly tends to have a high incidence of depression. That said, it has been vital to focus on preventing depression in the elderly in advance. Hence, this study aims to select the factors related to depression in low-income seniors identified in previous studies and to develop a prediction model. In this study, 2975 elderly people from low-income families were extracted using the 13th-year data of the Korea Welfare Panel Study (2018). Decision trees, logistic regression, neural networks, and random forest were applied to develop a predictive model among the numerous data mining techniques. In addition, the wrapper's stepwise backward elimination, which finds the optimal model by removing the least relevant factors, was applied. The evaluation of the model was confirmed via accuracy. It was verified that the final prediction model, in the case of a decision tree, showed the highest predictive power with an accuracy of 97.3%. Second, psychological factors, leisure life satisfaction, social support, subjective health awareness, and family support ranked higher than demographic factors influencing depression. Based on the results, an approach focused on psychological support is much needed to manage depression in low-income seniors. As predicting depression in the elderly varies on numerous influencing factors, using a decision tree may be beneficial to establish a firm prediction model to identify vital factors causing depression in the elderly population. [ABSTRACT FROM AUTHOR]

Online shopping has promoted the development of logistics and express delivery businesses. Express delivery stations are closely related to residents' daily lives, and it is an important topic for the study of urban consumption space and commercial service space. This paper analyzed the factors influencing the spatial distribution of terminal logistics space (express delivery stations) in the process of online shopping. The gradient boosting decision trees (GBDT) was selected for analyzing the factors influencing the distribution of express delivery stations. The results demonstrated that express delivery stations' distribution is mainly influenced by commercial retail and residential neighborhoods, showing a clustering toward consumer spaces and residential areas. This paper studied the association between express delivery stations and other functional spaces in the city, and established an analytical framework for the factors influencing the spatial distribution of express delivery stations. The research results help to improve the rationality and effectiveness of the setting and management of the terminal logistics space in the online shopping process. [ABSTRACT FROM AUTHOR]

The rapid development of the Internet of Things (IoT) has brought many conveniences to our daily life. However, it has also introduced various security risks that need to be addressed. The proliferation of IoT botnets is one of these risks. Most of researchers have had some success in IoT botnet detection using artificial intelligence (AI). However, they have not considered the impact of dynamic network data streams on the models in real-world environments. Over time, existing detection models struggle to cope with evolving botnets. To address this challenge, we propose an incremental learning approach based on Gradient Boosting Decision Trees (GBDT), called GBDT-IL, for detecting botnet traffic in IoT environments. It improves the robustness of the framework by adapting to dynamic IoT data using incremental learning. Additionally, it incorporates an enhanced Fisher Score feature selection algorithm, which enables the model to achieve a high accuracy even with a smaller set of optimal features, thereby reducing the system resources required for model training. To evaluate the effectiveness of our approach, we conducted experiments on the BoT-IoT, N-BaIoT, MedBIoT, and MQTTSet datasets. We compared our method with similar feature selection algorithms and existing concept drift detection algorithms. The experimental results demonstrated that our method achieved an average accuracy of 99.81% using only 25 features, outperforming similar feature selection algorithms. Furthermore, our method achieved an average accuracy of 96.88% in the presence of different types of drifting data, which is 2.98% higher than the best available concept drift detection algorithms, while maintaining a low average false positive rate of 3.02%. [ABSTRACT FROM AUTHOR]

Influence estimation analyzes how changes to the training data can lead to different model predictions; this analysis can help us better understand these predictions, the models making those predictions, and the data sets they're trained on. However, most influence-estimation techniques are designed for deep learning models with continuous parameters. Gradient-boosted decision trees (GBDTs) are a powerful and widely-used class of models; however, these models are black boxes with opaque decision-making processes. In the pursuit of better understanding GBDT predictions and generally improving these models, we adapt recent and popular influence-estimation methods designed for deep learning models to GBDTs. Specifically, we adapt representer-point methods and TracIn, denoting our new methods TREX and BoostIn, respectively; source code is available at https://github.com/jjbrophy47/tree_influence. We compare these methods to LeafInfluence and other baselines using 5 different evaluation measures on 22 real-world data sets with 4 popular GBDT implementations. These experiments give us a comprehensive overview of how different approaches to influence estimation work in GBDT models. We find BoostIn is an efficient influence-estimation method for GBDTs that performs equally well or better than existing work while being four orders of magnitude faster. Our evaluation also suggests the gold-standard approach of leave-one-out (LOO) retraining consistently identifies the single-most influential training example but performs poorly at finding the most influential set of training examples for a given target prediction. [ABSTRACT FROM AUTHOR]

In this research, we consider decision trees that incorporate standard queries with one feature per query as well as hypotheses consisting of all features' values. These decision trees are used to represent knowledge and are comparable to those investigated in exact learning, in which membership queries and equivalence queries are used. As an application, we look into the issue of creating decision trees for two cases: the sorting of a sequence that contains equal elements and multiple-value decision tables which are modified from UCI Machine Learning Repository. We contrast the efficiency of several forms of optimal (considering the parameter depth) decision trees with hypotheses for the aforementioned applications. We also investigate the efficiency of decision trees built by dynamic programming and by an entropy-based greedy method. We discovered that the greedy algorithm produces very similar results compared to the results of dynamic programming algorithms. Therefore, since the dynamic programming algorithms take a long time, we may readily apply the greedy algorithms. [ABSTRACT FROM AUTHOR]

Systems of decision rules and decision trees are widely used as a means for knowledge representation, as classifiers, and as algorithms. They are among the most interpretable models for classifying and representing knowledge. The study of relationships between these two models is an important task of computer science. It is easy to transform a decision tree into a decision rule system. The inverse transformation is a more difficult task. In this paper, we study unimprovable upper and lower bounds on the minimum depth of decision trees derived from decision rule systems with discrete attributes depending on the various parameters of these systems. To illustrate the process of transformation of decision rule systems into decision trees, we generalize well known result for Boolean functions to the case of functions of k-valued logic. [ABSTRACT FROM AUTHOR]

We describe a new simulated annealing algorithm to compute near-optimal oblique splits in the context of decision tree induction. The algorithm can be interpreted as a walk on the cells of a hyperplane arrangement defined by the observations in the training set. The cells of this hyperplane arrangement correspond to subsets of oblique splits that divide the feature space in the same manner and the vertices of this arrangement reveal multiple neighboring solutions. We use a pivoting strategy to iterate over the vertices and to explore this neighborhood. Embedding this neighborhood search in a simulated annealing framework allows to escape local minima and increases the probability of finding global optimal solutions. To overcome the problems related to degeneracy, we rely on a lexicographic pivoting scheme. Our experimental results indicate that our approach is well-suited for inducing small and accurate decision trees and capable of outperforming existing univariate and oblique decision tree induction algorithms. Furthermore, oblique decision trees obtained with this method are competitive with other popular prediction models. [ABSTRACT FROM AUTHOR]

The demand for explainable and transparent models increases with the continued success of reinforcement learning. In this article, we explore the potential of generating shallow decision trees (DTs) as simple and transparent surrogate models for opaque deep reinforcement learning (DRL) agents. We investigate three algorithms for generating training data for axis-parallel and oblique DTs with the help of DRL agents ("oracles") and evaluate these methods on classic control problems from OpenAI Gym. The results show that one of our newly developed algorithms, the iterative training, outperforms traditional sampling algorithms, resulting in well-performing DTs that often even surpass the oracle from which they were trained. Even higher dimensional problems can be solved with surprisingly shallow DTs. We discuss the advantages and disadvantages of different sampling methods and insights into the decision-making process made possible by the transparent nature of DTs. Our work contributes to the development of not only powerful but also explainable RL agents and highlights the potential of DTs as a simple and effective alternative to complex DRL models. [ABSTRACT FROM AUTHOR]

Recently, decision trees (DT) have been used as an explainable representation of controllers (a.k.a. strategies, policies, schedulers). Although they are often very efficient and produce small and understandable controllers for discrete systems, complex continuous dynamics still pose a challenge. In particular, when the relationships between variables take more complex forms, such as polynomials, they cannot be obtained using the available DT learning procedures. In contrast, support vector machines provide a more powerful representation, capable of discovering many such relationships, but not in an explainable form. Therefore, we suggest to combine the two frameworks to obtain an understandable representation over richer, domain-relevant algebraic predicates. We demonstrate and evaluate the proposed method experimentally on established benchmarks. [ABSTRACT FROM AUTHOR]

The literature provides multiple measures of diversity along a single demographic dimension, but when it comes to studying the interaction of multiple diversity types (e.g. age, gender, and race), the field of useable measures diminishes. We present the use of decision trees as a machine learning technique to automatically identify the interactions across diversity types to predict different levels of a dependent variable. In order to demonstrate the power of decision trees, we use five types of surface-level diversity (age, gender, education level, religion, and region of origin) measured via the standardized Blau index as independent variables and knowledge sharing as the dependent variable. The results of our decision tree approach relative to linear regression show that decision trees serve as a powerful tool to identify key demographic faultlines without a priori specification of a model structure. [ABSTRACT FROM AUTHOR]

The issues of low accuracy, poor generality, high cost of transformer fault early warning, and the subjective nature of empirical judgments made by field maintenance personnel are difficult to solve with the traditional measurement methods used during the development of the transformer. To construct a transformer fault early warning analysis, this study recommends a data-fusion-based decision tree approach for merging electrical quantity signals with a non-electrical amount of vibration signals. By merging a decision tree inference with actual operation data, a clustering center, and an early warning model, this method creates a transformer fault early warning model with self-learning ability and adaptive capabilities. After reasonable verification, the method becomes more universal and interpretable, and it can successfully conduct an early warning of transformer faults. [ABSTRACT FROM AUTHOR]

The precise characterization of geological bodies in fracture-vuggy carbonates is challenging due to their high complexity and heterogeneous distribution. This study aims to present the hybrid of Visual Geometry Group 16 (VGG-16) pre-trained by Gradient-Boosting Decision Tree (GBDT) models as a novel approach for predicting and generating karst cavities with high accuracy on various scales based on uncertainty assessment from a small dataset. Seismic wave impedance images were used as input data. Their manual interpretation was used to build GBDT classifiers for Light Gradient-Boosting Machine (LightGBM) and Unbiased Boosting with Categorical Features (CatBoost) for predicting the karst cavities and unconformities. The results show that the LightGBM was the best GBDT classifier, which performed excellently in karst cavity interpretation, giving an F1-score between 0.87 and 0.94 and a micro-G-Mean ranging from 0.92 to 0.96. Furthermore, the LightGBM performed better in cave prediction than Linear Regression (LR) and Multilayer Perceptron (MLP). The prediction of karst cavities according to the LightGBM model was performed well according to the uncertainty quantification. Therefore, the hybrid VGG16 and GBDT algorithms can be implemented as an improved approach for efficiently identifying geological features within similar reservoirs worldwide. [ABSTRACT FROM AUTHOR]

The research concerns data collected in independent sets—more specifically, in local decision tables. A possible approach to managing these data is to build local classifiers based on each table individually. In the literature, many approaches toward combining the final prediction results of independent classifiers can be found, but insufficient efforts have been made on the study of tables' cooperation and coalitions' formation. The importance of such an approach was expected on two levels. First, the impact on the quality of classification—the ability to build combined classifiers for coalitions of tables should allow for the learning of more generalized concepts. In turn, this should have an impact on the quality of classification of new objects. Second, combining tables into coalitions will result in reduced computational complexity—a reduced number of classifiers will be built. The paper proposes a new method for creating coalitions of local tables and generating an aggregated classifier for each coalition. Coalitions are generated by determining certain characteristics of attribute values occurring in local tables and applying the Pawlak conflict analysis model. In the study, the classification and regression trees with Gini index are built based on the aggregated table for one coalition. The system bears a hierarchical structure, as in the next stage the decisions generated by the classifiers for coalitions are aggregated using majority voting. The classification quality of the proposed system was compared with an approach that does not use local data cooperation and coalition creation. The structure of the system is parallel and decision trees are built independently for local tables. In the paper, it was shown that the proposed approach provides a significant improvement in classification quality and execution time. The Wilcoxon test confirmed that differences in accuracy rate of the results obtained for the proposed method and results obtained without coalitions are significant, with a p level = 0.005. The average accuracy rate values obtained for the proposed approach and the approach without coalitions are, respectively: 0.847 and 0.812; so the difference is quite large. Moreover, the algorithm implementing the proposed approach performed up to 21-times faster than the algorithm implementing the approach without using coalitions. [ABSTRACT FROM AUTHOR]

In this article we propose a boosting algorithm for regression with functional explanatory variables and scalar responses. The algorithm uses decision trees constructed with multiple projections as the "base-learners", which we call "functional multi-index trees". We establish identifiability conditions for these trees and introduce two algorithms to compute them. We use numerical experiments to investigate the performance of our method and compare it with several linear and nonlinear regression estimators, including recently proposed nonparametric and semiparametric functional additive estimators. Simulation studies show that the proposed method is consistently among the top performers, whereas the performance of existing alternatives can vary substantially across different settings. In a real example, we apply our method to predict electricity demand using price curves and show that our estimator provides better predictions compared to its competitors, especially when one adjusts for seasonality. [ABSTRACT FROM AUTHOR]

As a critical part of desision tree (DT） growth, its nodes can be split to grow by either axis- or nonaxis-aligned way based on such splitting criteria as purity and misclassification error. However, these have nothing to do with the geometric structure of data, e. g. multicentric data or single-center data. In order to compensate for this, two splitting criteria are proposed by combining the between-class margin in the same inner node (BCM） and the between-node margin within the same class (BNM） respectively with the purity measure in weighting and the two-step method. Unlike traditional greedy growth of DT which only finds the current locally optimal splitting point, the proposed method first selects the top-k purity splitting nodes, then determines the optimal one by maximizing BCM and minimizing BNM. Since not only alleviating purity-based local optimality but also considering global structures of the data, our method greatly improves the division of descendant nodes and generalization of the formed trees, while enhancing the interpretability. In addition, two aforementioned splitting criteria can be combined to further boost the performance. The comparison results on 21 benchmark datasets show an improvement in predictive performance of new trees with reduction in complexity, while also are competitive with many other DTs using hybrid splitting criteria. [ABSTRACT FROM AUTHOR]

The determination of distance is fundamental in astrophysics. Gamma-ray sources are poorly characterized in this sense, as the limited angular resolution and poor photon-count statistics in gamma-ray astronomy makes it difficult to associate them to a multiwavelength object with known redshift. Taking the 1794 active galactic nuclei (AGNs) with known redshift from the Fermi -LAT latest AGN catalogue, 4LAC–DR3, we employ machine learning techniques to predict the distance of the rest of AGNs based on their spectral and spatial properties. The state-of-the-art catboost algorithm reaches an average 0.56 R2 score with 0.46 root-mean-squared error, predicting an average redshift value of z avg = 0.63, with a maximum z max = 1.97. We use the shap explainer package to gain insights into the variables influence on the outcome, and also study the extragalactic background light implications. In a second part, we use this regression model to predict the redshift of the unassociated sample of the latest LAT point-source catalogue, 4FGL–DR3, using the results of a previous paper to determine the possible AGNs within them. [ABSTRACT FROM AUTHOR]

Inertial confined fusion experiments at the National Ignition Facility have recently entered a new regime approaching ignition. Improved modeling and exploration of the experimental parameter space were essential to deepening our understanding of the mechanisms that degrade and amplify the neutron yield. The growing prevalence of machine learning in fusion studies opens a new avenue for investigation. In this paper, we have applied the Gradient-Boosted Decision Tree machine-learning architecture to further explore the parameter space and find correlations with the neutron yield, a key performance indicator. We find reasonable agreement between the measured and predicted yield, with a mean absolute percentage error on a randomly assigned test set of 35.5%. This model finds the characteristics of the laser pulse to be the most influential in prediction, as well as the hohlraum laser entrance hole diameter and an enhanced capsule fabrication technique. We used the trained model to scan over the design space of experiments from three different campaigns to evaluate the potential of this technique to provide design changes that could improve the resulting neutron yield. While these data-driven model cannot predict ignition without examples of ignited shots in the training set, it can be used to indicate that an unseen shot design will at least be in the upper range of previously observed neutron yields. [ABSTRACT FROM AUTHOR]

In this paper, we study arbitrary subword-closed languages over the alphabet { 0 , 1 } (binary subword-closed languages). For the set of words L (n) of the length n belonging to a binary subword-closed language L, we investigate the depth of the decision trees solving the recognition and the membership problems deterministically and nondeterministically. In the case of the recognition problem, for a given word from L (n) , we should recognize it using queries, each of which, for some i ∈ { 1 , ... , n } , returns the ith letter of the word. In the case of the membership problem, for a given word over the alphabet { 0 , 1 } of the length n, we should recognize if it belongs to the set L (n) using the same queries. With the growth of n, the minimum depth of the decision trees solving the problem of recognition deterministically is either bounded from above by a constant or grows as a logarithm, or linearly. For other types of trees and problems (decision trees solving the problem of recognition nondeterministically and decision trees solving the membership problem deterministically and nondeterministically), with the growth of n, the minimum depth of the decision trees is either bounded from above by a constant or grows linearly. We study the joint behavior of the minimum depths of the considered four types of decision trees and describe five complexity classes of binary subword-closed languages. [ABSTRACT FROM AUTHOR]

Machine learning classification algorithms, such as decision trees and random forests, are commonly used in many applications. Clients who want to classify their data send them to a server that performs their inference using a trained model. The client must trust the server and provide the data in plaintext. Moreover, if the classification is done at a third-party cloud service, the model owner also needs to trust the cloud service. In this paper, we propose a protocol for privately evaluating decision trees. The protocol uses a novel private comparison function based on fully homomorphic encryption over the torus (TFHE) scheme and a programmable bootstrapping technique. Our comparison function for 32-bit and 64-bit integers is 26% faster than the naive TFHE implementation. The protocol is designed to be non-interactive and is less complex than the existing interactive protocols. Our experiment results show that our technique scales linearly with the depth of the decision tree and efficiently evaluates large decision trees on real datasets. Compared with the state of the art, ours is the only non-interactive protocol to evaluate a decision tree with high precision on encrypted parameters. The final download bandwidth is also 50% lower than the state of the art. [ABSTRACT FROM AUTHOR]

In this article, we present a new machine learning model by imitation based on the linguistic description of complex phenomena. The idea consists of, first, capturing the behaviour of human players by creating a computational perception network based on the execution traces of the games and, second, representing it using fuzzy logic (linguistic variables and if-then rules). From this knowledge, a set of data (dataset) is automatically created to generate a learning model based on decision trees. This model will be used later to automatically control the movements of a bot. The result is an artificial agent that mimics the human player. We have implemented, tested and evaluated this technology from two different points of view: performance by using classical metrics (accuracy, ROC area and PRC area) and believability by using a Turing test for trained bots. The results obtained are interesting and promising, showing that this method can be a good alternative to design and implement the behaviour of intelligent agents in video game development. [ABSTRACT FROM AUTHOR]

The deepest all-sky survey available in the γ-ray band – the last release of the Fermi-LAT catalogue (4FGL-DR3) based on the data accumulated in 12 years contains more than 6600 sources. The largest population among the sources is blazar subclass – 3743, 60.1 per cent of which are classified as BL Lacertae objects (BL Lacs) or Flat Spectrum Radio Quasars (FSRQs), while the rest are listed as blazar candidates of uncertain type (BCU) as their firm optical classification is lacking. The goal of this study is to classify BCUs using different machine learning algorithms, which are trained on the spectral and temporal properties of already classified BL Lacs and FSRQs. Artificial Neural Networks, XGBoost , and LightGBM algorithms are employed to construct predictive models for BCU classification. Using 18 input parameters of 2219 BL Lacs and FSRQs, we train (80 per cent of the sample) and test (20 per cent) these algorithms and find that LightGBM model, state-of-the-art classification algorithm based on gradient boosting decision trees, provides the highest performance. Based on our best model, we classify 825 BCUs as BL Lac candidates and 405 as FSRQ candidates, however, 190 remain without a clear prediction, but the percentage of BCUs in 4FGL is reduced to 5.1 per cent. The γ-ray photon index, synchrotron peak frequency, and high-energy peak frequency of a large sample are used to investigate the relationship between FSRQs and BL Lacs (LBLs, IBLs, and HBLs). [ABSTRACT FROM AUTHOR]

The objective of this research is to present a methodology proposal for the diagnosis of failures in the industry based on decision trees and fuzzy logic. Decision trees have among their advantages simplicity and ease of interpretation. While fuzzy logic provides capabilities for managing the uncertainty of the knowledge provided by process experts. This new methodology makes the contribution of offering a method to avoid the need to have a large training database during the development process for the diagnosticator, which will imply the use of a fault symptom table. In addition, the PMML standard is used to transmit the information of the model obtained in the configuration application to the application that performs the online monitoring tasks. The effectiveness of the methodology is analyzed in a real case, specifically in the diagnosis of failures of a water treatment process for injection in the pharmaceutical industry. [ABSTRACT FROM AUTHOR]

Background/Objectives: The aim was to develop a decision tree and a new prognostic tool to predict cancer-specific survival in patients with urothelial bladder cancer treated with radical cystectomy. Methods: A total of 11,834 patients with bladder cancer treated with radical cystectomy between 2004 and 2019 from the SEER database were randomly split into the derivation (n = 7889) and validation cohorts (n = 3945). Survival curves were estimated using conditional decision tree analysis. We used Multiple Imputation by Chained Equations for the treatment of missing values and the pec package to compare the predictive performance. We extracted data from our model following CHARMS and assessed the risk of bias and applicability with PROBAST. Results: A total of 4824 (41%) patients died during the follow-up period due to bladder cancer. A decision tree was made and 12 groups were obtained. Patients with a higher AJCC stage and older age have a worse prognosis. The risk groups were summarized into high, intermediate and low risk. The integrated Brier scores between 0 and 191 months for the bootstrap estimates of the prediction error are the lowest for our conditional survival tree (0.189). The model showed a low risk of bias and low concern about applicability. The results must be externally validated. Conclusions: Decision tree analysis is a useful tool with significant discrimination. With this tool, we were able to stratify patients into 12 subgroups and 3 risk groups with a low risk of bias and low concern about applicability. [ABSTRACT FROM AUTHOR]

Introduction: Bronchopulmonary dysplasia (BPD) poses a substantial global health burden. Individualized treatment strategies based on early prediction of the development of BPD can mitigate preterm birth complications; however, previously suggested predictive models lack early postnatal applicability. We aimed to develop predictive models for BPD and mortality based on immediate postnatal clinical data. Methods: Clinical information on very preterm and very low birth weight infants born between 2008 and 2018 was extracted from a nationwide Japanese database. The gradient boosting decision trees (GBDT) algorithm was adopted to predict BPD and mortality, using predictors within the first 6 h postpartum. We assessed the temporal validity and evaluated model adequacy using Shapley additive explanations (SHAP) values. Results: We developed three predictive models using data from 39,488, 39,096, and 40,291 infants to predict "death or BPD," "death or severe BPD," and "death before discharge," respectively. These well-calibrated models achieved areas under the receiver operating characteristic curve of 0.828 (95% CI: 0.828–0.828), 0.873 (0.873–0.873), and 0.887 (0.887–0.888), respectively, outperforming the multivariable logistic regression models. SHAP value analysis identified predictors of BPD, including gestational age, size at birth, male sex, and persistent pulmonary hypertension. In SHAP value-based case clustering, the "death or BPD" prediction model stratified infants by gestational age and persistent pulmonary hypertension, whereas the other models for "death or severe BPD" and "death before discharge" commonly formed clusters of low mortality, extreme prematurity, low Apgar scores, and persistent pulmonary hypertension of the newborn. Conclusions: GBDT models for predicting BPD and mortality, designed for use within 6 h postpartum, demonstrated superior prognostic performance. SHAP value-based clustering, a data-driven approach, formed clusters of clinical relevance. These findings suggest the efficacy of a GBDT algorithm for the early postnatal prediction of BPD. [ABSTRACT FROM AUTHOR]

AbstractImpurity functions are crucial in decision trees. These functions help determine the impurity level of a node in a decision tree, guiding the splitting criteria. However, two primary ambiguities have surrounded impurity functions: (1) the question of their non negativity and (2) the debate over their concavity. In this paper, we address these uncertainties by delving into the characteristics of impurity functions. We establish that the non negativity of an impurity function is inconsequential. Through counter examples, we disprove the equivalence between an impurity function and a concave function. We identify an impurity function that is not concave and a concave function that is not an impurity function. Interestingly, we find an impurity function that results in a negative impurity reduction. Furthermore, we validate several significant properties of impurity functions. For example, we demonstrate that when an impurity function is concave, the impurity reduction remains nonnegative for multiway divisions. We also discuss the sufficient conditions for a concave function to be an impurity function. Our numerical results further indicate that a positive linear combination of the two most popular impurity functions, namely Gini Index and Entropy, may surpass the individual performance of each when applied to the well-known German credit dataset. [ABSTRACT FROM AUTHOR]

In recent years, transformative technological advances have reshaped countless sectors, including higher education, leaving a lasting and indelible mark. Artificial intelligence (AI) and the Internet of Things (IoT) coming together has proven to be an influential driving force for the development of creative solutions to the ever-evolving challenges faced by educational institutions. Traditional classroom teaching methods have encountered unprecedented hurdles with the widespread integration of the Internet into education. China's commitment to "governing the nation by law" has elevated the need for legal expertise. As society evolves, pedagogical approaches must adapt. Information and communication technology trends, combined with the rise of the IoT, are reshaping education and offering tools to bolster campus security, monitor vital assets, and enrich classroom accessibility. This paper proposes an AI-based hybrid model and an IoT-infused teaching approach personalized for legal education in colleges and universities, employing the decision tree algorithm. The study's focal point is the development of legal professionals possessing the robust capabilities and qualities society seeks, using teaching methods compatible with modern information technology. The foundation of this endeavor lies in the hybrid teaching mode, specifically Internet-based legal research. Within this platform, educators can disseminate instructional materials, videos, and exercises, with the option to enlist teaching assistants to aid in managing student learning. Comparing varied and traditional teaching classes, the passing rates were 96.75% and 95.36%, respectively. Data analysis using decision trees identifies factors influencing these rates. The study also evaluates students' accuracy in five in-class exercises; in the mixed class, correctness rates were 96.4%, 93.5%, 82.5%, 64.2%, and 62.8%. The proposed method is compared with other machine learning algorithms (SVM, KNN, LR, RF) and Decision Trees to evaluate the educational impact of the mixed teaching approach comprehensively. [ABSTRACT FROM AUTHOR]

In ordinal classification problems, the class value exhibits a natural order. Usually, these problems are solved as multiclass classification problems while discarding the ordering form of the class. Recently, several research studies have proposed novel methods for ordinal classification problems while aiming to predict a predefined objective value. These methods are based on modification of the splitting criteria of decision tree-based algorithms. These methods consider the ordinal nature of the data and the magnitude of the potential classification error from a predefined predicted objective. This research aims to consolidate these methods and generalize them for any decision tree-based method while constructing decision trees according to any predefined objective value calculated from the training dataset. Furthermore, to evaluate the proposed method, a comprehensive experimental study is performed based on known ordinal datasets. The proposed method is found to significantly outperform its counterpart nonordinal models. This observation confirms that ordering information benefits ordinal methods and improves their performance. Additionally, the ordinal decision tree-based methods achieved competitive performance compared to state-of-the-art ordinal techniques. The results are validated with various performance measures that are commonly used for ordinal and non-ordinal classification problems. [ABSTRACT FROM AUTHOR]

Introduction: In population health surveillance research, survey data are commonly analyzed using regression methods; however, these methods have limited ability to examine complex relationships. In contrast, decision tree models are ideally suited for segmenting populations and examining complex interactions among factors, and their use within health research is growing. This article provides a methodological overview of decision trees and their application to youth mental health survey data. Methods: The performance of two popular decision tree techniques, the classification and regression tree (CART) and conditional inference tree (CTREE) techniques, is compared to traditional linear and logistic regression models through an application to youth mental health outcomes in the COMPASS study. Data were collected from 74 501 students across 136 schools in Canada. Anxiety, depression and psychosocial well-being outcomes were measured along with 23 sociodemographic and health behaviour predictors. Model performance was assessed using measures of prediction accuracy, parsimony and relative variable importance. Results: Decision tree and regression models consistently identified the same sets of most important predictors for each outcome, indicating a general level of agreement between methods. Tree models had lower prediction accuracy but were more parsimonious and placed greater relative importance on key differentiating factors. Conclusion: Decision trees provide a means of identifying high-risk subgroups to whom prevention and intervention efforts can be targeted, making them a useful tool to address research questions that cannot be answered by traditional regression methods. [ABSTRACT FROM AUTHOR]

We present a novel online decision-making solution, where the optimal path of a given decision tree is dynamically found based on the contextual bandits analysis. At each round, the learner finds a path in the decision tree by making a sequence of decisions following the tree structure and receives an outcome when a terminal node is reached. At each decision node, the environment information is observed to hint on which child node to visit, resulting in a better outcome. The objective is to learn the context-specific optimal decision for each decision node to maximize the accumulated outcome. In this paper, we propose Dynamic Path Identifier (DPI), a learning algorithm where the contextual bandit is applied to every decision node, and the observed outcome is used as the reward of the previous decisions of the same round. The technical difficulty of DPI is the high exploration challenge caused by the width (i.e., the number of paths) of the tree as well as the large context space. We mathematically prove that DPI's regret per round approached zero as the number of the rounds approaches infinity. We also prove that the regret is not a function of the number of paths in the tree. Numerical evaluations are provided to complement the theoretical analysis. [ABSTRACT FROM AUTHOR]

Currently, abdominal aortic aneurysm (AAA) diagnosis mainly relies on the analysis of the image data, such as Doppler ultrasonic and computed tomography (CT). Once AAA has formed, it may rupture and lead to death at any time. Surgical or endovascular treatment was the only method, but it has a high complication rate and poses a huge economic burden to patients. The gradient boosted decision trees (GBDT) model proposed in this paper is used to predict the probability and risk factors that lead to AAA, and the prediction accuracy of the algorithm is able to reach as high as 96%. This study selected 15 related AAA features as training samples. After the training, age, triglycerides (TG), blood pressure (BP), low-density lipoprotein cholesterol (LDL-C), blood glucose (Glu), and body mass index (BMI) are found to have a direct impact on AAA. For individuals with a high AAA probability, the risk factors that contribute the most to the AAA probability can be determined with the GBDT model. This study presents the GBDT model that effectively predicts the probability and risk factors of early AAA, which enables an early intervention and control of these risk factors against incidence of AAA. [ABSTRACT FROM AUTHOR]

Pipeline transport is widely used in industrial production and daily life. To reduce the waste of resources and economic losses caused by pipeline leaks, leak detection, localization and estimation systems are implemented in liquid pipelines. To minimize leak interpretation errors, leak detection algorithms based on artificial intelligence (AI) and data analysis have been developed. This study proposes a scheme for the detection and localization of multiple sequential leaks, based on the combination of two techniques such as decision trees and support vector machines. The results show that the proposed models have high accuracy, precision, recall and F1 score of 99.9%, 99.7%, respectively, which are better than the traditional classification model. [ABSTRACT FROM AUTHOR]

Cannabis use continues to escalate among emerging adults and college attendance may be a risk factor for use. Severe cases of cannabis use can escalate to a cannabis use disorder, which is associated with worse psychosocial functioning. Predictors of cannabis use consequences and cannabis use disorder symptom severity have been identified; however, they typically employ a narrow set of predictors and rely on linear models. Machine learning is well suited for exploratory data analyses of high-dimensional data. This study applied decision tree learning to identify predictors of cannabis user status, negative cannabis-related consequences, and cannabis use disorder symptoms. Undergraduate college students (N = 7000) were recruited from nine universities in nine states across the U.S. Among the 7 trees, 24 splits created by 15 distinct predictors were identified. Consistent with prior research, one's beliefs about cannabis were strong predictors of user status. Negative reinforcement cannabis use motives were the most consistent predictors of cannabis use disorder symptoms, and past month cannabis use was the most consistent predictor of probable cannabis use disorder. Typical frequency of cannabis use was the only predictor of negative cannabis-related consequences. Our results demonstrate that decision trees are a useful methodological tool for identifying targets for future clinical research. [ABSTRACT FROM AUTHOR]

Decision tree (DT)-based machine learning (ML) algorithms are one of the preferred solutions for real-time internet traffic classification in terms of their easy implementation on hardware. However, the rapid increase in today's newly developed applications and the resulting diversity in internet traffic greatly increases the size of DTs. Therefore, the tree-based hardware classifiers cannot keep up with this growth in terms of resource usage and classification speed. To alleviate the problem, we propose to group application classes by certain rules and create an individual small DT per each group. In this article, a pipelined organization of multiple DT data structures, called pipelined decision trees, is proposed as a scalable solution to tree-based traffic classification. We also propose two distinct algorithms, namely confusion matrix-based class aggregation and leaf count-based class aggregation algorithms, to set group creation rules that allows traffic classification on pipelined smaller DTs in a hierarchical order. We further designed an hardware engine on field programmable gate arrays, which can search those pipelined trees within a single clock cycle by transforming them into bit vectors and implementing multiple range comparisons in parallel. Our architecture with 12 classes can run in 928.88 giga bit per second and achieve 96.04% accuracy. [ABSTRACT FROM AUTHOR]