Search

Copyright of Journal of Forest & Environment is the property of Journal of Forest & Environment Editorial and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Statistical approaches in linguistics seem to have gained in importance in recent times, especially in the field of Corpus Linguistics. In particular, the last ten years have seen an upsurge of linguists being dedicated to statistical methods and the improvement of statistical knowledge. This has repeatedly been described as 'the quantitative turn' in linguistics. In the present paper, we assess how real this quantitative turn actually is and whether statistics can be considered the 'new normal' in (corpus) linguistics. To this end, we have analyzed the contributions to six high-impact journals (Corpora, Corpus Linguistics and Linguistic Theory, ICAME Journal, English World-Wide, Journal of English Linguistics, and Language Variation and Change) for a period of eleven years (January 2011 until December 2021). Our results suggest that, indeed, statistical methods seem to be on the rise in linguistic studies. However, their frequency strongly varies between the journals, and, in general, we have identified some room for improvement in the use of advanced statistical methods, in particular the discussion of true prediction. [ABSTRACT FROM AUTHOR]

An ensemble n-sub-epidemic modeling framework that integrates sub-epidemics to capture complex temporal dynamics has demonstrated powerful forecasting capability in previous works. This modeling framework can characterize complex epidemic patterns, including plateaus, epidemic resurgences, and epidemic waves characterized by multiple peaks of different sizes. In this tutorial paper, we introduce and illustrate SubEpiPredict, a user-friendly MATLAB toolbox for fitting and forecasting time series data using an ensemble n-sub-epidemic modeling framework. The toolbox can be used for model fitting, forecasting, and evaluation of model performance of the calibration and forecasting periods using metrics such as the weighted interval score (WIS). We also provide a detailed description of these methods including the concept of the n-sub-epidemic model, constructing ensemble forecasts from the top-ranking models, etc. For the illustration of the toolbox, we utilize publicly available daily COVID-19 death data at the national level for the United States. The MATLAB toolbox introduced in this paper can be very useful for a wider group of audiences, including policymakers, and can be easily utilized by those without extensive coding and modeling backgrounds. [ABSTRACT FROM AUTHOR]

Machine learning is increasingly and ubiquitously being used in the medical domain. Evaluation metrics like accuracy, precision, and recall may indicate the performance of the models but not necessarily the reliability of their outcomes. This paper assesses the effectiveness of a number of machine learning algorithms applied to an important dataset in the medical domain, specifically, mental health, by employing explainability methodologies. Using multiple machine learning algorithms and model explainability techniques, this work provides insights into the models' workings to help determine the reliability of the machine learning algorithm predictions. The results are not intuitive. It was found that the models were focusing significantly on less relevant features and, at times, unsound ranking of the features to make the predictions. This paper therefore argues that it is important for research in applied machine learning to provide insights into the explainability of models in addition to other performance metrics like accuracy. This is particularly important for applications in critical domains such as healthcare. [ABSTRACT FROM AUTHOR]

In the pursuit of ecological validity, current business process simulation methods are calibrated to data from existing processes. This is important for realistic what-if analysis in the context of these processes. However, this is not always the "right tool for the job." To test hypotheses in the area of predictive process monitoring, it can be more helpful to simulate event-log data from a theoretical process, where all aspects can be manipulated. One example is when assessing the influence of process complexity or variability on the performance of a new prediction method. In this case, the ability to include control variables and systematically change process characteristics is a key to fully understanding their influence. Calibrating a simulation model from observed data alone can in these cases be limiting. This paper proposes a simulation framework, Synthetic Business Process Simulation (SynBPS), a Python library for the generation of event-log data from synthetic processes. Aspects such as process complexity, stability, trace distribution, duration distribution, and case arrivals can be fully controlled by the user. The overall architecture is described in detail, and a demonstration of the framework is presented. [ABSTRACT FROM AUTHOR]

Modeling of the soil–structure interface has been a critical issue in geotechnical engineering. Numerous studies have simulated complex soil–structure interface behaviors. These models usually are assessed by direct comparisons between the simulations and experiments. However, little work has been done to compare the specific interface behaviors simulated by different interface models. This paper evaluated some frequently recognized interface behaviors for six different interface models. These models either were adopted from the existing literature or modified from the existing soil models, including the exponential model, hyperbolic model, hypoplastic model, MCC model, SANISAND model, and SIMSAND model. Global comparisons and effects of the soil density, normal stiffness, and shearing rate were investigated to evaluate the interface models based on Fontainebleau sand–steel interface experiments and kaolin clay–steel interface experiments. The limitations and advantages of different models under different conditions were discussed. [ABSTRACT FROM AUTHOR]

Item response theory models and applications are affected by many sources of variability, including errors associated with item parameter estimation. Metric stability analysis (MSA) is one method to evaluate the effects of item parameter standard errors that quantifies how well a model determines the latent trait metric. This paper describes how to evaluate MSA in dichotomous and polytomous data and describes a Bayesian implementation of MSA that does not require a positive definite variance–covariance matrix among item parameters. MSA analyses are illustrated in the context of an oral-health-related quality of life measure administered before and after prosthodontic treatment. The R code to implement the methods described in this paper is provided. [ABSTRACT FROM AUTHOR]

Although data modeling methods for conceptual cost estimation are proven to be effective in academia, they are not adopted by construction practitioners as expected. To understand this fact and find solutions to the challenge of implementing modeling methods, a review of the modeling process is needed. Fifty-one most relevant studies were filtered out from the Web of Science and ASCE. Referencing two established data mining frameworks, namely, CRISP-DM and KDD, this paper identifies the key tasks of implementing conceptual cost estimation models. The results of reviewing key tasks show that the literature did not provide sufficient solutions to data preparation and model evaluation. Critical judgments on the accomplishment and deficiencies of the current conceptual cost estimation studies, from the perspective of data modeling process for the first time, is the main contribution of this paper. Other contributions include the elaboration of the body of knowledge to guide practitioners to implement advanced cost estimation, as well as recommendations on future studies of improving data quality and integration with data management systems to achieve the data models' best capacity. [ABSTRACT FROM AUTHOR]

In the field of computer research, the increase of data in result of societal progress has been remarkable, and the management of this data and the analysis of linked businesses have grown in popularity. There are numerous practical uses for the capability to extract key characteristics from secondary property data and utilize these characteristics to forecast home prices. Using regression methods in machine learning to segment the data set, examine the major factors affecting it, and forecast home prices is the most popular method for examining pricing information. It is challenging to generate precise forecasts since many of the regression models currently being utilized in research are unable to efficiently collect data on the distinctive elements that correlate y with a high degree of house price movement. In today's forecasting studies, ensemble learning is a very prevalent and well-liked study methodology. The regression integration computation of large housing datasets can use a lot of computer resources as well as computation time, and ensemble learning uses more resources and calls for more machine support in integrating diverse models. The Average Model suggested in this paper uses the concept of fusion to produce integrated analysis findings from several models, combining the best benefits of separate models. The Average Model has a strong applicability in the field of regression prediction and significantly increases computational efficiency. The technique is also easier to replicate and very effective in regression investigations. Before using regression processing techniques, this work creates an average of different regression models using theAM(Average Model) algorithm in a novelway. By evaluating essential models with 90% accuracy, this technique significantly increases the accuracy of house price predictions. The experimental results show that the AM algorithm proposed in this paper has lower prediction error than other comparison algorithms, and the prediction accuracy is greatly improved compared with other algorithms, and has a good experimental effect in house price prediction. [ABSTRACT FROM AUTHOR]

This paper presents a pioneering methodology for refining product recommender systems, introducing a synergistic integration of unsupervised models—K-means clustering, content-based filtering (CBF), and hierarchical clustering—with the cutting-edge GPT-4 large language model (LLM). Its innovation lies in utilizing GPT-4 for model evaluation, harnessing its advanced natural language understanding capabilities to enhance the precision and relevance of product recommendations. A flask-based API simplifies its implementation for e-commerce owners, allowing for the seamless training and evaluation of the models using CSV-formatted product data. The unique aspect of this approach lies in its ability to empower e-commerce with sophisticated unsupervised recommender system algorithms, while the GPT model significantly contributes to refining the semantic context of product features, resulting in a more personalized and effective product recommendation system. The experimental results underscore the superiority of this integrated framework, marking a significant advancement in the field of recommender systems and providing businesses with an efficient and scalable solution to optimize their product recommendations. [ABSTRACT FROM AUTHOR]

With the warming and humidification process in the Northwest Arid Zone over the past 30 years, the runoff of a vast majority of rivers has been affected to different degrees. In this paper, the runoff from the Weigan River, a typical inland river in the arid zone of Northwest China, is taken as an example, and seven types of CMIP6 data are selected with the help of a SWAT model to predict the runoff volume of the Weigan River in the next 30 years under climate change. The results show that (1) the SWAT model can simulate the runoff from the Weigan River well and has good applicability in this study area. (2) With an increase in radiative forcing, the temperature, precipitation and runoff in the study area show an increasing trend. (3) Under the four radiative forcing scenarios in 2030 and 2050, the runoff from the Weigan River out of the mountain is predicted to be maintained at 25.68 to 30.89 × 108 m3, which is an increase of 1.35% to 21.91% compared with the current runoff, of which the contribution to the increase in future runoff caused by the changes in temperature and precipitation is 68.71% and 27.24%, respectively. It is important to explore the impact of climate change on the runoff from the Weigan River to understand the impact of climate change on the Northwest Arid Region scientifically and rationally, and to provide a scientific basis for evaluating the risk of climate change and formulating policies to deal with it. [ABSTRACT FROM AUTHOR]

This paper addresses a problem of optimal error quantification in the framework of robust control theory in the 1 setup. The upper bounds of biased external disturbance and the gains of coprime factor perturbations in a discrete-time linear time invariant SISO plant are assumed to be unknown. The computation of optimal data-consistent upper bounds under a known bias of external disturbance has been simplified to linear programming. This allows for the computation of optimal estimates in real-time and their application to achieve optimal robust steady-state tracking even when facing an unknown bias in the external disturbance. The presented results have been illustrated through computer simulations. [ABSTRACT FROM AUTHOR]

In this paper, a set of best practice data sharing guidelines for wind turbine fault detection model evaluation is developed, which can help practitioners overcome the main challenges of digitalisation. Digitalisation is one of the key drivers for reducing costs and risks over the whole wind energy project life cycle. One of the largest challenges in successfully implementing digitalisation is the lack of data sharing and collaboration between organisations in the sector. In order to overcome this challenge, a new collaboration framework called WeDoWind was developed in recent work. The main innovation of this framework is the way it creates tangible incentives to motivate and empower different types of people from all over the world to share data and knowledge in practice. In this present paper, the challenges related to comparing and evaluating different SCADA-data-based wind turbine fault detection models are investigated by carrying out a new case study, the "WinJi Gearbox Fault Detection Challenge", based on the WeDoWind framework. A total of six new solutions were submitted to the challenge, and a comparison and evaluation of the results show that, in general, some of the approaches (Particle Swarm Optimisation algorithm for constructing health indicators, performance monitoring using Deep Neural Networks, Combined Ward Hierarchical Clustering and Novelty Detection with Local Outlier Factor and Time-to-failure prediction using Random Forest Regression) appear to exhibit high potential to reach the goals of the Challenge. However, there are a number of concrete things that would have to have been done by the Challenge providers and the Challenge moderators in order to ensure success. This includes enabling access to more details of the different failure types, access to multiple data sets from more wind turbines experiencing gearbox failure, provision of a model or rule relating fault detection times or a remaining useful lifetime to the estimated costs for repairs, replacements and inspections, provision of a clear strategy for training and test periods in advance, as well as provision of a pre-defined template or requirements for the results. These learning outcomes are used directly to define a set of best practice data sharing guidelines for wind turbine fault detection model evaluation. The guidelines can be used by researchers in the sector in order to improve model evaluation and data sharing in the future. [ABSTRACT FROM AUTHOR]

Copyright of Advances in Atmospheric Sciences is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

For this paper, a new global atmospheric model (Global-to-Regional Integrated forecast SysTem; GRIST) with improved sub-grid scale orographic parameterization was verified and assessed, with an emphasis on the precipitation caused by typhoons. Four typical typhoon cases were selected for the verification of the model. The results indicate that, compared to the control experiments, the sensitivity experiments consistently simulated the trends in the three-hour cumulative precipitation changes and the high-value regions of total precipitation better. However, the improved experiments only had an ameliorating effect on the cumulative precipitation modelling biases for Typhoon LEKIMA and Typhoon HAGUPIT, not all of them. Precipitation bias is smaller on flat land than that on mountainous land, but the precipitation bias on windward/leeward slopes depends on the typhoon case. Precipitation modelling accuracy varies considerably between flat and mountainous terrain but very little between windward and leeward slopes. The precipitation simulation is poor for all terrains, with large precipitation thresholds in three typhoon cases, but for Typhoon HOTA, after improving the terrain, the model has the ability to forecast the heavy rainfall scenarios of the mountainous terrain. [ABSTRACT FROM AUTHOR]

Using knowledge graph embedding models (KGEMs) is a popular approach for predicting links in knowledge graphs (KGs). Traditionally, the performance of KGEMs for link prediction is assessed using rank-based metrics, which evaluate their ability to give high scores to ground-truth entities. However, the literature claims that the KGEM evaluation procedure would benefit from adding supplementary dimensions to assess. That is why, in this paper, we extend our previously introduced metric Sem@K that measures the capability of models to predict valid entities w.r.t. domain and range constraints. In particular, we consider a broad range of KGs and take their respective characteristics into account to propose different versions of Sem@K. We also perform an extensive study to qualify the abilities of KGEMs as measured by our metric. Our experiments show that Sem@K provides a new perspective on KGEM quality. Its joint analysis with rank-based metrics offers different conclusions on the predictive power of models. Regarding Sem@K, some KGEMs are inherently better than others, but this semantic superiority is not indicative of their performance w.r.t. rank-based metrics. In this work, we generalize conclusions about the relative performance of KGEMs w.r.t. rank-based and semantic-oriented metrics at the level of families of models. The joint analysis of the aforementioned metrics gives more insight into the peculiarities of each model. This work paves the way for a more comprehensive evaluation of KGEM adequacy for specific downstream tasks. [ABSTRACT FROM AUTHOR]

The ultimate strength of spherical shells under external pressure has been an attractive topic in the field of marine structures. However, recent studies have revealed that the current theoretical approaches are hardly compatible with the actual hulls, which indicates that the ultimate strength models still need to be reevaluated and unified. To address this challenge, a comparative study has been conducted in this paper. Various analytical approaches and codified rules are compared through screened experiment data that have realistic imperfections and different shape parameters in the range generally applied for marine structures. The model evaluation criteria have been established by implementing statistical model uncertainty factors in terms of bias and coefficient of variation. A comparison of the results has been made from three different aspects including shell shape, R/t and geometrical parameters to evaluate their performance. Based on the calculation outcome, analysis has been made to study the theory behind those models and determine their limitations and recommended application range. [ABSTRACT FROM AUTHOR]

Background: Clinical prediction models are often not evaluated properly in specific settings or updated, for instance, with information from new markers. These key steps are needed such that models are fit for purpose and remain relevant in the long-term. We aimed to present an overview of methodological guidance for the evaluation (i.e., validation and impact assessment) and updating of clinical prediction models. Methods: We systematically searched nine databases from January 2000 to January 2022 for articles in English with methodological recommendations for the post-derivation stages of interest. Qualitative analysis was used to summarize the 70 selected guidance papers. Results: Key aspects for validation are the assessment of statistical performance using measures for discrimination (e.g., C-statistic) and calibration (e.g., calibration-in-the-large and calibration slope). For assessing impact or usefulness in clinical decision-making, recent papers advise using decision-analytic measures (e.g., the Net Benefit) over simplistic classification measures that ignore clinical consequences (e.g., accuracy, overall Net Reclassification Index). Commonly recommended methods for model updating are recalibration (i.e., adjustment of intercept or baseline hazard and/or slope), revision (i.e., re-estimation of individual predictor effects), and extension (i.e., addition of new markers). Additional methodological guidance is needed for newer types of updating (e.g., meta-model and dynamic updating) and machine learning-based models. Conclusion: Substantial guidance was found for model evaluation and more conventional updating of regression-based models. An important development in model evaluation is the introduction of a decision-analytic framework for assessing clinical usefulness. Consensus is emerging on methods for model updating. [ABSTRACT FROM AUTHOR]

Monthly and seasonal precipitation forecasts can potentially assist disaster risk reduction and water resource management. The aim of this study is to assess the skill of an ensemble framework for monthly and seasonal precipitation forecasts over Iran by focusing on system design and model performance evaluation. The ensemble framework presented in this paper is based on a one‐way double‐nested model that uses Weather Research and Forecasting (WRF) modelling system to downscale the second version of the NCEP Climate Forecast System (CFSv2). The performance is evaluated for October–April period at 1‐, 2‐ and 3‐month lead time. Multiple initial conditions, model parameters and physics are used to construct ensemble members. Using quantile mapping (QM) method, the outputs of the model are bias corrected. This methodology is applied for two periods: (i) climatology from 2000 to 2019 to evaluate the model's ability to precipitation forecast on a monthly and seasonal time scale; (ii) the forecast for 2020 to evaluate the model's performance operationally. The model evaluation is performed using the continuous (e.g., RMSE, r, MBE, NSE) and categorical (e.g., POD, FAR, PC, Heidke skill score) assessment metrics. We conclude that model outputs were improved by the QM bias correction method. According to results, the proposed ensemble framework can accurately predict amount of monthly and seasonal precipitation in Iran with an accuracy of 58 to 45% for lead‐1 to 3. For all three lead times, the averaged NSE, CC, MBE, and RMSE were 0.4, 0.56, −15.5, and 41.6, indicating that the framework has reasonable performance. Our results suggest that precipitation forecast accuracy varies with lead time, so the accuracy for lead‐1 is higher than lead‐2 and lead‐3. Additionally, the model's accuracy differs in various regions of the country and decreases in the spring. Using the approach for an operational case, it was found that the spatial features of precipitation predicted by the framework were close to those observed. [ABSTRACT FROM AUTHOR]

Background Problem Method Results Conclusion Software defect prediction (SDP) is a topic actively researched in the software engineering community. Within‐project defect prediction (WPDP) involves using labeled modules from previous versions of the same project to train classifiers. Over time, many defect prediction models have been evaluated under the WPDP scenario.Data duplication poses a significant challenge in current WPDP evaluation procedures. Unchanged modules, characterized by identical executable source code, are frequently present in both target and source versions during experimentation. However, it is still unclear how and to what extent the presence of unchanged modules affects the performance assessment of WPDP models and the comparison of multiple WPDP models.In this paper, we provide a method to detect and remove unchanged modules from defect datasets and unveil the impact of data duplication in WPDP on model evaluation.The experiments conducted on 481 target versions from 62 projects provide evidence that data duplication significantly affects the reported performance values of individual learners in WPDP. However, when ranking multiple WPDP models based on prediction performance, the impact of removing unchanged instances is not substantial. Nevertheless, it is important to note that removing unchanged instances does have a slight influence on the selection of models with better generalization.We recommend that future WPDP studies take into consideration the removal of unchanged modules from target versions when evaluating the performance of their models. This practice will enhance the reliability and validity of the results obtained in WPDP research, leading to improved understanding and advancements in defect prediction models. [ABSTRACT FROM AUTHOR]

Car-following models describe how a driver follows the leading vehicle in the same lane. They serve as the cornerstone of microscopic traffic-flow simulations and play an essential role in analyzing human factors in traffic casualty, congestion, efficiency, and emissions. An extensive and continuously growing number of car-following models in the literature raises the requirement to evaluate and compare different models objectively. Generally, a car-following model is evaluated after model parameter calibration: the optimal residual between the calibrated model output and the measured counterpart is used as a metric to assess a car-following model’s performance. However, model parameter calibration, usually formed as a numerical optimization problem, suffers from several issues, such as local optimality and heavy computational burden. More importantly, different formulations of the cost function can lead to distinct calibration outcomes and contradictory conclusions of the model evaluation results. This paper proposes instead a purely algebraic framework for evaluating a class of car-following models whose parameters can be linearly identified. Car-following models with nonlinear relationships among parameters, e.g., the behavioral car-following models, are out of the scope of analysis in this paper. Algebraic manipulations performed on a model finally produce a system error index, which is a uniform metric for evaluating and comparing different car-following models. During the whole process, no cost function needs to be designed a priori, and no computationally expensive numerical optimization is involved. Three car-following models are evaluated and compared under the proposed algebraic framework. [ABSTRACT FROM AUTHOR]

The findings of hydrological modeling studies depend on which model was used. Although hydrological model selection is a crucial step, experience suggests that hydrologists tend to stick to the model they have experience with, and rarely switch to competing models, although these models might be more adequate given the study objectives. To gain quantitative insights into model selection, we explored the use of seven rainfall‐runoff models based on the abstract of 1,529 peer‐reviewed papers published between 1991 and 2018. The models selected were the Hydrologiska Byråns Vattenbalansavdelning model (HBV), the Variable Infiltration Capacity model (VIC), the mesoscale Hydrological model (mHM), the TOPography‐based hydrologic model (TOPMODEL), the Precipitation Runoff Modelling System (PRMS), the Génie Rural model à 4 paramètres Journaliers (GR4J), and the Sacramento soil moisture accounting model. We provide quantitative evidence of regional preferences in model use across the world and demonstrate that specific models are consistently preferred by certain institutes. Model attachment is particularly strong. In ~74% of the studies, the model selected can be predicted solely based on the affiliation of the first author. The influence of adequacy on the model selection process is less clear. Our data reveal that each model is used across a wide range of purposes, landscapes, and temporal and spatial scales (i.e., as a model of everything and everywhere). Model intercomparisons can provide guidance for model selection and improve model adequacy, but they are still rare (because each model must usually be setup individually) and the insights they provide are currently limited (because they are rarely controlled experiments). We suggest that moving from fixed‐structure models to modular modeling frameworks (master templates for model generation) can overcome these issues, enable a more collaborative and responsive model development environment, and result in improved model adequacy. Key Points: Text mining of 1,500+ peer‐reviewed articles enabled us to relate hydrological models to institutes, regions, and research topicsWe provide evidence of regional preferences in model use across the world and underline the decisive influence of legacy on model selectionWe reflect on current tendencies and future model development strategies and advocate for a broader use of modular modeling frameworks [ABSTRACT FROM AUTHOR]

Though tropical cyclone (TC) models have been routinely evaluated against track and intensity observations, little work has been performed to validate modeled TC wind fields over land. In this paper, we present a simple framework for evaluating simulated low‐level inland winds with in‐situ observations and existing TC structure theory. The Automated Surface Observing Systems, Florida Coastal Monitoring Program, and best track data are used to generate a theory‐predicted wind profile that reasonably represents the observed radial distribution of TC wind speeds. We quantitatively and qualitatively evaluated the modeled inland TC wind fields, and described the model performance with a set of simple indicators. The framework was used to examine the performance of a high‐resolution two‐way nested Geophysical Fluid Dynamics Laboratory model on recent U.S. landfalling TCs. Results demonstrate the capacity of using this framework to assess the modeled TC low‐level wind field in the absence of dense inland observations. Plain Language Summary: Some of the biggest human impacts of tropical cyclone (TC) winds come after the TC makes landfall. A skillful prediction of the radial distribution of winds is essential for forecasting TC‐induced inland hazards. However, the forecast skill of numerical hurricane models on inland TC wind fields has rarely been evaluated since it is challenging to collect wind observations during landfall, and the network of regular weather observations is too spread out to capture the strongest winds associated with a TC. This inhibits the improvement of forecast models and limits our understanding of the TC's inland evolution. Our work combines available inland in‐situ wind observations over the southeastern U.S. with existing TC structure theory, and presents a new "optimal" estimate of the post‐landfall winds. Our framework is found to be useful for evaluating the post‐landfall TC winds in hurricane forecast models. In addition, the new evaluation technique can intuitively demonstrate how well the model simulates TC intensity and structure. Key Points: We introduce a new framework for evaluating modeled inland tropical cyclone (TC) wind fields with observation‐based, theory‐predicted wind profilesThe theory‐predicted wind profile well represents the observed radial distribution of inland TC wind speedsWe propose simple indicators to summarize the model performance on inland wind field predictions [ABSTRACT FROM AUTHOR]

Model evaluation is critical in deep learning. However, the traditional model evaluation approach is susceptible to issues of untrustworthiness, including insecure data and model sharing, insecure model training, incorrect model evaluation, centralized model evaluation, and evaluation results that can be tampered easily. To minimize these untrustworthiness issues, this paper proposes a blockchain-based model evaluation framework. The framework consists of an access control layer, a storage layer, a model training layer, and a model evaluation layer. The access control layer facilitates secure resource sharing. To achieve fine-grained and flexible access control, an attribute-based access control model combining the idea of a role-based access control model is adopted. A smart contract is designed to manage the access control policies stored in the blockchain ledger. The storage layer ensures efficient and secure storage of resources. Resource files are stored in the IPFS, with the encrypted results of their index addresses recorded in the blockchain ledger. Another smart contract is designed to achieve decentralized and efficient management of resource records. The model training layer performs training on users' servers, and, to ensure security, the training data must have records in the blockchain. The model evaluation layer utilizes the recorded data to evaluate the recorded models. A method in the smart contract of the storage layer is designed to enable evaluation, with scores automatically uploaded as a resource attribute. The proposed framework is applied to deep learning-based motion object segmentation, demonstrating its key functionalities. Furthermore, we validated the storage strategy adopted by the framework, and the trustworthiness of the framework is also analyzed. [ABSTRACT FROM AUTHOR]

Since the beginning of the twenty-first century, people's living standards have been continuously improved, followed by changes in diet structure and living habits. These changes have affected the body's endocrine system, causing lesions in the pancreatic tissue. Among these pancreatic tissue diseases, pancreatic cancer is the most harmful to human health because of its inability to find and high mortality within 1 year. At present, in the diagnosis of pancreatic cancer, medical imaging and pathological puncture are the main methods of diagnosis. These methods have a high diagnostic rate for patients with advanced pancreatic cancer, but it is difficult to apply to the diagnosis of early pancreatic cancer. In response to these problems, this paper proposes a pancreatic cancer diagnosis model based on the fusion of distribution estimation algorithm and genetic algorithm. By collecting pathological data of patients with pancreatic cancer from a hospital oncology, pathological data include clinical manifestations of pancreatic cancer patients, serum tumor markers, etc., after data preprocessing, input models, and then use different machine learning classification algorithms to make pancreatic cancer for diagnosis. By evaluating the diagnosis results of each classification algorithm, an optimal classification algorithm is obtained and applied to the diagnosis model of pancreatic cancer. The results show that compared with other classification algorithms, the model using classification algorithm has the highest accuracy, recall rate and harmonic mean, and the diagnostic performance is the best. The results show that the diagnostic model constructed in this paper has a very high application value in the early auxiliary pre-diagnosis of pancreatic cancer. [ABSTRACT FROM AUTHOR]

Species distribution modeling (SDM) is widely used in ecology and conservation. Currently, the most available data for SDM are species presence‐only records (available through digital databases). There have been many studies comparing the performance of alternative algorithms for modeling presence‐only data. Among these, a 2006 paper from Elith and colleagues has been particularly influential in the field, partly because they used several novel methods (at the time) on a global data set that included independent presence–absence records for model evaluation. Since its publication, some of the algorithms have been further developed and new ones have emerged. In this paper, we explore patterns in predictive performance across methods, by reanalyzing the same data set (225 species from six different regions) using updated modeling knowledge and practices. We apply well‐established methods such as generalized additive models and MaxEnt, alongside others that have received attention more recently, including regularized regressions, point‐process weighted regressions, random forests, XGBoost, support vector machines, and the ensemble modeling framework biomod. All the methods we use include background samples (a sample of environments in the landscape) for model fitting. We explore impacts of using weights on the presence and background points in model fitting. We introduce new ways of evaluating models fitted to these data, using the area under the precision‐recall gain curve, and focusing on the rank of results. We find that the way models are fitted matters. The top method was an ensemble of tuned individual models. In contrast, ensembles built using the biomod framework with default parameters performed no better than single moderate performing models. Similarly, the second top performing method was a random forest parameterized to deal with many background samples (contrasted to relatively few presence records), which substantially outperformed other random forest implementations. We find that, in general, nonparametric techniques with the capability of controlling for model complexity outperformed traditional regression methods, with MaxEnt and boosted regression trees still among the top performing models. All the data and code with working examples are provided to make this study fully reproducible. [ABSTRACT FROM AUTHOR]

Due to the high cost of repairing defective programs, many researches focus on automatic program repair (APR). In recent years, the new trend of APR is to apply neural networks to mine the relations between defective programs and corresponding patches automatically, which is known as neural program repair (NPR). The community, however, ignores some important properties that could impact the applicability of NPR systems, such as robustness. For semantic‐identical buggy programs, NPR systems may produce totally different patches. In this paper, we propose an evaluation tool named RobustNPR, the first NPR robustness evaluation tool. RobustNPR employs several mutators to generate semantic‐identical mutants of defective programs. For an original defective program and its mutant, it checks two aspects of NPR: (a) Can NPR fix mutants when it can fix the original defective program? and (b) can NPR generate semantic‐identical patches for the original program and the mutant? Then, we evaluate four SOTA NPR models and analyze the results. From the results, we find that even for the best‐performing model, 20.16% of the repair success is unreliable, which indicates that the robustness of NPR is not perfect. In addition, we find that the robustness of NPR is correlated with model settings and other factors. [ABSTRACT FROM AUTHOR]

The Tucker–Lewis index (TLI; Tucker & Lewis, 1973), also known as the non-normed fit index (NNFI; Bentler & Bonett, 1980), is one of the numerous incremental fit indices widely used in linear mean and covariance structure modeling, particularly in exploratory factor analysis, tools popular in prevention research. It augments information provided by other indices such as the root-mean-square error of approximation (RMSEA). In this paper, we develop and examine an analogous index for categorical item level data modeled with item response theory (IRT). The proposed Tucker–Lewis index for IRT (TLIRT) is based on Maydeu-Olivares and Joe's (2005) M 2 family of limited-information overall model fit statistics. The limited-information fit statistics have significantly better Chi-square approximation and power than traditional full-information Pearson or likelihood ratio statistics under realistic situations. Building on the incremental fit assessment principle, the TLIRT compares the fit of model under consideration along a spectrum of worst to best possible model fit scenarios. We examine the performance of the new index using simulated and empirical data. Results from a simulation study suggest that the new index behaves as theoretically expected, and it can offer additional insights about model fit not available from other sources. In addition, a more stringent cutoff value is perhaps needed than Hu and Bentler's (1999) traditional cutoff criterion with continuous variables. In the empirical data analysis, we use a data set from a measurement development project in support of cigarette smoking cessation research to illustrate the usefulness of the TLIRT. We noticed that had we only utilized the RMSEA index, we could have arrived at qualitatively different conclusions about model fit, depending on the choice of test statistics, an issue to which the TLIRT is relatively more immune. [ABSTRACT FROM AUTHOR]

Fake information in social media frequently causes social issues. The amount of fake information is smaller than that of real information, this leads to class imbalance. Some improved classification methods and metrics to resolve the imbalance and evaluate model performance have been proposed, respectively. However, the existing metrics for classification methods have many limitations. This paper proposes the robust metric, L-measure, that can reasonably evaluate all models with binary class imbalance with different IRs. L-measure also require less computation than the Matthews correlation coefficient. Finally, this paper demonstrates the validity of the proposed metric under different IRs with examples from UCI and Kaggle. [ABSTRACT FROM AUTHOR]

Using a big data set of venture capital financing and related startup firms from Crunchbase, this paper develops a machinelearning model called CapitalVX (for "Capital Venture eXchange") to predict the outcomes for startups, i.e., whether they will exit successfully through an IPO or acquisition, fail, or remain private. Using a large feature set, the out-of-sample accuracy of predictions on startup outcomes and follow-on funding is 80e89%. This research suggests that VC/PE firms may be able to benefit from using machine learning to screen potential investments using publicly available information, diverting this time instead into mentoring and monitoring the investments they make. [ABSTRACT FROM AUTHOR]

The diagnosis and prognosis of cardiovascular disease are crucial medical tasks to ensure correct classification, which helps cardiologists provide proper treatment to the patient. Machine learning applications in the medical niche have increased as they can recognize patterns from data. Using machine learning to classify cardiovascular disease occurrence can help diagnosticians reduce misdiagnosis. This research develops a model that can correctly predict cardiovascular diseases to reduce the fatality caused by cardiovascular diseases. This paper proposes a method of k-modes clustering with Huang starting that can improve classification accuracy. Models such as random forest (RF), decision tree classifier (DT), multilayer perceptron (MP), and XGBoost (XGB) are used. GridSearchCV was used to hypertune the parameters of the applied model to optimize the result. The proposed model is applied to a real-world dataset of 70,000 instances from Kaggle. Models were trained on data that were split in 80:20 and achieved accuracy as follows: decision tree: 86.37% (with cross-validation) and 86.53% (without cross-validation), XGBoost: 86.87% (with cross-validation) and 87.02% (without cross-validation), random forest: 87.05% (with cross-validation) and 86.92% (without cross-validation), multilayer perceptron: 87.28% (with cross-validation) and 86.94% (without cross-validation). The proposed models have AUC (area under the curve) values: decision tree: 0.94, XGBoost: 0.95, random forest: 0.95, multilayer perceptron: 0.95. The conclusion drawn from this underlying research is that multilayer perceptron with cross-validation has outperformed all other algorithms in terms of accuracy. It achieved the highest accuracy of 87.28%. [ABSTRACT FROM AUTHOR]

Semi-empirical kernel-driven models have been widely used to characterize anisotropic reflectance due to their simple form and physically meaningful approximation. Recently, several kernel-driven models have been coupled with topographic effects to improve the fitting of bidirectional reflectance over rugged terrains. However, extensive evaluations of the various models' performances are required before their subsequent application in remote sensing. Three typical kernel-driven BRDF models over snow-free rugged terrains such as the RTLSR, TCKD, and the KDST-adjusted TCKD (KDST-TCKD) were investigated in this paper using simulated and observed BRFs. Against simulated data, the fitting error (NIR/Red RMSE) of the RTLSR gradually increases from 0.0358/0.0342 to 0.0471/0.0516 with mean slopes (α) increases from 9.13° to 33.40°. However, the TCKD and KDST-TCKD models perform an overall better fitting accuracy: the fitting errors of TCKD gradually decreased from 0.0366/0.0337 to 0.0252/0.0292, and the best fit from the KDST-TCDK model with NIR/Red RMSE decreased from 0.0192/0.0269 to 0.0169/0.0180. When compared to the sandbox data (α from 8.4° to 30.36°), the NIR/Red RMSE of the RTLSR model ranges from 0.0147/0.0085 to 0.0346/0.0165, for the TCKD model from 0.0144/0.0086 to 0.0298/0.0154, and for the KDST-TCKD model from 0.0137/0.0082 to 0.0234/0.0149. Using MODIS data, the TCKD and KDST-TCKD models show more significant improvements compared to the RTLSR model in rugged terrains. Their RMSE differences are within 0.003 over a relatively flat terrain (α < 10°). When α is large (20°–30° and >30°), the RMSE of the TCKD model has a decrease of around 0.01 compared to that of the RTLSR; for KDST-TCKD, it is approximately 0.02, and can even reach 0.0334 in the savannas. Therefore, the TCKD and KDST-TCKD models have an overall better performance than the RTLSR model in rugged terrains, especially in the case of large mean slopes. Among them, the KDST-TCKD model performs the best due to its consideration of topographic effects, geotropic growth, and component spectra. [ABSTRACT FROM AUTHOR]