Search

Western Monarch butterflies (Danaus plexippus) migrate from inland breeding ranges to coastal overwintering grounds in California. Given that migratory individuals may make multi-scale habitat selection decisions, we considered a multi-scale species distribution model (SDM) using range-wide climatic and local landscape-level predictors of migratory and overwintering habitat and community-science presence data. The range-wide model output was included as a predictor in the local-scale model, generating multi-scale habitat suitability. The top range-wide predictor was the minimum temperature in December, contributing 83.7% to the model, and was positively associated with presence. At the local scale, the strongest predictors of presence were the range-wide output and percent coverage of low and medium levels of development, contributing > 95%, with 61–63% from the range-wide output, with local-scale suitability coinciding with the California coastal zones. Development's positive association with overwintering monarch presence was counterintuitive. It is likely that our local-scale model is overfit to these development zones, but it is unclear whether this overfitting resulted from modeler choices, monarchs overwintering close to human development, biased detection near human development, or a combination of these factors. Therefore, alternative approaches to collecting local-scale attribute data are suggested while recognizing the primacy of climate in restricting overwinter sites. [ABSTRACT FROM AUTHOR]

We consider situations where a user feeds her attributes to a machine learning method that tries to predict her best option based on a random sample of other users. The predictor is incentive-compatible if the user has no incentive to misreport her covariates. Focusing on the popular Lasso estimation technique, we borrow tools from high-dimensional statistics to characterize sufficient conditions that ensure that Lasso is incentive compatible in the asymptotic case. We extend our results to a new nonlinear machine learning technique, Generalized Linear Model Structured Sparsity estimators. Our results show that incentive compatibility is achieved if the tuning parameter is kept above some threshold in the case of asymptotics. [ABSTRACT FROM AUTHOR]

Western Monarch butterflies (Danaus plexippus) migrate from inland breeding ranges to coastal overwintering grounds in California. Given that migratory individuals may make multi-scale habitat selection decisions, we considered a multi-scale species distribution model (SDM) using range-wide climatic and local landscape-level predictors of migratory and overwintering habitat and community-science presence data. The range-wide model output was included as a predictor in the local-scale model, generating multi-scale habitat suitability. The top range-wide predictor was the minimum temperature in December, contributing 83.7% to the model, and was positively associated with presence. At the local scale, the strongest predictors of presence were the range-wide output and percent coverage of low and medium levels of development, contributing > 95%, with 61–63% from the range-wide output, with local-scale suitability coinciding with the California coastal zones. Development’s positive association with overwintering monarch presence was counterintuitive. It is likely that our local-scale model is overfit to these development zones, but it is unclear whether this overfitting resulted from modeler choices, monarchs overwintering close to human development, biased detection near human development, or a combination of these factors. Therefore, alternative approaches to collecting local-scale attribute data are suggested while recognizing the primacy of climate in restricting overwinter sites.

Construct-based models have become a mainstay of management and information systems research. However, these models are likely overfit to the data samples upon which they are estimated, making them risky to use in explanatory, prescriptive, or predictive ways outside a given sample. Empirical researchers currently lack tools to analyze why and how their models may not generalize out of sample. We propose a composite overfit analysis (COA) framework that applies predictive tools to describe the sources and ramifications of overfit in terms of the focal concepts important to empirical researchers: cases, constructs, and causal paths. The COA framework begins by using a leave-one-out crossvalidation procedure to identify cases with unusually high predictive error given their in-sample fit—a difference we describe as predictive deviance. The framework then employs a novel deviance tree method to group deviant cases that have similar predictive deviance and for similar theoretical reasons. We then employ a leave-deviant-group-out method, which sequentially analyzes how each deviant group affects model parameters, thereby identifying potentially unstable paths in the model. We can then infer descriptive reasons for why and how overfit affects a given model and data sample using the grouping criteria of the deviance tree, construct scores of deviant groups, and resulting unstable paths. These insights allow researchers to identify unexpected behavior that could define boundary conditions of their theory or point to new theoretical phenomena. We demonstrate the practical utility of our analytical framework on a technology adoption model in a new context. This paper was accepted by Dongjun Wu, information systems. Funding: This work was partially supported by the Ministry of Science and Technology, Taiwan [Grants 109-2811-H-007-503 and 108-2410-H-007-091-MY3] and by the 2021 Arts and Social Sciences Benefaction Fund of Trinity College Dublin, Ireland. Supplemental Material: The data files and online appendix are available at https://doi.org/10.1287/mnsc.2023.4705. [ABSTRACT FROM AUTHOR]

Background: The framework of adaptive radiation therapy (ART) was crafted to address the underlying sources of intra‐patient variation that were observed throughout numerous patients' radiation sessions. ART seeks to minimize the consequential dosimetric uncertainty resulting from this daily variation, commonly through treatment planning re‐optimization. Re‐optimization typically consists of manual evaluation and modification of previously utilized optimization criteria. Ideally, frequent treatment plan adaptation through re‐optimization on each day's computed tomography (CT) scan may improve dosimetric accuracy and minimize dose delivered to organs at risk (OARs) as the planning target volume (PTV) changes throughout the course of treatment. Purpose: Re‐optimization in its current form is time‐consuming and inefficient. In response to this ART bottleneck, we propose a deep learning based adaptive dose prediction model that utilizes a head and neck (H&N) patient's initial planning data to fine‐tune a previously trained population model towards a patient‐specific model. Our fine‐tuned, patient‐specific (FT‐PS) model, which is trained using the intentional deep overfit learning (IDOL) method, may enable clinicians and treatment planners to rapidly evaluate relevant dosimetric changes daily and re‐optimize accordingly. Methods: An adaptive population (AP) model was trained using adaptive data from 33 patients. Separately, 10 patients were selected for training FT‐PS models. The previously trained AP model was utilized as the base model weights prior to re‐initializing model training for each FT‐PS model. Ten FT‐PS models were separately trained by fine‐tuning the previous model weights based on each respective patient's initial treatment plan. From these 10 patients, 26 ART treatment plans were withheld from training as the test dataset for retrospective evaluation of dose prediction performance between the AP and FT‐PS models. Each AP and FT‐PS dose prediction was compared against the ground truth dose distribution as originally generated during the patient's course of treatment. Mean absolute percent error (MAPE) evaluated the dose differences between a model's prediction and the ground truth. Results: MAPE was calculated within the 10% isodose volume region of interest for each of the AP and FT‐PS models dose predictions and averaged across all test adaptive sessions, yielding 5.759% and 3.747% respectively. MAPE differences were compared between AP and FT‐PS models across each test session in a test of statistical significance. The differences were statistically significant in a paired t‐test with two‐tailed p‐value equal to 3.851×10−9$3.851 \times {10}^{ - 9}$ and 95% confidence interval (CI) equal to [−2.483, −1.542]. Furthermore, MAPE was calculated using each individually segmented structure as an ROI. Nineteen of 24 structures demonstrated statistically significant differences between the AP and FT‐PS models. Conclusion: We utilized the IDOL method to fine‐tune a population‐based dose prediction model into an adaptive, patient‐specific model. The averaged MAPE across the test dataset was 5.759% for the population‐based model versus 3.747% for the fine‐tuned, patient‐specific model, and the difference in MAPE between models was found to be statistically significant. Our work demonstrates the feasibility of patient‐specific models in adaptive radiotherapy, and offers unique clinical benefit by utilizing initial planning data that contains the physician's treatment intent. [ABSTRACT FROM AUTHOR]

MRI has been extensively used to identify anatomical and functional differences in Autism Spectrum Disorder (ASD). Yet, many of these findings have proven difficult to replicate because studies rely on small cohorts and are built on many complex, undisclosed, analytic choices. We conducted an international challenge to predict ASD diagnosis from MRI data, where we provided preprocessed anatomical and functional MRI data from > 2,000 individuals. Evaluation of the predictions was rigorously blinded. 146 challengers submitted prediction algorithms, which were evaluated at the end of the challenge using unseen data and an additional acquisition site. On the best algorithms, we studied the importance of MRI modalities, brain regions, and sample size. We found evidence that MRI could predict ASD diagnosis: the 10 best algorithms reliably predicted diagnosis with AUC∼0.80 – far superior to what can be currently obtained using genotyping data in cohorts 20-times larger. We observed that functional MRI was more important for prediction than anatomical MRI, and that increasing sample size steadily increased prediction accuracy, providing an efficient strategy to improve biomarkers. We also observed that despite a strong incentive to generalise to unseen data, model development on a given dataset faces the risk of overfitting: performing well in cross-validation on the data at hand, but not generalising. Finally, we were able to predict ASD diagnosis on an external sample added after the end of the challenge (EU-AIMS), although with a lower prediction accuracy (AUC=0.72). This indicates that despite being based on a large multisite cohort, our challenge still produced biomarkers fragile in the face of dataset shifts.

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

When the in-sample Sharpe ratio is obtained by optimizing over a k-dimensional parameter space, it is a biased estimator for what can be expected on unseen data (out-of-sample). We derive (1) an unbiased estimator adjusting for both sources of bias: noise fit and estimation error. We then show (2) how to use the adjusted Sharpe ratio as model selection criterion analogously to the Akaike Information Criterion (AIC). Selecting a model with the highest adjusted Sharpe ratio selects the model with the highest estimated out-of-sample Sharpe ratio in the same way as selection by AIC does for the log-likelihood as a measure of fit. [ABSTRACT FROM AUTHOR]

MR brain image categorization has been an active research domain from the last decade. Several techniques have been devised in the past for MR image categorization, starting from classical to the deep learning methods like convolutional neural networks (CNNs). Classical machine learning methods need handcrafted features to perform classification. The CNNs, on the other hand, perform classification by extracting image features directly from raw images via tuning the parameters of the convolutional and pooling layer. The features extracted by CNN strongly depend on the size of the training dataset. If the training dataset is small, CNN tends to overfit after several epochs. So, deep CNNs (DCNNs) with transfer learning have evolved. The prime objective of the present work is to explore the capability of different pre-trained DCNN models with transfer learning for pathological brain image classification. Various pre-trained DCNNs, namely Alexnet, Resnet50, GoogLeNet, VGG-16, Resnet101, VGG-19, Inceptionv3, and InceptionResNetV2, were used in the present study. The last few layers of these models were replaced to accommodate new image categories for our application. These models were extensively evaluated on data from Harvard, clinical, and benchmark Figshare repository. The dataset was then partitioned in the ratio 60:40 for training and testing. The validation on the test set reveals that the pre-trained Alexnet with transfer learning exhibited the best performance in less time compared to other proposed models. The proposed method is more generic as it does not need any handcrafted features and can achieve an accuracy value of 100%, 94%, and 95.92% for three datasets. Other performance measures used in the study include sensitivity, specificity, precision, false positive rate, error, F-score, Mathew correlation coefficient, and area under the curve. The results are compared with both the traditional machine learning methods and those using CNN. [ABSTRACT FROM AUTHOR]

Multinomial Logistic Regression (MLR) has been advocated for developing clinical prediction models that distinguish between three or more unordered outcomes. We present a full-factorial simulation study to examine the predictive performance of MLR models in relation to the relative size of outcome categories, number of predictors and the number of events per variable. It is shown that MLR estimated by Maximum Likelihood yields overfitted prediction models in small to medium sized data. In most cases, the calibration and overall predictive performance of the multinomial prediction model is improved by using penalized MLR. Our simulation study also highlights the importance of events per variable in the multinomial context as well as the total sample size. As expected, our study demonstrates the need for optimism correction of the predictive performance measures when developing the multinomial logistic prediction model. We recommend the use of penalized MLR when prediction models are developed in small data sets or in medium sized data sets with a small total sample size (ie, when the sizes of the outcome categories are balanced). Finally, we present a case study in which we illustrate the development and validation of penalized and unpenalized multinomial prediction models for predicting malignancy of ovarian cancer. [ABSTRACT FROM AUTHOR]

MRI has been extensively used to identify anatomical and functional differences in Autism Spectrum Disorder (ASD). Yet, many of these findings have proven difficult to replicate because studies rely on small cohorts and are built on many complex, undisclosed, analytic choices. We conducted an international challenge to predict ASD diagnosis from MRI data, where we provided preprocessed anatomical and functional MRI data from > 2,000 individuals. Evaluation of the predictions was rigorously blinded. 146 challengers submitted prediction algorithms, which were evaluated at the end of the challenge using unseen data and an additional acquisition site. On the best algorithms, we studied the importance of MRI modalities, brain regions, and sample size. We found evidence that MRI could predict ASD diagnosis: the 10 best algorithms reliably predicted diagnosis with AUC∼0.80 – far superior to what can be currently obtained using genotyping data in cohorts 20-times larger. We observed that functional MRI was more important for prediction than anatomical MRI, and that increasing sample size steadily increased prediction accuracy, providing an efficient strategy to improve biomarkers. We also observed that despite a strong incentive to generalise to unseen data, model development on a given dataset faces the risk of overfitting: performing well in cross-validation on the data at hand, but not generalising. Finally, we were able to predict ASD diagnosis on an external sample added after the end of the challenge (EU-AIMS), although with a lower prediction accuracy (AUC=0.72). This indicates that despite being based on a large multisite cohort, our challenge still produced biomarkers fragile in the face of dataset shifts. [ABSTRACT FROM AUTHOR]

Abstract: In metabolomics research a large number of metabolites are measured that reflect the cellular state under the experimental conditions studied. In many occasions the experiments are performed according to an experimental design to make sure that sufficient variation is induced in the metabolite concentrations. However, as metabolomics is a holistic approach, also a large number of metabolites are measured in which no variation is induced by the experimental design. The presence of such non-induced metabolites hampers traditional data analysis methods as PCA to estimate the true model of the induced variation. The greediness of PCA leads to a clear overfit of the metabolomics data and can lead to a bad selection of important metabolites. In this paper we explore how, why and how severe PCA overfits data with an underlying experimental design. Recently new data analysis methods have been introduced that can use prior information of the system to reduce the overfit. We show that incorporation of prior knowledge of the system under investigation leads to a better estimation of the true underlying structure and to less overfit. The experimental design information together with ASCA is used to improve the analysis of metabolomics data. To show the improved model estimation property of ASCA a thorough simulation study is used and the results are extended to a microbial metabolomics batch fermentation study. The ASCA model is much less affected by the non-induced variation and measurement error than PCA, leading to a much better model of the induced variation. [Copyright &y& Elsevier]

Abstract: The effectiveness and the “indeterminacy” of artificial neural networks (ANNs) are discussed. In the way of determination of parameters of different models a clear distinction should be made between two approaches: (i) parameter estimation (classical approach) and (ii) ANN approach with the aim of prediction (mainly interpolation). In the classical approach different regression procedures are used, while in latter one the so-called “training” using back propagation algorithm is applied. The latter approach is widely employed in combinatorial materials science for “information mining” purposes. Parameters obtained in classical approach should have a definite physical meaning. It also means that in the absence of a presumed hypothetical model the classical regression procedures cannot be applied. The advantage of the usage of ANNs is that it does not require a casual model for the system investigated. However, parameters obtained by ANN have no physical meaning. Generally, in ANNs the number of parameters to be determined is significantly higher than the number of data in the training set. It has to be emphasized that in ANN approach the good predictive ability has superior importance to physical meaning of parameters. [Copyright &y& Elsevier]

Multinomial Logistic Regression (MLR) has been advocated for developing clinical prediction models that distinguish between three or more unordered outcomes. We present a full-factorial simulation study to examine the predictive performance of MLR models in relation to the relative size of outcome categories, number of predictors and the number of events per variable. It is shown that MLR estimated by Maximum Likelihood yields overfitted prediction models in small to medium sized data. In most cases, the calibration and overall predictive performance of the multinomial prediction model is improved by using penalized MLR. Our simulation study also highlights the importance of events per variable in the multinomial context as well as the total sample size. As expected, our study demonstrates the need for optimism correction of the predictive performance measures when developing the multinomial logistic prediction model. We recommend the use of penalized MLR when prediction models are developed in small data sets or in medium sized data sets with a small total sample size (ie, when the sizes of the outcome categories are balanced). Finally, we present a case study in which we illustrate the development and validation of penalized and unpenalized multinomial prediction models for predicting malignancy of ovarian cancer., (© 2019 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.)