Search

Objective A model that can predict reliably the risk of pre‐eclampsia (PE)‐related pregnancy complications does not exist. The aim of this study was to develop and validate internally a clinical prediction model to predict the risk of a composite outcome of PE‐related maternal and fetal complications within 7, 14 and 30 days of testing in women with suspected or confirmed PE. Methods The data for this study were derived from a prospective, multicenter, observational cohort study on women with a singleton pregnancy and suspected or confirmed PE at 20 to

Background: Early palliative care team consultation has been shown to reduce costs of hospital care. The objective of this study was to investigate the association between palliative care team (PCT) consultation and the content and costs of hospital care in patients with advanced cancer. Material and Methods: A prospective, observational study was conducted in 12 Dutch hospitals. Patients with advanced cancer and an estimated life expectancy of less than 1 year were included. We compared hospital care during 3 months of follow-up for patients with and without PCT involvement. Propensity score matching was used to estimate the effect of PCTs on costs of hospital care. Additionally, gamma regression models were estimated to assess predictors of hospital costs. Results: We included 535 patients of whom 126 received PCT consultation. Patients with PCT had a worse life expectancy (life expectancy

Objective To provide an overview of prediction models for the risk of developing endometrial cancer in women of the general population or for the presence of endometrial cancer in symptomatic women. Methods We systematically searched the Embase and Pubmed database until September 2017 for relevant publications. We included studies describing the development, the external validation, or the updating of a multivariable model for predicting endometrial cancer in the general population or symptomatic women. Results Out of 2756 references screened, 14 studies were included. We found two prediction models for developing endometrial cancer in the general population (risk models) and one extension. Eight studies described the development of models for symptomatic women (diagnostic models), one comparison of the performance of two diagnostic models and two external validation. Sample size varied from 60 (10 with cancer) to 201,811 (855 with cancer) women. The age of the women was included as a predictor in almost all models. The risk models included epidemiological variables related to the reproductive history of women, hormone use, BMI, and smoking history. The diagnostic models also included clinical predictors, such as endometrial thickness and recurrent bleeding. The concordance statistic (c), assessing the discriminative ability, varied from 0.68 to 0.77 in the risk models and from 0.73 to 0.957 in the diagnostic models. Methodological information was often limited, especially on the handling of missing data, and the selection of predictors. One risk model and four diagnostic models were externally validated. Conclusions Only a few models have been developed to predict endometrial cancer in asymptomatic or symptomatic women. The usefulness of most models is unclear considering methodological shortcomings and lack of external validation. Future research should focus on external validation and extension with new predictors or biomarkers, such as genetic and epigenetic markers.

Summary Vitamin K antagonists (VKAs) used for the prevention and treatment of thromboembolic disease, increase the risk of bleeding complications. We developed and validated a model to predict the risk of an international normalised ratio (INR) ≥ 4·5 during a hospital stay. Adult patients admitted to a tertiary hospital and treated with VKAs between 2006 and 2010 were analysed. Bleeding risk was operationalised as an INR value ≥4·5. Multivariable logistic regression analysis was used to assess the association between potential predictors and an INR ≥ 4·5 and validated in an independent cohort of patients from the same hospital between 2011 and 2014. We identified 8996 admissions of patients treated with VKAs, of which 1507 (17%) involved an INR ≥ 4·5. The final model included the following predictors: gender, age, concomitant medication and several biochemical parameters. Temporal validation showed a c statistic of 0·71. We developed and validated a clinical prediction model for an INR ≥ 4·5 in VKA‐treated patients admitted to our hospital. The model includes factors that are collected during routine care and are extractable from electronic patient records, enabling easy use of this model to predict an increased bleeding risk in clinical practice.

AimTo develop a prediction model for preconception identification of women at risk of gestational diabetes mellitus (GDM).MethodsData from a prospective cohort, the Australian Longitudinal Study on Women’s Health, were used. Nulliparous women aged 18–23 who reported a pregnancy up to age 37–42 were included. Preconception predictors of GDM during a first pregnancy were selected using logistic regression. Regression coefficients were multiplied by a shrinkage factor estimated with bootstrapping to improve prediction in external populations.ResultsAmong 6504 women, 314 (4.8%) developed GDM during their first pregnancy. The final prediction model included age at menarche, proposed age at future first pregnancy, ethnicity, body mass index, diet, physical activity, polycystic ovary syndrome, and family histories of type 1 or 2 diabetes and GDM. The model showed good discriminative ability with a C-statistic of 0.79 (95% CI 0.76, 0.83) after internal validation. More than half of the women (58%) were classified to be at risk of GDM (>2% predicted risk), with corresponding sensitivity and specificity values of 91% and 43%.ConclusionsNulliparous women at risk of GDM in a future first pregnancy can be accurately identified based on preconception lifestyle and health-related characteristics. Further studies are needed to test our model in other populations.

PURPOSE: Patients with nonmuscle invasive bladder cancer are followed with frequent cystoscopies. In this study FGFR3, TERT and OTX1 were investigated as a diagnostic urinary marker combination during followup of patients with primary nonmuscle invasive bladder cancer.MATERIALS AND METHODS: In this international, multicenter, prospective study 977 patients with nonmuscle invasive bladder cancer were included. A total of 2,496 urine samples were collected prior to cystoscopy during regular visits. Sensitivity was estimated to detect concomitant recurrences. Kaplan-Meier curves were used to estimate the development of future recurrences after urinalysis and a negative cystoscopy.RESULTS: Sensitivity of the assay combination for recurrence detection was 57% in patients with primary low grade, nonmuscle invasive bladder cancer. However, sensitivity was 83% for recurrences that were pT1 or muscle invasive bladder cancer. Of the cases 2% progressed to muscle invasive bladder cancer. Sensitivity for recurrence detection in patients with primary high grade disease was 72% and 7% of them had progression to muscle invasive bladder cancer. When no concomitant tumor was found by cystoscopy, positive urine samples were more frequently followed by a recurrence over time compared to a negative urine sample (58% vs 36%, p CONCLUSIONS: Recurrences in patients with primary nonmuscle invasive bladder cancer can be detected by a combination of urine assays. This study supports the value of urinalysis as an alternative diagnostic tool in patients presenting with low grade tumors and as a means to identify high stage tumors earlier.

Prediction models fitted with logistic regression often show poor performance when applied in populations other than the development population. Model updating may improve predictions. Previously suggested methods vary in their extensiveness of updating the model. We aim to define a strategy in selecting an appropriate update method that considers the balance between the amount of evidence for updating in the new patient sample and the danger of overfitting. We consider recalibration in the large (re-estimation of model intercept); recalibration (re-estimation of intercept and slope) and model revision (re-estimation of all coefficients) as update methods. We propose a closed testing procedure that allows the extensiveness of the updating to increase progressively from a minimum (the original model) to a maximum (a completely revised model). The procedure involves multiple testing with maintaining approximately the chosen type I error rate. We illustrate this approach with three clinical examples: patients with prostate cancer, traumatic brain injury and children presenting with fever. The need for updating the prostate cancer model was completely driven by a different model intercept in the update sample (adjustment: 2.58). Separate testing of model revision against the original model showed statistically significant results, but led to overfitting (calibration slope at internal validation = 0.86). The closed testing procedure selected recalibration in the large as update method, without overfitting. The advantage of the closed testing procedure was confirmed by the other two examples. We conclude that the proposed closed testing procedure may be useful in selecting appropriate update methods for previously developed prediction models

Objective Validation of clinical prediction models traditionally refers to the assessment of model performance in new patients. We studied different approaches to geographic and temporal validation in the setting of multicenter data from two time periods. Study Design and Setting We illustrated different analytic methods for validation using a sample of 14,857 patients hospitalized with heart failure at 90 hospitals in two distinct time periods. Bootstrap resampling was used to assess internal validity. Meta-analytic methods were used to assess geographic transportability. Each hospital was used once as a validation sample, with the remaining hospitals used for model derivation. Hospital-specific estimates of discrimination (c-statistic) and calibration (calibration intercepts and slopes) were pooled using random-effects meta-analysis methods. I 2 statistics and prediction interval width quantified geographic transportability. Temporal transportability was assessed using patients from the earlier period for model derivation and patients from the later period for model validation. Results Estimates of reproducibility, pooled hospital-specific performance, and temporal transportability were on average very similar, with c-statistics of 0.75. Between-hospital variation was moderate according to I 2 statistics and prediction intervals for c-statistics. Conclusion This study illustrates how performance of prediction models can be assessed in settings with multicenter data at different time periods.

Introduction: Experimental studies have shown that palliative care team (PCT) involvement can improve quality of life (QoL) and symptom burden of patients with advanced cancer. It is unclear to what extent this effect is sustained in daily practice of hospital care. Objective: This observational study aims to investigate the effect of PCT consultation on QoL and symptom burden of hospitalized patients with advanced cancer in daily practice. Methods: After admission to 1 of 9 participating hospitals, patients with advanced cancer for whom the attending physician answered “no” to the Surprise Question were invited to complete a questionnaire, including the EORTC QLQ-C15-PAL, at 6 points in time, until 3 months after admission. Outcomes were compared between patients who received PCT consultation and patients who did not, taking into account differences in baseline characteristics. Results: A total of 164 patients consented to participate, of whom 32 received PCT consultation. Of these patients, 108 were able to complete a questionnaire at day 14, of whom 19 after receiving PCT consultation. After adjusting for baseline differences, EORTC QLQ-C15-PAL scores for pain, appetite, and emotional functioning at day 14 were more favorable for patients who received a PCT consultation. Conclusion: PCT consultation decreased patients’ symptom burden and tends to have a positive effect on QoL of hospitalized patients with advanced cancer, even if the PCT is consulted late in the patient’s disease trajectory.

Background Discriminative ability is an important aspect of prediction model performance, but challenging to assess in clustered (e.g., multicenter) data. Concordance (c)-indexes may be too extreme within small clusters. We aimed to define a new approach for the assessment of discriminative ability in clustered data. Methods We assessed discriminative ability of a prediction model for the binary outcome mortality after traumatic brain injury within centers of the CRASH trial. With multilevel logistic regression analysis, we estimated cluster-specific calibration slopes which we used to obtain the recently proposed calibrated model-based concordance (c-mbc) within each cluster. We compared the c-mbc with the naïve c-index in centers of the CRASH trial and in simulations of clusters with varying calibration slopes. Results The c-mbc was less extreme in distribution than the c-index in 19 European centers (internal validation; n = 1716) and 36 non-European centers (external validation; n = 3135) of the CRASH trial. In simulations, the c-mbc was biased but less variable than the naïve c-index, resulting in lower root mean squared errors. Conclusions The c-mbc, based on multilevel regression analysis of the calibration slope, is an attractive alternative to the c-index as a measure of discriminative ability in multicenter studies with patient clusters of limited sample size. Electronic supplementary material The online version of this article (10.1186/s41512-019-0055-8) contains supplementary material, which is available to authorized users.

BACKGROUND: Hypertension, even during childhood, increases the risk of developing atherosclerosis and cardiovascular disease. Therefore, starting prevention of hypertension early in the life course could be beneficial. Prediction models might be useful for identifying children at increased risk of developing hypertension, which may enable targeted primordial prevention of cardiovascular disease.OBJECTIVE: To provide an overview of childhood prediction models for future hypertension.METHODS: Embase and Medline were systematically searched. Studies were included that were performed in the general population, and that reported on development or validation of a multivariable model for children to predict future high blood pressure, prehypertension or hypertension. Data were extracted using the CHARMS checklist for prediction modelling studies.RESULTS: Out of 12 780 reviewed records, six studies were included in which 18 models were presented. Five studies predicted adulthood hypertension, and one predicted adolescent prehypertension/hypertension. BMI and current blood pressure were most commonly included as predictors in the final models. Considerable heterogeneity existed in timing of prediction (from early childhood to late adolescence) and outcome measurement. Important methodological information was often missing, and in four studies information to apply the model in new individuals was insufficient. Reported area under the ROC curves ranged from 0.51 to 0.74. As none of the models were validated, generalizability could not be confirmed.CONCLUSION: Several childhood prediction models for future hypertension were identified, but their value for practice remains unclear because of suboptimal methods, limited information on performance, or the lack of external validation. Further validation studies are indicated.This is an open access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal. http://creativecommons.org/licenses/by-nc-nd/4.0.

The aim of this study was to identify predictive factors and develop a model to assess individualized risk of postnatal surgical intervention in patients with antenatal hydronephrosis. This is a retrospective cohort study of 694 infants with prenatally detected congenital anomalies of kidney and urinary tract with a median follow-up time of 37 months. The main event of interest was postnatal surgical intervention. A predictive model was developed using Cox model with internal validation by bootstrap technique. Of 694 patients, 164 (24%) infants underwent surgical intervention in a median age of 7.8 months. Predictors of the surgical intervention in the model were: baseline glomerular filtration rate, associated hydronephrosis, presence of renal damage and the severity of renal pelvic dilatation. The optimism corrected c statistic for the model was 0.84 (95%CI, 0.82-0.87). The predictive model may contribute to identify infants at high risk for surgical intervention. Further studies are necessary to validate the model in patients from other settings.

Information of an individual's epigenome can be useful in cancer screening to enable personalised decision making on participation, treatment options and further screening strategies. However, adding this information might result in complex risk predictions on multiple diseases, unsolicited findings and information on (past) environmental exposure and behaviour. This complicates informed consent procedures and may impede autonomous decision-making. In this article we investigate and identify the specific features of epigenetic risk-stratified cancer screening that challenge the current informed consent doctrine. Subsequently we describe current and new informed consent models and the principle of respect for autonomy and argue for a specific informed consent model for epigenetic risk-stratified screening programmes. Next, we propose a framework that guides the development of Patient Decision Aids (PDAs) to support informed consent and promote autonomous choices in the specific context of epigenetic cancer screening programmes.

Targeted screening for childhood high blood pressure may be more feasible than routine blood pressure measurement in all children to avoid unnecessary harms, overdiagnosis or costs. Targeting maybe based e.g. on being overweight, but information on other predictors may also be useful. Therefore, we aimed to develop a multivariable diagnostic prediction model to select children aged 9-10 years for blood pressure measurement. Data from 5359 children in a population-based prospective cohort study were used. High blood pressure was defined as systolic or diastolic blood pressure ≥ 95th percentile for gender, age, and height. Logistic regression with backward selection was used to identify the strongest predictors related to pregnancy, child, and parent characteristics. Internal validation was performed using bootstrapping. 227 children (4.2%) had high blood pressure. The diagnostic model included maternal hypertensive disease during pregnancy, maternal BMI, maternal educational level, parental hypertension, parental smoking, child birth weight standard deviation score (SDS), child BMI SDS, and child ethnicity. The area under the ROC curve was 0.73, compared to 0.65 when using only child overweight. Using the model and a cut-off of 5% for predicted risk, sensitivity and specificity were 59% and 76%; using child overweight only, sensitivity and specificity were 47% and 84%. In conclusion, our diagnostic prediction model uses easily obtainable information to identify children at increased risk of high blood pressure, offering an opportunity for targeted screening. This model enables to detect a higher proportion of children with high blood pressure than a strategy based on child overweight only.

ObjectiveTo determine the agreement between peripheral and central capillary refill time (pCRT/cCRT) and their diagnostic values for detecting serious bacterial infection (SBI) in febrile children attending the paediatric emergency department (ED).DesignProspective observational study.SettingPaediatric ED, Erasmus Medium Care-Sophia Children's hospital, the Netherlands.Patients1193 consecutively included, previously healthy, febrile children (1 month–16 years) with both pCRT measurements and cCRT measurements available. SBI diagnosis was based on abnormal radiographic findings and/or positive cultures from normally sterile locations in addition to clinical criteria.Main outcome measuresAgreement between pCRT and cCRT (Cohen's κ), overall and stratified for age and body temperature. The diagnostic value of pCRT and cCRT for SBI was assessed with logistic regression.ResultsOverall agreement was 0.35 (95% CI 0.27 to 0.43; considered ‘fair’). Although not significant, agreement was lower in children aged 1–39.5°C. Abnormal pCRT (>2 s) was observed in 153 (12.8%; 95% CI 10.9% to 14.7%) and abnormal cCRT in 55 (4.6%; 95% CI 3.4% to 5.8%) children. The OR of abnormal pCRT (>2 s) for predicting SBI was 1.10 (95% CI 0.65 to 1.84). For abnormal cCRT (>2 s), the OR was 0.43 (95% CI 0.13 to 1.39).ConclusionsThe pCRT and cCRT values showed only fair agreement in a general population of febrile children at the ED, and no significant association with age or body temperature was found. Only a small part of febrile children at risk for serious infections at the ED show abnormal CRT values. Both abnormal pCRT and cCRT (defined as >2 s) performed poorly and were non-significant in this study detecting SBI in a general population of febrile children.

Concordance measures are frequently used for assessing the discriminative ability of risk prediction models. The interpretation of estimated concordance at external validation is difficult if the case-mix differs from the model development setting. We aimed to develop a concordance measure that provides insight into the influence of case-mix heterogeneity and is robust to censoring of time-to-event data. We first derived a model-based concordance (mbc) measure that allows for quantification of the influence of case-mix heterogeneity on discriminative ability of proportional hazards and logistic regression models. This mbc can also be calculated including a regression slope that calibrates the predictions at external validation (c-mbc), hence assessing the influence of overall regression coefficient validity on discriminative ability. We derived variance formulas for both mbc and c-mbc. We compared the mbc and the c-mbc with commonly used concordance measures in a simulation study and in two external validation settings. The mbc was asymptotically equivalent to a previously proposed resampling-based case-mix corrected c-index. The c-mbc remained stable at the true value with increasing proportions of censoring, while Harrell's c-index and to a lesser extent Uno's concordance measure increased unfavorably. Variance estimates of mbc and c-mbc were well in agreement with the simulated empirical variances. We conclude that the mbc is an attractive closed-form measure that allows for a straightforward quantification of the expected change in a model's discriminative ability due to case-mix heterogeneity. The c-mbc also reflects regression coefficient validity and is a censoring-robust alternative for the c-index when the proportional hazards assumption holds. Copyright © 2016 John Wiley & Sons, Ltd.

OBJECTIVES: Risk stratification for chest pain patients at the emergency department is recommended in several guidelines. The history, ECG, age, risk factors, and troponin (HEART) score is based on medical literature and expert opinion to estimate the risk of a major adverse cardiac event. We aimed to assess the predictive effects of the 5 HEART components and to compare performances of the original HEART score and a model based on regression analysis. METHODS: We analyzed prospectively collected data from 2388 patients, of whom 407 (17%) had a major adverse cardiac event within 6 weeks (acute myocardial infarction, percutaneous coronary intervention, coronary artery bypass graft, significant stenosis with conservative treatment and death due to any cause). RESULTS: Univariate regression analysis showed the same ordering of predictive effects as used in the HEART score. Based on multivariable logistic regression analysis, an adjusted score showed slightly better calibration and discrimination (c statistic HEART, 0.83, HEART-adj, 0.85). In comparison to HEART, HEART-adj proved in a decision curve analysis clinically useful for decision thresholds over 25%. Nevertheless, the original HEART classified patients better than HEART-adj (net reclassification improvement = 14.1%). CONCLUSION: The previously chosen weights of the 5 elements of the HEART score are supported by multivariable statistical analyses, although some improvement in calibration and discrimination is possible by adapting the score. The gain in clinical usefulness is relatively small and supports the use of either the original or adjusted HEART score in daily practice.

The performance of previously developed models to predict non sentinel lymph node status for patients with sentinel lymph node-positive breast cancer was poor for our Dutch patient population. Therefore, a new model was developed that includes only 2 predictors, is easy to use, and can help clinicians and their patients in the clinical decision-making regarding axillary treatment. Background: Multiple predictive systems have previously been developed to identify the sentinel lymph node (SLN)-positive patients at low risk of additional axillary non-SLN involvement and for whom completion axillary lymph node dissection (ALND) could be avoided. However, previous studies showed that these tools had poor performance in Dutch patients with breast cancer, probably owing to variations in pathology settings and differences in population characteristics. The aim of the present study was to develop a predictive tool for the risk of non-SLN involvement in a Dutch population with SLN-positive breast cancer. Materials and Methods: The data from 513 patients with SLN-positive breast cancer at 10 participating hospitals, who had undergone ALND from January 2007 to December 2008 were studied. The uni- and multivariable associations of predictors for non-SLN metastases were analyzed, and a predictive model was developed. The discriminatory ability of the model was measured by the area under the receiver operating characteristic curve (AUC) and the agreement between predicted probabilities and observed frequencies was visualized by a calibration plot. Results: A predictive model was developed that included the 2 strongest predictors: the size of the SLN metastases in millimeters and the presence of a negative sentinel lymph node. The model showed good discriminative ability (AUC, 0.75) and good calibration over the complete range of predicted probabilities. Conclusion: We have developed a tool to predict additional non-SLN metastases in Dutch patients with SLN-positive breast cancer that is easy to use in daily clinical breast cancer practice. (C) 2016 Elsevier Inc. All rights reserved.

Objectives To develop prediction models for Chlamydia trachomatis (Ct) infection with different levels of detail in information, that is, from readily available data in registries and from additional questionnaires. Methods All inhabitants of Rotterdam and Amsterdam aged 16–29 were invited yearly from 2008 until 2011 for home-based testing. Their registry data included gender, age, ethnicity and neighbourhood-level socioeconomic status (SES). Participants were asked to fill in a questionnaire on education, sexually transmitted infection history, symptoms, partner information and sexual behaviour. We developed prediction models for Ct infection using first-time participant data—including registry variables only and with additional questionnaire variables—by multilevel logistic regression analysis to account for clustering within neighbourhoods. We assessed the discriminative ability by the area under the receiver operating characteristic curve (AUC). Results Four per cent (3540/80 385) of the participants was infected. The strongest registry predictors for Ct infection were young age (especially for women) and Surinamese, Antillean or sub-Saharan African ethnicity. Neighbourhood-level SES was of minor importance. Strong questionnaire predictors were low to intermediate education level, ethnicity of the partner (non-Dutch) and having sex with casual partners. When using a prediction model including questionnaire risk factors (AUC 0.74, 95% CI 0.736 to 0.752) for selective screening, 48% of the participating population needed to be screened to find 80% (95% CI 78.4% to 81.0%) of Ct infections. The model with registry risk factors only (AUC 0.67, 95% CI 0.656 to 0.675) required 60% to be screened to find 78% (95% CI 76.6% to 79.4%) of Ct infections. Conclusions A registry-based prediction model can facilitate selective Ct screening at population level, with further refinement at the individual level by including questionnaire risk factors.

Background— The predictive performance of static risk prediction models such as EuroSCORE deteriorates over time. We aimed to explore different methods for continuous updating of EuroSCORE (dynamic modeling) to improve risk prediction. Methods and Results— Data on adult cardiac surgery from 2007 to 2012 (n=95 240) were extracted from the Netherlands Association for Cardio-Thoracic Surgery database. The logistic EuroSCORE predicting in-hospital death was updated using 6 methods: recalibrating the intercept of the logistic regression model; recalibrating the intercept and joint effects of the prognostic factors; re-estimating all prognostic factor effects, re-estimating all prognostic factor effects, and applying shrinkage of the estimates; applying a test procedure to select either of these; and a Bayesian learning strategy. Models were updated with 1 or 3 years of data, in all cardiac surgery or within operation subgroups. Performance was tested in the subsequent year according to discrimination (area under the receiver operating curve, area under the curve) and calibration (calibration slope and calibration-in-the-large). Compared with the original EuroSCORE, all updating methods resulted in improved calibration-in-the-large (range −0.17 to 0.04 versus −1.13 to −0.97, ideally 0.0). Calibration slope (range 0.92–1.15) and discrimination (area under the curve range 0.83–0.87) were similar across methods. In small subgroups, such as aortic valve replacement and aortic valve replacement+coronary artery bypass grafting, extensive updating using 1 year of data led to poorer performance than using the original EuroSCORE. The choice of updating method had little effect on benchmarking results of all cardiac surgery. Conclusions— Several methods for dynamic modeling may result in good discrimination and superior calibration compared with the original EuroSCORE. For large populations, all methods are appropriate. For smaller subgroups, it is recommended to use data from multiple years or a Bayesian approach.

OBJECTIVES: This study aims to investigate the influence of the amount of clustering [intraclass correlation (ICC) = 0%, 5%, or 20%], the number of events per variable (EPV) or candidate predictor (EPV = 5, 10, 20, or 50), and backward variable selection on the performance of prediction models. STUDY DESIGN AND SETTING: Researchers frequently combine data from several centers to develop clinical prediction models. In our simulation study, we developed models from clustered training data using multilevel logistic regression and validated them in external data. RESULTS: The amount of clustering was not meaningfully associated with the models' predictive performance. The median calibration slope of models built in samples with EPV = 5 and strong clustering (ICC = 20%) was 0.71. With EPV = 5 and ICC = 0%, it was 0.72. A higher EPV related to an increased performance: the calibration slope was 0.85 at EPV = 10 and ICC = 20% and 0.96 at EPV = 50 and ICC = 20%. Variable selection sometimes led to a substantial relative bias in the estimated predictor effects (up to 118% at EPV = 5), but this had little influence on the model's performance in our simulations. CONCLUSION: We recommend at least 10 EPV to fit prediction models in clustered data using logistic regression. Up to 50 EPV may be needed when variable selection is performed. publisher: Elsevier articletitle: A simulation study of sample size demonstrated the importance of the number of events per variable to develop prediction models in clustered data journaltitle: Journal of Clinical Epidemiology articlelink: http://dx.doi.org/10.1016/j.jclinepi.2015.02.002 content_type: article copyright: Copyright © 2015 Elsevier Inc. All rights reserved. ispartof: Journal of Clinical Epidemiology vol:68 issue:12 pages:1406-1414 ispartof: location:United States status: published

An important aim of clinical prediction models is to positively impact clinical decision making and subsequent patient outcomes. The impact on clinical decision making and patient outcome can be quantified in prospective comparative—ideally cluster-randomized—studies, known as ‘impact studies’. However, such impact studies often require a lot of time and resources, especially when they are (cluster-)randomized studies. Before envisioning such large-scale randomized impact study, it is important to ensure a reasonable chance that the use of the prediction model by the targeted healthcare professionals and patients will indeed have a positive effect on both decision making and subsequent outcomes. We recently performed two differently designed, prospective impact studies on a clinical prediction model to be used in surgical patients. Both studies taught us new valuable lessons on several aspects of prediction model impact studies, and which considerations may guide researchers in their decision to conduct a prospective comparative impact study. We provide considerations on how to prepare a prediction model for implementation in practice, how to present the model predictions, and how to choose the proper design for a prediction model impact study. Electronic supplementary material The online version of this article (10.1186/s41512-018-0033-6) contains supplementary material, which is available to authorized users.

The incidence of cancer is continuing to rise and risk-tailored early diagnostic and/or primary prevention strategies are urgently required. The ideal risk-predictive test should: integrate the effects of both genetic and nongenetic factors and aim to capture these effects using an approach that is both biologically stable and technically reproducible; derive a score from easily accessible biological samples that acts as a surrogate for the organ in question; and enable the effectiveness of risk-reducing measures to be monitored. Substantial evidence has accumulated suggesting that the epigenome and, in particular, DNA methylation-based tests meet all of these requirements. However, the development and implementation of DNA methylation-based risk-prediction tests poses considerable challenges. In particular, the cell type specificity of DNA methylation and the extensive cellular heterogeneity of the easily accessible surrogate cells that might contain information relevant to less accessible tissues necessitates the use of novel methods in order to account for these confounding issues. Furthermore, the engagement of the scientific community with health-care professionals, policymakers and the public is required in order to identify and address the organizational, ethical, legal, social and economic challenges associated with the routine use of epigenetic testing.

Clinical risk prediction models are increasingly being developed and validated on multicenter datasets. In this article, we present a comprehensive framework for the evaluation of the predictive performance of prediction models at the center level and the population level, considering population-averaged predictions, center-specific predictions, and predictions assuming an average random center effect. We demonstrated in a simulation study that calibration slopes do not only deviate from one because of over- or underfitting of patterns in the development dataset, but also as a result of the choice of the model (standard versus mixed effects logistic regression), the type of predictions (marginal versus conditional versus assuming an average random effect), and the level of model validation (center versus population). In particular, when data is heavily clustered (ICC 20%), center-specific predictions offer the best predictive performance at the population level and the center level. We recommend that models should reflect the data structure, while the level of model validation should reflect the research question. ispartof: Statistical Methods in Medical Research vol:27 issue:6 pages:1723-1736 ispartof: location:England status: published

Background and objectives This study was conducted to evaluate the performance of available tools predicting non-sentinel lymph node (non-SLN) status in women with SLN positive breast cancer and to see if they can be safely used in everyday clinical practice. Methods Data of 220 women with breast cancer who underwent a SLN biopsy at the Maxima Medical Centre between 2000–2008 were analysed. Tools evaluated were: the models from Memorial Sloan Kettering Cancer Centre, Stanford, Mayo, Cambridge, Gur, and MOU, and the scores from Saidi, Tenon, and MDA. Model performance was assessed using calibration, discrimination and Nagelkerke's explained variation. Results The MSKCC nomogram showed best overall performance with best discrimination (AUC 0.69), second best calibration, and highest explained variation (31%). The 10% low risk threshold led to defining only 22% (38/176) of the women as being low risk while in fact 66% (116/176) were non-SLN negative. The false negative rate was 13% (5/38). Conclusions Current models for predicting non-SLN metastases in SLN positive breast cancer are not yet ready for implementation in general practice. Further research efforts should improve model performance in selecting patients or perhaps find a role in support in the paradigm shift to a “treat none unless” approach. J. Surg. Oncol. 2015 111:133–138. © 2015 Wiley Periodicals, Inc.

Brinkman-Stoppelenburg A, Vergouwe Y, Van der Heide A, Onwuteaka- Philipsen BD. SCEN-artsen en de zorgvuldigheidseisen. Huisarts Wet 2015;58(6):290-3. Consultatie door een onafhankelijk arts is een van de zorgvuldigheidseisen waaraan artsen moeten voldoen bij het uitvoeren van euthanasie. SCEN-artsen zijn opgeleid om deze consultatie te verlenen. Met dit onderzoek wilden wij nagaan waarom SCEN-artsen soms vinden dat er niet aan de zorgvuldigheidseisen is voldaan. Verder wilden we achterhalen welke factoren samenhangen met een negatief oordeel van SCEN-artsen. Gedurende vijf jaar (2006, 2008-2011) hebben we gestandaardiseerde vragenlijsten afgenomen bij alle SCEN-artsen in Nederland. Door middel van een multilevel logistische regressie hebben we gekeken welke factoren samenhingen met het oordeel van de SCEN-artsen over het al dan niet voldoen aan de zorgvuldigheidseisen. We hebben 1631 euthanasieverzoeken geanalyseerd, waarbij 415 SCEN-artsen betrokken waren. De kans dat aan de zorgvuldigheidseisen was voldaan nam af bij bepaalde patientkenmerken: ‘levensmoeheid’, ‘depressie’ en ‘niet tot last van familie/ omgeving willen zijn’. Fysiek lijden en een hogere leeftijd van de patient hingen samen met een grotere kans dat aan alle zorgvuldigheidseisen was voldaan. We hebben ook gekeken naar kenmerken van de SCEN-artsen zelf. Alleen hogere leeftijd lijkt enigszins samen te hangen met een wat grotere kans dat aan de zorgvuldigheidseisen is voldaan. We vonden geen samenhang tussen andere kenmerken van de SCEN-artsen en hun oordeel. Als er sprake is van psychisch lijden is er een grotere kans dat SCEN-artsen vinden dat niet aan de zorgvuldigheidseisen is voldaan. Het oordeel van SCEN-artsen hangt niet samen met persoonskenmerken als geslacht, achtergrond, ervaring als SCEN-arts, gevolgde scholing en of ze al dan niet consulent palliatieve zorg zijn.

Objectives: We aimed to compare modeling approaches to estimate the individual survival benefit of treatment with either coronary artery bypass graft surgery (CABG) or percutaneous coronary intervention (PCI) for patients with complex coronary artery disease. Study Design and Setting: We estimated survival with Cox regression models that included the treatment variable (CABG/PCI) interacting with either an internally developed overall prognostic index (PI) or with individual prognostic factors. We analyzed data of patients who were randomized in the Synergy between Percutaneous Coronary Intervention with Taxus and Cardiac Surgery trial (1,800 patients, 178 deaths). Results: A negligible interaction with the PI (P = 0.51) led to 4-year survival estimates in favor of CABG for all patients. In contrast, individual interactions indicated substantial relative treatment effect heterogeneity (overall interaction P = 0.004), and estimates of 4-year survival were numerically in favor of CABG for 1,275 of 1,800 patients (71%; 519 with 95% confidence). To test the more complex model with individual interactions, we first used penalized regression, resulting in smaller but largely Consistent individual estimates of the survival difference between CABG and PCI. Second, strong treatment interactions were confirmed at external validation in 2,891 patients from a multinational registry. Conclusion: Modeling strategies that omit interactions may result in misleading estimates of absolute treatment benefit for individual patients with the potential hazard of suboptimal decision making. (C) 2015 Elsevier Inc. All rights reserved.

Background In a large cluster-randomized trial on the impact of a prediction model, presenting the calculated risk of postoperative nausea and vomiting (PONV) on-screen (assistive approach) increased the administration of risk-dependent PONV prophylaxis by anaesthetists. This change in therapeutic decision-making did not improve the patient outcome; that is, the incidence of PONV. The present study aimed to quantify the effects of adding a specific therapeutic recommendation to the predicted risk (directive approach) on PONV prophylaxis decision-making and the incidence of PONV. Methods A prospective before–after study was conducted in 1483 elective surgical inpatients. The before-period included care-as-usual and the after-period included the directive risk-based (intervention) strategy. Risk-dependent effects on the administered number of prophylactic antiemetics and incidence of PONV were analysed by mixed-effects regression analysis. Results During the intervention period anaesthetists administered 0.5 [95% confidence intervals (CIs): 0.4–0.6] more antiemetics for patients identified as being at greater risk of PONV. This directive approach led to a reduction in PONV [odds ratio (OR): 0.60, 95% CI: 0.43–0.83], with an even greater reduction in PONV in high-risk patients (OR: 0.45, 95% CI: 0.28–0.72). Conclusions Anaesthetists administered more prophylactic antiemetics when a directive approach was used for risk-tailored intervention compared with care-as-usual. In contrast to the previously studied assistive approach, the increase in PONV prophylaxis now resulted in a lower PONV incidence, particularly in high-risk patients. When one aims for a truly ‘PONV-free hospital)', a more liberal use of prophylactic antiemetics must be accepted and lower-risk thresholds should be set for the actionable recommendations.

C-Reactive protein (CRP) is an important diagnostic marker for serious bacterial infections in febrile children. C-Reactive protein bedside testing could potentially accelerate the diagnostic evaluation and shorten length of stay (LOS).The aim of the study was to study the effect of introducing CRP bedside testing on the LOS of febrile children at the emergency department (ED).A prospective observational study with a preimplementation cohort (2008) with traditional CRP testing and a postimplementation cohort (2009-2011) in which CRP bedside testing was introduced.All previously healthy children with fever, aged 1 month to 16 years, attending the ED of a university hospital were included; non-ill-appearing children with an upper airway infection were not eligible for CRP bedside testing.Multivariable linear regression and propensity score analyses were used to determine the effect of CRP bedside testing on the logarithmic transformation length of stay [(log)LOS].The preimplementation cohort included 609 children of whom 286 (47%) had traditional CRP. The postimplementation cohort included the following 1330 children: 728 (55%) children had bedside CRP and 156 (12%) children had traditional CRP. Bedside CRP significantly lowered the median LOS of children in whom an additional diagnostic CRP test was performed, from 178 minutes (interquartile range, 135-232 minutes) to 148 minutes (interquartile range, 108-200 minutes) (30 minutes, 19% of total LOS). A significant reduction of 15% of the (log)LOS remained after adjusting for other determinants of (log)LOS; propensity score analysis showed a 16% reduction.C-Reactive protein bedside testing substantially lowered the LOS of children with fever at the ED in whom an additional diagnostic CRP test was performed.

Objectives: It is widely acknowledged that the performance of diagnostic and prognostic prediction models should be assessed in external validation studies with independent data from "different but related" samples as compared with that of the development sample. We developed a framework of methodological steps and statistical methods for analyzing and enhancing the interpretation of results from external validation studies of prediction models. Study Design and Setting: We propose to quantify the degree of relatedness between development and validation samples on a scale ranging from reproducibility to transportability by evaluating their corresponding case-mix differences. We subsequently assess the models' performance in the validation sample and interpret the performance in view of the case-mix differences. Finally, we may adjust the model to the validation setting. Results: We illustrate this three-step framework with a prediction model for diagnosing deep venous thrombosis using three validation samples with varying case mix. While one external validation sample merely assessed the model's reproducibility, two other samples rather assessed model transportability. The performance in all validation samples was adequate, and the model did not require extensive updating to correct for miscalibration or poor fit to the validation settings. Conclusion: The proposed framework enhances the interpretation of findings at external validation of prediction models. (C) 2015 The Authors. Published by Elsevier Inc.

Recurrence of bladder cancer can occur repeatedly in the same patient after treatment of the primary tumor. Models predicting the risk of a next recurrence may inform individualized decision-making on surveillance frequency. We aimed to assess the usefulness of extensions of the Cox proportional hazards model for repeated events in this context. We analyzed 531 Dutch patients with bladder cancer (1990-2012) with information on 7 prespecified predictors at the time of diagnosis of the primary and recurrent tumors. We considered 3 aspects of model variants: how to model time to the repeated events (calendar time, gap time, elapsed time); the number of preceding events (predictor, stratum variable); and the within-subject correlation (ignored in a simple Cox model, robust standard errors in a variance-correction model, random effect in a frailty model). First to fourth recurrences of bladder cancer occurred in 313, 174, 103, and 66 patients, respectively, with median calendar follow-up times of 1.1, 2.5, 3.8, and 4.5 years, respectively. We focused on gap time in the detailed analyses, allowing for clinically meaningful predictions. Variance-correction models may be useful if predictor selection is part of the model development. Frailty models may be useful when within-subject correlation is strong.

Background The SYNTAX score (SXscore), an anatomical-based scoring tool reflecting the complexity of coronary anatomy, has established itself as an important long-term prognostic factor in patients undergoing percutaneous coronary intervention (PCI). The incorporation of clinical factors may further augment the utility of the SXscore to longer-term risk stratify the individual patient for clinical outcomes. Methods and results Patient-level merged data from >6000 patients in seven contemporary coronary stent trials was used to develop a logistic regression model—the Logistic Clinical SXscore—to predict 1-year risk for all-cause death and major adverse cardiac events (MACE). A core model (composed of the SXscore, age, creatinine clearance, and left ventricular ejection fraction) and an extended model [incorporating the core model and six additional (best performing) clinical variables] were developed and validated in a cross-validation procedure. The core model demonstrated a substantial improvement in predictive ability for 1-year all-cause death compared with the SXscore in isolation [area under the receiver operator curve (AUC): core model: 0.753, SXscore: 0.660]. A minor incremental benefit of the extended model was shown (AUC: 0.791). Consequently the core model alone was retained in the final the Logistic Clinical SXscore model. Validation plots confirmed the model predictions to be well calibrated. For 1-year MACE, the addition of clinical variables did not improve the predictive ability of the SXscore, secondary to the SXscore being the predominant determinant of all-cause revascularization. Conclusion The Logistic Clinical SXscore substantially enhances the prediction of 1-year mortality after PCI compared with the SXscore, and allows for an accurate personalized assessment of patient risk. The original version was incorrect. This article should have been published as a Clinical Research paper, not a Clinical Review paper. This error has been corrected in the present version.

BACKGROUND: Stability in baseline risk and estimated predictor effects both geographically and temporally is a desirable property of clinical prediction models. However, this issue has received little attention in the methodological literature. Our objective was to examine methods for assessing temporal and geographic heterogeneity in baseline risk and predictor effects in prediction models.METHODS: We studied 14,857 patients hospitalized with heart failure at 90 hospitals in Ontario, Canada, in two time periods. We focussed on geographic and temporal variation in baseline risk (intercept) and predictor effects (regression coefficients) of the EFFECT-HF mortality model for predicting 1-year mortality in patients hospitalized for heart failure. We used random effects logistic regression models for the 14,857 patients.RESULTS: The baseline risk of mortality displayed moderate geographic variation, with the hospital-specific probability of 1-year mortality for a reference patient lying between 0.168 and 0.290 for 95% of hospitals. Furthermore, the odds of death were 11% lower in the second period than in the first period. However, we found minimal geographic or temporal variation in predictor effects. Among 11 tests of differences in time for predictor variables, only one had a modestly significant P value (0.03).CONCLUSIONS: This study illustrates how temporal and geographic heterogeneity of prediction models can be assessed in settings with a large sample of patients from a large number of centers at different time periods.

SummaryBackground A third of patients with a first basal cell carcinoma (BCC) will develop subsequent (metachronous) BCCs. Objectives We studied the prognostic effect of the number of previous BCC diagnosis dates a patient has experienced and derived a prediction model to assess the risk of metachronous BCCs that may inform individualized decision making on surveillance. Methods We considered participants of north-western European ancestry from a prospective population-based cohort study (Rotterdam Study). After linkage with the Dutch Pathology Registry (PALGA) 1,077 patients with a first BCC were included. Candidate predictors for metachronous BCCs included patient, lifestyle, and tumour characteristics. The prognostic model was developed with Fine and Gray regression analysis to account for competing risk of death. We used bootstrapping to correct for within-patient correlation and statistical optimism in predictive performance. Results Second to fifth BCCs occurred in 293, 122, 58, and 36 patients, with median follow-up times of 3.0, 2.1, 1.7, and 1.8 years after the previous BCC, respectively. The risk of a new BCC was higher for patients with more metachronous BCCs. Having more than 1 BCC at diagnosis was another strong predictor of metachronous BCCs. Discriminative ability of the model was reasonable with optimism-corrected c-index of 0.70 at 3 years. Conclusions The number of previous BCC diagnosis dates was a strong prognostic factor and should be considered when predicting the risk of metachronous BCCs. When the number of previous BCC diagnosis dates is combined with other readily available characteristics into a prognostic model, patients at high risk of a new BCC can be identified. This article is protected by copyright. All rights reserved.

Item does not contain fulltext BACKGROUND: No golden diagnostic standard is available to diagnose chronic gastrointestinal ischemia (CGI). GOALS: We aimed to establish an accurate prediction model for CGI, based on clinical symptoms and radiologic evaluation of the amount of stenosis in the celiac artery (CA) and superior mesenteric artery (SMA) by means of computed tomography-angiography or magnetic resonance (MR)-angiography. STUDY: We prospectively included 436 consecutive patients with clinical suspicion of CGI in a tertiary referral center. Predictors for CGI were obtained by comparing clinical parameters to the diagnosis of CGI. Multivariable logistic regression was used to combine the strongest predictors in a model. A score chart based on the prediction model was provided to calculate the risk of CGI. RESULTS: CGI was present in 171/436 (39%) patients (67 y; range, 54 to 74 y; 27% male). Strongest predictors for CGI were female gender [odds ratio (OR)=1.44; 95% confidence interval (CI), 0.85-2.43], weight loss (OR=1.63, 95% CI, 0.98-2.72), concomitant cardiovascular disease (OR=1.70, 95% CI, 1.04-2.78), duration of symptoms (OR=0.88, 95% CI, 0.79-0.99), and stenosis of CA and SMA (50% to 70% stenosis CA: OR=1.33, 95% CI, 0.56-3.19; >70% stenosis CA: OR=5.79, 95% CI, 3.42-9.81; 50% to 70% stenosis SMA: OR=3.21, 95% CI, 0.81-12.74; >70% stenosis SMA: OR=4.39, 95% CI, 2.30-8.41). A model based on clinical symptoms alone showed limited discriminative ability for diagnosing CGI (c-statistic 0.62). Adding radiologic imaging of the mesenteric arteries improved the discriminative ability (c-statistic 0.79). CONCLUSIONS: Clinical symptoms alone are insufficient to predict the risk of CGI. Radiologic evaluation of the mesenteric arteries is essential. This tool may be useful for clinicians to assess the risk of CGI and to decide whether further diagnostic work-up for CGI is needed.

To assess the incremental value of a single determination of the serum levels of sFlt-1 (soluble Fms-like tyrosine kinase 1) and PlGF (placental growth factor) or their ratio, without using cutoff values, for the prediction of maternal and fetal/neonatal complications and pregnancy prolongation, 620 women with suspected/confirmed preeclampsia, aged 18 to 48 years, were included in a prospective, multicenter, observational cohort study. Women had singleton pregnancies and a median pregnancy duration of 34 (range, 20–41) weeks. Complications occurred in 118 women and 248 fetuses. The median duration between admission and delivery was 12 days. To predict prolongation, PlGF showed the highest incremental value ( R 2 =0.72) on top of traditional predictors (gestational age at inclusion, diastolic blood pressure, proteinuria, creatinine, uric acid, alanine transaminase, lactate dehydrogenase, and platelets) compared with R 2 =0.53 for the traditional predictors only. The sFlt-1/PlGF ratio showed the highest value to discriminate women with and without maternal complications (C-index=0.80 versus 0.63 for the traditional predictors only), and to discriminate fetal/neonatal complications (C-index=0.86 versus 0.78 for the traditional predictors only). Applying previously suggested cutoff values for the sFlt-1/PlGF ratio yielded lower incremental values than applying continuous values. In conclusion, sFlt-1 and PlGF are strong and independent predictors for days until delivery along with maternal and fetal/neonatal complications on top of the traditional criteria. Their use as continuous variables (instead of applying cutoff values for different gestational ages) should now be tested in a prospective manner, making use of an algorithm calculating the risk of an individual woman with suspected/confirmed preeclampsia to develop complications.