Search

Background: Early palliative care team consultation has been shown to reduce costs of hospital care. The objective of this study was to investigate the association between palliative care team (PCT) consultation and the content and costs of hospital care in patients with advanced cancer. Material and Methods: A prospective, observational study was conducted in 12 Dutch hospitals. Patients with advanced cancer and an estimated life expectancy of less than 1 year were included. We compared hospital care during 3 months of follow-up for patients with and without PCT involvement. Propensity score matching was used to estimate the effect of PCTs on costs of hospital care. Additionally, gamma regression models were estimated to assess predictors of hospital costs. Results: We included 535 patients of whom 126 received PCT consultation. Patients with PCT had a worse life expectancy (life expectancy <3 months: 62% vs. 31%, p <.01) and performance status (p <.01, e.g., WHO status higher than 2:54% vs. 28%) and more often had no more options for anti-tumour therapy (57% vs. 30%, p <.01). Hospital length of stay, use of most diagnostic procedures, medication and other therapeutic interventions were similar. The total mean hospital costs were €8,393 for patients with and €8,631 for patients without PCT consultation. Analyses using propensity scores to control for observed confounding showed no significant difference in hospital costs. Conclusions: PCT consultation for patients with cancer in Dutch hospitals often occurs late in the patients’ disease trajectories, which might explain why we found no effect of PCT consultation on costs of hospital care. Earlier consultation could be beneficial to patients and reduce costs of care.

Background and objective: The objective of this article is to externally validate and update a recently published score chart for chronic mesenteric ischemia (CMI). Methods: A multicenter prospective cohort analysis was conducted of 666 CMI-suspected patients referred to two Dutch specialized CMI centers. Multidisciplinary consultation resulted in expert-based consensus diagnosis after which CMI consensus patients were treated. A definitive diagnosis of CMI was established if successful treatment resulted in durable symptom relief. The absolute CMI risk was calculated and discriminative ability of the original chart was assessed by the c-statistic in the validation cohort. Thereafter the original score chart was updated based on the performance in the combined original and validation cohort with inclusion of celiac artery (CA) stenosis cause. Results: In 8% of low-risk patients, 39% of intermediate-risk patients and 94% of high-risk patients of the validation cohort, CMI was diagnosed. Discriminative ability of the original model was acceptable (c-statistic 0.79). The total score of the updated chart ranged from 0 to 28 points (low risk 19% absolute CMI risk, intermediate risk 45%, and high risk 92%). The discriminative ability of the updated chart was slightly better (c-statistic 0.80). Conclusion: The CMI prediction model performs and discriminates well in the validation cohort. The updated score chart has excellent discriminative ability and is useful in clinical decision making.

BACKGROUND: Hypertension, even during childhood, increases the risk of developing atherosclerosis and cardiovascular disease. Therefore, starting prevention of hypertension early in the life course could be beneficial. Prediction models might be useful for identifying children at increased risk of developing hypertension, which may enable targeted primordial prevention of cardiovascular disease. OBJECTIVE: To provide an overview of childhood prediction models for future hypertension. METHODS: Embase and Medline were systematically searched. Studies were included that were performed in the general population, and that reported on development or validation of a multivariable model for children to predict future high blood pressure, prehypertension or hypertension. Data were extracted using the CHARMS checklist for prediction modelling studies. RESULTS: Out of 12 780 reviewed records, six studies were included in which 18 models were presented. Five studies predicted adulthood hypertension, and one predicted adolescent prehypertension/hypertension. BMI and current blood pressure were most commonly included as predictors in the final models. Considerable heterogeneity existed in timing of prediction (from early childhood to late adolescence) and outcome measurement. Important methodological information was often missing, and in four studies information to apply the model in new individuals was insufficient. Reported area under the ROC curves ranged from 0.51 to 0.74. As none of the models were validated, generalizability could not be confirmed. CONCLUSION: Several childhood prediction models for future hypertension were identified, but their value for practice remains unclear because of suboptimal methods, limited information on performance, or the lack of external validation. Further validation studies are indicated.

Vitamin K antagonists (VKAs) used for the prevention and treatment of thromboembolic disease, increase the risk of bleeding complications. We developed and validated a model to predict the risk of an international normalised ratio (INR) ≥ 4·5 during a hospital stay. Adult patients admitted to a tertiary hospital and treated with VKAs between 2006 and 2010 were analysed. Bleeding risk was operationalised as an INR value ≥4·5. Multivariable logistic regression analysis was used to assess the association between potential predictors and an INR ≥ 4·5 and validated in an independent cohort of patients from the same hospital between 2011 and 2014. We identified 8996 admissions of patients treated with VKAs, of which 1507 (17%) involved an INR ≥ 4·5. The final model included the following predictors: gender, age, concomitant medication and several biochemical parameters. Temporal validation showed a c statistic of 0·71. We developed and validated a clinical prediction model for an INR ≥ 4·5 in VKA-treated patients admitted to our hospital. The model includes factors that are collected during routine care and are extractable from electronic patient records, enabling easy use of this model to predict an increased bleeding risk in clinical practice.

Clinical risk prediction models are increasingly being developed and validated on multicenter datasets. In this article, we present a comprehensive framework for the evaluation of the predictive performance of prediction models at the center level and the population level, considering population-averaged predictions, center-specific predictions, and predictions assuming an average random center effect. We demonstrated in a simulation study that calibration slopes do not only deviate from one because of over- or underfitting of patterns in the development dataset, but also as a result of the choice of the model (standard versus mixed effects logistic regression), the type of predictions (marginal versus conditional versus assuming an average random effect), and the level of model validation (center versus population). In particular, when data is heavily clustered (ICC 20%), center-specific predictions offer the best predictive performance at the population level and the center level. We recommend that models should reflect the data structure, while the level of model validation should reflect the research question.

Objective: To provide an overview of prediction models for the risk of developing endometrial cancer in women of the general population or for the presence of endometrial cancer in symptomatic women. Methods: We systematically searched the Embase and Pubmed database until September 2017 for relevant publications. We included studies describing the development, the external validation, or the updating of a multivariable model for predicting endometrial cancer in the general population or symptomatic women. Results: Out of 2756 references screened, 14 studies were included. We found two prediction models for developing endometrial cancer in the general population (risk models) and one extension. Eight studies described the development of models for symptomatic women (diagnostic models), one comparison of the performance of two diagnostic models and two external validation. Sample size varied from 60 (10 with cancer) to 201,811 (855 with cancer) women. The age of the women was included as a predictor in almost all models. The risk models included epidemiological variables related to the reproductive history of women, hormone use, BMI, and smoking history. The diagnostic models also included clinical predictors, such as endometrial thickness and recurrent bleeding. The concordance statistic (c), assessing the discriminative ability, varied from 0.68 to 0.77 in the risk models and from 0.73 to 0.957 in the diagnostic models. Methodological information was often limited, especially on the handling of missing data, and the selection of predictors. One risk model and four diagnostic models were externally validated. Conclusions: Only a few models have been developed to predict endometrial cancer in asymptomatic or symptomatic women. The usefulness of most models is unclear considering methodological shortcomings and lack of external validation. Future research should focus on external validation and extension with new predictors or biomarkers, such as genetic and epigenetic

Background: The double-blind, placebo-controlled food challenge test (DBPCFC) is the gold standard in cashew nut allergy. This test is costly, time consuming and not without side effects. Analysis of IgE reactivity to cashew nut components may reduce the need for food challenge tests. Methods: In a prospective and multicentre study, children with suspected cashew nut allergy underwent a DBPCFC with cashew nut. Specific IgE to cashew nut and to the components Ana o 1, 2 and 3 were determined. A skin prick test (SPT) with cashew nut extract was performed. The association between the outcome of the food challenge test and specific IgE to Ana o 1, 2 and 3 was assessed with logistic regression analyses, unadjusted and adjusted for other diagnostic variables. Discriminative ability was quantified with a concordance index (c). Results: A total of 173 children (103 boys, 60%) with a median age of 9 years were included. About 79% had a positive challenge test outcome. A steep rise in the risk of a positive challenge was observed for specific IgE to each individual component Ana o 1, 2 and 3 with estimated risks up to approximately 100%. Median values of Ana o 1, 2, 3 were 1.29 kU/l (range 0–100 kU/l), 4.77 kU/l (range 0–100 kU/l) and 8.33 kU/l (range 0–100 kU/l) respectively and varied significantly (p < 0.001). Specific IgE to Ana o 1, 2 and 3 was better distinguished between cashew-allergic and tolerant children (c = 0.87, 0.85 and 0.89, respectively) than specific IgE to cashew nut or SPT (c = 0.76 and 0.83, respectively). Conclusion: The major cashew nut allergens Ana o 1, 2 and 3 are each individually predictive for the outcome of food challenge tests in cashew-allergic children.

Objectives Prediction models may facilitate risk-based management of health care conditions. In a large cluster-randomized trial, presenting calculated risks of postoperative nausea and vomiting (PONV) to physicians (assistive approach) increased risk-based management of PONV. This increase did not improve patient outcome - that is, PONV incidence. This prompted us to explore how prediction tools guide the decision-making process of physicians. Study Design and Setting Using mixed methods, we interviewed eight physicians to understand how predicted risks were perceived by the physicians and how they influenced decision making. Subsequently, all 57 physicians of the trial were surveyed for how the presented risks influenced their perceptions. Results Although the prediction tool made physicians more aware of PONV prevention, the physicians reported three barriers to use predicted risks in their decision making. PONV was not considered an outcome of utmost importance; decision making on PONV prophylaxis was mostly intuitive rather than risk based; prediction models do not weigh benefits and risks of prophylactic drugs. Conclusion Combining probabilistic output of the model with their clinical experience may be difficult for physicians, especially when their decision-making process is mostly intuitive. Adding recommendations to predicted risks (directive approach) was considered an important step to facilitate the uptake of a prediction tool.

Objective: Validation of clinical prediction models traditionally refers to the assessment of model performance in new patients. We studied different approaches to geographic and temporal validation in the setting of multicenter data from two time periods. Study Design and Setting: We illustrated different analytic methods for validation using a sample of 14,857 patients hospitalized with heart failure at 90 hospitals in two distinct time periods. Bootstrap resampling was used to assess internal validity. Meta-analytic methods were used to assess geographic transportability. Each hospital was used once as a validation sample, with the remaining hospitals used for model derivation. Hospital-specific estimates of discrimination (c-statistic) and calibration (calibration intercepts and slopes) were pooled using random-effects meta-analysis methods. I2 statistics and prediction interval width quantified geographic transportability. Temporal transportability was assessed using patients from the earlier period for model derivation and patients from the later period for model validation. Results: Estimates of reproducibility, pooled hospital-specific performance, and temporal transportability were on average very similar, with c-statistics of 0.75. Between-hospital variation was moderate according to I2 statistics and prediction intervals for c-statistics. Conclusion: This study illustrates how performance of prediction models can be assessed in settings with multicenter data at different time periods.

Background: New markers hold the promise of improving risk prediction for individual patients. We aimed to compare the performance of different strategies to extend a previously developed prediction model with a new marker. Methods: Our motivating example was the extension of a risk calculator for prostate cancer with a new marker that was available in a relatively small dataset. Performance of the strategies was also investigated in simulations. Development, marker and test sets with different sample sizes originating from the same underlying population were generated. A prediction model was fitted using logistic regression in the development set, extended using the marker set and validated in the test set. Extension strategies considered were re-estimating individual regression coefficients, updating of predictions using conditional likelihood ratios (LR) and imputation of marker values in the development set and subsequently fitting a model in the combined development and marker sets. Sample sizes considered for the development and marker set were 500 and 100, 500 and 500, and 100 and 500 patients. Discriminative ability of the extended models was quantified using the concordance statistic (c-statistic) and calibration was quantified using the calibration slope. Results: All strategies led to extended models with increased discrimination (c-statistic increase from 0.75 to 0.80 in test sets). Strategies estimating a large number of parameters (re-estimation of all coefficients and updating using conditional LR) led to overfitting (calibration slope below 1). Parsimonious methods, limiting the number of coefficients to be re-estimated, or applying shrinkage after model revision, limited the amount of overfitting. Combining the development and marker set using imputation of missing marker values approach led to consistently good performing models in all scenarios. Similar results were observed in the motivating example. Conclusion: When the sample with the new marke

Background: Palliative sedation is the widely-used intervention of administering sedating agents to induce a state of unconsciousness to take away a dying patient's perception of otherwise irrelievable symptoms. However, it remains questionable whether this ethically complex intervention is beneficial for patients and whether the associated lack of communication in the last phase of life has a negative impact on relatives' wellbeing. Methods: An observational questionnaire study was conducted among relatives of a consecutive sample of patients who died a non-sudden death in the Erasmus MC Cancer Institute or in the hospice 'Laurens Cadenza' (both in Rotterdam) between 2010 and 2013. Results: Relatives filled in questionnaires regarding 151 patients who had been sedated and 90 patients who had not been sedated. The median time since all patients had passed away was 21 (IQR 14-32) months. No significant differences were found in relatives' assessments of the quality of end-of-life care, patients' quality of life in the last week before death and their quality of dying, between patients who did and did not receive sedation, or in relatives' satisfaction with their own life, their general health and their mental wellbeing after the patient's death. Conclusions: The use of sedation in these patients appears to have no negative effect on be

Background The work ability index (WAI) is a frequently used tool in occupational health to identify workers at risk for a reduced work performance and for work-related disability. However, information about the prognostic value of the WAI to identify workers at risk for sickness absence is scarce. Objectives To investigate the prognostic value of the WAI for sickness absence, and whether the discriminative ability differs across demographic subgroups. Methods At baseline, the WAI (score 7-49) was assessed among 1,331 office workers from a Dutch financial service company. Sickness absence was registered during 12-months follow-up and categorised as 0 days, 0

Objectives To validate and optimize a referral rule to identify primary care patients with chronic low back pain (CLBP) suspected for axial spondyloarthritis (axSpA). Design Cross-sectional study with data from 19 Dutch primary care practices for development and 38 for validation. Participants Primary care patients aged 18-45 years with CLBP existing more than three months a

Background: Patients with an advanced incurable disease are often hospitalised for some time during the last phase of life. Care in hospitals is generally focussed at curing disease and prolonging life and may therefore not in all cases adequately address the needs of such patients. We present the COMPASS study, a study on the effects and costs of consultation teams for palliative care in hospitals. This observational study aims to investigate the use, effects and costs of PCT consultation services for hospitalized patients with incurable cancer in the Netherlands. Methods/design: The study consists of 3 parts: 1. A questionnaire, interviews and a focus group discussion to investigate the characteristics of PCT consultation in 12 hospitals. PCTs will register their activities to calculate the costs of PCT consultation. 2. Cancer patients for whom the attending physician would not be surprised that they would die within 12 month will be included in a medical file search in three hospitals. Medical records will be investigated to compare care, treatment and hospital costs between patients with and patients without PCT consultation. 3. In the other nine hospitals, we will perform a longitudinal study, and compare quality of life between 100 patients for w

Objectives We aimed to compare nonlinear modeling methods for handling continuous predictors for reproducibility and transportability of prediction models. Study Design and Setting We analyzed four cohorts of previously unscreened men who underwent prostate biopsy for diagnosing prostate cancer. Continuous predictors of prostate cancer included prostate-specific antigen and prostate volume. The logistic regression models included linear terms, logarithmic terms, fractional polynomials of degree one or two (FP1 and FP2), or restricted cubic splines (RCS) with three or five knots (RCS3 and RCS5). The resulting models were internally validated by bootstrap resampling and externally validated in the cohorts not used at model development. Performance was assessed with the area under the receiver operating characteristic curve (AUC) and the calibration component of the Brier score (CAL). Results At internal validation models with FP2 or RCS5 showed slightly better performance than the other models (typically 0.004 difference in AUC and 0.001 in CAL). At external validation models containing logarithms, FP1, or RCS3 showed better performance (differences 0.01 and 0.002). Conclusion Flexible nonlinear modeling methods led to better model performance at internal validation. However, when application of the model is intended across a wide range of settings, less flexible functions may be more appropriate to maximize external validity.

Objectives It is widely acknowledged that the performance of diagnostic and prognostic prediction models should be assessed in external validation studies with independent data from "different but related" samples as compared with that of the development sample. We developed a framework of methodological steps and statistical methods for analyzing and enhancing the interpretation of results from external validation studies of prediction models. Study Design and Setting We propose to quantify the degree of relatedness between devel

__Objectives:__ To assess the impact of a clinical decision model for febrile children at risk for serious bacterial infections (SBI) attending the emergency department (ED). __Methods:__ Randomized controlled trial with 439 febrile children, aged 1 month-16 years, attending the pediatric ED of a Dutch university hospital during 2010-2012. Febrile children were randomly assigned t

Clinical prediction models provide risk estimates for the presence of disease (diagnosis) or an event in the future course of disease (prognosis) for individual patients. Although publications that present and evaluate such models are becoming more frequent, the methodology is often suboptimal. We propose that seven steps should be considered in developing prediction models: (i) consideration of the research question and initial data inspection; (ii) coding of predictors; (iii) model specification; (iv) model estimation; (v) evaluation of model performance; (vi) internal validation; and (vii) model presentation. The validity of a prediction model is ideally assessed in fully independent data, where we propose four key measures to evaluate model performance: calibration-in-the-large, or the model intercept (A); calibration slope (B); discrimination, with a concordance statistic (C); and clinical usefulness, with decision-curve analysis (D). As an application, we develop and validate prediction models for 30-day mortality in patients with an acute myocardial infarction. This illustrates the usefulness of the proposed framework to strengthen the methodological rigour and quality for prediction models in cardiovascular research.

Background: The discriminative ability of a risk model is often measured by Harrell's concordance-index (c-index). The c-index estimates for two randomly chosen subjects the probability that the model predicts a higher risk for the subject with poorer outcome (concordance probability). When data are clustered, as in multicenter data, two types of concordance are distinguished: concordance in subjects from the same cluster (within-cluster concordance probability) and concordance in subjects from different clusters (between-cluster concordance probability). We argue that the within-cluster concordance probability is most relevant when a risk model supports decisions within clusters (e.g. who should be treated in a particular center). We aimed to explore different approaches to estimate the within-cluster concordance probability in clustered data. Methods. We used data of the CRASH

Objectives: It is widely acknowledged that the performance of diagnostic and prognostic prediction models should be assessed in external validation studies with independent data from "different but related" samples as compared with that of the development sample. We developed a framework of methodological steps and statistical methods for analyzing and enhancing the interpretation of results from external validation studies of prediction models. Study Design and Se

Objective: To derive, cross validate, and externally validate a clinical prediction model that assesses the risks of different serious bacterial infections in children with fever at the emergency department. Design: Prospective observational diagnostic study. Setting: Three paediatric emergency care units: two in the Netherlands and one in the United Kingdom. Participants: Children with fever, aged 1 month to 15 years, at three paediatric emergency care units: Rotterdam (n=1750) and the Hague (n=967), the Netherlands, and Coventry (n=487), United Kingdom. A prediction model was constructed using multivariable polytomous logistic regression analysis and included the predefined predictor variables age, duration of fever, tachycardia, temperature, tachypnoea, ill appearance, chest wall retractions, prolonged capillary refill time (>3 seconds), oxygen saturation <94%, and C reactive protein. Main outcome measures: Pneumonia, other serious bacterial infections (SBIs, including septicaemia/meningitis, urinary tract infections, and others), and no SBIs. Results: Oxygen saturation <94% and presence of tachypnoea were important predictors of pneumonia. A raised C reactive protein level predicted the presence of both pneumonia and other SBIs, whereas chest wall retractions and oxygen saturation <94% were useful to rule out the presence of other SBIs. Discriminative ability (C statistic) to predict pneumonia was 0.81 (95% confidence interval 0.73 to 0.88); for other SBIs this was even better: 0.86 (0.79 to 0.92). Risk thresholds of 10% or more were useful to identify children with serious bacterial infections; risk thresholds less than 2.5% were useful to rule out the presence of serious bacterial infections. External validation showed good discrimination for the prediction of pneumonia (0.81, 0.69 to 0.93); discriminative ability for the prediction of other SBIs was lower (0.69, 0.53 to 0.86). Conclusion: A validated prediction model, including clinical signs, symptoms, and C

Background: Diagnostic and prognostic literature is overwhelmed with studies reporting univariable predictor-outcome associations. Currently, methods to incorporate such information in the construction of a prediction model are underdeveloped and unfamiliar to many researchers. Methods. This article aims to improve upon an adaptation method originally proposed by Greenland (1987) and Steyerberg (2000) to incorporate previously published univariable associations in the construction of a novel prediction model. The proposed method improves upon the variance estimation component by reconfiguring the adaptation process in established theory and making it more robust. Different variants of the proposed method were tested in a simulation study, where performance was measured by comparing estimated associations with their predefined values according to the Mean Squared Error and coverage of the 90% confidence intervals. Results: Results demonstrate that performance of estimated multivariable associations considerably improves for small datasets where external evidence is included. Although the error of estimated associations decreases with increasing amount of individual participant data, it does not disappear completely, even in very large datasets. Conclusions: The proposed method to aggregate previously published univariable associations with individual participant data in the construction of a novel prediction models outperforms established approaches and is especially worthwhile when relatively limited individual participant data are available.

Background: We investigated the reporting and methods of prediction studies, focusing on aims, designs, participant selection, outcomes, predictors, statistical power, statistical methods, and predictive performance measures. Methods and Findings: We used a full hand search to identify all prediction studies published in 2008 in six high impact general medical journals. We developed a comprehensive item list to systematically score conduct and reporting of the studies, based on recent recommendations for prediction research. Two reviewers independently scored the studies. We retrieved 71 papers for full text review: 51 were predictor finding studies, 14 were prediction model development studies, three addressed an external validation of a previously developed model, and three reported on a model's impact on participant outcome. Study design was unclear in 15% of studies, and a prospective cohort was used in most studies (60%). Descriptions of the participants and definitions of predictor and outcome were generally good. Despite many recommendations against doing so, continuous predictors were often dichotomized (32% of studies). The number of events per predictor as a measure of statistical power could not be determined in 67% of the studies; of the remainder, 53% had fewer than the commonly recommended value of ten events per predictor. Methods for a priori selection of candidate predictors were described in most studies (68%). A substantial number of studies relied on a p-value cut-off of p<0.05 to select predictors in the multivariable analyses (29%). Predictive model performance measures, i.e., calibration and discrimination, were reported in 12% and 27% of studies, respectively. Conclusions: The majority of prediction studies in high impact journals do not follow current methodological recommendations, limiting their reliability and applicability. Please see later in the article for the Editors' Summary.

OBJECTIVES: To validate externally 2 prognostic models for stillbirth and neonatal death in very preterm infants who are either known to be alive at the onset of labor or admitted for neonatal intensive care. PATIENTS AND METHODS: All infants, with gestational age 22 to 32 weeks, of European ethnicity, known to be alive at the onset of labor (n = 17 582) and admitted for neonatal intensive care (n = 11 578), who were born in the Netherlands between January 1, 2000, and December 31, 2007. The main outcome measures were stillbirth or death within 28 days for infants known to be alive at the onset of labor and death before discharge from the NICU for infants admitted for intensive care. Model performance was studied with calibration plots and c statistic. RESULTS: Of the infants known to be alive at the onset of labor, 16.7% (n = 2939) died during labor or within 28 days of birth, and 7.8% (n = 908) of the infants admitted for neonatal intensive care died before discharge from intensive care. The prognostic model for infants known to be alive at the onset of labor showed good calibration and excellent discrimination (c statistic 0.92). The prognostic model for infants admitted for neonatal intensive care showed good calibration and good discrimination (c statistic 0.82). CONCLUSIONS: The 2 prognostic models for stillbirth and neonatal death in very preterm Dutch infants showed good performance, suggesting their use in clinical practice in the Netherlands and possibly other Western countries. Copyright

Background: Several clinical risk scores (CRSs) for the outcome of patients with colorectal liver metastases have been validated, but not in patients undergoing neoadjuvant chemotherapy. Therefore, this study evaluates the predictive value of these CRSs in this specific group. Methods: Between January 2000 and December 2008, all patients undergoing a metastasectomy were analyzed and divided into two groups: 193 patients did not receive neoadjuvant chemotherapy (group A), and 159 patients received neoadjuvant chemotherapy (group B). In group B, the CRSs were calculated before and after administration of neoadjuvant chemotherapy. Results were evaluated by using the CRSs proposed by Nordlinger et al., Fong et al., Nagashima et al., and Konopke et al. Results: In groups A and B, the overall median survival was 43 and 47 months, respectively (P = 0.648). In group A, all CRSs used were of statistically significant predictive value. Before administration of neoadjuvant chemotherapy, only the Nordlinger score was of predictive value. After administration of neoadjuvant chemotherapy, all CRSs were of predictive value again, except for the Konopke score. Conclusions: Traditional CRSs are not a reliable prognostic tool when used in patients before treatment with neoadjuvant chemotherapy. However, CRSs assessed after the administration of neoadjuvant chemotherapy are useful to predict prognosis.

Various performance measures related to calibration and discrimination are available for the assessment of risk models. When the validity of a risk model is assessed in a new population, estimates of the model's performance can be influenced in several ways. The regression coefficients can be incorrect, which indeed results in an invalid model. However, the distribution of patient characteristics (case mix) may also influence the performance of the model. Here the authors consider a number of typical situations that can be encountered in external validation studies. Theoretical relations between differences in development and validation samples and performance measures are studied by simulation. Benchmark values for the performance measures are proposed to disentangle a case-mix effect from incorrect regression coefficients, when interpreting the model's estimated performance in validation samples. The authors demonstrate the use of the benchmark va

We assessed the external validity of a prediction rule for nonseminomatous testicular cancer patients. The rule was developed to predict the probability of retroperitoneal metastases being benign (only necrosis/fibrosis) after chemotherapy treatment. Patients with a high probability of benign residual masses might be offered surveillance as oppo

linical prediction models combine patient characteristics to predict the probability of having a certain disease (diagnosis) or the probability that a particular disease state will occur (prognosis). The predicted probability of the diagnostic or prognostic outcome may assist the clinician in decision making for patient care. Before a prediction model can reliably be applied in clinical practice, the performance of the model in new patients needs to be studied (‘external validity’). This thesis described several theoretical and practical aspects of the external validation of clinical prediction models. The objectives were (i) to describe aspects of model validity and relevant performance measures; (ii) to estimate the power of these performance measures; (iii) to externally validate a prediction model for residual mass histology in testicular cancer; and (iv) to update this model with all available information. Three aspects of model performance are discussed: calibration, discrimination, and clinical usefulness. The external validity of a model does not only depend on the new patients for whom the model is applied, but also on the development process of the model. Therefore, this thesis contained also some illustrations of model development aspects. The development process of the prediction model for residual mass histology in nonseminomatous testicular germ cell cancer was described; the model predicts the probability that a residual retroperitoneal mass contains only benign tissue after cis-platin based chemotherapy for metastatic tumour.