Search

This study was conducted to evaluate the performance of available tools predicting non-sentinel lymph node (non-SLN) status in women with SLN positive breast cancer and to see if they can be safely used in everyday clinical practice.Data of 220 women with breast cancer who underwent a SLN biopsy at the Máxima Medical Centre between 2000-2008 were analysed. Tools evaluated were: the models from Memorial Sloan Kettering Cancer Centre, Stanford, Mayo, Cambridge, Gur, and MOU, and the scores from Saidi, Tenon, and MDA. Model performance was assessed using calibration, discrimination and Nagelkerke's explained variation.The MSKCC nomogram showed best overall performance with best discrimination (AUC 0.69), second best calibration, and highest explained variation (31%). The 10% low risk threshold led to defining only 22% (38/176) of the women as being low risk while in fact 66% (116/176) were non-SLN negative. The false negative rate was 13% (5/38).Current models for predicting non-SLN metastases in SLN positive breast cancer are not yet ready for implementation in general practice. Further research efforts should improve model performance in selecting patients or perhaps find a role in support in the paradigm shift to a "treat none unless" approach.

Background: Patients with an advanced incurable disease are often hospitalised for some time during the last phase of life. Care in hospitals is generally focussed at curing disease and prolonging life and may therefore not in all cases adequately address the needs of such patients. We present the COMPASS study, a study on the effects and costs of consultation teams for palliative care in hospitals. This observational study aims to investigate the use, effects and costs of PCT consultation services for hospitalized patients with incurable cancer in the Netherlands. Methods/design: The study consists of 3 parts: 1. A questionnaire, interviews and a focus group discussion to investigate the characteristics of PCT consultation in 12 hospitals. PCTs will register their activities to calculate the costs of PCT consultation. 2. Cancer patients for whom the attending physician would not be surprised that they would die within 12 month will be included in a medical file search in three hospitals. Medical records will be investigated to compare care, treatment and hospital costs between patients with and patients without PCT consultation. 3. In the other nine hospitals, we will perform a longitudinal study, and compare quality of life between 100 patients for whom a PCT was consulted with 200 patients without PCT consultation. Propensity score matching will be used to adjust for differences between both patient groups. Patients will be followed for three months after inclusion. Quality of life will be assessed with the Palliative Outcome Scale, the EuroQol-5d and the EORTC-QLQ-C15 PAL. Satisfaction with care in the hospital is measured with the IN-PATSAT32. The cost impact of PCT consultation will also be explored. Discussion: This is the first multicenter study on PCT consultation in the Netherlands. The study will give valuable insight in the process, effects and costs of PCT consultation in hospitals. It is anticipated that PCT consultation has a positive effect on patients' quality of life and satisfaction with care and will lead to less hospital care costs.

Background The work ability index (WAI) is a frequently used tool in occupational health to identify workers at risk for a reduced work performance and for work-related disability. However, information about the prognostic value of the WAI to identify workers at risk for sickness absence is scarce. Objectives To investigate the prognostic value of the WAI for sickness absence, and whether the discriminative ability differs across demographic subgroups. Methods At baseline, the WAI (score 7-49) was assessed among 1,331 office workers from a Dutch financial service company. Sickness absence was registered during 12-months follow-up and categorised as 0 days, 0< days< 5, 5 = 15 days in one year. Associations between WAI and sickness absence were estimated by multinomial regression analyses. Discriminative ability of the WAI was assessed by the Area Under the Curve (AUC) and Ordinal c-index (ORC). Test characteristics were determined for dichotomised outcomes. Additional analyses were performed for separate WAI dimensions, and subgroup analyses for demographic groups. Results A lower WAI was associated with sickness absence (>= 15 days vs. 0 days: per point lower WAI score OR=1.27; 95% CI 1.21-1.33). The WAI showed reasonable ability to discriminate between categories of sickness absence (ORC=0.65; 95% CI 0.63-0.68). Highest discrimination was found for comparing workers with >= 15 sick days with 0 sick days (AUC=0.77) or with 1-5 sick days (AUC=0.69). At the cut-off for poor work ability (WAI = 15 sick days was 7.5%, the specificity 99.6%, and the positive predictive value 82%. The performance was similar across demographic subgroups. Conclusions The WAI could be used to identify workers at high risk for prolonged sickness absence. However, due to low sensitivity many workers will be missed. Hence, additional factors are required to better identify workers at highest risk.

One of the purposes to perform an oral food challenge (FC) test is to avoid unnecessary elimination of food allergens. In case of a negative FC test result, the food can be introduced. It is, however, unknown if patients act according to the outcome of the test. This study evaluates the rate of introduction of peanut, hazelnut, cow’s milk or hen’s egg allergens after a negative FC test. We investigated the introduction rate of children (0–18 years) with a negative FC test visiting the Department of Allergology, Erasmus Medical Centre Rotterdam from 2008 till 2013 and the factors that influence the rate of introduction. Patients were asked to complete a comprehensive questionnaire about their FC test. In total, 157 (38 % girls, mean age during challenge 6.9 years) participated in the study. Of these FC tests, 104 (56 %) were followed by a successful introduction, 30 (16 %) by a partly introduction (traces or processed foods) and 52 (28 %) by a failed introduction. Peanut and hazelnut showed a statistically significant lower successful introduction rate. Age, gender, symptoms during FC test, dietary advice and time period to introduction significantly influenced the rate of introduction. One fourth of the children with failure of introducing foods experienced symptoms during the introduction. Conclusion: More than one quarter of all children with a negative FC test result did not introduce the food. The FC test in its current form does not achieve its objective for this group of children. What is Known: • When the outcome of a food challenge test is negative, the food should be introduced in the diet of the child. • Failure of this introduction has negative consequences for the health of the child. What is New: • Failure of introduction of foods after a negative challenge test is reported in almost 25 % of the challenged children. • Failure of introduction after a negative challenge test is significant associated with gender, age, allergens, symptoms during OFC (according to the parents), advice, time start eating the food, and symptoms during introduction.

BackgroundClinical prediction rules (CPRs) to identify children with serious infections lack validation in low-prevalence populations, which hampers their implementation in primary care practice.AimTo evaluate the diagnostic value of published CPRs for febrile children in primary care.Design and settingObservational cohort study among febrile children (MethodAlarm signs of serious infection and clinical management were extracted from routine clinical practice data and manually recoded with a structured electronic data-entry program. Eight CPRs were selected from literature. CPR-variables were matched with alarm signs and CPRs were applied to the GPC-population. 'Referral to emergency department (ED)' was used as a proxy outcome measure for 'serious infection'. CPR performance was assessed by calibration analyses, sensitivity, specificity, and area under the ROC-curve (ROC-area).ResultsA total of 9794 GPC-contacts were eligible, 54% male, median age 2.3 years (interquartile range 1.0-4.6 years) and 8.1% referred to ED. Frequencies of CPR-variables varied from 0.5% (cyanosis, drowsy) to 25% (temperature = 40 degrees C). Alarm signs frequently included in CPRs were 'ill appearance', 'inconsolable', and 'abnormal circulatory or respiratory signs'. The height of the CPR's predicted risks generally corresponded with being (or not being) referred to the ED in practice. However, calibration-slopes indicated that three CPRs underestimated the risk of serious infection in the GPC-population. Sensitivities ranged from 42% to 54%, specificities from 68% to 89%. ROC-areas ranged from 0.52 to 0.81, with best performance of CPRs for children agedConclusionPublished CPRs performed moderately well in the primary out-of-hours care population. Advice is given on how to improve translation of CPRs to primary care practice.

Objectives: We aimed to compare nonlinear modeling methods for handling continuous predictors for reproducibility and transportability of prediction models. Study Design and Setting: We analyzed four cohorts of previously unscreened men who underwent prostate biopsy for diagnosing prostate cancer. Continuous predictors of prostate cancer included prostate-specific antigen and prostate volume. The logistic regression models included linear terms, logarithmic terms, fractional polynomials of degree one or two (FP1 and FP2), or restricted cubic splines (RCS) with three or five knots (RCS3 and RCS5). The resulting models were internally validated by bootstrap resampling and externally validated in the cohorts not used at model development. Performance was assessed with the area under the receiver operating characteristic curve (AUC) and the calibration component of the Brier score (CAL). Results: At internal validation models with FP2 or RCS5 showed slightly better performance than the other models (typically 0.004 difference in AUC and 0.001 in CAL). At external validation models containing logarithms, FPI, or RCS3 showed better performance (differences 0.01 and 0.002). Conclusion: Flexible nonlinear modeling methods led to better model performance at internal validation. However, when application of the model is intended across a wide range of settings, less flexible functions may be more appropriate to maximize external validity. (C) 2015 Elsevier Inc. All rights reserved.

Objectives To assess the impact of a clinical decision model for febrile children at risk for serious bacterial infections (SBI) attending the emergency department (ED). Methods Randomized controlled trial with 439 febrile children, aged 1 month-16 years, attending the pediatric ED of a Dutch university hospital during 2010-2012. Febrile children were randomly assigned to the intervention (clinical decision model; n=219) or the control group (usual care; n=220). The clinical decision model included clinical symptoms, vital signs, and C-reactive protein and provided high/low-risks for “pneumonia” and “other SBI”. Nurses were guided by the intervention to initiate additional tests for high-risk children. The clinical decision model was evaluated by 1) area-under-the-receiver-operating-characteristic-curve (AUC) to indicate discriminative ability and 2) feasibility, to measure nurses’ compliance to model recommendations. Primary patient outcome was defined as correct SBI diagnoses. Secondary process outcomes were defined as length of stay; diagnostic tests; antibiotic treatment; hospital admission; revisits and medical costs. Results The decision model had good discriminative ability for both pneumonia (n=33; AUC 0.83 (95% CI 0.75-0.90)) and other SBI (n=22; AUC 0.81 (95% CI 0.72-0.90)). Compliance to model recommendations was high (86%). No differences in correct SBI determination were observed. Application of the clinical decision model resulted in less full-blood-counts (14% vs. 22%, p-value

When patients die relatives and healthcare professionals may appreciate the quality of the dying phase differently, but comparisons are rare. In a cross-sectional study (June 2009–July 2012) the experiences of bereaved relatives, physicians, and nurses concerning the quality of dying in a large Dutch university hospital were compared, and the relation to communication was explored. Measurements were concordance on the quality of dying (QOD) (0–10 scale), awareness of impending death, and end-of-life communication. Results. Data on all three perspectives were available for 200 patients. Concordance in general was poor. Relatives’ scores for QOD (median 7; IQR 5–8) were lower than physicians and nurses’ (both median 7; IQR 6–8) (P=0.002). 48% of the relatives, 77% of the physicians, and 73% of the nurses had been aware of impending death. Physicians more often reported to have informed patients and relatives of end-of-life issues than relatives reported. When both physicians and relatives reported about such discussion, relatives’ awareness of impending death and presence at the patient’s deathbed were more likely. Conclusion. Relatives, physicians, and nurses seem to have their “own truth” about the dying phase. Professionals should put more emphasis on the collaboration with relatives and on verification of relative’s understanding.

Summary Objective To describe treatment and outcome of epilepsy in children with tuberous sclerosis complex (TSC). Methods Seventy-one children with TSC and epilepsy treated at the ENCORE TSC Expertise Center between 1988 and 2014 were included. Patient characteristics and duration and effectiveness of antiepileptic treatments were extracted from our clinical database. Correlations were made between recurrence of seizures after response to treatment, and several patient characteristics. Results Median age at time of inclusion was 9.4 years (range 0.9–18.0). Seizure history showed that 55 children (77%) of 71 became seizure-free for longer than 1 month, and 21 (30%) of 71 for longer than 24 months. Remission of seizures was associated with higher IQ, and a trend was observed between seizure remission and age at onset of seizures. A total of 19 antiepileptic drugs (AEDs) were used. Valproic acid, vigabatrin, levetiracetam, and carbamazepine were used most frequently. Nonpharmacologic therapies (ketogenic diet, epilepsy surgery, and vagus nerve stimulation) were used 13 times. Epilepsy surgery was most effective, with four of five children becoming seizure-free. AEDs prescribed as first and second treatment were most effective. Valproic acid was prescribed most frequently as first and second treatment, followed by vigabatrin. Thirty-one children had infantile spasms, preceded by focal seizures in 18 children (58%). Vigabatrin was used by 29 children (94%), and was first treatment in 15 (48%). Vigabatrin was more effective than other AEDs when prescribed as first treatment. Significance We showed that, although 77% of children with epilepsy due to TSC reached seizure remission, usually after their first or second AED, this was sustained for at least 24 months in only 38%. Almost half of those with 24 months of remission later had relapse of seizures. Our results support vigabatrin as first choice drug, and show the need for better treatment options for these children.

Recently, a prediction rule for developing neurological sequelae after childhood bacterial meningitis was developed on a small derivation set. Before implementing in practice a prediction rule must first be tested in new patients (external validation). Our aim was to study the external validity of this rule and, if necessary, to update the rule. The prediction rule was tested on newly available data (validation set) by assessing the rule's calibration and discrimination. We updated the prediction rule by adding extra predictors and re-estimating the regression coefficients of the original predictors in the combined datasets. The rule showed poor agreement between predicted risks and observed frequencies. The ROC area was 0.65 (95% CI 0.57-0.72), which was statistically significantly lower than in the derivation set (0.87 (0.78-0.96)), p-value

We have validated two scoring systems for predicting postoperative nausea and vomiting, derived by Apfel et al. and Koivuranta et al. from 1388 adult inpatients undergoing a wide range of surgical procedures. The predictive accuracy of the scoring systems was evaluated in terms of the ability to discriminate between patients with and without postoperative nausea and vomiting (discrimination) and agreement between observed and predicted outcomes (calibration). Discrimination and calibration were less than expected based on previous reports, with both scoring systems providing risk predictions that were too extreme. The area under the ROC curve was 0.63 for Apfel et al.'s scoring system and 0.66 for Koivuranta et al.'s scoring system. Neither of the scoring systems provided a risk threshold for administering anti-emetic prophylaxis that yielded satisfying results in terms of predictive values, sensitivity and specificity. Hence, in their original forms, the scoring systems do not guarantee accurate prediction of the risk of postoperative nausea and vomiting in other patient populations. Koivuranta et al.'s scoring system appears to be more robust across different populations.

Background and Objectives: The performance of a prediction model is usually worse in external validation data compared to the development data. We aimed to determine at which effective sample sizes (i.e., number of events) relevant differences in model performance can be detected with adequate power. Methods: We used a logistic regression model to predict the probability that residual masses of patients treated for metastatic testicular cancer contained only benign tissue. We performed standard power calculations and Monte Carlo simulations to estimate the numbers of events that are required to detect several types of model invalidity with 80% power at the 5% significance level. Results: A validation sample with 111 events was required to detect that a model predicted too high probabilities, when predictions were on average 1.5 times too high on the odds scale. A decrease in discriminative ability of the model, indicated by a decrease in the c-statistic from 0.83 to 0.73, required 81 to 106 events, depending on the specific scenario. Conclusion: We suggest a minimum of 100 events and 100 nonevents for external validation samples. Specific hypotheses may, however, require substantially higher effective sample sizes to obtain adequate power.

Reliable estimates of hospitalization costs for severe respiratory syncytial virus (RSV) infection are necessary to perform economic analyses of preventive strategies of severe RSV disease. We aimed to develop a model that predicts anticipated mean RSV hospitalization costs of groups of young children at risk for hospitalization, but not yet hospitalized, based on readily available child characteristics.We determined real direct medical costs of RSV hospitalization from a societal perspective, using a bottom-up strategy, in 3458 infants and young children hospitalized for severe RSV disease during the RSV seasons 1996-1997 to 1999-2000 in the Southwest of the Netherlands. We used a linear regression model to predict anticipated mean RSV hospitalization costs of groups of children at risk, based on 4 child characteristics [age, gestational age, birth weight and bronchopulmonary dysplasia (BPD)], expressed in EC Euros as of the year 2000.The mean RSV hospitalization costs of all patients were 3110 Euros. RSV hospitalization costs were higher for patients with lower gestational age (5555 Euros; gestational age,/=28 weeks), lower birth weight (3895 Euros; birth weight/=2500 g), BPD (5785 Euros; with BPD) and young age (4730 Euros; first month of life). The linear regression model had an adjusted R of 0.08. This indicates a low explanatory ability for hospitalization costs of individual children. However, the model could accurately estimate the anticipated mean hospitalization costs of groups of children with the same characteristics.RSV hospitalization costs were substantial, especially of specific high risk groups. Anticipated mean hospitalization costs of groups of children at risk for RSV hospitalization, but not yet hospitalized, could well be estimated with 4 child characteristics (age, gestational age, birth weight and BPD). These estimated costs can be used for economic analyses of preventive strategies for severe RSV disease.

Introduction The sFlt-1/PlGF ratio has emerged as a biomarker for the timely diagnosis of preeclampsia and prediction of pregnancy outcomes. In most reports the ratio has been compared with the traditional criteria (proteinuria and hypertension) of PE instead of assessment of its additional value. Objectives To assess whether a single determination of the serum values of sFlt-1, PlGF and their ratio improves prediction of pregnancy prolongation in days and maternal and fetal complications in women with suspected or clinically confirmed PE. Methods In this ongoing Dutch multicenter study (600 pts to be enrolled) blood was drawn on admission for suspected or clinically confirmed preeclampsia. Using the automated Elecsys system values of sFlt-1 and PlGF were measured postpartum to prevent influence of this information on decision-making of the treating physicians. Clinical characteristics and pregnancy outcomes were retrieved from medical records. Maternal complications included HELLP syndrome, cerebral/ pulmonary edema, acute renal failure, stroke or hemorrhagic problems. Fetal complications included prematurity, intrauterine growth restriction, admission to neonatal intensive care, infant respiratory distress syndrome and necrotizing enterocolitis. The added value of sFlt-1, PlGF and their ratio on already known predictors as maternal and gestational age, gravidity, blood pressure, proteinuria and diagnosis at inclusion was determined with multivariable regression analysis. Results Complete data of 472 patients (age 18–48 yrs., singleton pregnancies, median pregnancy duration 34 weeks (range 20–41 weeks) were available. Adverse outcome occurred in approximately 99 (21%) of the women and in 188 (40%) of the fetus. The biomarkers sFlt-, PlGF and their ratio showed clear additive value for the prediction of pregnancy prolongation and maternal and fetal complications. Compared to the sFlt-1 and PlGF values the additive value of the ratio to predict fetal complications, was two-fold higher. Prediction of pregnancy prolongation was most improved by PlGF. Conclusions The biomarkers sFlt-1, PlGF and sFlt-1/PlGF ratio are strong predictors for maternal and fetal complications and pregnancy prolongation. Particularly the sFlt-1/PlGF ratio has much added value for the prediction of fetal complications and pregnancy prolongation.

Purpose: Patients with clinical stage I nonseminomatous testicular germ cell tumor should ideally receive adjuvant therapy only when they are at high risk for occult metastasis. We aimed to quantify the importance of predictors for occult metastasis by performing a systematic review of the relevant literature. In addition, we reviewed published multivariable models and risk-adapted treatment policies. Patients and Methods: We identified 23 publications between 1979 and 2001, reporting a total of 2,587 patients. Twenty-nine percent of the patients (759 of 2,587 patients) had occult metastases, which was diagnosed either at retroperitoneal lymph node dissection (n = 193) or during follow-up (n = 566). Odds ratios (OR) were pooled using meta-analysis techniques. Results: The presence of vascular invasion of the primary tumor cells had the strongest effect (OR, 5.2; 95% CI, 4.0 to 6.8). Immunohistochemical staining of the primary tumor cells with the MIB-1 monoclonal antibody showing proliferative activity was a promising predictor (OR, 4.7; 95% CI, 2.0 to 11). Intermediate effects were found for embryonal carcinoma in the primary tumor (OR, 2.9; 95% CI, 2.0 to 4.4) and a high pathologic stage of the tumor (OR, 2.6; 95% CI, 1.8 to 3.8). Size of the primary tumor and age of the patient had weaker though also statistically significant associations with occult metastasis. Until now, multivariable models often included vascular invasion and embryonal carcinoma with one or two weaker predictors. None of the published risk-adapted treatment policies included MIB-1 staining. Conclusion: Several strong predictors for occult metastasis were identified. A risk-adapted treatment policy should be developed that incorporates all relevant predictors so that adjuvant therapy is targeted better to those with occult metastases.

We assessed the external validity of a prediction rule for nonseminomatous testicular cancer patients. The rule was developed to predict the probability of retroperitoneal metastases being benign (only necrosis/fibrosis) after chemotherapy treatment. Patients with a high probability of benign residual masses might be offered surveillance as opposed to patients with a low probability, who should undergo retroperitoneal lymph node dissection (RPLND). We compared the observed histology with the predicted probability in 105 patients with good prognosis germ cell cancer who underwent RPLND between 1995 and 1998. We found that predicted probabilities higher than 5% were in good agreement with the observed frequencies of benign masses. The area under the receiver operating characteristic curve was 0.76, suggesting that the rule could reasonably discriminate between benign masses and tumour. However, nearly all predicted probabilities (n = 10 1) were lower than 70%, which might be considered as the lowest value at which surveillance offers a reasonable alternative to RPLND. Further, 35% of patients currently under surveillance (84 out of 24 1) had predicted probabilities lower than 70%. In conclusion, the clinical relevance of the prediction rule was limited for the patients who underwent RPLND; use of the rule would change the policy from RPLND to surveillance in only a few. On the other hand, the rule might support selection of patients for RPLND, who currently are under surveillance. (C) 2003 Cancer Research UK.

Clinical prediction models provide risk estimates for the presence of disease (diagnosis) or an event in the future course of disease (prognosis) for individual patients. Although publications that present and evaluate such models are becoming more frequent, the methodology is often suboptimal. We propose that seven steps should be considered in developing prediction models: (i) consideration of the research question and initial data inspection; (ii) coding of predictors; (iii) model specification; (iv) model estimation; (v) evaluation of model performance; (vi) internal validation; and (vii) model presentation. The validity of a prediction model is ideally assessed in fully independent data, where we propose four key measures to evaluate model performance: calibration-in-the-large, or the model intercept (A); calibration slope (B); discrimination, with a concordance statistic (C); and clinical usefulness, with decision-curve analysis (D). As an application, we develop and validate prediction models for 30-day mortality in patients with an acute myocardial infarction. This illustrates the usefulness of the proposed framework to strengthen the methodological rigour and quality for prediction models in cardiovascular research.

Prediction models fitted with logistic regression often show poor performance when applied in populations other than the development population. Model updating may improve predictions. Previously suggested methods vary in their extensiveness of updating the model. We aim to define a strategy in selecting an appropriate update method that considers the balance between the amount of evidence for updating in the new patient sample and the danger of overfitting. We consider recalibration in the large (re-estimation of model intercept); recalibration (re-estimation of intercept and slope) and model revision (re-estimation of all coefficients) as update methods. We propose a closed testing procedure that allows the extensiveness of the updating to increase progressively from a minimum (the original model) to a maximum (a completely revised model). The procedure involves multiple testing with maintaining approximately the chosen type I error rate. We illustrate this approach with three clinical examples: patients with prostate cancer, traumatic brain injury and children presenting with fever. The need for updating the prostate cancer model was completely driven by a different model intercept in the update sample (adjustment: 2.58). Separate testing of model revision against the original model showed statistically significant results, but led to overfitting (calibration slope at internal validation = 0.86). The closed testing procedure selected recalibration in the large as update method, without overfitting. The advantage of the closed testing procedure was confirmed by the other two examples. We conclude that the proposed closed testing procedure may be useful in selecting appropriate update methods for previously developed prediction models. Copyright © 2016 John WileySons, Ltd.

C-reactive protein (CRP) and procalcitonin (PCT) are useful diagnostic tools to estimate the risk of serious bacterial infection (SBI) in febrile children at the emergency department (ED). The Lab-score combines these 2 biomarkers with urinalysis in an easy to use validated model. Kinetics of inflammatory markers suggests a differentiating role of duration of disease.: Appraisal of the diagnostic role of CRP and PCT in febrile children at risk of SBI, determining the differentiating value of duration of fever, and validating and updating the Lab-score.In this prospective observational study previously healthy children with fever, 1 month to 16 years of age, attending the EDs of a university hospital and a teaching hospital (Rotterdam, the Netherlands) between 2009 and 2012 were included. Standardized information on clinical signs and symptoms, CRP, PCT and urinalysis were collected prospectively. Logistic multivariable regression analysis was used to assess diagnostic performance. The original Lab-score included CRP, PCT and urinalysis and the total score ranged 0-9 points.One thousand eighty-four children were included, median age was 1.6 years (interquartile range: 0.8-3.5), 170 children (16%) had SBI. CRP [receiver operating characteristic (ROC)-area 0.77 (95% confidence interval [CI]: 0.69-0.85)] and PCT [ROC-area 0.75 (95% CI: 0.67-0.83)] were both strong predictors of SBI. Duration of fever had no added diagnostic value to CRP and PCT. The Lab-score performed well [ROC area 0.79 (95% CI: 0.72-0.87)], but threshold values performed similar to often used cutoffs of single biomarkers. An updated Lab-score improved only moderately [ROC area 0.83 (95% CI: 0.76-0.90)]. PCT did not alter post-test probabilities for SBI substantially in patients with low (20 mg/L) or elevated CRP (≥ 100 mg/L) levels (67% of population).CRP and PCT were both strong predictors of SBI. The original and updated Lab-score performed well, but thresholds values lacked diagnostic value for ruling out SBI. Depending on clinical risk thresholds, diagnostic testing can be limited to CRP or PCT, rather than both, in many febrile children.

Objectives This study aimed to update the Logistic Clinical SYNTAX score to predict 3-year survival after percutaneous coronary intervention (PCI) and compare the performance with the SYNTAX score alone. Background The SYNTAX score is a well-established angiographic tool to predict long-term outcomes after PCI. The Logistic Clinical SYNTAX score, developed by combining clinical variables with the anatomic SYNTAX score, has been shown to perform better than the SYNTAX score alone in predicting 1-year outcomes after PCI. However, the ability of this score to predict long-term survival is unknown. Methods Patient-level data (N = 6,304, 399 deaths within 3 years) from 7 contemporary PCI trials were analyzed. We revised the overall risk and the predictor effects in the core model (SYNTAX score, age, creatinine clearance, and left ventricular ejection fraction) using Cox regression analysis to predict mortality at 3 years. We also updated the extended model by combining the core model with additional independent predictors of 3-year mortality (i.e., diabetes mellitus, peripheral vascular disease, and body mass index). Results The revised Logistic Clinical SYNTAX models showed better discriminative ability than the anatomic SYNTAX score for the prediction of 3-year mortality after PCI (c-index: SYNTAX score, 0.61; core model, 0.71; and extended model, 0.73 in a cross-validation procedure). The extended model in particular performed better in differentiating low-and intermediate-risk groups. Conclusions Risk scores combining clinical characteristics with the anatomic SYNTAX score substantially better predict 3-year mortality than the SYNTAX score alone and should be used for long-term risk stratification of patients undergoing PCI. (C) 2014 by the American College of Cardiology Foundation

Background: The discriminative ability of a risk model is often measured by Harrell's concordance-index (c-index). The c-index estimates for two randomly chosen subjects the probability that the model predicts a higher risk for the subject with poorer outcome (concordance probability). When data are clustered, as in multicenter data, two types of concordance are distinguished: concordance in subjects from the same cluster (within-cluster concordance probability) and concordance in subjects from different clusters (between-cluster concordance probability). We argue that the within-cluster concordance probability is most relevant when a risk model supports decisions within clusters (e.g. who should be treated in a particular center). We aimed to explore different approaches to estimate the within-cluster concordance probability in clustered data. Methods: We used data of the CRASH trial (2,081 patients clustered in 35 centers) to develop a risk model for mortality after traumatic brain injury. To assess the discriminative ability of the risk model within centers we first calculated cluster-specific c-indexes. We then pooled the cluster-specific c-indexes into a summary estimate with different meta-analytical techniques. We considered fixed effect meta-analysis with different weights (equal; inverse variance; number of subjects, events or pairs) and random effects meta-analysis. We reflected on pooling the estimates on the log-odds scale rather than the probability scale. Results: The cluster-specific c-index varied substantially across centers (IQR = 0.70-0.81; I-2 = 0.76 with 95% confidence interval 0.66 to 0.82). Summary estimates resulting from fixed effect meta-analysis ranged from 0.75 (equal weights) to 0.84 (inverse variance weights). With random effects meta- analysis - accounting for the observed heterogeneity in c-indexes across clusters - we estimated a mean of 0.77, a between-cluster variance of 0.0072 and a 95% prediction interval of 0.60 to 0.95. The normality assumptions for derivation of a prediction interval were better met on the probability than on the log-odds scale. Conclusion: When assessing the discriminative ability of risk models used to support decisions at cluster level we recommend meta- analysis of cluster-specific c-indexes. Particularly, random effects meta- analysis should be considered.

Item does not contain fulltext STUDY QUESTION: How well does the recently developed UK model predicting the success rate of IVF treatment (the 2011 Nelson model) perform in comparison with a UK model developed in the early 1990s (the Templeton model)? SUMMARY ANSWER: Both models showed similar performance, after correction for the increasing success rate over time of IVF. WHAT IS KNOWN ALREADY: For counselling couples undergoing IVF treatment it is of paramount importance to be able to predict success. Several prediction models for the chance of success after IVF treatment have been developed. So far, the Templeton model has been recommended as the best approach after having been validated in several independent patient data sets. The Nelson model, developed in 2011 and characterized by the largest development sample containing the most recently treated couples, may well perform better. STUDY DESIGN, SIZE, DURATION: We tested both models in couples that were included in a national cohort study carried out in the Netherlands between the beginning of January 2002 and the end of December 2004. PARTICIPANTS/MATERIALS, SETTING, METHODS: We analysed the IVF cycles of Dutch couples with primary infertility (n = 5176). The chance of success was calculated using the two UK models that had been developed using the information collected in the Human Fertilisation and Embryology Authority database. Women were treated in 1991-1994 (Templeton) or 2003-2007 (Nelson). The outcome of success for both UK models is the occurrence of a live birth after IVF but the outcome in the Dutch data is an ongoing pregnancy. In order to make the outcomes compatible, we used a factor to convert the chance of live birth to ongoing pregnancy and use the overall terms 'success or no success after IVF'. The discriminative ability and the calibration of both models were assessed, the latter before and after adjustment for time trends in IVF success rates. MAIN RESULTS AND THE ROLE OF CHANCE: The two models showed a similarly limited degree of discriminative ability on the tested data (area under the receiver operating characteristic curve 0.597 for the Templeton model and 0.590 for the Nelson model). The Templeton model underestimated the success rate (observed 21% versus predicted 14%); the Nelson model overestimated the success rate (observed 21% versus predicted 29%). When the models were adjusted for the changing success rates over time, the calibration of both models considerably improved (Templeton observed 21% versus predicted 20%; Nelson observed 21% versus predicted 24%). LIMITATIONS, REASONS FOR CAUTION: We could only test the models in couples with primary infertility because detailed information on secondary infertile couples was lacking in the Dutch data. This shortcoming may have negatively influenced the performance of the Nelson model. WIDER IMPLICATIONS OF THE FINDINGS: The changes in success rates over time should be taken into account when assessing prediction models for estimating the success rate of IVF treatment. In patients with primary infertility, the choice to use the Templeton or Nelson model is arbitrary.

Recurrent abdominal pain (RAP) in children is generally believed to be functional. In practice, many children with RAP become pain-free with laxative therapy. The aims of the study were to establish the role of (occult) constipation in RAP and to investigate whether patients diagnosed with (occult) constipation could be identified by history and physical examination. During 2 years, all patients (age 4-16 years, secondary referral) fulfilling Apley criteria of RAP were included. After exclusion of gastrointestinal infections and food intolerance, laxatives were advised when pain persisted. (Occult) constipation was defined as 'abdominal pain disappearing with laxative treatment and not reappearing within a 6 month follow up period'; 'occult constipation' was diagnosed in patients who did not fulfil the Rome criteria of constipation. Two hundred children (87 M; median age 8.8 years) were evaluated. (Occult) constipation was found in 92 patients (46 %). Of these, 18 had considerable relief of pain when treated for a somatic cause but experienced complete relief only after laxative measures; they were considered to have two diagnoses. Using multivariate analysis, a simple model was developed with cystitis in past history, early satiety and flatulence as predictors for (occult) constipation. The risk of (occult) constipation ranged from 18/58 if no predictor was present to 4/4 if all three were present. Conclusion: Laxatives played a pivotal role in the recovery of patients with RAP. We developed a simple model to identify patients at risk of having (occult) constipation.

Aims/hypothesis Type 1 diabetes is associated with a higher risk of major vascular complications and death. A reliable method that predicted these outcomes early in the disease process would help in risk classification. We therefore developed such a prognostic model and quantified its performance in independent cohorts. Methods Data were analysed from 1,973 participants with type 1 diabetes followed for 7 years in the EURODIAB Prospective Complications Study. Strong prognostic factors for major outcomes were combined in a Weibull regression model. The performance of the model was tested in three different prospective cohorts: the Pittsburgh Epidemiology of Diabetes Complications study (EDC, n¿=¿554), the Finnish Diabetic Nephropathy study (FinnDiane, n¿=¿2,999) and the Coronary Artery Calcification in Type 1 Diabetes study (CACTI, n¿=¿580). Major outcomes included major CHD, stroke, end-stage renal failure, amputations, blindness and all-cause death. Results A total of 95 EURODIAB patients with type 1 diabetes developed major outcomes during follow-up. Prognostic factors were age, HbA1c, WHR, albumin/creatinine ratio and HDL-cholesterol level. The discriminative ability of the model was adequate, with a concordance statistic (C-statistic) of 0.74. Discrimination was similar or even better in the independent cohorts, the C-statistics being: EDC, 0.79; FinnDiane, 0.82; and CACTI, 0.73. Conclusions/interpretation Our prognostic model, which uses easily accessible clinical features can discriminate between type 1 diabetes patients who have a good or a poor prognosis. Such a prognostic model may be helpful in clinical practice and for risk stratification in clinical trials.

The performance of a predictive model is overestimated when simply determined on the sample of subjects that was used to construct the model. Several internal validation methods are available that aim to provide a more accurate estimate of model performance in new subjects. We evaluated several variants of split-sample, cross-validation and bootstrapping methods with a logistic regression model that included eight predictors for 30-day mortality after an acute myocardial infarction. Random samples with a size between n = 572 and n = 9165 were drawn from a large data set (GUSTO-I; n = 40,830; 2851 deaths) to reflect modeling in data sets with between 5 and 80 events per variable. Independent performance was determined on the remaining subjects. Performance measures included discriminative ability, calibration and overall accuracy. We found that split-sample analyses gave overly pessimistic estimates of performance, with large variability. Cross-validation on 10% of the sample had low bias and low variability, but was not suitable for all performance measures. Internal validity could best be estimated with bootstrapping, which provided stable estimates with low bias. We conclude that split-sample validation is inefficient, and recommend bootstrapping for estimation of internal validity of a predictive logistic regression model.

After chemotherapy for metastatic non-seminomatous testicular cancer, surgical resection is a generally accepted treatment to remove remnants of the initial metastases, since residual tumour may still be present (mature teratoma or viable cancer cells). In this paper, we review the development and external validation of a logistic regression model to predict the absence of residual tumour. Three sources of information were used. A quantitative review identified six relevant predictors from 19 published studies (996 resections). Second, a development data set included individual data of 544 patients from six centres. This data set was used to assess the predictive relationships of five continuous predictors, which resulted in dichotomization for two, and a log, square root, and linear transformation for three other predictors. The multiple logistic regression coefficients were reduced with a shrinkage factor (0.95) to improve calibration, based on a bootstrapping procedure. Third, a validation data set included 172 more recently treated patients. The model showed adequate calibration and good discrimination in the development and in the validation sample (areas under the ROC curve 0.83 and 0.82). This study illustrates that a careful modelling strategy may result in an adequate predictive model. Further study of model validity may stimulate application in clinical practice.

This study aimed to update the Logistic Clinical SYNTAX score to predict 3-year survival after percutaneous coronary intervention (PCI) and compare the performance with the SYNTAX score alone.The SYNTAX score is a well-established angiographic tool to predict long-term outcomes after PCI. The Logistic Clinical SYNTAX score, developed by combining clinical variables with the anatomic SYNTAX score, has been shown to perform better than the SYNTAX score alone in predicting 1-year outcomes after PCI. However, the ability of this score to predict long-term survival is unknown.Patient-level data (N = 6,304, 399 deaths within 3 years) from 7 contemporary PCI trials were analyzed. We revised the overall risk and the predictor effects in the core model (SYNTAX score, age, creatinine clearance, and left ventricular ejection fraction) using Cox regression analysis to predict mortality at 3 years. We also updated the extended model by combining the core model with additional independent predictors of 3-year mortality (i.e., diabetes mellitus, peripheral vascular disease, and body mass index).The revised Logistic Clinical SYNTAX models showed better discriminative ability than the anatomic SYNTAX score for the prediction of 3-year mortality after PCI (c-index: SYNTAX score, 0.61; core model, 0.71; and extended model, 0.73 in a cross-validation procedure). The extended model in particular performed better in differentiating low- and intermediate-risk groups.Risk scores combining clinical characteristics with the anatomic SYNTAX score substantially better predict 3-year mortality than the SYNTAX score alone and should be used for long-term risk stratification of patients undergoing PCI.

Clinical prediction models have been shown to have moderate sensitivity and specificity, yet their use will depend on implementation in clinical practice. The authors hypothesized that implementation of a prediction model for postoperative nausea and vomiting (PONV) would lower the PONV incidence by stimulating anesthesiologists to administer more "risk-tailored" prophylaxis to patients.A single-center, cluster-randomized trial was performed in 12,032 elective surgical patients receiving anesthesia from 79 anesthesiologists. Anesthesiologists were randomized to either exposure or nonexposure to automated risk calculations for PONV (without patient-specific recommendations on prophylactic antiemetics). Anesthesiologists who treated less than 50 enrolled patients were excluded during the analysis to avoid too small clusters, yielding 11,613 patients and 57 anesthesiologists (intervention group: 5,471 and 31; care-as-usual group: 6,142 and 26). The 24-h incidence of PONV (primary outcome) and the number of prophylactic antiemetics administered per patient were studied for risk-dependent differences between allocation groups.There were no differences in PONV incidence between allocation groups (crude incidence intervention group 41%, care-as-usual group 43%; odds ratio, 0.97; 95% CI, 0.87-1.1; risk-dependent odds ratio, 0.92; 95% CI, 0.80-1.1). Nevertheless, intervention-group anesthesiologists administered more prophylactic antiemetics (rate ratio, 2.0; 95% CI, 1.6-2.4) and more risk-tailored than care-as-usual-group anesthesiologists (risk-dependent rate ratio, 1.6; 95% CI, 1.3-2.0).Implementation of a PONV prediction model did not reduce the PONV incidence despite increased antiemetic prescription in high-risk patients by anesthesiologists. Before implementing prediction models into clinical practice, implementation studies that include patient outcomes as an endpoint are needed.

Calibration, that is, whether observed outcomes agree with predicted risks, is important when evaluating risk prediction models. For dichotomous outcomes, several tools exist to assess different aspects of model calibration, such as calibration-in-the-large, logistic recalibration, and (non-)parametric calibration plots. We aim to extend these tools to prediction models for polytomous outcomes. We focus on models developed using multinomial logistic regression (MLR): outcome Y with k categories is predicted using k - 1 equations comparing each category i (i = 2, … ,k) with reference category 1 using a set of predictors, resulting in k - 1 linear predictors. We propose a multinomial logistic recalibration framework that involves an MLR fit where Y is predicted using the k - 1 linear predictors from the prediction model. A non-parametric alternative may use vector splines for the effects of the linear predictors. The parametric and non-parametric frameworks can be used to generate multinomial calibration plots. Further, the parametric framework can be used for the estimation and statistical testing of calibration intercepts and slopes. Two illustrative case studies are presented, one on the diagnosis of malignancy of ovarian tumors and one on residual mass diagnosis in testicular cancer patients treated with cisplatin-based chemotherapy. The risk prediction models were developed on data from 2037 and 544 patients and externally validated on 1107 and 550 patients, respectively. We conclude that calibration tools can be extended to polytomous outcomes. The polytomous calibration plots are particularly informative through the visual summary of the calibration performance.

Phlebectomy of varicose tributaries is usually considered an additional treatment after or during saphenous ablation. As phlebectomies alone affect the hemodynamics of the venous system, this treatment can be effective as primary intervention in selected patients. The objective of this study was to analyze hemodynamic, clinical, and patient-reported outcomes after phlebectomies in a prospective multicenter study to determine predictors for treatment success, that is, restoration of great saphenous vein (GSV) competence.Patients with symptomatic GSV and tributary incompetence (reflux0.5 second) at the level of the thigh were included. Duplex ultrasound (DUS) was used to assess GSV and tributary characteristics, and a reflux elimination test was performed. Three and 12 months after phlebectomy of the tributary, reflux and GSV diameter were evaluated with DUS. Clinical outcome measures were C class of the Clinical, Etiologic, Anatomic, and Pathologic (CEAP) classification and Venous Clinical Severity Score; patients' reported outcome was determined by the Aberdeen Varicose Vein Questionnaire. To evaluate differences between the success and failure groups, baseline DUS characteristics, Venous Clinical Severity Score, CEAP class, and Aberdeen Varicose Vein Questionnaire score were compared. Multivariable logistic regression including all clinically relevant variables following a backward variable elimination process was used to determine predictors for success. The model was internally validated by 1000 bootstrap samples.The study included 94 patients (65 women, 29 men) with a mean age of 53 years. The majority had C2 or C3 disease. One year after treatment, GSV reflux had disappeared in 50% of patients (P .01), and GSV diameter had decreased significantly (P .01). Clinical outcome and Aberdeen Varicose Vein Questionnaire score improved significantly (P .01) and symptoms had disappeared in 66%. Of 47 patients with persisting GSV incompetence, 15 did not receive additional treatment because they were asymptomatic. Independent predictors for success were low C class of the CEAP classification, low number of refluxing GSV segments, small diameter of the GSV above the tributary, and positive reflux elimination test result (P .0001). The reflux elimination test appeared to be an important independent predictor, with65% chance of success when the result was positive.At 1-year follow-up, treatment with single phlebectomies of a large tributary was effective to abolish GSV reflux in 50% of patients and to free 66% of patients from symptoms. Patients with limited disease progression and mild DUS alterations are most likely to benefit from this approach.

Published clinical prediction models are often ignored during the development of novel prediction models despite similarities in populations and intended usage. The plethora of prediction models that arise from this practice may still perform poorly when applied in other populations. Incorporating prior evidence might improve the accuracy of prediction models and make them potentially better generalizable. Unfortunately, aggregation of prediction models is not straightforward, and methods to combine differently specified models are currently lacking. We propose two approaches for aggregating previously published prediction models when a validation dataset is available: model averaging and stacked regressions. These approaches yield user-friendly stand-alone models that are adjusted for the new validation data. Both approaches rely on weighting to account for model performance and between-study heterogeneity but adopt a different rationale (averaging versus combination) to combine the models. We illustrate their implementation in a clinical example and compare them with established methods for prediction modeling in a series of simulation studies. Results from the clinical datasets and simulation studies demonstrate that aggregation yields prediction models with better discrimination and calibration in a vast majority of scenarios, and results in equivalent performance (compared to developing a novel model from scratch) when validation datasets are relatively large. In conclusion, model aggregation is a promising strategy when several prediction models are available from the literature and a validation dataset is at hand. The aggregation methods do not require existing models to have similar predictors and can be applied when relatively few data are at hand.

Background: When study data are clustered, standard regression analysis is considered inappropriate and analytical techniques for clustered data need to be used. For prediction research in which the interest of predictor effects is on the patient level, random effect regression models are probably preferred over standard regression analysis. It is well known that the random effect parameter estimates and the standard logistic regression parameter estimates are different. Here, we compared random effect and standard logistic regression models for their ability to provide accurate predictions. Methods. Using an empirical study on 1642 surgical patients at risk of postoperative nausea and vomiting, who were treated by one of 19 anesthesiologists (clusters), we developed prognostic models either with standard or random intercept logistic regression. External validity of these models was assessed in new patients from other anesthesiologists. We supported our results with simulation studies using intra-class correlation coefficients (ICC) of 5%, 15%, or 30%. Standard performance measures and measures adapted for the clustered data structure were estimated. Results: The model developed with random effect analysis showed better discrimination than the standard approach, if the cluster effects were used for risk prediction (standard c-index of 0.69 versus 0.66). In the external validation set, both models showed similar discrimination (standard c-index 0.68 versus 0.67). The simulation study confirmed these results. For datasets with a high ICC (≥15%), model calibration was only adequate in external subjects, if the used performance measure assumed the same data structure as the model development method: standard calibration measures showed good calibration for the standard developed model, calibration measures adapting the clustered data structure showed good calibration for the prediction model with random intercept. Conclusion: The models with random intercept discriminate better than the standard model only if the cluster effect is used for predictions. The prediction model with random intercept had good calibration within clusters. © 2013 Bouwmeester et al.; licensee BioMed Central Ltd.

OBJECTIVE: To derive, cross validate, and externally validate a clinical prediction model that assesses the risks of different serious bacterial infections in children with fever at the emergency department. DESIGN: Prospective observational diagnostic study. SETTING: Three paediatric emergency care units: two in the Netherlands and one in the United Kingdom. PARTICIPANTS: Children with fever, aged 1 month to 15 years, at three paediatric emergency care units: Rotterdam (n=1750) and the Hague (n=967), the Netherlands, and Coventry (n=487), United Kingdom. A prediction model was constructed using multivariable polytomous logistic regression analysis and included the predefined predictor variables age, duration of fever, tachycardia, temperature, tachypnoea, ill appearance, chest wall retractions, prolonged capillary refill time (>3 seconds), oxygen saturation

Background The purpose of this study was to update and external validation of a prognostic model that is able to predict the survival probability of newly diagnosed patients with head and neck cancer. Methods Our original prognostic model is based on historical data of 1371 patients with primary head and neck cancer, diagnosed and treated in the Leiden University Medical Center, between 1981 and 1999. The model contains the predictors age, sex, tumor site, TNM-classification, prior tumors, and comorbidity. We updated the model with follow-up data until January 2010. The updated model was then externally validated in 598 patients with head and neck cancer from the Siteman Cancer Center/Barnes-Jewish Hospital, St. Louis, Missouri. Results Median follow-up was 5.5 years (range, 0–25.5). Only 2.5% of patients were lost to follow-up. During follow-up 1099 patients (80.2%) passed away. Discrimination of the updated prognostic model was good, with a C-index of 0.73 after internal validation. The discrimination was slightly lower in the external validation set (C-index, 0.69). The predicted 2-year and 5-year survival rates correlated satisfactorily with some slight deviations from the perfect calibration line. Conclusions We used recent follow-up information to update the Leiden prognostic model for newly diagnosed patients with head and neck cancer. The model showed acceptably good calibration and discrimination results in internal and external validation procedures. © 2012 Wiley Periodicals, Inc. Head Neck, 2013

Recently, sex-specific prediction models for low hemoglobin (Hb) deferral have been developed in Dutch whole blood donors. In the present study, we validated and updated the models in a cohort of Irish whole blood donors.Prospectively collected data from 45,031 Irish whole blood donors were used. Hb cutoff levels for donation were approximately 0.35 mmol/L lower in Ireland than the Dutch cutoff levels (8.07 mmol/L vs. 8.40 mmol/L in men; 7.45 mmol/L vs. 7.80 mmol/L in women). The predictive performance of the models was assessed with calibration plots, calibration-in-the-large, and the concordance (c)-statistic. The models were updated by revising the strength of the individual predictors in the models.A total of 613 men (2.4%) and 1624 women (8.4%) were deferred from donation because of a low Hb level. Validation demonstrated underestimation of predicted risks and lower c-statistics for men and women compared to the Dutch cohort. The strength of most predictive factors, particularly previous Hb level, was lower in Irish donors. The updated models showed a c-statistic of 0.83 (95% confidence interval [CI], 0.81-0.84) for men and 0.76 (95% CI, 0.74-0.77) for women.The performance of Dutch prediction models for Hb deferral was limited when validated in Irish whole blood donors. Updating the models resulted in different predictor effects. This improved mainly the model calibration; the improvement in discrimination was small.

To identify axial spondyloarthritis (SpA) in Dutch primary care patients with chronic low back pain (CLBP), and to design a simple referral model for general practitioners (GPs) that would identify patients at risk for axial SpA.Patients (ages 20-45 years) with CLBP were identified from GP records. Assessments included inflammatory back pain questionnaires, medical interviews, physical examinations, HLA-B27, C-reactive protein level, conventional radiography, and magnetic resonance imaging. The outcome measure was axial SpA defined by the Assessment of SpondyloArthritis international Society (ASAS) criteria. Multivariable regression analysis with bootstrapping was used to develop the referral model.A total of 364 patients (mean ± SD age 36.3 ± 6.8 years) was recruited with a median symptom duration of 9.0 years. Eighty-six patients (24%) fulfilled the ASAS criteria for axial SpA. Of all potential determinants, the ASAS inflammatory back pain questionnaire, good response to nonsteroidal antiinflammatory drugs, family history of SpA, and symptom duration were identified as most relevant for diagnosing axial SpA by multivariable regression analysis related to axial SpA. The shrunken regression coefficients were, respectively, 1.04, 0.83, 0.73, and 0.23. The combination of these 4 items proved a useful area under the receiver operating characteristic curve of 0.75 (SE 0.03). In a simplified score model, at the suggested cutoff value of 1.5, the sensitivity was 83% and specificity was 59%.This study shows that 1 of 4 primary care patients with CLBP was classified as having axial SpA. A preselection in primary care based on a combination of clinical items may be useful to facilitate the identification of patients at risk of axial SpA.

Background Nested case–control studies become increasingly popular as they can be very efficient for quantifying the diagnostic accuracy of costly or invasive tests or (bio)markers. However, they do not allow for direct estimation of the test’s predictive values or post-test probabilities, let alone for their confidence intervals (CIs). Correct estimates of the predictive values itself can easily be obtained using a simple correction by the (inverse) sampling fractions of the cases and controls. But using this correction to estimate the corresponding standard error (SE), falsely increases the number of patients that are actually studied, yielding too small CIs. We compared different approaches for estimating the SE and thus CI of predictive values or post-test probabilities of diagnostic test results in a nested case–control study. Methods We created datasets based on a large, previously published diagnostic study on 2 different tests (D-dimer test and calf difference test) with a nested case–control design. We compared six different approaches; the approaches were: 1. the standard formula for the SE of a proportion, 2. adaptation of the standard formula with the sampling fraction, 3. A bootstrap procedure, 4. A approach, which uses the sensitivity, the specificity and the prevalence, 5. Weighted logistic regression, and 6. Approach 4 on the log odds scale. The approaches were compared with respect to coverage of the CI and CI-width. Results The bootstrap procedure (approach 3) showed good coverage and relatively small CI widths. Approaches 4 and 6 showed some undercoverage, particularly for the D-dimer test with frequent positive results (positive results around 70%). Approaches 1, 2 and 5 showed clear overcoverage at low prevalences of 0.05 and 0.1 in the cohorts for all case–control ratios. Conclusion The results from our study suggest that a bootstrap procedure is necessary to assess the confidence interval for the predictive values or post-test probabilities of diagnostic tests results in studies using a nested case–control design.

Objectives Prediction models may perform poorly in a new setting. We aimed to determine which model updating methods should be applied for models predicting polytomous outcomes, which often suffer from one or more categories with low prevalence. Study Design and Setting We used case studies on testicular and ovarian tumors. The original regression models were based on 502 and 2,037 patients and validated on 273 and 1,107 patients, respectively. The polytomous models combined dichotomous models for category A vs. B + C and B vs. C (sequential dichotomous modeling). Simple recalibration, revision, and redevelopment methods were considered. To assess discrimination (using dichotomous and polytomous c-statistics) and calibration (by comparing observed and expected prevalences) of these methods, the validation data were divided into updating and test parts. Five hundred such divisions were randomly generated, and the average test set results reported. Results None of the updating methods could improve discrimination of the original models, but recalibration, revision, and redevelopment strongly improved calibration. Redevelopment was unstable with respect to overfitting and performance. Conclusion Simple dichotomous updating methods behaved well when applied to polytomous models. Our results suggest that recalibration is preferred, but larger validation sets may make revision or redevelopment a sensible alternative.

Blood donors that meet the hemoglobin (Hb) criteria for donation may have undetected subclinical iron deficiency. The aim of this study was to assess the prevalence of subclinical iron deficiency in whole blood donors with Hb levels above cutoff levels for donation by measuring zinc protoporphyrin (ZPP) levels. In addition, prevalence rates based on other iron variables were assessed for comparison.The study population comprised 5280 Dutch whole blood donors, who passed the Hb criteria for donation. During donor screening, Hb levels were measured in capillary samples (finger prick), and venous blood samples were taken for measurements of ZPP and other iron variables. These variables included ferritin, transferrin saturation, soluble transferrin receptor (sTfR), hepcidin, red blood cell mean corpuscular volume (MCV), and mean cell Hb (MCH).With a ZPP cutoff level of at least 100 μmol/mol heme, subclinical iron deficiency was present in 6.9% of male donors and in 9.8% of female donors. Based on other iron variables, iron deficiency was also observed. Prevalence rates ranged from 4.8% (based on transferrin saturation) to 27.4% (based on hepcidin concentration) in men and from 5.6% (based on sTfR concentration) to 24.7% (based on hepcidin concentration) in women.Results from this study showed that subclinical iron deficiency is prevalent among blood donors that meet the Hb criteria for blood donation, based on ZPP levels and on other iron variables. This finding needs attention because these donors are at increased risk of developing iron deficiency affecting Hb formation and other cellular processes.

Increased zinc protoporphyrin (ZPP) levels can indicate iron deficiency and may be predictive for low hemoglobin (Hb) deferral in blood donors. Prediction models for Hb deferral in whole blood donors have already been developed. In this study, we examined if addition of ZPP to these prediction models improves risk estimation of Hb deferral.This study included 4598 Dutch whole blood donors. Information on ZPP levels measured at the previous visit was added to the existing prediction models to estimate the risk of Hb deferral. Models were compared using the following measures: concordance (c)-statistic, continuous net reclassification improvement (NRI), and clinical net benefit (NB).Seventy-six male donors (2.9%) and 69 female donors (3.5%) were deferred because of a low Hb level. Previous ZPP level was associated with risk of Hb deferral (odds ratio for interquartile range of previous ZPP level, men 2.0 [95% confidence interval {CI}, 1.7-2.3]; women 2.2 [95% CI, 1.9-2.4]) in a multivariable risk model. Addition of ZPP into the models resulted in an increase of the c-statistic from 0.93 to 0.94 for men and from 0.80 to 0.85 for women. The added value of ZPP was confirmed by measures of clinical usefulness. NRI for men was 0.42, and for women, 0.56. At relevant threshold probabilities between 10 and 15%, NB was higher for models considering ZPP.This study shows that ZPP measurements obtained at the previous visit may have added value in the risk prediction of Hb deferral in whole blood donors.

Clinical prediction models are increasingly used to complement clinical reasoning and decision-making in modern medicine, in general, and in the cardiovascular domain, in particular. To these ends, developed models first and foremost need to provide accurate and (internally and externally) validated estimates of probabilities of specific health conditions or outcomes in the targeted individuals. Subsequently, the adoption of such models by professionals must guide their decision-making, and improve patient outcomes and the cost-effectiveness of care. In the first paper of this series of two companion papers, issues relating to prediction model development, their internal validation, and estimating the added value of a new (bio)marker to existing predictors were discussed. In this second paper, an overview is provided of the consecutive steps for the assessment of the model's predictive performance in new individuals (external validation studies), how to adjust or update existing models to local circumstances or with new predictors, and how to investigate the impact of the uptake of prediction models on clinical decision-making and patient outcomes (impact studies). Each step is illustrated with empirical examples from the cardiovascular field.

Each year, approximately 5% of the invited blood donors is eventually deferred from donation because of low hemoglobin (Hb) levels. Estimating the risk of Hb deferral in blood donors can be helpful in the management of the donation program. We developed and validated a prediction model for Hb deferral in whole blood donors, separately for men and women.Data from a Dutch prospective cohort of 220,946 whole blood donors were used to identify predictors for Hb deferral using multivariable logistic regression analyses. Validity of the prediction models was assessed with a cross-validation.A total of 12,865 donors (5.8%) were deferred because of a low Hb level. The strongest predictors of Hb deferral were Hb level measured at the previous visit, age, seasonality, difference in Hb levels between the previous two visits, time since the previous visit, deferral at the previous visit, and the total number of whole blood donations in the past 2 years for both men and women. The prediction models had an area under the receiver operating characteristic curve of 0.89 for men and 0.84 for women. Cross-validation showed similar results and good calibration.Using a limited number of easy-to-measure characteristics enables a good prediction of Hb deferral risk in whole blood donors. The prediction models may guide the decision which donors to invite for a next donation and for which donors the invitation should be postponed. Potentially, this could decrease the number of Hb deferrals in blood donors.

Prediction models are increasingly used to complement clinical reasoning and decision making in modern medicine in general, and in the cardiovascular domain in particular. Developed models first and foremost need to provide accurate and (internally and externally) validated estimates of probabilities of specific health conditions or outcomes in targeted patients. The adoption of such models must guide physician's decision making and an individual's behaviour, and consequently improve individual outcomes and the cost-effectiveness of care. In a series of two articles we review the consecutive steps generally advocated for risk prediction model research. This first article focuses on the different aspects of model development studies, from design to reporting, how to estimate a model's predictive performance and the potential optimism in these estimates using internal validation techniques, and how to quantify the added or incremental value of new predictors or biomarkers (of whatever type) to existing predictors. Each step is illustrated with empirical examples from the cardiovascular field.

In this paper, we focus on measures to evaluate discrimination of prediction models for ordinal outcomes. We review existing extensions of the dichotomous c-index-which is equivalent to the area under the receiver operating characteristic (ROC) curve--suggest a new measure, and study their relationships. The volume under the ROC surface (VUS) scores sets of cases including one case from each outcome category. VUS considers sets as either correctly or incorrectly ordered by the model. All other existing measures assess pairs of cases. We propose an ordinal c-index (ORC) that is set-based but, contrary to VUS, scores sets more gradually by indicating the closeness of the model-based ordering to the perfect ordering. As a result, the ORC does not decrease rapidly as the number of outcome categories increases. It turns out that the ORC can be rewritten as the average of pairwise c-indexes. Hence, the ORC has both a set- and pair-based interpretation. There are several relationships between the existing measures, leading to only two types of existing measures: a prevalence-weighted average of pairwise c-indexes and the VUS. Our suggested measure ORC positions itself in between as it is set-based but turns out to equal an unweighted average of pairwise c-indexes. The measures are demonstrated through a case study on the prediction of six-month outcome after traumatic brain injury. In conclusion, the set-based nature and graded scoring system make the ORC an attractive measure with a simple interpretation, together with its prevalence-independence that is a natural property of a discrimination measure.

The discriminative ability of risk models for dichotomous outcomes is often evaluated with the concordance index (c-index). However, many medical prediction problems are polytomous, meaning that more than two outcome categories need to be predicted. Unfortunately such problems are often dichotomized in prediction research. We present a perspective on the evaluation of discriminative ability of polytomous risk models, which may instigate researchers to consider polytomous prediction models more often. First, we suggest a “discrimination plot” as a tool to visualize the model’s discriminative ability. Second, we discuss the use of one overall polytomous c-index versus a set of dichotomous measures to summarize the performance of the model. Third, we address several aspects to consider when constructing a polytomous c-index. These involve the assessment of concordance in pairs versus sets of patients, weighting by outcome prevalence, the value related to models with random performance, the reduction to the dichotomous c-index for dichotomous problems, and interpretation. We illustrate these issues on case studies dealing with ovarian cancer (four outcome categories) and testicular cancer (three categories). We recommend the use of a discrimination plot together with an overall c-index such as the Polytomous Discrimination Index. If the overall c-index suggests that the model has relevant discriminative ability, pairwise c-indexes for each pair of outcome categories are informative. For pairwise c-indexes we recommend the ‘conditional-risk’ method which is consistent with the analytical approach of the multinomial logistic regression used to develop polytomous risk models.

Background Diagnostic and prognostic literature is overwhelmed with studies reporting univariable predictor-outcome associations. Currently, methods to incorporate such information in the construction of a prediction model are underdeveloped and unfamiliar to many researchers. Methods This article aims to improve upon an adaptation method originally proposed by Greenland (1987) and Steyerberg (2000) to incorporate previously published univariable associations in the construction of a novel prediction model. The proposed method improves upon the variance estimation component by reconfiguring the adaptation process in established theory and making it more robust. Different variants of the proposed method were tested in a simulation study, where performance was measured by comparing estimated associations with their predefined values according to the Mean Squared Error and coverage of the 90% confidence intervals. Results Results demonstrate that performance of estimated multivariable associations considerably improves for small datasets where external evidence is included. Although the error of estimated associations decreases with increasing amount of individual participant data, it does not disappear completely, even in very large datasets. Conclusions The proposed method to aggregate previously published univariable associations with individual participant data in the construction of a novel prediction models outperforms established approaches and is especially worthwhile when relatively limited individual participant data are available.