Search

Researchers often perform data-driven variable selection when modeling the associations between an outcome and multiple independent variables in regression analysis. Variable selection may improve the interpretability, parsimony and/or predictive accuracy of a model. Yet variable selection can also have negative consequences, such as false exclusion of important variables or inclusion of noise variables, biased estimation of regression coefficients, underestimated standard errors and invalid confidence intervals, as well as model instability. While the potential advantages and disadvantages of variable selection have been discussed in the literature for decades, few large-scale simulation studies have neutrally compared data-driven variable selection methods with respect to their consequences for the resulting models. We present the protocol for a simulation study that will evaluate different variable selection methods: forward selection, stepwise forward selection, backward elimination, augmented backward elimination, univariable selection, univariable selection followed by backward elimination, and penalized likelihood approaches (Lasso, relaxed Lasso, adaptive Lasso). These methods will be compared with respect to false inclusion and/or exclusion of variables, consequences on bias and variance of the estimated regression coefficients, the validity of the confidence intervals for the coefficients, the accuracy of the estimated variable importance ranking, and the predictive performance of the selected models. We consider both linear and logistic regression in a low-dimensional setting (20 independent variables with 10 true predictors and 10 noise variables). The simulation will be based on real-world data from the National Health and Nutrition Examination Survey (NHANES). Publishing this study protocol ahead of performing the simulation increases transparency and allows integrating the perspective of other experts into the study design., Competing Interests: The authors have declared that no competing interests exist., (Copyright: © 2024 Ullmann et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

Background: Kidney transplantation is the preferred treatment for eligible patients with kidney failure who need renal replacement therapy. However, it remains unclear whether the anticipated survival benefit from kidney transplantation is different for women and men., Methods: We included all dialysis patients recorded in the Austrian Dialysis and Transplant Registry who were waitlisted for their first kidney transplant between 2000 and 2018. In order to estimate the causal effect of kidney transplantation on 10-year restricted mean survival time, we mimicked a series of controlled clinical trials and applied inverse probability of treatment and censoring weighted sequential Cox models., Results: This study included 4408 patients (33% female) with a mean age of 52 years. Glomerulonephritis was the most common primary renal disease both in women (27%) and men (28%). Kidney transplantation led to a gain of 2.22 years (95% CI 1.88 to 2.49) compared with dialysis over a 10-year follow-up. The effect was smaller in women (1.95 years, 95% CI 1.38 to 2.41) than in men (2.35 years, 95% CI 1.92 to 2.70) due to a better survival on dialysis. Across ages the survival benefit of transplantation over a follow-up of 10 years was smaller in younger women and men and increased with age, showing a peak for both women and men aged about 60 years., Conclusions: There were few differences in survival benefit by transplantation between females and males. Females had better survival than males on the waitlist receiving dialysis and similar survival to males after transplantation., (© The Author(s) 2023. Published by Oxford University Press on behalf of the ERA.)

TP53 is the most commonly mutated gene in cancer and has been shown to form amyloid-like aggregates, similar to key proteins in neurodegenerative diseases. Nonetheless, the clinical implications of p53 aggregation remain unclear. Here, we investigated the presence and clinical relevance of p53 aggregates in serous ovarian cancer (OC). Using the p53-Seprion-ELISA, p53 aggregates were detected in 46 out of 81 patients, with a detection rate of 84.3% in patients with missense mutations. High p53 aggregation was associated with prolonged progression-free survival. We found associations of overall survival with p53 aggregates, but they did not reach statistical significance. Interestingly, p53 aggregation was significantly associated with elevated levels of p53 autoantibodies and increased apoptosis, suggesting that high levels of p53 aggregates may trigger an immune response and/or exert a cytotoxic effect. To conclude, for the first time, we demonstrated that p53 aggregates are an independent prognostic marker in serous OC. P53-targeted therapies based on the amount of these aggregates may improve the patient's prognosis., (© 2023. The Author(s), under exclusive licence to Springer Nature Limited.)

Importance: Kidney transplant is considered beneficial in terms of survival compared with continued dialysis for patients with kidney failure. However, randomized clinical trials are infeasible, and available evidence from cohort studies is at high risk of bias., Objective: To compare restricted mean survival times (RMSTs) between patients who underwent transplant and patients continuing dialysis across transplant candidate ages and depending on waiting time, applying target trial emulation methods., Design, Setting, and Participants: In this retrospective cohort study, patients aged 18 years or older appearing on the wait list for their first single-organ deceased donor kidney transplant between January 1, 2000, and December 31, 2018, in Austria were evaluated. Available data were obtained from the Austrian Dialysis and Transplant Registry and Eurotransplant and included repeated updates on wait-listing status and relevant covariates. Data were analyzed between August 1, 2019, and December 23, 2021., Exposures: A target trial was emulated in which patients were randomized to either receive the transplant immediately (treatment group) or to continue dialysis and never receive a transplant (control group) at each time an organ became available., Main Outcomes and Measures: The primary outcome was time from transplant allocation to death. Effect sizes in terms of RMSTs were obtained using a sequential Cox approach., Results: Among the 4445 included patients (2974 men [66.9%]; mean [SD] age, 52.2 [13.2] years), transplant was associated with increased survival time across all considered ages compared with continuing dialysis and remaining on the wait list within a 10-year follow-up. The estimated RMST differences were 0.57 years (95% CI, -0.14 to 1.84 years) at age 20 years, 3.01 years (95% CI, 2.50 to 3.54 years) at age 60 years, and 2.48 years (95% CI, 1.88 to 3.04 years) at age 70 years. The survival benefit for patients who underwent transplant across ages was independent of waiting time., Conclusions and Relevance: The findings of this study suggest that kidney transplant prolongs the survival time of persons with kidney failure across all candidate ages and waiting times.

Objective: To identify and critically appraise risk prediction models for living donor solid organ transplant counselling., Study Design and Setting: We systematically reviewed articles describing the development or validation of prognostic risk prediction models about living donor solid organ (kidney and liver) transplantation indexed in Medline until April 4, 2021. Models were eligible if intended to predict, at transplant counselling, any outcome occurring after transplantation or donation in recipients or donors. Duplicate study selection, data extraction, assessment for risk of bias and quality of reporting was done using the CHARMS checklist, PRISMA recommendations, PROBAST tool, and TRIPOD Statement., Results: We screened 4691 titles and included 49 studies describing 68 models (35 kidney, 33 liver transplantation). We identified 49 new risk prediction models and 19 external validations of existing models. Most models predicted recipients outcomes (n = 38, 75%), e.g., kidney graft loss (29%), or mortality of liver transplant recipients (55%). Many new models (n = 46, 94%) and external validations (n = 17, 89%) had a high risk of bias because of methodological weaknesses. The quality of reporting was generally poor., Conclusion: We advise against applying poorly developed, reported, or validated prediction models. Future studies could validate or update the few identified methodologically appropriate models., (Copyright © 2022 The Author(s). Published by Elsevier Inc. All rights reserved.)

Ovarian cancer (OC) is the most lethal genital malignancy in women. We aimed to develop and validate new proteomic-based models for non-invasive diagnosis of OC. We also compared them to the modified Risk of Ovarian Malignancy Algorithm (ROMA-50), the Copenhagen Index (CPH-I) and our earlier Proteomic Model 2017. Biomarkers were assessed using bead-based multiplex technology (Luminex®) in 356 women (250 with malignant and 106 with benign ovarian tumors) from five European centers. The training cohort included 279 women from three centers, and the validation cohort 77 women from two other centers. Of six previously studied serum proteins (CA125, HE4, osteopontin [OPN], prolactin, leptin, and macrophage migration inhibitory factor [MIF]), four contributed significantly to the Proteomic Model 2021 (CA125, OPN, prolactin, MIF), while leptin and HE4 were omitted by the algorithm. The Proteomic Model 2021 revealed a c-index of 0.98 (95% CI 0.96, 0.99) in the training cohort; however, in the validation cohort it only achieved a c-index of 0.82 (95% CI 0.72, 0.91). Adding patient age to the Proteomic Model 2021 constituted the Combined Model 2021, with a c-index of 0.99 (95% CI 0.97, 1) in the training cohort and a c-index of 0.86 (95% CI 0.78, 0.95) in the validation cohort. The Full Combined Model 2021 (all six proteins with age) yielded a c-index of 0.98 (95% CI 0.97, 0.99) in the training cohort and a c-index of 0.89 (95% CI 0.81, 0.97) in the validation cohort. The validation of our previous Proteomic Model 2017, as well as the ROMA-50 and CPH-I revealed a c-index of 0.9 (95% CI 0.82, 0.97), 0.54 (95% CI 0.38, 0.69) and 0.92 (95% CI 0.85, 0.98), respectively. In postmenopausal women, the three newly developed models all achieved a specificity of 1.00, a positive predictive value (PPV) of 1.00, and a sensitivity of >0.9. Performance in women under 50 years of age (c-index below 0.6) or with normal CA125 (c-index close to 0.5) was poor. CA125 and OPN had the best discriminating power as single markers. In summary, the CPH-I, the two combined 2021 Models, and the Proteomic Model 2017 showed satisfactory diagnostic accuracies, with no clear superiority of either model. Notably, although combining values of only four proteins with age, the Combined Model 2021 performed comparably to the Full Combined Model 2021. The models confirmed their exceptional diagnostic performance in women aged ≥50. All models outperformed the ROMA-50.

Background: While machine learning (ML) algorithms may predict cardiovascular outcomes more accurately than statistical models, their result is usually not representable by a transparent formula. Hence, it is often unclear how specific values of predictors lead to the predictions. We aimed to demonstrate with graphical tools how predictor-risk relations in cardiovascular risk prediction models fitted by ML algorithms and by statistical approaches may differ, and how sample size affects the stability of the estimated relations., Methods: We reanalyzed data from a large registry of 1.5 million participants in a national health screening program. Three data analysts developed analytical strategies to predict cardiovascular events within 1 year from health screening. This was done for the full data set and with gradually reduced sample sizes, and each data analyst followed their favorite modeling approach. Predictor-risk relations were visualized by partial dependence and individual conditional expectation plots., Results: When comparing the modeling algorithms, we found some similarities between these visualizations but also occasional divergence. The smaller the sample size, the more the predictor-risk relation depended on the modeling algorithm used, and also sampling variability played an increased role. Predictive performance was similar if the models were derived on the full data set, whereas smaller sample sizes favored simpler models., Conclusion: Predictor-risk relations from ML models may differ from those obtained by statistical models, even with large sample sizes. Hence, predictors may assume different roles in risk prediction models. As long as sample size is sufficient, predictive accuracy is not largely affected by the choice of algorithm., (© 2021. The Author(s).)

Statistical models are often fitted to obtain a concise description of the association of an outcome variable with some covariates. Even if background knowledge is available to guide preselection of covariates, stepwise variable selection is commonly applied to remove irrelevant ones. This practice may introduce additional variability and selection is rarely certain. However, these issues are often ignored and model stability is not questioned. Several resampling-based measures were proposed to describe model stability, including variable inclusion frequencies (VIFs), model selection frequencies, relative conditional bias (RCB), and root mean squared difference ratio (RMSDR). The latter two were recently proposed to assess bias and variance inflation induced by variable selection. Here, we study the consistency and accuracy of resampling estimates of these measures and the optimal choice of the resampling technique. In particular, we compare subsampling and bootstrapping for assessing stability of linear, logistic, and Cox models obtained by backward elimination in a simulation study. Moreover, we exemplify the estimation and interpretation of all suggested measures in a study on cardiovascular risk. The VIF and the model selection frequency are only consistently estimated in the subsampling approach. By contrast, the bootstrap is advantageous in terms of bias and precision for estimating the RCB as well as the RMSDR. Though, unbiased estimation of the latter quantity requires independence of covariates, which is rarely encountered in practice. Our study stresses the importance of addressing model stability after variable selection and shows how to cope with it., (© 2020 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Although separate prediction models for donors and recipients were previously published, we identified a need to predict outcomes of donor/recipient simultaneously, as they are clearly not independent of each other. We used characteristics from transplantations performed at the Oslo University Hospital from 1854 live donors and from 837 recipients of a live donor kidney transplant to derive Cox models for predicting donor mortality up to 20 years, and recipient death, and graft loss up to 10 years. The models were developed using the multivariable fractional polynomials algorithm optimizing Akaike's information criterion, and optimism-corrected performance was assessed. Age, year of donation, smoking status, cholesterol and creatinine were selected to predict donor mortality (C-statistic of 0.81). Linear predictors for donor mortality served as summary of donor prognosis in recipient models. Age, sex, year of transplantation, dialysis vintage, primary renal disease, cerebrovascular disease, peripheral vascular disease and HLA mismatch were selected to predict recipient mortality (C-statistic of 0.77). Age, dialysis vintage, linear predictor of donor mortality, HLA mismatch, peripheral vascular disease and heart disease were selected to predict graft loss (C-statistic of 0.66). Our prediction models inform decision-making at the time of transplant counselling and are implemented as online calculators., (© 2020 The Authors. Transplant International published by John Wiley & Sons Ltd on behalf of Steunstichting ESOT.)

Equations predicting the risk of occurrence of cardiovascular disease (CVD) are used in primary care to identify high-risk individuals among the general population. To improve the predictive performance of such equations, we updated the Framingham general CVD 1991 and 2008 equations and the Pooled Cohort equations for atherosclerotic CVD within five years in a contemporary cohort of individuals who participated in the Austrian health-screening program from 2009-2014. The cohort comprised 1.7 M individuals aged 30-79 without documented CVD history. CVD was defined by hospitalization or death from cardiovascular cause. Using baseline and follow-up data, we recalibrated and re-estimated the equations. We evaluated the gain in discrimination and calibration and assessed explained variation. A five-year general CVD risk of 4.61% was observed. As expected, discrimination c-statistics increased only slightly and ranged from 0.73-0.79. The two original Framingham equations overestimated the CVD risk, whereas the original Pooled Cohort equations underestimated it. Re-estimation improved calibration of all equations adequately, especially for high-risk individuals. Half of the individuals were reclassified into another risk category using the re-estimated equations. Predictors in the re-estimated Framingham equations explained 7.37% of the variation, whereas the Pooled Cohort equations explained 5.81%. Age was the most important predictor.

Objectives: Neurologic outcome prediction in out-of-hospital cardiac arrest survivors is highly limited due to the lack of consistent predictors of clinically relevant brain damage. The present study aimed to identify novel biomarkers of neurologic recovery to improve early prediction of neurologic outcome., Design: Prospective, single-center study, SETTING:: University-affiliated tertiary care center., Patients: We prospectively enrolled 96 out-of-hospital cardiac arrest survivors into our study., Interventions: None., Measurements and Main Results: Neurologic outcome was assessed by the Cerebral Performance Categories score. To identify plasma biomarkers for poor neurologic outcome (Cerebral Performance Categories score ≥ 3), we performed a three-step proteomics strategy of preselection by shotgun analyses, crosschecking in brain tissue samples, and verification by targeted proteomic analyses using a multistep statistical modeling approach. Sixty-three patients (66%) had a poor neurologic outcome. Out of a total of 299 proteins, we identified α-enolase, 14-3-3 protein ζ/δ, cofilin-1, and heat shock cognate 71 kDa protein as novel biomarkers for poor neurologic outcome. The implementation of these biomarkers into a clinical multimarker model, consisting of previously identified covariates associated to outcome, resulted in a significant improvement of neurologic outcome prediction (C-index, 0.70; explained variation, 11.9%; p for added value, 0.019)., Conclusions: This study identified four novel biomarkers for the prediction of poor neurologic outcome in out-of-hospital cardiac arrest survivors. The implementation of α-enolase, 14-3-3 protein ζ/δ, cofilin-1, and heat shock cognate 71 kDa protein into a multimarker predictive model along with previously identified risk factors significantly improved neurologic outcome prediction. Each of the proteomically identified biomarkers did not only outperform current risk stratification models but may also reflect important pathophysiologic pathways undergoing during cerebral ischemia.

Background: Cardiovascular prevention guidelines advocate the use of statistical risk equations to predict individual cardiovascular risk. However, predictive accuracy and clinical value of existing equations may differ in populations other than the one used for their development. Using baseline and follow-up data of the Austrian health-screening program, we assessed discrimination, calibration, and clinical utility of three widely recommended equations-the Framingham 1991 and 2008 general cardiovascular disease (CVD) equations, and the Pooled Cohort equations predicting atherosclerotic CVD., Methods: The validation cohort comprised 1.7 M individuals aged 30-79, without documented CVD history who participated in the program from 2009 to 2014. CVD events were defined by a cardiovascular cause of hospitalization or death., Results: The observed five-year general CVD risk was 4.66%. Discrimination c-indices (0.72-0.78) were slightly lower than those reported for the development cohorts. C-indices for women were always higher than for men. CVD risk was overestimated by the Framingham 2008 equation, but underestimated by the Pooled Cohort equations. The Framingham 1991 equation was well-calibrated, especially for individuals up to 64 years. If applied to recommend health interventions at a predicted five-year risk between 5 and 10%, the equations were clinically useful with their net benefits, weighting true positives against false positives, ranging from 0.13 to 3.43%., Conclusion: The equations can discriminate high-risk from low-risk individuals, but predictive accuracy (especially for high-risk individuals) might be improved by recalibration. The Framingham 1991 equation yielded the most accurate predictions., (Copyright © 2018 Elsevier B.V. All rights reserved.)

Statistical models support medical research by facilitating individualized outcome prognostication conditional on independent variables or by estimating effects of risk factors adjusted for covariates. Theory of statistical models is well-established if the set of independent variables to consider is fixed and small. Hence, we can assume that effect estimates are unbiased and the usual methods for confidence interval estimation are valid. In routine work, however, it is not known a priori which covariates should be included in a model, and often we are confronted with the number of candidate variables in the range 10-30. This number is often too large to be considered in a statistical model. We provide an overview of various available variable selection methods that are based on significance or information criteria, penalized likelihood, the change-in-estimate criterion, background knowledge, or combinations thereof. These methods were usually developed in the context of a linear regression model and then transferred to more generalized linear models or models for censored survival data. Variable selection, in particular if used in explanatory modeling where effect estimates are of central interest, can compromise stability of a final model, unbiasedness of regression coefficients, and validity of p-values or confidence intervals. Therefore, we give pragmatic recommendations for the practicing statistician on application of variable selection methods in general (low-dimensional) modeling problems and on performing stability investigations and inference. We also propose some quantities based on resampling the entire variable selection process to be routinely reported by software packages offering automated variable selection algorithms., (© 2017 The Authors. Biometrical Journal Published by WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.)

Background: Collections of electronic medical records (EMRs) can provide a rich source of information for renal health care research. However, their use in statistical analyses requires many preparatory steps, including coding of freetext entries and clear definitions of time windows for harvesting prognostic factors and outcomes. We analyse a large collection of EMRs to identify prognostic factors of adequate health care in diabetic patients at risk for chronic kidney disease, and discuss benefits and risks of such re-use of routine data., Methods: In a representative sample of 695 068 patient records collected in 58 Austrian general practitioners' offices, we could identify 31 374 patients with diabetes mellitus. As outcomes, we investigated whether a patient received a serum creatinine measurement, and the time elapsing between two consecutive serum creatinine measurements. Prognostic factors were defined by extracting previous diagnoses, laboratory measurements, drug prescriptions and demographic characteristics from the records., Results: Serum creatinine was measured annually in 44.4% of diabetic patients with previous signs of reduced kidney function and in 20.5% of the patients without such signs. Within 1 year after the first measurement, a follow-up measurement was made in 79.4 and 68.4% of the patients, respectively. Previous diagnoses, laboratory measurements, drug prescriptions and demographic characteristics explained 41% of the observed variance of kidney function monitoring. With 24% explained variance, previous referrals to laboratories were identified as the most important prognostic factor group., Conclusions: The analysis of large routine data collections poses various challenges, among which the need for coding free text into variables and various sources of biases are most demanding. However, routine data collections represent the daily practice of health care and offer many chances for renal health services and outcomes research., (© The Author 2015. Published by Oxford University Press on behalf of ERA-EDTA. All rights reserved.)