Search

Results of simulation studies evaluating the performance of statistical methods can have a major impact on the way empirical research is implemented. However, so far there is limited evidence of the replicability of simulation studies. Eight highly cited statistical simulation studies were selected, and their replicability was assessed by teams of replicators with formal training in quantitative methodology. The teams used information in the original publications to write simulation code with the aim of replicating the results. The primary outcome was to determine the feasibility of replicability based on reported information in the original publications and supplementary materials. Replicasility varied greatly: some original studies provided detailed information leading to almost perfect replication of results, whereas other studies did not provide enough information to implement any of the reported simulations. Factors facilitating replication included availability of code, detailed reporting or visualization of data-generating procedures and methods, and replicator expertise. Replicability of statistical simulation studies was mainly impeded by lack of information and sustainability of information sources. We encourage researchers publishing simulation studies to transparently report all relevant implementation details either in the research paper itself or in easily accessible supplementary material and to make their simulation code publicly available using permanent links.

Prediction models are used among others to inform medical decisions on interventions. Typically, individuals with high risks of adverse outcomes are advised to undergo an intervention while those at low risk are advised to refrain from it. Standard prediction models do not always provide risks that are relevant to inform such decisions: for example, an individual may be estimated to be at low risk because similar individuals in the past received an intervention which lowered their risk. Therefore, prediction models supporting decisions should target risks belonging to defined intervention strategies. Previous works on prediction under interventions assumed that the prediction model was used only at one time point to make an intervention decision. In clinical practice, intervention decisions are rarely made only once: they might be repeated, deferred, and reevaluated. This requires estimated risks under interventions that can be reconsidered at several potential decision moments. In the current work, we highlight key considerations for formulating estimands in sequential prediction under interventions that can inform such intervention decisions. We illustrate these considerations by giving examples of estimands for a case study about choosing between vaginal delivery and cesarean section for women giving birth. Our formalization of prediction tasks in a sequential, causal, and estimand context provides guidance for future studies to ensure that the right question is answered and appropriate causal estimation approaches are chosen to develop sequential prediction models that can inform intervention decisions., (© 2024 The Author(s). Biometrical Journal published by Wiley‐VCH GmbH.)

Telemedicine in heart failure (HF) management may positively impact health outcomes, but varied effects in studies hinder guidance in HF guidelines. Evidence on the effectiveness of telemedicine in HF subpopulations is limited. We conducted a scoping review to evaluate and synthesise evidence on the effectiveness of telemedicine across HF subpopulations that could guide telemedicine strategies in routine practice. Meta-analyses concerning randomised controlled trials (RCTs) with subgroup analyses on telemedicine effectives were identified in PubMed. We identified 15 RCTs, encompassing 21 different subgroups based on characteristics of HF patients. Findings varied across studies and no definite evidence was found about which patients benefit most from telemedicine. Subgroup definitions were inconsistent, not always a priori defined and subgroups contained few patients. Some studies found heterogeneous effects of telemedicine on mortality and hospitalisation across subgroups defined by: New York Heart Association (NYHA) classification, previous HF decompensation, implantable device, concurrent depression, time since hospital discharge and duration of HF. Patients represented in the RCTs were mostly male, aged 65-75 years, with HF with reduced ejection fraction and NYHA class II/III. Traditional RCTs have not been able to provide clinicians with guidance; continuous real-world evidence generation could enhance monitoring and identify who benefits from telemedicine., (© 2024. The Author(s).)

Background and Aims: Home treatment is considered safe in acute pulmonary embolism (PE) patients selected by a validated triage tool (e.g. simplified PE severity index score or Hestia rule), but there is uncertainty regarding the applicability in underrepresented subgroups. The aim was to evaluate the safety of home treatment by performing an individual patient-level data meta-analysis., Methods: Ten prospective cohort studies or randomized controlled trials were identified in a systematic search, totalling 2694 PE patients treated at home (discharged within 24 h) and identified by a predefined triage tool. The 14- and 30-day incidences of all-cause mortality and adverse events (combined endpoint of recurrent venous thromboembolism, major bleeding, and/or all-cause mortality) were evaluated. The relative risk (RR) for 14- and 30-day mortalities and adverse events is calculated in subgroups using a random effects model., Results: The 14- and 30-day mortalities were 0.11% [95% confidence interval (CI) 0.0-0.24, I2 = 0) and 0.30% (95% CI 0.09-0.51, I2 = 0). The 14- and 30-day incidences of adverse events were 0.56% (95% CI 0.28-0.84, I2 = 0) and 1.2% (95% CI 0.79-1.6, I2 = 0). Cancer was associated with increased 30-day mortality [RR 4.9; 95% prediction interval (PI) 2.7-9.1; I2 = 0]. Pre-existing cardiopulmonary disease, abnormal troponin, and abnormal (N-terminal pro-)B-type natriuretic peptide [(NT-pro)BNP] at presentation were associated with an increased incidence of 14-day adverse events [RR 3.5 (95% PI 1.5-7.9, I2 = 0), 2.5 (95% PI 1.3-4.9, I2 = 0), and 3.9 (95% PI 1.6-9.8, I2 = 0), respectively], but not mortality. At 30 days, cancer, abnormal troponin, and abnormal (NT-pro)BNP were associated with an increased incidence of adverse events [RR 2.7 (95% PI 1.4-5.2, I2 = 0), 2.9 (95% PI 1.5-5.7, I2 = 0), and 3.3 (95% PI 1.6-7.1, I2 = 0), respectively]., Conclusions: The incidence of adverse events in home-treated PE patients, selected by a validated triage tool, was very low. Patients with cancer had a three- to five-fold higher incidence of adverse events and death. Patients with increased troponin or (NT-pro)BNP had a three-fold higher risk of adverse events, driven by recurrent venous thromboembolism and bleeding., (© The Author(s) 2024. Published by Oxford University Press on behalf of the European Society of Cardiology.)

Drawing on our experiences conducting replications we describe the lessons we learned about replication studies and formulate recommendations for researchers, policy makers, and funders about the role of replication in science and how it should be supported and funded. We first identify a variety of benefits of doing replication studies. Next, we argue that it is often necessary to improve aspects of the original study, even if that means deviating from the original protocol. Thirdly, we argue that replication studies highlight the importance of and need for more transparency of the research process, but also make clear how difficult that is. Fourthly, we underline that it is worth trying out replication in the humanities. We finish by formulating recommendations regarding reproduction and replication research, aimed specifically at funders, editors and publishers, and universities and other research institutes.

Clinical prediction models provide risks of health outcomes that can inform patients and support medical decisions. However, most models never make it to actual implementation in practice. A commonly heard reason for this lack of implementation is that prediction models are often not externally validated. While we generally encourage external validation, we argue that an external validation is often neither sufficient nor required as an essential step before implementation. As such, any available external validation should not be perceived as a license for model implementation. We clarify this argument by discussing 3 common misconceptions about external validation. We argue that there is not one type of recommended validation design, not always a necessity for external validation, and sometimes a need for multiple external validations. The insights from this paper can help readers to consider, design, interpret, and appreciate external validation studies., Competing Interests: Declaration of competing interest There are no competing interests for any author., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

Objectives: To systematically evaluate the performance of COVID-19 prognostic models and scores for mortality risk in older populations across three health-care settings: hospitals, primary care, and nursing homes., Study Design and Setting: This retrospective external validation study included 14,092 older individuals of ≥70 years of age with a clinical or polymerase chain reaction-confirmed COVID-19 diagnosis from March 2020 to December 2020. The six validation cohorts include three hospital-based (CliniCo, COVID-OLD, COVID-PREDICT), two primary care-based (Julius General Practitioners Network/Academisch network huisartsgeneeskunde/Network of Academic general Practitioners, PHARMO), and one nursing home cohort (YSIS) in the Netherlands. Based on a living systematic review of COVID-19 prediction models using Prediction model Risk Of Bias ASsessment Tool for quality and risk of bias assessment and considering predictor availability in validation cohorts, we selected six prognostic models predicting mortality risk in adults with COVID-19 infection (GAL-COVID-19 mortality, 4C Mortality Score, National Early Warning Score 2-extended model, Xie model, Wang clinical model, and CURB65 score). All six prognostic models were validated in the hospital cohorts and the GAL-COVID-19 mortality model was validated in all three healthcare settings. The primary outcome was in-hospital mortality for hospitals and 28-day mortality for primary care and nursing home settings. Model performance was evaluated in each validation cohort separately in terms of discrimination, calibration, and decision curves. An intercept update was performed in models indicating miscalibration followed by predictive performance re-evaluation., Main Outcome Measure: In-hospital mortality for hospitals and 28-day mortality for primary care and nursing home setting., Results: All six prognostic models performed poorly and showed miscalibration in the older population cohorts. In the hospital settings, model performance ranged from calibration-in-the-large -1.45 to 7.46, calibration slopes 0.24-0.81, and C-statistic 0.55-0.71 with 4C Mortality Score performing as the most discriminative and well-calibrated model. Performance across health-care settings was similar for the GAL-COVID-19 model, with a calibration-in-the-large in the range of -2.35 to -0.15 indicating overestimation, calibration slopes of 0.24-0.81 indicating signs of overfitting, and C-statistic of 0.55-0.71., Conclusion: Our results show that most prognostic models for predicting mortality risk performed poorly in the older population with COVID-19, in each health-care setting: hospital, primary care, and nursing home settings. Insights into factors influencing predictive model performance in the older population are needed for pandemic preparedness and reliable prognostication of health-related outcomes in this demographic., Competing Interests: Declaration of competing interest All authors have completed the International Committee of Medical Journals Editors uniform disclosure form at https://www.icmje.org/disclosure-of-interest/ and declare: funding from the Netherlands Organisation for Scientific Research (ZonMw). KJ and FW have received grants from the program Leren van Data by theDutch Ministry of Health, Welfare and Sport(grantnumber:329517); all declare no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work., (Copyright © 2024 The Authors. Published by Elsevier Inc. All rights reserved.)

Background: During the COVID-19 pandemic, older patients in primary care were triaged based on their frailty or assumed vulnerability for poor outcomes, while evidence on the prognostic value of vulnerability measures in COVID-19 patients in primary care was lacking. Still, knowledge on the role of vulnerability is pivotal in understanding the resilience of older people during acute illness, and hence important for future pandemic preparedness. Therefore, we assessed the predictive value of different routine care-based vulnerability measures in addition to age and sex for 28-day mortality in an older primary care population of patients with COVID-19., Methods: From primary care medical records using three routinely collected Dutch primary care databases, we included all patients aged 70 years or older with a COVID-19 diagnosis registration in 2020 and 2021. All-cause mortality was predicted using logistic regression based on age and sex only (basic model), and separately adding six vulnerability measures: renal function, cognitive impairment, number of chronic drugs, Charlson Comorbidity Index, Chronic Comorbidity Score, and a Frailty Index. Predictive performance of the basic model and the six vulnerability models was compared in terms of area under the receiver operator characteristic curve (AUC), index of prediction accuracy and the distribution of predicted risks., Results: Of the 4,065 included patients, 9% died within 28 days after COVID-19 diagnosis. Predicted mortality risk ranged between 7-26% for the basic model including age and sex, changing to 4-41% by addition of comorbidity-based vulnerability measures (Charlson Comorbidity Index, Chronic Comorbidity Score), more reflecting impaired organ functioning. Similarly, the AUC of the basic model slightly increased from 0.69 (95%CI 0.66 - 0.72) to 0.74 (95%CI 0.71 - 0.76) by addition of either of these comorbidity scores. Addition of a Frailty Index, renal function, the number of chronic drugs or cognitive impairment yielded no substantial change in predictions., Conclusion: In our dataset of older COVID-19 patients in primary care, the 28-day mortality fraction was substantial at 9%. Six different vulnerability measures had little incremental predictive value in addition to age and sex in predicting short-term mortality., (© 2024. The Author(s).)

Introduction: Meta-analyses show postive effects of telemedicine in heart failure (HF) management on hospitalisation, mortality and costs. However, these effects are heterogeneous due to variation in the included HF population, the telemedicine components and the quality of the comparator usual care. Still, telemedicine is gaining acceptance in HF management. The current nationwide study aims to identify (1) in which subgroup(s) of patients with HF telemedicine is (cost-)effective and (2) which components of telemedicine are most (cost-)effective., Methods and Analysis: The RELEASE-HF ('REsponsible roLl-out of E-heAlth through Systematic Evaluation - Heart Failure') study is a multicentre, observational, registry-based cohort study that plans to enrol 6480 patients with HF using data from the HF registry facilitated by the Netherlands Heart Registration. Collected data include patient characteristics, treatment information and clinical outcomes, and are measured at HF diagnosis and at 6 and 12 months afterwards. The components of telemedicine are described at the hospital level based on closed-ended interviews with clinicians and at the patient level based on additional data extracted from electronic health records and telemedicine-generated data. The costs of telemedicine are calculated using registration data and interviews with clinicians and finance department staff. To overcome missing data, additional national databases will be linked to the HF registry if feasible. Heterogeneity of the effects of offering telemedicine compared with not offering on days alive without unplanned hospitalisations in 1 year is assessed across predefined patient characteristics using exploratory stratified analyses. The effects of telemedicine components are assessed by fitting separate models for component contrasts., Ethics and Dissemination: The study has been approved by the Medical Ethics Committee 2021 of the University Medical Center Utrecht (the Netherlands). Results will be published in peer-reviewed journals and presented at (inter)national conferences. Effective telemedicine scenarios will be proposed among hospitals throughout the country and abroad, if applicable and feasible., Trial Registration Number: NCT05654961., Competing Interests: Competing interests: JB received an independent research grant from Abbott for ISS paid to institution, and has had speaker engagements or advisory boards for AstraZeneca, Abbott, Bayer, Boehringer, Daiichi Sankyo, Novartis and Vifor in the past 5 years. RAdB has received research grants and/or fees from AstraZeneca, Abbott, Boehringer Ingelheim, Cardior Pharmaceuticals, Ionis Pharmaceuticals, Novo Nordisk and Roche, and has had speaker engagements with Abbott, AstraZeneca, Bayer, Bristol Myers Squibb, Novartis and Roche. CJWB has received speaker fees from AstraZeneca, Boehringer Ingelheim and Novartis. MLH is supported by the Dutch Heart Foundation (Dr E Dekker Senior Clinical Scientist Grant 2020T058) and CVON (2020B008 RECONNEXT). He received an investigator-initiated research grant from Vifor Pharma, an educational grant from Boehringer Ingelheim and Novartis, and speaker/consultancy fees from Abbott, AstraZeneca, Bayer, Boehringer Ingelheim, MSD, Novartis, Sankyo, Daiichi, Quinn and Vifor Pharma, all of which were not related to this study., (© Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY. Published by BMJ.)

A common view in epidemiology is that automated confounder selection methods, such as backward elimination, should be avoided as they can lead to biased effect estimates and underestimation of their variance. Nevertheless, backward elimination remains regularly applied. We investigated if and under which conditions causal effect estimation in observational studies can improve by using backward elimination on a prespecified set of potential confounders. An expression was derived that quantifies how variable omission relates to bias and variance of effect estimators. Additionally, 3960 scenarios were defined and investigated by simulations comparing bias and mean squared error (MSE) of the conditional log odds ratio, log(cOR), and the marginal log risk ratio, log(mRR), between full models including all prespecified covariates and backward elimination of these covariates. Applying backward elimination resulted in a mean bias of 0.03 for log(cOR) and 0.02 for log(mRR), compared to 0.56 and 0.52 for log(cOR) and log(mRR), respectively, for a model without any covariate adjustment, and no bias for the full model. In less than 3% of the scenarios considered, the MSE of the log(cOR) or log(mRR) was slightly lower (max 3%) when backward elimination was used compared to the full model. When an initial set of potential confounders can be specified based on background knowledge, there is minimal added value of backward elimination. We advise not to use it and otherwise to provide ample arguments supporting its use., (© 2022 The Authors. Biometrical Journal published by Wiley-VCH GmbH.)

Background: In patients clinically suspected of having pulmonary embolism (PE), physicians often rely on intuitive estimation ("gestalt") of PE presence. Although shown to be predictive, gestalt is criticized for its assumed variation across physicians and lack of standardization., Objectives: To assess the diagnostic accuracy of gestalt in the diagnosis of PE and gain insight into its possible variation., Methods: We performed an individual patient data meta-analysis including patients suspected of having PE. The primary outcome was diagnostic accuracy of gestalt for the diagnosis of PE, quantified as risk ratio (RR) between gestalt and PE based on 2-stage random-effect log-binomial meta-analysis regression as well as gestalts' sensitivity and specificity. The variability of these measures was explored across different health care settings, publication period, PE prevalence, patient subgroups (sex, heart failure, chronic lung disease, and items of the Wells score other than gestalt), and age., Results: We analyzed 20 770 patients suspected of having PE from 16 original studies. The prevalence of PE in patients with and without a positive gestalt was 28.8% vs 9.1%, respectively. The overall RR was 3.02 (95% CI, 2.35-3.87), and the overall sensitivity and specificity were 74% (95% CI, 68%-79%) and 61% (95% CI, 53%-68%), respectively. Although variation was observed across individual studies (I 2 , 90.63%), the diagnostic accuracy was consistent across all subgroups and health care settings., Conclusion: A positive gestalt was associated with a 3-fold increased risk of PE in suspected patients. Although variation was observed across studies, the RR of gestalt was similar across prespecified subgroups and health care settings, exemplifying its diagnostic value for all patients suspected of having PE., Competing Interests: Declaration of competing interests There are no competing interests to disclose., (Copyright © 2023 The Authors. Published by Elsevier Inc. All rights reserved.)

Purpose: Ideally, the objectives of a pharmacoepidemiologic comparative effectiveness or safety study should dictate its design and data analysis. This paper discusses how defining an estimand is instrumental to this process., Methods: We applied the ICH-E9 (Statistical Principles for Clinical Trials) R1 addendum on estimands - which originally focused on randomized trials - to three examples of observational pharmacoepidemiologic comparative effectiveness and safety studies. Five key elements specify the estimand: the population, contrasted treatments, endpoint, intercurrent events, and population-level summary measure., Results: Different estimands were defined for case studies representing three types of pharmacological treatments: (1) single-dose treatments using a case study about the effect of influenza vaccination versus no vaccination on mortality risk in an adult population of ≥60 years of age; (2) sustained-treatments using a case study about the effect of dipeptidyl peptidase 4 inhibitor versus glucagon-like peptide-1 agonist on hypoglycemia risk in treatment of uncontrolled diabetes; and (3) as needed treatments using a case study on the effect of nitroglycerin spray as-needed versus no nitroglycerin on syncope risk in treatment of stabile angina pectoris., Conclusions: The case studies illustrated that a seemingly clear research question can still be open to multiple interpretations. Defining an estimand ensures that the study targets a treatment effect that aligns with the treatment decision the study aims to inform. Estimand definitions further help to inform choices regarding study design and data-analysis and clarify how to interpret study findings., (© 2023 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons Ltd.)

Prognostic modelling techniques have rapidly evolved over the past decade and may greatly benefit patients supported with ExtraCorporeal Membrane Oxygenation (ECMO). Epidemiological and computational physiological approaches aim to provide more accurate predictive assessments of ECMO-related risks and benefits. Implementation of these approaches may produce predictive tools that can improve complex clinical decisions surrounding ECMO allocation and management. This Review describes current applications of prognostic models and elaborates on upcoming directions for their clinical applicability in decision support tools directed at improved allocation and management of ECMO patients. The discussion of these new developments in the field will culminate in a futuristic perspective leaving ourselves and the readers wondering whether we may " fly ECMO by wire " someday.

Background: The COVID-19 pandemic has a large impact worldwide and is known to particularly affect the older population. This paper outlines the protocol for external validation of prognostic models predicting mortality risk after presentation with COVID-19 in the older population. These prognostic models were originally developed in an adult population and will be validated in an older population (≥ 70 years of age) in three healthcare settings: the hospital setting, the primary care setting, and the nursing home setting., Methods: Based on a living systematic review of COVID-19 prediction models, we identified eight prognostic models predicting the risk of mortality in adults with a COVID-19 infection (five COVID-19 specific models: GAL-COVID-19 mortality, 4C Mortality Score, NEWS2 + model, Xie model, and Wang clinical model and three pre-existing prognostic scores: APACHE-II, CURB65, SOFA). These eight models will be validated in six different cohorts of the Dutch older population (three hospital cohorts, two primary care cohorts, and a nursing home cohort). All prognostic models will be validated in a hospital setting while the GAL-COVID-19 mortality model will be validated in hospital, primary care, and nursing home settings. The study will include individuals ≥ 70 years of age with a highly suspected or PCR-confirmed COVID-19 infection from March 2020 to December 2020 (and up to December 2021 in a sensitivity analysis). The predictive performance will be evaluated in terms of discrimination, calibration, and decision curves for each of the prognostic models in each cohort individually. For prognostic models with indications of miscalibration, an intercept update will be performed after which predictive performance will be re-evaluated., Discussion: Insight into the performance of existing prognostic models in one of the most vulnerable populations clarifies the extent to which tailoring of COVID-19 prognostic models is needed when models are applied to the older population. Such insight will be important for possible future waves of the COVID-19 pandemic or future pandemics., (© 2023. The Author(s).)

Purpose: It is challenging to generate and subsequently implement high-quality evidence in surgical practice. A first step would be to grade the strengths and weaknesses of surgical evidence and appraise risk of bias and applicability. Here, we described items that are common to different risk-of-bias tools. We explained how these could be used to assess comparative operative intervention studies in orthopedic trauma surgery, and how these relate to applicability of results., Methods: We extracted information from the Cochrane risk-of-bias-2 (RoB-2) tool, Risk Of Bias In Non-randomised Studies-of Interventions tool (ROBINS-I), and Methodological Index for Non-Randomized Studies (MINORS) criteria and derived a concisely formulated set of items with signaling questions tailored to operative interventions in orthopedic trauma surgery., Results: The established set contained nine items: population, intervention, comparator, outcome, confounding, missing data and selection bias, intervention status, outcome assessment, and pre-specification of analysis. Each item can be assessed using signaling questions and was explained using good practice examples of operative intervention studies in orthopedic trauma surgery., Conclusion: The set of items will be useful to form a first judgment on studies, for example when including them in a systematic review. Existing risk of bias tools can be used for further evaluation of methodological quality. Additionally, the proposed set of items and signaling questions might be a helpful starting point for peer reviewers and clinical readers., (© 2022. The Author(s).)

Although it is generally known that a (statistical) association between a factor, i.e., determinant or independent variable, and outcome, i.e., dependent variable, does not directly provide evidence of a causal relation, in practice the distinction between associative and causal relationships often becomes fuzzy when interpreting prognostic factor research. We provide suggestions for interpreting the findings of prognostic factor research. It is important to assess the purpose and design of the study, including the statistical analysis. The actual evidence that prognostic factor research can provide is easily overestimated. In particular when associations between factors and outcome are estimated in a multivariable analysis, causal or predictive qualities can easily but wrongfully be attributed to a prognostic factor. It is generally advisable to refrain from judgments on the causal of predictive qualities of a prognostic factor purely based on a prognostic factor study. Findings from prognostic factor research are usually a good starting point for follow-up research, while the direct applicability of such findings in daily medical practice is often limited.

Objective: To provide considerations for reporting and interpretation that can improve assessment of the credibility of exploratory analyses in aetiologic research., Design: Mini-review of the literature and account of exploratory research principles., Setting: This study focuses on a particular type of causal research, namely aetiologic studies, which investigate the causal effect of one or multiple risk factors on a particular health outcome or disease. The mini review included aetiologic research articles published in four epidemiology journals in the first issue of 2021: American Journal of Epidemiology , Epidemiology , European Journal of Epidemiology , and International Journal of Epidemiology , specifically focusing on observational studies of causal risk factors of diseases., Main Outcome Measures: Number of exposure-outcome associations reported, grouped by type of analysis (main, sensitivity, and additional)., Results: The journal articles reported many exposure-outcome associations: a mean number of 33 (range 1-120) exposure-outcome associations for the primary analysis, 30 (0-336) for sensitivity analyses, and 163 (0-1467) for additional analyses. Six considerations were discussed that are important in assessing the credibility of exploratory analyses: research problem, protocol, statistical criteria, interpretation of findings, completeness of reporting, and effect of exploratory findings on future causal research., Conclusions: Based on this mini-review, exploratory analyses in aetiologic research were not always reported properly. Six considerations for reporting of exploratory analyses in aetiologic research were provided to stimulate a discussion about their preferred handling and reporting. Researchers should take responsibility for the results of exploratory analyses by clearly reporting their exploratory nature and specifying which findings should be investigated in future research and how., Competing Interests: Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: support from the Netherlands Organisation for Scientific Research and Leiden University Medical Centre for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work., (© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: When a predictor variable is measured in similar ways at the derivation and validation setting of a prognostic prediction model, yet both differ from the intended use of the model in practice (i.e., "predictor measurement heterogeneity"), performance of the model at implementation needs to be inferred. This study proposed an analysis to quantify the impact of anticipated predictor measurement heterogeneity., Methods: A simulation study was conducted to assess the impact of predictor measurement heterogeneity across validation and implementation setting in time-to-event outcome data. The use of the quantitative prediction error analysis was illustrated using an example of predicting the 6-year risk of developing type 2 diabetes with heterogeneity in measurement of the predictor body mass index., Results: In the simulation study, calibration-in-the-large of prediction models was poor and overall accuracy was reduced in all scenarios of predictor measurement heterogeneity. Model discrimination decreased with increasing random predictor measurement heterogeneity., Conclusions: Heterogeneity of predictor measurements across settings of validation and implementation reduced predictive performance at implementation of prognostic models with a time-to-event outcome. When validating a prognostic model, the targeted clinical setting needs to be considered and analyses can be conducted to quantify the impact of anticipated predictor measurement heterogeneity on model performance at implementation., (© 2022. The Author(s).)

Background: Guidance reports for observational comparative effectiveness and drug safety research recommend implementing a new-user design whenever possible, since it reduces the risk of selection bias in exposure effect estimation compared to a prevalent-user design. The uptake of this guidance has not been studied extensively., Methods: We reviewed 89 observational effectiveness and safety cohort studies published in six pharmacoepidemiological journals in 2018 and 2019. We developed an extraction tool to assess how frequently new-user and prevalent-user designs were reported to be implemented. For studies that implemented a new-user design in both treatment arms, we extracted information about the extent to which the moment of meeting eligibility criteria, treatment initiation, and start of follow-up were reported to be aligned., Results: Of the 89 studies included, 40% reported implementing a new-user design for both the study exposure arm and the comparator arm, while 13% reported implementing a prevalent-user design in both arms. The moment of meeting eligibility criteria, treatment initiation, and start of follow-up were reported to be aligned in both treatment arms in 53% of studies that reported implementing a new-user design. We provided examples of studies that minimized the risk of introducing bias due to unclear definition of time origin in unexposed participants, immortal time, or a time lag., Conclusions: Almost half of the included studies reported implementing a new-user design. Implications of misalignment of study design origin were difficult to assess because it would require explicit reporting of the target estimand in original studies. We recommend that the choice for a particular study time origin is explicitly motivated to enable assessment of validity of the study., (© 2021 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons Ltd.)

Purpose: Exposure definitions vary across pharmacoepidemiological studies. Therefore, transparent reporting of exposure definitions is important for interpretation of published study results. We aimed to assess the quality of reporting of exposure to identify where improvement may be needed., Method: We systematically reviewed observational pharmacoepidemiological studies that used routinely collected health data, published in 2017 in six pharmacoepidemiological journals. Reporting of exposure was scored using 11 items of the ISPE-ISPOR guideline on reporting of pharmacoepidemiological studies., Results: Of the 91 studies included, all studies reported the type of exposure (100%), while most reported the exposure risk window (85%) and the exposure assessment window (98%). Operationalization of the exposure window was described infrequently: 16% (14/90) of the studies explicitly reported the presence or absence of an induction period if applicable, 11% (5/47), and 35% (17/49) reported how stockpiling and gaps between exposure episodes were handled, respectively, and 35% (17/49) explicitly mentioned the exposure extension. Switching/add-on was reported in 62% (50/81). How switching between drugs was dealt with and specific drug codes were reported in 52 (57%) and 24 (26%) studies, respectively., Conclusion: Publications of pharmacoepidemiological studies frequently reported the type of exposure, the exposure risk window, and the exposure assessment window. However, more details on exposure assessment are needed, especially when it concerns the operationalization of the exposure risk window (eg, the presence or absence of an induction period or exposure extension, handling of stockpiling and gaps, and specific codes), to allow for correct interpretation, reproducibility, and assessment of validity., (© 2020 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons Ltd.)

In this paper we study approaches for dealing with treatment when developing a clinical prediction model. Analogous to the estimand framework recently proposed by the European Medicines Agency for clinical trials, we propose a 'predictimand' framework of different questions that may be of interest when predicting risk in relation to treatment started after baseline. We provide a formal definition of the estimands matching these questions, give examples of settings in which each is useful and discuss appropriate estimators including their assumptions. We illustrate the impact of the predictimand choice in a dataset of patients with end-stage kidney disease. We argue that clearly defining the estimand is equally important in prediction research as in causal inference.

Objectives: The aim of this study was to quantify the impact of predictor measurement heterogeneity on prediction model performance. Predictor measurement heterogeneity refers to variation in the measurement of predictor(s) between the derivation of a prediction model and its validation or application. It arises, for instance, when predictors are measured using different measurement instruments or protocols., Study Design and Setting: We examined the effects of various scenarios of predictor measurement heterogeneity in real-world clinical examples using previously developed prediction models for diagnosis of ovarian cancer, mutation carriers for Lynch syndrome, and intrauterine pregnancy., Results: Changing the measurement procedure of a predictor influenced the performance at validation of the prediction models in nine clinical examples. Notably, it induced model miscalibration. The calibration intercept at validation ranged from -0.70 to 1.43 (0 for good calibration), whereas the calibration slope ranged from 0.50 to 1.67 (1 for good calibration). The difference in C-statistic and scaled Brier score between derivation and validation ranged from -0.08 to +0.08 and from -0.40 to +0.16, respectively., Conclusion: This study illustrates that predictor measurement heterogeneity can influence the performance of a prediction model substantially, underlining that predictor measurements used in research settings should resemble clinical practice. Specification of measurement heterogeneity can help researchers explaining discrepancies in predictive performance between derivation and validation setting., (Copyright © 2019 The Authors. Published by Elsevier Inc. All rights reserved.)

Preeclampsia (PE) is a hypertensive pregnancy disorder complicating up to 1-5% of pregnancies, and a major cause of maternal and fetal morbidity and mortality. In recent years, observational studies have consistently shown that PE carries an increased risk for the mother to develop cardiovascular and renal disease later in life. Women with a history of PE experience a 2-fold increased risk of long-term cardiovascular disease (CVD) and an approximate 5-12-fold increased risk of end-stage renal disease (ESRD). Recognition of PE as a risk factor for renal disease and CVD allows identification of a young population of women at high risk of developing of cardiovascular and renal disease. For this reason, current guidelines recommend cardiovascular screening and treatment for formerly preeclamptic women. However, these recommendations are based on low levels of evidence due to a lack of studies on screening and prevention in formerly preeclamptic women. This review lists the incidence of premature CVD and ESRD observed after PE and outlines observed abnormalities that might contribute to the increased CVD risk with a focus on kidney-related disturbances. We discuss gaps in current knowledge to guide optimal screening and prevention strategies. We emphasize the need for research on mechanisms of late disease manifestations, and on effective screening and therapeutic strategies aimed at reducing the late disease burden in formerly preeclamptic women., (© 2016 Authors; published by Portland Press Limited.)