Search

PURPOSE: Ideally, the objectives of a pharmacoepidemiologic comparative effectiveness or safety study should dictate its design and data analysis. This paper discusses how defining an estimand is instrumental to this process. METHODS: We applied the ICH-E9 (Statistical Principles for Clinical Trials) R1 addendum on estimands - which originally focused on randomized trials - to three examples of observational pharmacoepidemiologic comparative effectiveness and safety studies. Five key elements specify the estimand: the population, contrasted treatments, endpoint, intercurrent events, and population-level summary measure. RESULTS: Different estimands were defined for case studies representing three types of pharmacological treatments: (1) single-dose treatments using a case study about the effect of influenza vaccination versus no vaccination on mortality risk in an adult population of ≥60 years of age; (2) sustained-treatments using a case study about the effect of dipeptidyl peptidase 4 inhibitor versus glucagon-like peptide-1 agonist on hypoglycemia risk in treatment of uncontrolled diabetes; and (3) as needed treatments using a case study on the effect of nitroglycerin spray as-needed versus no nitroglycerin on syncope risk in treatment of stabile angina pectoris. CONCLUSIONS: The case studies illustrated that a seemingly clear research question can still be open to multiple interpretations. Defining an estimand ensures that the study targets a treatment effect that aligns with the treatment decision the study aims to inform. Estimand definitions further help to inform choices regarding study design and data-analysis and clarify how to interpret study findings.

Additional file 1.

Additional file 3.

Additional file 2.

Additional file 2. Recorded data.

Additional file 3: Table S1. Supplement group characteristics and results: ‘Adduction group’.

ObjectiveTo quantify prediction model performance in relation to data preparation choices when using electronic health records (EHR).Study Design and SettingCox proportional hazards models were developed for predicting the first-ever main adverse cardiovascular events using Dutch primary care EHR data. The reference model was based on a 1-year run-in period, cardiovascular events were defined based on both EHR diagnosis and medication codes, and missing values were multiply imputed. We compared data preparation choices based on (i) length of the run-in period (2- or 3-year run-in); (ii) outcome definition (EHR diagnosis codes or medication codes only); and (iii) methods addressing missing values (mean imputation or complete case analysis) by making variations on the derivation set and testing their impact in a validation set.ResultsWe included 89,491 patients in whom 6,736 first-ever main adverse cardiovascular events occurred during a median follow-up of 8 years. Outcome definition based only on diagnosis codes led to a systematic underestimation of risk (calibration curve intercept: 0.84; 95% CI: 0.83–0.84), while complete case analysis led to overestimation (calibration curve intercept: −0.52; 95% CI: −0.53 to −0.51). Differences in the length of the run-in period showed no relevant impact on calibration and discrimination.ConclusionData preparation choices regarding outcome definition or methods to address missing values can have a substantial impact on the calibration of predictions, hampering reliable clinical decision support. This study further illustrates the urgency of transparent reporting of modeling choices in an EHR data setting.

Additional file 1. Simulation study assessing the impact of predictor measurement heterogeneity across validation and implementation setting in time-to-event outcome data.

Additional file 1 Supplementary material to ���Identification of causal effects in case-control studies���.

Additional file 2. Sensitivity analysis assessing the impact of anticipated predictor measurement heterogeneity across validation and implementation setting in time-to-event outcome data. An example on prediction of incident diabetes type 2.

Choosing a noninferiority margin is one of the main challenges when designing a noninferiority trial. The European Medicines Agency (EMA) published a guidance report on the choice of margins in 2005. Nonetheless, in 2008 and 2009 they did not accept 41% (35 of 86) of the noninferiority margins that were proposed by pharmaceutical companies in the context of scientific-advice letters. In this study, we focus on whether the EMA's recommendations were followed by pharmaceutical companies, and on a possible relationship with eventual drug approval. Five of the 35 unaccepted margins were equivalence margins; we considered only the 30 unaccepted noninferiority margins in our analysis. Twelve of these margins were defined based on clinical and statistical considerations (the approach recommended by the EMA) and were rejected due to unacceptable clinical considerations. The other 18 margins were rejected because they were considered too wide. The EMA's recommendations were followed in the cases of 10 of the 15 margins (67%) for which information on follow-through of recommendations was available. The main reason for ignoring the EMA's recommendation in the other 5 cases was that the margins had been accepted by the US Food and Drug Administration. The proportions of approved drugs for which recommendations were and were not followed were similar, yet numbers were too low for formal statistical testing. This study shows that the main concern of regulators with regard to noninferiority trials was the strictness of margins from a clinical perspective. Future studies using more recent data, including data on the US Food and Drug Administration, may help in assessing the impact of guideline recommendations on noninferiority margins used for drug approval and may assist in reaching consensus among regulators about the choice of margins.

In observational research treatment effects, the average treatment effect (ATE) estimator may be biased if a confounding variable is misclassified. We discuss the impact of classification error in a dichotomous confounding variable in analyses using marginal structural models estimated using inverse probability weighting (MSMs-IPW) and compare this with its impact in conditional regression models, focusing on a point-treatment study with a continuous outcome. Expressions were derived for the bias in the ATE estimator from a MSM-IPW and conditional model by using the potential outcome framework. Based on these expressions, we propose a sensitivity analysis to investigate and quantify the bias due to classification error in a confounding variable in MSMs-IPW. Compared to bias in the ATE estimator from a conditional model, the bias in MSM-IPW can be dissimilar in magnitude but the bias will always be equal in sign. A simulation study was conducted to study the finite sample performance of MSMs-IPW and conditional models if a confounding variable is misclassified. Simulation results showed that confidence intervals of the treatment effect obtained from MSM-IPW are generally wider and coverage of the true treatment effect is higher compared to a conditional model, ranging from over coverage if there is no classification error to smaller under coverage when there is classification error. The use of the bias expressions to inform a sensitivity analysis was demonstrated in a study of blood pressure lowering therapy. It is important to consider the potential impact of classification error in a confounding variable in studies of treatment effects and a sensitivity analysis provides an opportunity to quantify the impact of such errors on causal conclusions. An online tool for sensitivity analyses was developed: https://lindanab.shinyapps.io/SensitivityAnalysis., Comment: 25 pages, 3 figures, 3 tables

Objectives Findings from nonrandomized studies on safety or efficacy of treatment in patient subgroups may trigger postlaunch randomized clinical trials (RCTs). In the analysis of such RCTs, results from nonrandomized studies are typically ignored. This study explores the trade-off between bias and power of Bayesian RCT analysis incorporating information from nonrandomized studies. Study Design and Setting A simulation study was conducted to compare frequentist with Bayesian analyses using noninformative and informative priors in their ability to detect interaction effects. In simulated subgroups, the effect of a hypothetical treatment differed between subgroups (odds ratio 1.00 vs. 2.33). Simulations varied in sample size, proportions of the subgroups, and specification of the priors. Results As expected, the results for the informative Bayesian analyses were more biased than those from the noninformative Bayesian analysis or frequentist analysis. However, because of a reduction in posterior variance, informative Bayesian analyses were generally more powerful to detect an effect. In scenarios where the informative priors were in the opposite direction of the RCT data, type 1 error rates could be 100% and power 0%. Conclusion Bayesian methods incorporating data from nonrandomized studies can meaningfully increase power of interaction tests in postlaunch RCTs.

It is widely acknowledged that the predictive performance of clinical prediction models should be studied in patients that were not part of the data in which the model was derived. Out-of-sample performance can be hampered when predictors are measured differently at derivation and external validation. This may occur, for instance, when predictors are measured using different measurement protocols or when tests are produced by different manufacturers. Although such heterogeneity in predictor measurement between deriviation and validation data is common, the impact on the out-of-sample performance is not well studied. Using analytical and simulation approaches, we examined out-of-sample performance of prediction models under various scenarios of heterogeneous predictor measurement. These scenarios were defined and clarified using an established taxonomy of measurement error models. The results of our simulations indicate that predictor measurement heterogeneity can induce miscalibration of prediction and affects discrimination and overall predictive accuracy, to extents that the prediction model may no longer be considered clinically useful. The measurement error taxonomy was found to be helpful in identifying and predicting effects of heterogeneous predictor measurements between settings of prediction model derivation and validation. Our work indicates that homogeneity of measurement strategies across settings is of paramount importance in prediction research., Comment: 32 pages, 4 figures

In their recent Health Services Research article titled "Squeezing the Balloon: Propensity Scores and Unmeasured Covariate Balance," Brooks and Ohsfeldt (2013) addressed an important topic on the balancing property of the propensity score (PS) with respect to unmeasured covariates. They concluded that PS methods that balance measured covariates between treated and untreated subjects exacerbate imbalance in unmeasured covariates that are unrelated to measured covariates. Furthermore, they emphasized that for PS algorithms, an imbalance on unmeasured covariates between treatment and untreated subjects is a necessary condition to achieve balance on measured covariates between the groups. We argue that these conclusions are the results of their assumptions on the mechanism of treatment allocation. In addition, we discuss the underlying assumptions of PS methods, their advantages compared with multivariate regression methods, as well as the interpretation of the effect estimates from PS methods.