Search

To extend previous simulations on the performance of propensity score (PS) weighting and trimming methods to settings without and with unmeasured confounding, Poisson outcomes, and various strengths of treatment prediction (PS c statistic), we simulated studies with a binary intended treatment T as a function of 4 measured covariates. We mimicked treatment withheld and last-resort treatment by adding 2 "unmeasured" dichotomous factors that directed treatment to change for some patients in both tails of the PS distribution. The number of outcomes Y was simulated as a Poisson function of T and confounders. We estimated the PS as a function of measured covariates and trimmed the tails of the PS distribution using 3 strategies ("Crump," "Stürmer," and "Walker"). After trimming and reestimation, we used alternative PS weights to estimate the treatment effect (rate ratio): inverse probability of treatment weighting, standardized mortality ratio (SMR)-treated, SMR-untreated, the average treatment effect in the overlap population (ATO), matching, and entropy. With no unmeasured confounding, the ATO (123%) and "Crump" trimming (112%) improved relative efficiency compared with untrimmed inverse probability of treatment weighting. With unmeasured confounding, untrimmed estimates were biased irrespective of weighting method, and only Stürmer and Walker trimming consistently reduced bias. In settings where unmeasured confounding (e.g., frailty) may lead physicians to withhold treatment, Stürmer and Walker trimming should be considered before primary analysis., (© The Author(s) 2021. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

Confounding can cause substantial bias in nonexperimental studies that aim to estimate causal effects. Propensity score methods allow researchers to reduce bias from measured confounding by summarizing the distributions of many measured confounders in a single score based on the probability of receiving treatment. This score can then be used to mitigate imbalances in the distributions of these measured confounders between those who received the treatment of interest and those in the comparator population, resulting in less biased treatment effect estimates. This methodology was formalized by Rosenbaum and Rubin in 1983 and, since then, has been used increasingly often across a wide variety of scientific disciplines. In this review article, we provide an overview of propensity scores in the context of real-world evidence generation with a focus on their use in the setting of single treatment decisions, that is, choosing between two therapeutic options. We describe five aspects of propensity score analysis: alignment with the potential outcomes framework, implications for study design, estimation procedures, implementation options, and reporting. We add context to these concepts by highlighting how the types of comparator used, the implementation method, and balance assessment techniques have changed over time. Finally, we discuss evolving applications of propensity scores., (© 2020 John Wiley & Sons, Ltd.)

Oncology drug development increasingly relies on single-arm clinical trials. External controls (ECs) derived from electronic health record (EHR) databases may provide additional context. Patients from a US-based oncology EHR database were aligned with patients from randomized controlled trials (RCTs) and trial-specific eligibility criteria were applied to the EHR dataset. Overall survival (OS) in the EC-derived control arm was compared with OS in the RCT experimental arm. The primary outcome was OS, defined as time from randomization or treatment initiation (EHR) to death. Cox regression models were used to obtain effect estimates using EHR data. EC-derived hazard ratio estimates aligned closely with those from the corresponding RCT with one exception. Comparing log HRs among all RCT and EC results gave a Pearson correlation coefficient of 0.86. Properly selected control arms from contemporaneous EHR data could be used to put single-arm trials of OS in advanced non-small cell lung cancer into context., (© 2019 The Authors. Clinical Pharmacology & Therapeutics published by Wiley Periodicals, Inc. on behalf of American Society for Clinical Pharmacology and Therapeutics.)

Pharmacoepidemiologic and pharmacoeconomic analysis of health care databases has become a vital source of evidence to support health care decision making and efficient management of health care organizations. However, decision makers often consider studies done in nonrandomized health care databases more difficult to review than randomized trials because many design choices need to be considered. This is perceived as an important barrier to decision making about the effectiveness and safety of medical products. Design flaws in longitudinal database studies are avoidable but can be unintentionally obscured in the convoluted prose of methods sections, which often lack specificity. We propose a simple framework of graphical representation that visualizes study design implementations in a comprehensive, unambiguous, and intuitive way; contains a level of detail that enables reproduction of key study design variables; and uses standardized structure and terminology to simplify review and communication to a broad audience of decision makers. Visualization of design details will make database studies more reproducible, quicker to review, and easier to communicate to a broad audience of decision makers.

Study size has typically been planned based on statistical power and therefore has been heavily influenced by the philosophy of statistical hypothesis testing. A worthwhile alternative is to plan study size based on precision, for example by aiming to obtain a desired width of a confidence interval for the targeted effect. This article presents formulas for planning the size of an epidemiologic study based on the desired precision of the basic epidemiologic effect measures.

Scientific knowledge changes rapidly, but the concepts and methods of the conduct of research change more slowly. To stimulate discussion of outmoded thinking regarding the conduct of research, I list six misconceptions about research that persist long after their flaws have become apparent. The misconceptions are: 1) There is a hierarchy of study designs; randomized trials provide the greatest validity, followed by cohort studies, with case-control studies being least reliable. 2) An essential element for valid generalization is that the study subjects constitute a representative sample of a target population. 3) If a term that denotes the product of two factors in a regression model is not statistically significant, then there is no biologic interaction between those factors. 4) When categorizing a continuous variable, a reasonable scheme for choosing category cut-points is to use percentile-defined boundaries, such as quartiles or quintiles of the distribution. 5) One should always report P values or confidence intervals that have been adjusted for multiple comparisons. 6) Significance testing is useful and important for the interpretation of data. These misconceptions have been perpetuated in journals, classrooms and textbooks. They persist because they represent intellectual shortcuts that avoid more thoughtful approaches to research problems. I hope that calling attention to these misconceptions will spark the debates needed to shelve these outmoded ideas for good.

Frailty, a poorly measured confounder in older patients, can promote treatment in some situations and discourage it in others. This can create unmeasured confounding and lead to nonuniform treatment effects over the propensity score (PS). The authors compared bias and mean squared error for various PS implementations under PS trimming, thereby excluding persons treated contrary to prediction. Cohort studies were simulated with a binary treatment T as a function of 8 covariates X. Two of the covariates were assumed to be unmeasured strong risk factors for the outcome and present in persons treated contrary to prediction. The outcome Y was simulated as a Poisson function of T and all X's. In analyses based on measured covariates only, the range of PS's was trimmed asymmetrically according to the percentile of PS in treated patients at the lower end and in untreated patients at the upper end. PS trimming reduced bias due to unmeasured confounders and mean squared error in most scenarios assessed. Treatment effect estimates based on PS range restrictions do not correspond to a causal parameter but may be less biased by such unmeasured confounding. Increasing validity based on PS trimming may be a unique advantage of PS's over conventional outcome models.

The attraction of being able to use the internet for the recruitment of an epidemiologic cohort stems mainly from cost efficiency and convenience. The pregnancy planning study ('Snart-Gravid')-a prospective cohort study of Danish women planning a pregnancy-was conducted to evaluate the feasibility and cost efficiency of using internet-based recruitment and follow-up. Feasibility was assessed by examining patient accrual data over time, questionnaire-specific response rates and losses to follow-up. The relative cost efficiency was examined by comparing the study costs with those of an alternative non internet-based study approach. The target recruitment of 2,500 participants over 6 months was achieved using advertisements on a health-related website, supported by a coordinated media strategy at study initiation. Questionnaire cycle-specific response rates ranged from 87 to 90% over the 12-month follow-up. At 6 months, 87% of women had a known outcome or were still under follow-up; at 12 months the figure was 82%. The study cost of $400,000 ($160 per enrolled subject) compared favorably with the estimated cost to conduct the same study using a conventional non-internet based approach ($322 per subject). The gain in efficiency with the internet-based approach appeared to be even more substantial with longer follow-up and larger study sizes. The successful conduct of this pilot study suggests that the internet may be a useful tool to recruit and follow subjects in prospective cohort studies.

Background: Nonexperimental studies of drug effects in large automated databases can provide timely assessment of real-life drug use, but are prone to confounding by variables that are not contained in these databases and thus cannot be controlled., Objectives: To describe how information on additional confounders from validation studies can help address the problem of unmeasured confounding in the main study., Research Design: Review types of validation studies that allow adjustment for unmeasured confounding and illustrate these with an example., Subjects: Main study: New Jersey residents age 65 years or older hospitalized between 1995 and 1997, who filled prescriptions within Medicaid or a pharmaceutical assistance program. Validation study: representative sample of Medicare beneficiaries., Measures: Association between nonsteroidal antiinflammatory drugs (NSAIDs) and mortality., Results: Validation studies are categorized as internal (ie, additional information is collected on participants of the main study) or external. Availability of information on disease outcome will affect choice of analytic strategies. Using an external validation study without data on disease outcome to adjust for unmeasured confounding, propensity score calibration (PSC) leads to a plausible estimate of the association between NSAIDs and mortality in the elderly, if the biases caused by measured and unmeasured confounders go in the same direction., Conclusions: Estimates of drug effects can be adjusted for confounders that are not available in the main, but can be measured in a validation study. PSC uses validation data without information on disease outcome under a strong assumption. The collection and integration of validation data in pharmacoepidemiology should be encouraged.

The most recent (Fifth) revision of the Declaration of Helsinki, adopted in October 2000 by the World Medical Association (WMA), reinforces the longstanding prohibition against offering placebo instead of effective therapy. The WMA left no doubt that if a beneficial treatment for a condition has already been recognised, it is unethical to offer placebo in place of such treatment to anyone in a study of the same condition. We have previously drawn attention to the discrepancy between the spirit of the Declaration and the common practice of using placebo controls in randomised trials even if effective treatment exists. Despite the mandates of the Declaration of Helsinki and concern from ethicists and scientists, the US Food and Drug Administration (FDA) continues to demand and defend placebo-controlled evidence of efficacy and safety for the development of many new pharmaceuticals, even if effective therapy exists. We suggest that the FDA's arguments defending their practice are insufficient to justify medical research that violates the Declaration of Helsinki.

Epidemiologic methods developed to control confounding in non-experimental studies are equally applicable for experiments. In experiments, most confounding is usually controlled by random allocation of subjects to treatment groups, but randomization does not preclude confounding except for extremely large studies, the degree of confounding expected being inversely related to the size of the treatment groups. In experiments, as in non-experimental studies, the extent of confounding for each risk indicator should be assessed, and if sufficiently large, controlled. Confounding is properly assessed by comparing the unconfounded effect estimate to the crude effect estimate; a common error is to assess confounding by statistical tests of significance. Assessment of confounding involves its control as a prerequisite. Control is most readily and cogently achieved by stratification of the data, though with many factors to control simultaneously, multivariate analysis or a combination of multivariate analysis and stratification might be necessary.