Search

There is a compelling need to evaluate the real-world health effects of medical products outside of tightly controlled preapproval clinical trials. This is done through pharmacoepidemiology, which is the study of the health effects of medical products (including drugs, biologicals, and medical devices and diagnostics) in populations, often using nonrandomized designs. Recent developments in pharmacoepidemiology span changes in the focus of research questions, research designs, data used, and statistical analysis methods. Developments in these areas are thought to improve the value of the evidence produced by such studies, and are prompting greater use of real-world evidence to inform clinical, regulatory, and reimbursement decisions.

Increased interest in real-world evidence (RWE) for clinical and regulatory decision making and the need to evaluate long-term benefits and risks of pharmaceutical products raise the importance of understanding the use of external controls (ECs) for uncontrolled extensions of randomized controlled trials (RCTs). We searched clinicaltrials.gov from 2009 to 2019 for uncontrolled extensions and assessed the use of ECs in the trial protocol registry and PubMed. We present characteristics of identified uncontrolled extensions, their adoption of ECs, and a qualitative appraisal of published uncontrolled extensions with ECs according to good pharmacoepidemiologic practice. The number of uncontrolled extensions increased slightly across the study period, resulting in a total of 1,115 studies. Most originated from phase III RCTs (62.2%) and specified safety outcomes (61.9% among those with specified outcomes). Most uncontrolled extensions incorporated no control group with only 7 out of 1,115 (0.6%) employing ECs. For those studies with ECs, all involved treatments for rare conditions and assessment of effectiveness. Attempts to balance comparison groups varied from none mentioned to propensity score matching. We noted consistent deficiencies in outcome ascertainment methods and approaches to address attrition bias. The contrast of the large and growing number of uncontrolled extensions with the small number of studies that utilized ECs showed clear opportunities for enhancement in design, measurement, and analysis of uncontrolled extensions to allow causal inferences on long-term treatment effects. As extensions continue to expand within RWE regulatory frameworks, development of guidelines for use of EC with uncontrolled extensions is needed.

Prospectively designed studies that make use of real-world data (RWD) are increasingly being used for studies with regulatory implications, including both premarket and post-market studies, aided b...

BackgroundFamotidine has been posited as a potential treatment for COVID-19. We compared the incidence of COVID-19 outcomes (i.e., death; and death or intensive services use) among hospitalized famotidine users vs. proton pump inhibitors (PPIs) users, hydroxychloroquine users or famotidine non-users separately.MethodsWe constructed a retrospective cohort study using data from COVID-19 Premier Hospital electronic health records. Study population were COVID-19 hospitalized patients aged 18 years or older. Famotidine, PPI and hydroxychloroquine exposure groups were defined as patients dispensed any medication containing one of the three drugs on the day of admission. The famotidine non-user group was derived from the same source population with no history of exposure to any drug with famotidine as an active ingredient prior to or on the day of admission. Time-at-risk was defined based on the intention-to-treat principle starting 1 day after admission to 30 days after admission. For each study comparison group, we fit a propensity score (PS) model through large-scale regularized B logistic regression. The outcome was modeled using a survival model.ResultsWe identified 2193 users of PPI, 5950 users of the hydroxychloroquine, 1816 users of famotidine and 26,820 non-famotidine users. After PS stratification, the hazard ratios for death were as follows: famotidine vs no famotidine HR 1.03 (0.89-1.18); vs PPIs: HR 1.14 (0.94-1.39); vs hydroxychloroquine:1.03 (0.85-1.24). Similar results were observed for the risk of death or intensive services use.ConclusionWe found no evidence of a reduced risk of COVID-19 outcomes among hospitalized COVID-19 patients who used famotidine compared to those who did not or compared to PPI or hydroxychloroquine users.

Purpose Clinical trials compare outcomes among patients receiving study treatment with comparators drawn from the same source. These internal controls are missing in single arm trials and from long‐term extensions (LTE) of trials including only the treatment arm. An external control group derived from a different setting is then required to assess safety or effectiveness. Methods We present examples of external control groups that demonstrate some of the issues that arise and make recommendations to address them through careful assessment of the data source fitness for use, design, and analysis steps. Results Inclusion and exclusion criteria and context that produce a trial population may result in trial patients with different clinical characteristics than are present in an external comparison group. If these differences affect the risk of outcomes, then a comparison of outcome occurrence will be confounded. Further, patients who continue into LTE may differ from those initially entering the trial due to treatment effects. Application of appropriate methods is needed to make valid inferences when such treatment or selection effects are present. Outcome measures in a trial may be ascertained and defined differently from what can be obtained in an external comparison group. Differences in sensitivity and specificity for identification or measurement of study outcomes leads to information bias that can also invalidate inferences. Conclusion This review concentrates on threats to the valid use of external control groups both in the scenarios of single arm trials and LTE of randomized controlled trials, along with methodological approaches to mitigate them.

Introduction Observational studies estimating severe outcomes for paracetamol versus ibuprofen use have acknowledged the specific challenge of channeling bias. A previous study relying on negative controls suggested that using large-scale propensity score (LSPS) matching may mitigate bias better than models using limited lists of covariates. Objective The aim was to assess whether using LSPS matching would enable the evaluation of paracetamol, compared to ibuprofen, and increased risk of myocardial infarction, stroke, gastrointestinal (GI) bleeding, or acute renal failure. Study design and setting In a new-user cohort study, we used two propensity score model strategies for confounder controls. One replicated the approach of controlling for a hand-picked list. The second used LSPSs based on all available covariates for matching. Positive and negative controls assessed residual confounding and calibrated confidence intervals. The data source was the Clinical Practices Research Datalink (CPRD). Results A substantial proportion of negative controls were statistically significant after propensity score matching on the publication covariates, indicating considerable systematic error. LSPS adjustment was less biased, but residual error remained. The calibrated estimates resulted in very wide confidence intervals, indicating large uncertainty in effect estimates once residual error was incorporated. Conclusions For paracetamol versus ibuprofen, when using LSPS methods in the CPRD, it is only possible to distinguish true effects if those effects are large (hazard ratio > 2). Due to their smaller hazard ratios, the outcomes under study cannot be differentiated from null effects (represented by negative controls) even if there were a true effect. Based on these data, we conclude that we are unable to determine whether paracetamol is associated with an increased risk of myocardial infarction, stroke, GI bleeding, and acute renal failure compared to ibuprofen, due to residual confounding. Electronic supplementary material The online version of this article (10.1007/s40264-020-00950-3) contains supplementary material, which is available to authorized users.

Objective: Observational evidence suggests that patients with type 2 diabetes mellitus (T2DM) are at increased risk for acute pancreatitis (AP) versus those without T2DM. A small number of AP events were reported in clinical trials of the sodium glucose co-transporter 2 inhibitor canagliflozin, though no imbalances were observed between treatment groups. This observational study evaluated risk of AP among new users of canagliflozin compared with new users of six classes of other antihyperglycemic agents (AHAs). Methods: Three US claims databases were analyzed based on a prespecified protocol approved by the European Medicines Agency. Propensity score adjustment controlled for imbalances in baseline covariates. Cox regression models estimated the hazard ratio of AP with canagliflozin compared with other AHAs using on-treatment (primary) and intent-to-treat approaches. Sensitivity analyses assessed robustness of findings. Results: Across the three databases, there were between 12,023–80,986 new users of canagliflozin; the unadjusted incidence rates of AP (per 1000 person-years) were between 1.5–2.2 for canagliflozin and 1.1–6.6 for other AHAs. The risk of AP was generally similar for new users of canagliflozin compared with new users of glucagon-like peptide-1 receptor agonists, dipeptidyl peptidase-4 inhibitors, sulfonylureas, thiazolidinediones, insulin, and other AHAs, with no consistent between-treatment differences observed across databases. Intent-to-treat and sensitivity analysis findings were qualitatively consistent with on-treatment findings. Conclusions: In this large observational study, incidence rates of AP in patients with T2DM treated with canagliflozin or other AHAs were generally similar, with no evidence suggesting that canagliflozin is associated with increased risk of AP compared with other AHAs.

ImportanceAlthough peer review is an important component of publication for new research, the viability of this process has been questioned, particularly with the added stressors of the COVID-19 pandemic.ObjectiveTo characterize rates of peer reviewer acceptance of invitations to review manuscripts, reviewer turnaround times, and editor-assessed quality of reviews before and after the start of the COVID-19 pandemic at a large, open-access general medical journal.Design, Setting, and ParticipantsThis retrospective, pre-post cohort study examined all research manuscripts submitted to JAMA Network Open between January 1, 2019, and June 29, 2021, either directly or via transfer from other JAMA Network journals, for which at least 1 peer review of manuscript content was solicited. Measures were compared between the period before the World Health Organization declaration of a COVID-19 pandemic on March 11, 2020 (14.3 months), and the period during the pandemic (15.6 months) among all reviewed manuscripts and between pandemic-period manuscripts that did or did not address COVID-19.Main Outcomes and MeasuresFor each reviewed manuscript, the number of invitations sent to reviewers, proportions of reviewers accepting invitations, time in days to return reviews, and editor-assessed quality ratings of reviews were determined.ResultsIn total, the journal sought review for 5013 manuscripts, including 4295 Original Investigations (85.7%) and 718 Research Letters (14.3%); 1860 manuscripts were submitted during the prepandemic period and 3153 during the pandemic period. Comparing the prepandemic with the pandemic period, the mean (SD) number of reviews rated as high quality (very good or excellent) per manuscript increased slightly from 1.3 (0.7) to 1.5 (0.7) (P P Conclusions and RelevanceIn this cohort study, the speed and editor-reported quality of peer reviews in an open-access general medical journal improved modestly during the initial year of the pandemic. Additional study will be necessary to understand how the pandemic has affected reviewer burden and fatigue.

Objectives We propose a one-shot, privacy-preserving distributed algorithm to perform logistic regression (ODAL) across multiple clinical sites. Materials and Methods ODAL effectively utilizes the information from the local site (where the patient-level data are accessible) and incorporates the first-order (ODAL1) and second-order (ODAL2) gradients of the likelihood function from other sites to construct an estimator without requiring iterative communication across sites or transferring patient-level data. We evaluated ODAL via extensive simulation studies and an application to a dataset from the University of Pennsylvania Health System. The estimation accuracy was evaluated by comparing it with the estimator based on the combined individual participant data or pooled data (ie, gold standard). Results Our simulation studies revealed that the relative estimation bias of ODAL1 compared with the pooled estimates was Conclusions This study demonstrates that ODAL is privacy-preserving and communication-efficient with small bias and high statistical efficiency.

Outbreaks of emerging pathogens pose unique methodological and practical challenges for the design, implementation, and evaluation of vaccine efficacy trials. Lessons learned from COVID-19 highlight the need for innovative and flexible study design and application to quickly identify promising candidate vaccines. Trial design strategies should be tailored to the dynamics of the specific pathogen, location of the outbreak, and vaccine prototypes, within the regional socioeconomic constraints. Mathematical and statistical models can assist investigators in designing infectious disease clinical trials. We introduce key challenges for planning, evaluating, and modelling vaccine efficacy trials for emerging pathogens.

ObjectivesGenerating and using real-world evidence (RWE) is a pragmatic solution for evaluating health technologies. RWE is recognized by regulators, health technology assessors, clinicians, and manufacturers as a valid source of information to support their decision-making. Well-designed registries can provide RWE and become more powerful when linked with electronic health records and administrative databases in coordinated registry networks (CRNs). Our objective was to create a framework of maturity of CRNs and registries, so guiding their development and the prioritization of funding.Design, setting, and participantsWe invited 52 stakeholders from diverse backgrounds including patient advocacy groups, academic, clinical, industry and regulatory experts to participate on a Delphi survey. Of those invited, 42 participated in the survey to provide feedback on the maturity framework for CRNs and registries. An expert panel reviewed the responses to refine the framework until the target consensus of 80% was reached. Two rounds of the Delphi were distributed via Qualtrics online platform from July to August 2020 and from October to November 2020.Main outcome measuresConsensus on the maturity framework for CRNs and registries consisted of seven domains (unique device identification, efficient data collection, data quality, product life cycle approach, governance and sustainability, quality improvement, and patient-reported outcomes), each presented with five levels of maturity.ResultsOf 52 invited experts, 41 (79.9%) responded to round 1; all 41 responded to round 2; and consensus was reached for most domains. The expert panel resolved the disagreements and final consensus estimates ranged from 80.5% to 92.7% for seven domains.ConclusionsWe have developed a robust framework to assess the maturity of any CRN (or registry) to provide reliable RWE. This framework will promote harmonization of approaches to RWE generation across different disciplines and health systems. The domains and their levels may evolve over time as new solutions become available.

Focus of Presentation In a prospective meta-analysis (PMA), studies are included before their results are known. This can reduce risk of publication bias and selective outcome reporting, and enables researchers to harmonise their research efforts. Despite rising numbers, there is little guidance on how to conduct PMA. We, the Cochrane PMA Methods Group, developed step-by-step guidance based on a scoping review, and expert opinions and experiences. Each step is illustrated with a recent case study. Findings We describe seven steps for conducting PMA. After developing a protocol (Steps 1), a systematic search for eligible planned/ongoing studies should be conducted, including a search of registries, medical databases and contacting stakeholders (Step 2-3). These studies are then invited to form a collaboration (Step 4), ideally including a steering and data analysis committee. Next, important study features such as common core outcomes and confounders are agreed upon (Step 5). This reduces heterogeneity and increases the number of available outcomes for meta-analysis. Certainty of evidence is assessed by adapting tools such as GRADE (Step 6). Results should be reported using adapted versions of reporting tools such as PRISMA (Step 7). Conclusions/Implications PMA reduce many problems of traditional retrospective systematic reviews and meta-analysis. Updated guidance and recent technical advances will help increase their numbers further. Key messages PMA are ‘next generation systematic reviews’ that allow for greatly improved use of data, whilst reducing bias and research waste. This step-by-step guidance will enable more researchers to conduct successful PMA.

Purpose To compare the incidence of diabetic ketoacidosis (DKA) among patients with type 2 diabetes mellitus (T2DM) who were new users of sodium glucose co‐transporter 2 inhibitors (SGLT2i) versus other classes of antihyperglycemic agents (AHAs). Methods Patients were identified from four large US claims databases using broad (all T2DM patients) and narrow (intended to exclude patients with type 1 diabetes or secondary diabetes misclassified as T2DM) definitions of T2DM. New users of SGLT2i and seven groups of comparator AHAs were matched (1:1) on exposure propensity scores to adjust for imbalances in baseline covariates. Cox proportional hazards regression models, conditioned on propensity score‐matched pairs, were used to estimate hazard ratios (HRs) of DKA for new users of SGLT2i versus other AHAs. When I2

Introduction Psoriasis Longitudinal Assessment and Registry (PSOLAR) was designed in 2007 as the first disease-based registry for patients with psoriasis. Objective The aim of this study was to discuss methodological limitations and post hoc analyses in long-term safety registries using learnings from analyses of a potential safety risk for major adverse cardiovascular events (MACE) in PSOLAR. Methods PSOLAR is an international observational study of over 12,000 psoriasis patients that was conducted to meet postmarketing safety commitments for infliximab and ustekinumab. A recent annual review of registry data indicated a potential MACE risk for ustekinumab vs. non-biologics based on prespecified COX model regression analyses, which yielded an adjusted hazard ratio (HR) of 1.533 (95% confidence interval [CI] 1.103–2.131). Therefore, we conducted a comprehensive review of key statistical methodology and implemented post hoc analytical methods to address specific limitations. Results The following limiting factors were identified: (1) inclusion of both prevalent and incident (new) users of biologics; (2) unanticipated imbalances in patient characteristics between treatment cohorts at baseline; (3) limited availability of relevant clinical data after enrollment; and (4) divergence of characteristics associated with outcomes among comparator groups over time. The analysis was modified to include only incident users, propensity scores were used to weight HRs, and adalimumab was deemed a more clinically appropriate comparator. The revised HR was 0.820 (95% CI 0.532–1.265), indicating no meaningful increase in MACE risk for ustekinumab. Conclusion Our results, which do not support a causal association between ustekinumab exposure and MACE risk, underscore the need for ongoing assessment of analytical methods in long-term observational studies.

Cochrane systematic reviews and meta-analyses are regarded as the ‘gold standard’ for high-quality information and are widely used to inform healthcare policy and practice. The nature of how conventional systematic reviews are conceived and conducted after at least some of the included studies are completed means that reviewers can inadvertently introduce bias when faced with heterogeneous studies that cannot be easily synthesized. Prospective meta-analysis (PMA) is now gaining traction as a means of reducing research waste and producing meaningful and less biased evidence syntheses. PMA has been lauded as a ‘next- generation’ method, and Ioannidis has argued that “all primary original research may be designed, executed, and interpreted as prospective meta-analysis”.

Objective We developed and evaluated a privacy-preserving One-shot Distributed Algorithm to fit a multicenter Cox proportional hazards model (ODAC) without sharing patient-level information across sites. Materials and Methods Using patient-level data from a single site combined with only aggregated information from other sites, we constructed a surrogate likelihood function, approximating the Cox partial likelihood function obtained using patient-level data from all sites. By maximizing the surrogate likelihood function, each site obtained a local estimate of the model parameter, and the ODAC estimator was constructed as a weighted average of all the local estimates. We evaluated the performance of ODAC with (1) a simulation study and (2) a real-world use case study using 4 datasets from the Observational Health Data Sciences and Informatics network. Results On the one hand, our simulation study showed that ODAC provided estimates nearly the same as the estimator obtained by analyzing, in a single dataset, the combined patient-level data from all sites (ie, the pooled estimator). The relative bias was Conclusions ODAC is a privacy-preserving and noniterative method for implementing time-to-event analyses across multiple sites. It provides estimates on par with the pooled estimator and substantially outperforms the meta-analysis estimator when the event is uncommon, making it extremely suitable for studying rare events and diseases in a distributed manner.

In late 2018, the Food and Drug Administration (FDA) outlined a framework for evaluating the possible use of real-world evidence (RWE) to support regulatory decision-making. This framework was created to facilitate studies that would generate high-quality RWE, including pragmatic clinical trials (PCTs), which are randomized trials designed to inform clinical or policy decisions by assessing the real-world effectiveness of an intervention. There is general agreement among experts that the use of existing healthcare and patient-generated data holds promise for making randomized trials more efficient, less costly, and more generalizable. Yet the benefits of relying on real-world data sources must be weighed against difficulties with ensuring data integrity and completeness. Additionally, appropriately monitoring patient safety in randomized trials of new drugs using healthcare system data that might not be available in real time can be quite difficult. Recognizing that these and other concerns are critical to the development and acceptability of PCTs, a group of stakeholders from academia, industry, professional organizations, regulatory bodies, government agencies, and patient advocates discussed a path forward for PCT growth and sustainability at a think tank meeting entitled "Monitoring and Analyzing Data from Pragmatic Streamlined Randomized Clinical Trials," which took place in January 2019 (Washington, DC). The goals of this meeting were to: (1) evaluate study design and methodological options specific to PCTs that have the potential to yield high-quality evidence; (2) discuss best practices to ensure data quality in PCTs; and (3) identify appropriate methods for study monitoring. Proceedings from the think tank meeting are summarized in this manuscript.

In a prospective meta-analysis (PMA), study selection criteria, hypotheses, and analyses are specified before the results of the studies related to the PMA research question are known, reducing many of the problems associated with a traditional (retrospective) meta-analysis. PMAs have many advantages: they can help reduce research waste and bias, and they are adaptive, efficient, and collaborative. Despite an increase in the number of health research articles labelled as PMAs, the methodology remains rare, novel, and often misunderstood. This paper provides detailed guidance on how to address the key elements for conducting a high quality PMA with a case study to illustrate each step.

In a recent study, an indirect treatment comparison was performed to examine the relative efficacy and tolerability of aripiprazole once monthly and paliperidone palmitate once monthly. The authors concluded that the results may suggest relative advantages for aripiprazole once monthly over paliperi

Sodium glucose co-transporter 2 inhibitors (SGLT2i) are indicated for treatment of type 2 diabetes mellitus (T2DM); some SGLT2i have reported cardiovascular benefit, and some have reported risk of below-knee lower extremity (BKLE) amputation. This study examined the real-world comparative effectiveness within the SGLT2i class and compared with non-SGLT2i antihyperglycaemic agents.Data from 4 large US administrative claims databases were used to characterize risk and provide population-level estimates of canagliflozin's effects on hospitalization for heart failure (HHF) and BKLE amputation vs other SGLT2i and non-SGLT2i in T2DM patients. Comparative analyses using a propensity score-adjusted new-user cohort design examined relative hazards of outcomes across all new users and a subpopulation with established cardiovascular disease.Across the 4 databases (142 800 new users of canagliflozin, 110 897 new users of other SGLT2i, 460 885 new users of non-SGLT2i), the meta-analytic hazard ratio estimate for HHF with canagliflozin vs non-SGLT2i was 0.39 (95% CI, 0.26-0.60) in the on-treatment analysis. The estimate for BKLE amputation with canagliflozin vs non-SGLT2i was 0.75 (95% CI, 0.40-1.41) in the on-treatment analysis and 1.01 (95% CI, 0.93-1.10) in the intent-to-treat analysis. Effects in the subpopulation with established cardiovascular disease were similar for both outcomes. No consistent differences were observed between canagliflozin and other SGLT2i.In this large comprehensive analysis, canagliflozin and other SGLT2i demonstrated HHF benefits consistent with clinical trial data, but showed no increased risk of BKLE amputation vs non-SGLT2i. HHF and BKLE amputation results were similar in the subpopulation with established cardiovascular disease. This study helps further characterize the potential benefits and harms of SGLT2i in routine clinical practice to complement evidence from clinical trials and prior observational studies.

Introduced in the 1950s, acetaminophen is one of the most widely used antipyretics and analgesics worldwide. In 1999, the International Agency for Research on Cancer (IARC) reviewed the epidemiologic studies of acetaminophen and the data were judged to be “inadequate” to conclude that it is carcinogenic. In 2019 the California Office of Environmental Health Hazard Assessment initiated a review process on the carcinogenic hazard potential of acetaminophen. To inform this review process, the authors performed a comprehensive literature search and identified 136 epidemiologic studies, which for most cancer types suggest no alteration in risk associated with acetaminophen use. For 3 cancer types, renal cell, liver, and some forms of lymphohematopoietic, some studies suggest an increased risk; however, multiple factors unique to acetaminophen need to be considered to determine if these results are real and clinically meaningful. The objective of this publication is to analyze the results of these epidemiologic studies using a framework that accounts for the inherent challenge of evaluating acetaminophen, including, broad population-wide use in multiple disease states, challenges with exposure measurement, protopathic bias, channeling bias, and recall bias. When evaluated using this framework, the data do not support a causal association between acetaminophen use and cancer.

Aims To examine the incidence of amputation in patients with type 2 diabetes mellitus (T2DM) treated with sodium glucose co‐transporter 2 (SGLT2) inhibitors overall, and canagliflozin specifically, compared with non‐SGLT2 inhibitor antihyperglycaemic agents (AHAs). Materials and Methods Patients with T2DM newly exposed to SGLT2 inhibitors or non‐SGLT2 inhibitor AHAs were identified using the Truven MarketScan database. The incidence of below‐knee lower extremity (BKLE) amputation was calculated for patients treated with SGLT2 inhibitors, canagliflozin, or non‐SGLT2 inhibitor AHAs. Patients newly exposed to canagliflozin and non‐SGLT2 inhibitor AHAs were matched 1:1 on propensity scores, and a Cox proportional hazards model was used for comparative analysis. Negative controls (outcomes not believed to be associated with any AHA) were used to calibrate P values. Results Between April 1, 2013 and October 31, 2016, 118 018 new users of SGLT2 inhibitors, including 73 024 of canagliflozin, and 226 623 new users of non‐SGLT2 inhibitor AHAs were identified. The crude incidence rates of BKLE amputation were 1.22, 1.26 and 1.87 events per 1000 person‐years with SGLT2 inhibitors, canagliflozin and non‐SGLT2 inhibitor AHAs, respectively. For the comparative analysis, 63 845 new users of canagliflozin were matched with 63 845 new users of non‐SGLT2 inhibitor AHAs, resulting in well‐balanced baseline covariates. The incidence rates of BKLE amputation were 1.18 and 1.12 events per 1000 person‐years with canagliflozin and non‐SGLT2 inhibitor AHAs, respectively; the hazard ratio was 0.98 (95% confidence interval 0.68–1.41; P = .92, calibrated P = .95). Conclusions This real‐world study observed no evidence of increased risk of BKLE amputation for new users of canagliflozin compared with non‐SGLT2 inhibitor AHAs in a broad population of patients with T2DM.

Introduction Over-the-counter analgesics such as paracetamol and ibuprofen are among the most widely used, and having a good understanding of their safety profile is important to public health. Prior observational studies estimating the risks associated with paracetamol use acknowledge the inherent limitations of these studies. One threat to the validity of observational studies is channeling bias, i.e. the notion that patients are systematically exposed to one drug or the other, based on current and past comorbidities, in a manner that affects estimated relative risk. Objectives The aim of this study was to examine whether evidence of channeling bias exists in observational studies that compare paracetamol with ibuprofen, and, if so, the extent to which confounding adjustment can mitigate this bias. Study Design and Setting In a cohort of 140,770 patients, we examined whether those who received any paracetamol (including concomitant users) were more likely to have prior diagnoses of gastrointestinal (GI) bleeding, myocardial infarction (MI), stroke, or renal disease than those who received ibuprofen alone. We compared propensity score distributions between drugs, and examined the degree to which channeling bias could be controlled using a combination of negative control disease outcome models and large-scale propensity score matching. Analyses were conducted using the Clinical Practice Research Datalink. Results The proportions of prior MI, GI bleeding, renal disease, and stroke were significantly higher in those prescribed any paracetamol versus ibuprofen alone, after adjusting for sex and age. We were not able to adequately remove selection bias using a selected set of covariates for propensity score adjustment; however, when we fit the propensity score model using a substantially larger number of covariates, evidence of residual bias was attenuated. Conclusions Although using selected covariates for propensity score adjustment may not sufficiently reduce bias, large-scale propensity score matching offers a novel approach to consider to mitigate the effects of channeling bias. Electronic supplementary material The online version of this article (doi:10.1007/s40264-017-0581-7) contains supplementary material, which is available to authorized users.

Background: A key step in the design of a randomised controlled trial (RCT) is the estimation of the number of participants needed. By far the most common approach is to specify a target difference and then estimate the corresponding sample size; this sample size is chosen to provide reassurance that the trial will have high statistical power to detect such a difference between the randomised groups (at the planned statistical significance level). The sample size has many implications for the conduct of the study, as well as carrying scientific and ethical aspects to its choice. Despite the critical role of the target difference for the primary outcome in the design of an RCT, the manner in which it is determined has received little attention. This article reports the protocol of the Difference ELicitation in TriAls (DELTA2) project, which will produce guidance on the specification and reporting of the target difference for the primary outcome in a sample size calculation for RCTs. Methods/design: The DELTA2 project has five components: systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2); a Delphi study (stage 3); a 2-day consensus meeting bringing together researchers, funders and patient representatives, as well as one-off engagement sessions at relevant stakeholder meetings (stage 4); and the preparation and dissemination of a guidance document (stage 5). Discussion: Specification of the target difference for the primary outcome is a key component of the design of an RCT. There is a need for better guidance for researchers and funders regarding specification and reporting of this aspect of trial design. The aim of this project is to produce consensus based guidance for researchers and funders.

OBJECTIVES Doubling on average every 6 years, hypertension-related meta-analyses are now published twice weekly and are often considered the highest level of evidence for clinical practice. However, some hypertension specialists and guideline authors view meta-analyses with skepticism. This article evaluates the quality of hypertension-related meta-analyses of clinical trials. METHODS A systematic search was conducted for meta-analyses of clinical trials recently published over 3.3 years. Specific criteria reproducibly assessed 26 features in the four domains of meta-analysis quality, domains justified by fundamental analytics and extensive research: analyzing trial quality, analyzing heterogeneity, analyzing publication bias, and providing transparency. RESULTS A total of 143 meta-analyses were identified. A total of 44% had 8+ deficient features with no relation to journal impact factor: odds ratio relating 8+ deficient features to the upper third versus lower third of impact factor = 1.3 (95% confidence limit 0.6-2.9). A total of 56% had all four domains deficient. Quality did not improve over time. Thirty articles (21%) reported statistically significant results (P

Background The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the ‘target difference’ and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups. Objective The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives. Methods The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document. Results Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered. Conclusions Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items. Funding Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC–National Institute for Health Research Methodology Research programme.

Observational evidence suggests that patients with type 2 diabetes mellitus (T2DM) are at increased risk of AP. A small number of AP events were reported in clinical trials of the SGLT2 inhibitor CANA, with no imbalances noted between treatment groups. This observational study evaluated the risk of AP among new users of CANA vs. 6 classes of other AHAs. Three U.S. claims databases were analyzed based on a prespecified protocol approved by the European Medicines Agency. Propensity score adjustment controlled for imbalances in baseline covariates. Cox regression models were used to estimate the hazard ratio of AP with CANA vs. other AHAs using on-treatment and intent-to-treat (ITT) approaches. Sensitivity analyses assessed the robustness of study findings. Across the 3 databases, there were between 12,023 to 80,986 new users of CANA and the crude incidence rates of AP (events per 1000 person-years) were between 1.5 to 2.2 for CANA and 1.1 to 6.6 for other AHAs. AP risk in patients newly treated with CANA or other AHAs was generally similar (Table). Findings from the ITT approach and sensitivity analyses were qualitatively consistent with the on-treatment approach. In this large observational study, incidence rates of AP in patients with T2DM treated with CANA or other AHAs were generally similar, with no evidence to suggest that CANA is associated with an increased risk of AP compared with other AHAs. Disclosure Z. Yuan: Employee; Self; Janssen Research & Development. F. Defalco: Employee; Self; Janssen Research & Development. L. Wang: Employee; Self; Johnson & Johnson. L.L. Hester: Employee; Self; Janssen Research & Development. J.N. Swerdel: Employee; Self; Janssen Pharmaceuticals, Inc. A. Freedman: Employee; Self; Janssen Research & Development. P. Ryan: Employee; Self; Janssen Research & Development. M. Schuemie: Employee; Self; Janssen Research & Development. R. Qiu: Employee; Self; Janssen Research & Development. J. Yee: Employee; Self; Janssen Research & Development. G. Meininger: Employee; Self; Janssen Research & Development. J.A. Berlin: Employee; Self; Janssen Research & Development. Stock/Shareholder; Self; Janssen Research & Development. N. Rosenthal: Employee; Self; Janssen Research & Development. Funding Janssen Research & Development, LLC

Introduction: Diabetic ketoacidosis (DKA) is a serious acute metabolic complication of diabetes. Rare DKA events have occurred in patients taking sodium glucose co-transporter 2 inhibitors (SGLT2i). This study evaluated the risk of DKA in patients with type 2 diabetes mellitus (T2DM) taking SGLT2i versus other antihyperglycemic agents (AHAs) in clinical practice. Methods: This study, per protocol reviewed and approved by the European Medicines Agency, identified patients from 4 large US claims databases using broad and narrow definitions of T2DM; the broad definition captured all patients with a T2DM diagnosis and the narrow definition was intended to exclude T1DM misclassified as T2DM. DKA was identified from diagnosis codes in inpatient or emergency room claims. Eligible new users of SGLT2i and 7 groups of AHA comparators were matched (1:1) on exposure propensity scores (PS) to adjust for imbalances in baseline covariates. Cox proportional hazard models conditioned on PS-matched pairs were used to estimate hazard ratios (HR) of DKA risk for new users of SGLT2i versus other AHAs. P values were calibrated using negative control outcomes to address potential residual bias. Pooled HRs were calculated when I 2 was Results: The number of new users of SGLT2i in each database ranged from 11,141 to 152,728 using the broad T2DM definition and from 7,779 to 130,708 using the narrow definition. Across databases, the unadjusted incidence rates of DKA (events per 1000 patient-years) ranged from 2.75 to 8.84 with SGLT2i and 1.38 to 15.82 with other AHAs using the broad T2DM definition and from 1.15 to 3.91 with SGLT2i and 0.75 to 7.94 with other AHAs using the narrow definition. Using the broad T2DM definition, a significantly increased risk of DKA was observed among new users of SGLT2i versus 5 groups of other AHAs; when using the narrow definition, an increased risk of DKA with SGLT2i was observed only compared with sulfonylureas ( Figure ). Conclusion: In this claims database study, an increased risk of DKA was observed for new users of SGLT2i versus new users of several non-SGLT2i AHAs when T2DM was defined broadly. When T2DM was defined narrowly to exclude possible misdiagnosed T1DM patients, an increased risk of DKA with SGLT2i was observed compared to sulfonylureas.

Objectives Data Abstraction Assistant (DAA) is a software for linking items abstracted into a data collection form for a systematic review to their locations in a study report. We conducted a randomized cross-over trial that compared DAA-facilitated single-data abstraction plus verification (“DAA verification”), single data abstraction plus verification (“regular verification”), and independent dual data abstraction plus adjudication (“independent abstraction”). Study Design and Setting This study is an online randomized cross-over trial with 26 pairs of data abstractors. Each pair abstracted data from six articles, two per approach. Outcomes were the proportion of errors and time taken. Results Overall proportion of errors was 17% for DAA verification, 16% for regular verification, and 15% for independent abstraction. DAA verification was associated with higher odds of errors when compared with regular verification (adjusted odds ratio [OR] = 1.08; 95% confidence interval [CI]: 0.99–1.17) or independent abstraction (adjusted OR = 1.12; 95% CI: 1.03–1.22). For each article, DAA verification took 20 minutes (95% CI: 1–40) longer than regular verification, but 46 minutes (95% CI: 26 to 66) shorter than independent abstraction. Conclusion Independent abstraction may only be necessary for complex data items. DAA provides an audit trail that is crucial for reproducible research.

Many observational studies explore the association between acetaminophen and cancer, but known limitations such as vulnerability to channeling, protopathic bias, and uncontrolled confounding hamper the interpretability of results. To help understand the potential magnitude of bias, we identify key design choices in these observational studies and specify 10 study design variants that represent different combinations of these design choices. We evaluate these variants by applying them to 37 negative controls - outcome presumed not to be caused by acetaminophen - as well as 4 cancer outcomes in the Clinical Practice Research Datalink (CPRD) database. The estimated odds and hazards ratios for the negative controls show substantial bias in the evaluated design variants, with far fewer of the 95% confidence intervals containing 1 than the nominal 95% expected for negative controls. The effect-size estimates for the cancer outcomes are comparable to those observed for the negative controls. A comparison of exposed and unexposed reveals many differences at baseline for which most studies do not correct. We observe that the design choices made in many of the published observational studies can lead to substantial bias. Thus, caution in the interpretation of published studies of acetaminophen and cancer is recommended.

In alignment with the International Council of Harmonization’s strategic goals, a public-private consortium has developed a structured template for planning and reporting on the implementation of real world evidence (RWE) studies of the safety and effectiveness of treatments. The template serves as a guiding tool for designing and conducting reproducible RWE studies; set clear expectations for transparent communication of RWE methods; reduce misinterpretation of prose that lacks specificity; allow reviewers to quickly orient and find key information; and facilitate reproducibility, validity assessment, and evidence synthesis. The template is intended for use with studies of the effectiveness and safety of medical products and is compatible with multiple study designs, data sources, reporting guidelines, checklists, and bias assessment tools.

A key challenge in benefit–risk (B-R) assessment of a medication is the multitude of data sources and the changing quality and relevance of these sources during the medication lifecycle. At the time of regulatory approval, B-R assessment is largely based on data from controlled clinical trials and preclinical studies. Following approval, data informing B-R accumulate in a broader, larger sample of subjects in clinical practice and post-approval studies, and it can be challenging to know how to appropriately aggregate and compare these datasets. In this article, we critically evaluate different data sources that may be used in B-R assessment, including controlled trials, observational data, and spontaneous reports. We demonstrate how these sources may be used to create an effects table (summary of evidence of key efficacy and safety data), using dabigatran, a nonvitamin K antagonist oral anticoagulant (NOAC), as a case example. We discuss how to compare quantitatively across studies when population...

The Yale University Open Data Access (YODA) Project has facilitated access to clinical trial data since 2013. The purpose of this article is to provide an overview of the Project, describe key decisions that were made when establishing data sharing policies, and suggest how our experience and the experiences of our first two data generator partners, Medtronic, Inc. and Johnson & Johnson, can be used to enhance other ongoing or future initiatives.

Background This article on choosing the target difference for a randomised controlled trial (RCT) and undertaking and reporting the sample size calculation has been dual published in the BMJ and BMC Trials journals. A key step in the design of a RCT is the estimation of the number of participants needed in the study. The most common approach is to specify a target difference between the treatments for the primary outcome and then calculate the required sample size. The sample size is chosen to ensure that the trial will have a high probability (adequate statistical power) of detecting a target difference between the treatments should one exist. The sample size has many implications for the conduct and interpretation of the study. Despite the critical role that the target difference has in the design of a RCT, the way in which it is determined has received little attention. In this article, we summarise the key considerations and messages from new guidance for researchers and funders on specifying the target difference, and undertaking and reporting a RCT sample size calculation. Methods The DELTA2 (Difference ELicitation in TriAls) project comprised five major components: systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2); a Delphi study (stage 3); a two-day consensus meeting bringing together researchers, funders and patient representatives (stage 4); and the preparation and dissemination of a guidance document (stage 5). Results and Discussion The key messages from the DELTA2 guidance on specifying the target difference are presented. Recommendations for specifying the target difference and the subsequent reporting of the sample size calculation are provided.

The aim of this document is to provide practical guidance on the choice of target difference used in the sample size calculation of a randomised controlled trial (RCT). Guidance is provided with a definitive trial, one that seeks to provide a useful answer, in mind and not those of a more exploratory nature. The term “target difference” is taken throughout to refer to the difference that is used in the sample size calculation (the one that the study formally “targets”). Please see the glossary for definitions and clarification with regards other relevant concepts. In order to address the specification of the target difference, it is appropriate, and to some degree necessary, to touch on related statistical aspects of conducting a sample size calculation. Generally the discussion of other aspects and more technical details is kept to a minimum, with more technical aspects covered in the appendices and referencing of relevant sources provided for further reading.The main body of this guidance assumes a standard RCT design is used; formally, this can be described as a two-arm parallel-group superiority trial. Most RCTs test for superiority of the interventions, that is, whether or not one of the interventions is superior to the other (See Box 1 for a formal definition of superiority, and of the two most common alternative approaches). Some common alternative trial designs are considered in Appendix 3. Additionally, it is assumed in the main body of the text that the conventional (Neyman-Pearson) approach to the sample size calculation of an RCT is being used. Other approaches (Bayesian, precision and value of information) are briefly considered in Appendix 2 with reference to the specification of the target difference.

Sodium glucose co-transporter 2 inhibitors (SGLT2i) are indicated for treatment of T2DM; some SGLT2i have reported a CV benefit and some reported a risk of BKA. U.S. claims databases were analyzed using a prespecified protocol to examine CANA-associated effects on BKA and hospitalization for heart failure (HHF) vs. other SGLT2i and non-SGLT2i. Analyses used a propensity score adjusted new user design with numerous sensitivity analyses. The 4 databases included 142K new users of CANA, 110K of other SGLT2i, and 460K of non-SGLT2i AHAs. Meta-analysis results are reported when heterogeneity across databases was not substantial (I2 Disclosure P. Ryan: Employee; Self; Janssen Research & Development, LLC. J.B. Buse: Other Relationship; Self; ADOCIA, AstraZeneca, Dexcom, Inc., Elcelyx Therapeutics, Inc., Eli Lilly and Company, Fractyl Laboratories, Inc., Intarcia Therapeutics, Inc., Lexicon Pharmaceuticals, Inc., Metavention, NovaTarg, Novo Nordisk A/S, Sanofi, VTV Therapeutics. Research Support; Self; Boehringer Ingelheim GmbH, Johnson & Johnson Services, Inc., Theracos, Inc.. Other Relationship; Self; Shenzhen Hightide Biopharmaceutical, Ltd.. Research Support; Self; National Heart, Lung, and Blood Institute, National Center for Advancing Translational Sciences. Other Relationship; Self; National Institute of Diabetes and Digestive and Kidney Diseases, American Diabetes Association. Research Support; Self; Patient-Centered Outcomes Research Institute. Other Relationship; Self; National Institute of Environmental Health Sciences. M. Schuemie: Employee; Self; Janssen Research & Development, LLC. F. Defalco: Employee; Self; Janssen Research & Development, LLC. Z. Yuan: Employee; Self; Janssen Research & Development, LLC. P. Stang: Employee; Self; Janssen Research & Development, LLC. J.A. Berlin: Employee; Self; Johnson & Johnson, LLC. N. Rosenthal: Employee; Self; Janssen Research & Development.

Importance: While guidance on statistical principles for clinical trials exists, there is an absence of guidance covering the required content of statistical analysis plans (SAPs) to support transparency and reproducibility.Objective: To develop recommendations for a minimum set of items that should be addressed in SAPs for clinical trials, developed with input from statisticians, previous guideline authors, journal editors, regulators, and funders.Design: Funders and regulators (n = 39) of randomized trials were contacted and the literature was searched to identify existing guidance; a survey of current practice was conducted across the network of UK Clinical Research Collaboration–registered trial units (n = 46, 1 unit had 2 responders) and a Delphi survey (n = 73 invited participants) was conducted to establish consensus on SAPs. The Delphi survey was sent to statisticians in trial units who completed the survey of current practice (n = 46), CONSORT (Consolidated Standards of Reporting Trials) and SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) guideline authors (n = 16), pharmaceutical industry statisticians (n = 3), journal editors (n = 9), and regulators (n = 2) (3 participants were included in 2 groups each), culminating in a consensus meeting attended by experts (N = 12) with representatives from each group. The guidance subsequently underwent critical review by statisticians from the surveyed trial units and members of the expert panel of the consensus meeting (N = 51), followed by piloting of the guidance document in the SAPs of 5 trials.Findings: No existing guidance was identified. The registered trials unit survey (46 responses) highlighted diversity in current practice and confirmed support for developing guidance. The Delphi survey (54 of 73, 74% participants completing both rounds) reached consensus on 42% (n = 46) of 110 items. The expert panel (N = 12) agreed that 63 items should be included in the guidance, with an additional 17 items identified as important but may be referenced elsewhere. Following critical review and piloting, some overlapping items were combined, leaving 55 items.Conclusions and Relevance: Recommendations are provided for a minimum set of items that should be addressed and included in SAPs for clinical trials. Trial registration, protocols, and statistical analysis plans are critically important in ensuring appropriate reporting of clinical trials.