Search

Kristian Thorlund,1,2 Louis Dron,2 Jay JH Park,2,3 Edward J Mills1,2 1Department of Health Research Methods, Evidence & Impact (HEI), McMaster University, Hamilton, ON, Canada; 2MTEK Sciences, Vancouver, BC, Canada; 3Department of Medicine, University of British Columbia, Vancouver, BC, CanadaCorrespondence: Kristian ThorlundMTEK Sciences, 802-777 West Broadway, Vancouver, BC V5Z 1J5, CanadaEmail thorluk@mcmaster.caAbstract: There has been a rapid expansion in the use of non-randomized evidence in the regulatory approval of treatments globally. An emerging set of methodologies have been utilized to provide greater insight into external control data used for these purposes, collectively known as synthetic control methods. Through this paper, we provide the reader with a set of key questions to help assess the quality of literature publications utilizing synthetic control methodologies. Common challenges and real-life examples of synthetic controls are provided throughout, alongside a critical appraisal framework with which to assess future publications.Keywords: synthetic control, RCTs, real-world evidence

Jay JH Park,1 Kristian Thorlund,2,3 Edward J Mills2,3 1Department of Medicine, University of British Columbia, Vancouver, BC, Canada; 2Department of Health Research Methods, Evidence, and Impact (HEI), McMaster University, Hamilton, ON, Canada; 3The Bill and Melinda Gates Foundation, Seattle, WA, USA Abstract: Adaptive clinical trials are an innovative trial design aimed at reducing resources, decreasing time to completion and number of patients exposed to inferior interventions, and improving the likelihood of detecting treatment effects. The last decade has seen an increasing use of adaptive designs, particularly in drug development. They frequently differ importantly from conventional clinical trials as they allow modifications to key trial design components during the trial, as data is being collected, using preplanned decision rules. Adaptive designs have increased likelihood of complexity and also potential bias, so it is important to understand the common types of adaptive designs. Many clinicians and investigators may be unfamiliar with the design considerations for adaptive designs. Given their complexities, adaptive trials require an understanding of design features and sources of bias. Herein, we introduce some common adaptive design elements and biases and specifically address response adaptive randomization, sample size reassessment, Bayesian methods for adaptive trials, seamless trials, and adaptive enrichment using real examples. Keywords: adaptive designs, response adaptive randomization, sample size reassessment, Bayesian adaptive trials, seamless trials, adaptive enrichment

Chakrapani Balijepalli,1,2 Eric Druyts,2 Gaye Siliman,2 Michel Joffres,1 Kristian Thorlund,2,3 Edward J Mills3 1Faculty of Health Sciences, Simon Fraser University, Burnaby, 2Precision Health Economics, Vancouver, BC, 3Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada Objective: To understand the severity and potential impact of heterogeneity in definitions of hypoglycemia used in diabetes research, we aimed to review the hypoglycemia definitions adopted in randomized controlled trials (RCTs).Methods: We reviewed 109 RCTs included in the Canadian Agency for Drugs and Technologies in Health reports for the second- and third-line therapy for the patients with type 2 diabetes (T2D).Results: Nearly 60% (n=66) of the studies reviewed presented the definitions for overall hypoglycemia, and another 20% (n=22) of the studies reported the results for hypoglycemia but did not report a definition. Among these 66 studies, only 9 (14%) followed the American Diabetes Association/European Medicines Agency specified guidelines to define hypoglycemia, with an exact threshold of plasma glucose ≤3.9 mmol/L. Fifty-two of the 66 studies (79%) used a threshold considerably lower than the recommended ≤3.9 mmol/L, and 16 studies used a threshold between 3.8 and 4.0 mmol/L. The proportion of the trials that used a cutoff value of

Kristian Thorlund,1–3 Eric Druyts,1,4 Kabirraaj Toor,1,5 Jeroen P Jansen,1,6 Edward J Mills1,3 1Redwood Outcomes, Vancouver, BC, 2Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada; 3Stanford Prevention Research Center, Stanford University, Stanford, CA, USA; 4Department of Medicine, Faculty of Medicine, 5School of Population and Public Health, Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada; 6Department of Public Health and Community Medicine, Tufts University, Boston, MA, USA Introduction: Network meta-analysis (NMA) is an extension of conventional pairwise meta-analysis that allows for simultaneous comparison of multiple interventions. Well-established drug class efficacies have become commonplace in many disease areas. Thus, for reasons of ethics and equipoise, it is not practical to randomize patients to placebo or older drug classes. Unique randomized clinical trial designs are an attempt to navigate these obstacles. These alternative designs, however, pose challenges when attempting to incorporate data into NMAs. Using ulcerative colitis as an example, we illustrate an example of a method where data provided by these trials are used to populate treatment networks. Methods: We present the methods used to convert data from the PURSUIT trial into a typical parallel design for inclusion in our NMA. Data were required for three arms: golimumab 100 mg; golimumab 50 mg; and placebo. Golimumab 100 mg induction data were available; however, data regarding those individuals who were nonresponders at induction and those who were responders at maintenance were not reported, and as such, had to be imputed using data from the rerandomization phase. Golimumab 50 mg data regarding responses at week 6 were not available. Existing relationships between the available components were used to impute the expected proportions in this missing subpopulation. Data for placebo maintenance response were incomplete, as all induction nonresponders were assigned to golimumab 100 mg. Data from the PURSUIT trial were combined with ACT-1 and ULTRA-2 trial data to impute missing information. Discussion: We have demonstrated methods for converting results from alternative study designs to more conventional parallel randomized clinical trials. These conversions allow for indirect treatment comparisons that are informed by a wider array of evidence, adding to the precision of estimates. Keywords: adaptive, network meta-analysis, indirect treatment comparison, ulcerative colitis, golimumab

Kristian Thorlund,1,2 Eric Druyts,3,4 Edward J Mills1,4 1Stanford Prevention Research Center, Stanford University, Stanford, CA, USA; 2Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada; 3School of Population and Public Health, University of British Columbia, Vancouver, BC, Canada; 4Faculty of Health Sciences, University of Ottawa, Ottawa, ON, Canada Background: Randomized clinical trials (RCTs) of interventions for the hepatitis C virus have historically used sustained virological response (SVR) at 24 weeks after treatment (SVR24) as the key effect measure. However, recent RCTs investigating the efficacy of new direct acting agents (DAAs) have used SVR at 12 weeks after treatment (SVR12). While there is evidence to suggest SVR24 and SVR12 are similar in patients receiving new DAAs, this is unlikely to be true for patients receiving backbone peginterferon-ribavirin control treatment. Establishing the difference between SVR12 and SVR24 for patients receiving peginterferon-ribavirin treatment is therefore necessary to avoid biased interpretations of the benefits of newer DAAs. Methods: We searched the MEDLINE®, Embase™, and Cochrane CENTRAL for RCTs with a peginterferon-ribavirin arm that used SVR24 and/or SVR12. As no RCTs reported on both, we pooled SVR12 and SVR24 proportions using conventional meta-analysis. Proportions were pooled separately for peginterferon alpha-2a and alpha-2b. Further, a Bayesian meta-regression model was employed to estimate the difference between SVR12 and SVR24. Results: Thirty-five RCTs including a peginterferon arm were identified. Twenty-four trials included a peginterferon alpha-2a plus ribavirin arms, of which 20 reported SVR24 and five reported SVR12. Seventeen trials included a peginterferon alpha-2b plus ribavirin arm, of which 16 reported SVR24 and one reported SVR12. Using Bayesian meta-regression, the pooled SVR12 was 6% higher than SVR24 with peginterferon alpha-2a (53% versus 47%) and 5% higher with peginterferon alpha-2b (45% versus 40%) and 95% credible intervals (CrIs) were only marginally overlapping. The meta-regression also demonstrated a marginally significant relative risk of 1.13 (95% CrI 0.99–1.26) of SVR12 versus SVR24. The conventional pairwise meta-analyses were consistent with these findings. Conclusion: Considering the relatively large difference observed between SVR12 and SVR24, it seems reasonable to insist that future clinical trials report both to allow for complete transparency and clarity in their interpretation. Keywords: sustained virological response, meta-regression, direct acting antivirals

Eric Druyts,1 Kristian Thorlund,2,3 Samantha Humphreys,4 Michaela Lion,4 Curtis L Cooper,5 Edward J Mills1,31Faculty of Health Sciences, University of Ottawa, Ottawa, Ontario, Canada; 2Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada; 3Stanford Prevention Research Centre, Department of Medicine, Stanford University, Palo Alto, CA, USA; 4Merck Sharp and Dohme Ltd, UK; 5Division of Infectious Diseases, Department of Medicine, The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, CanadaAbstract: Indirect treatment comparison (ITC) and multiple treatment comparison (MTC) meta-analyses are increasingly being used to estimate the comparative effectiveness of interventions when head-to-head data do not exist. ITC meta-analyses can be conducted using simple methodology to compare two interventions. MTC meta-analyses can be conducted using more complex methodology, often employing Bayesian approaches, to compare multiple interventions. As the number of ITC and MTC meta-analyses increase, it is common to find multiple analyses evaluating the same interventions in similar therapeutic areas. Depending on the choice of the methodological approach, the conclusions about relative treatment efficacy may differ. Such situations create uncertainty for decision makers. An illustration of this is provided by four ITC and MTC meta-analyses assessing the efficacy of boceprevir and telaprevir for chronic hepatitis C virus infection. This paper examines why these evaluations provide discordant results by examining specific methodological issues that can strengthen or weaken inferences.Keywords: indirect treatment comparison, multiple treatment comparison, meta-analysis, hepatitis C virus

Kristian Thorlund,1 Eric Druyts,2 J Antonio Aviña-Zubieta,3,4 Edward J Mills1,21Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada; 2Faculty of Health Sciences, University of Ottawa, Ottawa, Ontario, Canada; 3Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada; 4Division of Rheumatology, Department of Medicine, University of British Columbia, Vancouver, British Columbia, CanadaObjective: To evaluate the comparative effectiveness of available tumor necrosis factor-a inhibitors (anti-TNFs) for the management of psoriatic arthritis (PsA) in patients with an inadequate response to disease-modifying antirheumatic drugs (DMARDs).Methods: We used an exhaustive search strategy covering randomized clinical trials, systematic reviews and health technology assessments (HTA) published on anti-TNFs for PsA. We performed indirect comparisons of the available anti-TNFs (adalimumab, etanercept, golimumab, and infliximab) measuring relative risks (RR) for the psoriatic arthritis response criteria (PsARC), mean differences (MDs) for improvements from baseline for the Health Assessment Questionnaire (HAQ) by PsARC responders and non-responders, and MD for the improvements from baseline for the psoriasis area and severity index (PASI). When the reporting of data on intervention group response rates and improvements were incomplete, we used straightforward conversions based on the available data.Results: We retrieved data from 20 publications representing seven trials, as well as two HTAs. All anti-TNFs were significantly better than control, but the indirect comparison did not reveal any statistically significant difference between the anti-TNFs. For PsARC response, golimumab yielded the highest RR and etanercept the second highest; adalimumab and infliximab both yielded notably smaller RRs. For HAQ improvement, etanercept and infliximab yielded the largest MD among PsARC responders. For PsARC nonresponders, etanercept, infliximab, and golimumab yielded similar MDs, and adalimumab a notably lower MD. For PASI improvement, infliximab yielded the largest MD and golimumab the second largest, while etanercept yielded the smallest MD. In some instances, the estimated magnitudes of effect were notably different from the estimates of previous HTA indirect comparisons.Conclusion: There is insufficient statistical evidence to demonstrate differences in effectiveness between available anti-TNFs for PsA. Effect estimates seem sensitive to the analytic approach, and this uncertainty should be taken into account in future economic evaluations.Keywords: anti-tumour necrosis factor drugs, biologic DMARDs, indirect comparison meta-analysis, psoriatic arthritis, health assessment questionnaire, psoriatic arthritis response criteria, psoriasis area and severity index

Kristian Thorlund,1 Eric Druyts,2 Antoine C El Khoury,3 Edward J Mills21Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, 2Faculty of Health Sciences, University of Ottawa, Ottawa, Ontario, Canada; 3Merck Sharp and Dohme, Whitehouse Station, NJ, USABackground: Boceprevir and telaprevir have recently showed dramatically better treatment outcomes than conventional PEGylated interferon plus ribavirin for the treatment of hepatitis C virus genotype 1, but the average cost per patient is unknown.Methods: In the UK context, we performed a budget impact analysis to estimate the average per patient cost of adding boceprevir or telaprevir to PEGylated interferon plus ribavirin therapy. We considered both standard-duration therapy and response-guided therapy regimens of boceprevir and telaprevir for treatment-naïve and treatment-experienced patients. Our model utilized monthly discontinuation rates. We built a Bayesian Markov model to account for uncertainty associated with the clinical input and cost data.Results: The total average cost of response-guided therapy with boceprevir is £22,850 and £25,060 for treatment-naïve and treatment-experienced patients, respectively. By comparison, the total average cost of response-guided therapy with telaprevir was £29,930 and £31,880 for treatment-naïve and treatment-experienced patients, respectively, whereas the total average cost of standard-duration boceprevir is £34,680 and £34,350 and for telaprevir was £32,530 and £31,680 for treatment-naïve and treatment experienced patients, respectively.Conclusion: Our results demonstrate that response-guided therapy with boceprevir is notably less costly than response-guided therapy with telaprevir. Our results also suggest that the standard treatment duration of boceprevir is slightly more costly than the standard treatment duration of telaprevir.Keywords: hepatitis C, budget impact analysis, boceprevir, telaprevir

Kristian Thorlund1, Edward Mills1,21Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, 2Faculty of Health Sciences, University of Ottawa, Ottawa, ON, CanadaBackground: Many medical interventions are administered in the form of treatment combinations involving two or more individual drugs (eg, drug A + drug B). When the individual drugs and drug combinations have been compared in a number of randomized clinical trials, it is possible to quantify the comparative effectiveness of all drugs simultaneously in a multiple treatment comparison (MTC) meta-analysis. However, current MTC models ignore the dependence between drug combinations (eg, A + B) and the individual drugs that are part of the combination. In particular, current models ignore the possibility that drug effects may be additive, ie, the property that the effect of A and B combined is equal to the sum of the individual effects of A and B. Current MTC models may thus be suboptimal for analyzing data including drug combinations when their effects are additive or approximately additive. However, the extent to which the additivity assumption can be violated before the conventional model becomes the more optimal approach is unknown. The objective of this study was to evaluate the comparative statistical performance of the conventional MTC model and the additive effects MTC model in MTC scenarios where additivity holds true, is mildly violated, or is strongly violated.Methods: We simulated MTC scenarios in which additivity held true, was mildly violated, or was strongly violated. For each scenario we simulated 500 MTC data sets and applied the conventional and additive effects MTC models in a Bayesian framework. Under each scenario we estimated the proportion of treatment effect estimates that were 20% larger than ‘the truth' (ie, % overestimates), the proportion that were 20% smaller than ‘the truth' (ie, % underestimates),The coverage of the 95% credible intervals, and the statistical power. We did this for all the comparisons under both models. Results: Under true additivity, the additive effects model is superior to the conventional model. Under mildly violated additivity, the additive model generally yields more overestimates or underestimates for a subset of treatment comparisons, but comparable coverage and greater power. Under strongly violated additivity, the proportion of overestimates or underestimates and coverage is considerably worse with the additive effects model.Conclusion: The additive MTC model is statistically superior when additivity holds true. The two models are comparably advantageous in terms of a bias-precision trade-off when additivity is only mildly violated. When additivity is strongly violated, the additive effects model is statistically inferior.Keywords: multiple treatment, meta-analysis, additive effects, statistical modeling

Curtis L Cooper1, Eric Druyts2, Kristian Thorlund3, Jean B Nachega4, Antoine C El Khoury5, Christopher O'Regan6, Edward J Mills2,31Division of Infectious Diseases, Faculty of Medicine, 2Faculty of Health Sciences, University of Ottawa, Ottawa, 3Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada; 4Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, 5Merck and Co, Inc, Whitehouse Station, NJ, 6Merck Sharp and Dohme Ltd, Hoddesdon, Hertfordshire, United KingdomBackground: The aim of this study was to examine the relative efficacy and safety of boceprevir and telaprevir, when used in combination with pegylated interferon alpha and ribavirin, using an indirect comparison meta-analysis.Methods: Published phase II and phase III randomized placebo-controlled trials examining the efficacy of boceprevir and telaprevir in chronic hepatitis C virus genotype 1 infected adult populations were included. The primary outcomes were sustained virologic response, relapse, and discontinuation of all study drugs. Secondary outcomes included the adverse events of anemia, neutropenia, rash, and pruritus. Results: Four boceprevir trials and six telaprevir trials were included. No significant differences were observed for sustained virologic response among either naïve (relative risk [RR] 1.14, 95% confidence interval [CI] 0.93–1.37, P = 0.20) or experienced patients (RR 0.81, 95% CI 0.52–1.23, P = 0.30). Similarly, for relapse among naïve (RR 0.80, 95% CI 0.18–3.45, P = 0.77) and experienced patients (RR 1.71, 95% CI 0.90–3.24, P = 0.10), or discontinuation of therapy for naïve (RR 0.80, 95% CI 0.28–2.29, P = 0.72) and experienced patients (RR 0.88, 95% CI 0.69–1.12, P = 0.30). Telaprevir was more likely to be associated with rash and pruritus, and boceprevir was more likely to be associated with neutropenia in certain patient populations.Conclusion: Boceprevir and telaprevir appear comparable in terms of sustained virologic response, relapse, or discontinuation of therapy for patients treated with standard-dose therapy durations and response-guided therapy durations.Keywords: direct-acting antivirals, boceprevir, telaprevir, hepatitis C, meta-analysis

Edward J Mills1, Nick Bansback2,8, Isabella Ghement3, Kristian Thorlund4, Steven Kelly5, Milo A Puhan6, James Wright71Faculty of Health Sciences, University of Ottawa, Ottawa, ON, Canada; 2Centre for Health Evaluation and Outcomes Sciences (CHEOS), University of British Columbia, Vancouver, BC, Canada; 3Ghement Statistical Consulting Company, Richmond, BC, Canada; 4Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada; 5Department of Outcomes Research and Evidence Based Medicine, Pfizer Ltd, Walton Oaks, UK; 6Department of Epidemiology, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA; 7Department of Oncology and Medicine, McMaster University, Hamilton, ON, Canada; 8School of Population and Public Health, University of British Columbia, Vancouver, BC, CanadaAbstract: The use of meta-analysis has become increasingly useful for clinical and policy decision making. A recent development in meta-analysis, multiple treatment comparison (MTC) meta-analysis, provides inferences on the comparative effectiveness of interventions that may have never been directly evaluated in clinical trials. This new approach may be confusing for clinicians and methodologists and raises specific challenges relevant to certain areas of medicine. This article addresses the methodological concepts of MTC meta-analysis, including issues of heterogeneity, choice of model, and adequacy of sample sizes. We address domain-specific challenges relevant to disciplines of medicine, including baseline risks of patient populations. We conclude that MTC meta-analysis is a useful tool in the context of comparative effectiveness and requires further study, as its utility and transparency will likely predict its uptake by the research and clinical community.Keywords: network, multiple treatment comparison, mixed treatment comparison, meta-analysis

Thorlund K, Druyts E, Aviña-Zubieta JA, et al. Biologics: Targets and Therapy. 2012;6:417–427. On page 418, note that under Search strategy, Merck-Shire-Dome should have been listed as Merck, Sharp, and Dohme.On page 421, Figure 2, the relative risk for golimumab versus placebo for the PsARC response was incorrectly listed as 2.45, and should be 3.45. The mean difference for etanercept versus placebo for the PASI response was incorrectly listed as 2.13, and should be 3.13. The x-axis of the PASI forest plot was incorrectly labeled with mean difference values of 1, 2, 3 and 4. The correct values are 3, 6, 9 and 12.Read the original article