Search

When a clinical trial stops early for benefit, the maximum likelihood estimate (MLE) of the treatment effect may be subject to overestimation bias. Several authors have proposed adjusting for this bias using the conditional MLE, which is obtained by conditioning on early stopping. However, this approach has a fundamental problem in that the adjusted estimate may not be in the direction of benefit, even though the study has stopped early due to benefit. In this paper, we address this problem by embedding both the MLE and the conditional MLE within a broader class of penalised likelihood estimates, and choosing a member of the class that is a favourable compromise between the two. This penalised MLE, and its associated confidence interval, always lie in the direction of benefit when the study stops early for benefit. We study its properties using both simulations and analyses of the ENZAMET trial in metastatic prostate cancer. Conditional on stopping early for benefit, the method is found to have good unbiasedness and coverage properties, along with very favourable efficiency at earlier interim analyses. We recommend the penalised MLE as a supplementary analysis to a conventional primary analysis when a clinical trial stops early for benefit.

Background Adaptive platform trials allow randomized controlled comparisons of multiple treatments using a common infrastructure and the flexibility to adapt key design features during the study. Nonetheless, they have been criticized due to the potential for time trends in the underlying risk level of the population. Such time trends lead to confounding between design features and risk level, which may introduce bias favoring one or more treatments. This is particularly true when experimental treatments are not all randomized during the same time period as the control, leading to the potential for bias from non-concurrent controls. Methods Two analysis methods addressing this bias are stratification and adjustment. Stratification uses only comparisons between treatment cohorts randomized during identical time periods and does not use non-concurrent randomizations. Adjustment uses a modeled analysis including time period adjustment, allowing all data to be used, even from periods without concurrent randomization. We show that these competing approaches may be embedded in a common framework using network meta-analysis principles. We interpret the stages between adaptations in a platform trial as separate fixed design trials. This allows platform trials to be viewed as networks of direct randomized comparisons and indirect non-randomized comparisons. Network meta-analysis methodology can be re-purposed to aggregate the total information from a platform trial and to transparently decompose this total information into direct randomized evidence and indirect non-randomized evidence. This allows sensitivity to indirect information to be assessed and the two analysis methods to be clearly compared. Results Simulations of platform trials were analyzed using a network approach implemented in the netmeta package in R. The results demonstrated bias of unadjusted methods in the presence of time trends in risk level. Adjustment and stratification were both unbiased when direct evidence and indirect evidence were consistent. Network tests of inconsistency may be used to diagnose inconsistency when it exists. In an illustrative network analysis of one of the treatment comparisons from the STAMPEDE platform trial in metastatic prostate cancer, indirect comparisons using non-concurrent controls were inconsistent with the information from direct randomized comparisons. This supports the primary analysis approach of STAMPEDE, which used only direct randomized comparisons. Conclusion Network meta-analysis provides a natural methodology for analyzing the network of direct and indirect treatment comparisons from a platform trial. Such analyses provide transparent separation of direct and indirect evidence, allowing assessment of the impact of non-concurrent controls. We recommend time-stratified analysis of concurrently controlled comparisons for primary analyses, with time-adjusted analyses incorporating non-concurrent controls reserved for secondary analyses. However, regardless of which methodology is used, a network analysis provides a useful supplement to the primary analysis.

The CONSORT Statement says that data-driven early stopping of a clinical trial is likely to weaken the inferences that can be drawn from the trial. The GRADE guidelines go further, saying that early stopping is a study limitation that carries the risk of bias, and recommending sensitivity analyses in which trials stopped early are omitted from evidence synthesis. Despite extensive debate in the literature over these issues, the existence of clear recommendations in high profile guidelines makes it inevitable that systematic reviewers will consider sensitivity analyses investigating the impact of early stopping. The purpose of this article is to assess methodologies for conducting such sensitivity analyses, and to make recommendations about how the guidelines should be interpreted. We begin with a clarifying overview of the impacts of early stopping on treatment effect estimation in single studies and meta-analyses. We then warn against naive approaches for conducting sensitivity analyses, including simply omitting trials stopped early from meta-analyses. This approach underestimates treatment effects, which may have serious implications if cost-effectiveness analyses determine whether treatments are made widely available. Instead, we discuss two unbiased approaches to sensitivity analysis, one of which is straightforward but statistically inefficient, and the other of which achieves greater statistical efficiency by making use of recent methodological developments in the analysis of clinical trials. We end with recommendations for interpreting: (a) the CONSORT Statement on reporting of reasons for early stopping, and (b) the GRADE guidelines on sensitivity analyses assessing the impact of early stopping.

Clinical trialists may regard an observed imbalance on a prognostic covariate as sufficiently troubling to warrant action. Objective To elucidate the issues associated with selecting, and switching between, an unadjusted versus an adjusted analysis in response to an observed covariate imbalance. Study design and setting Simulation study performed under the null hypothesis of no treatment effect using data from a large secondary prevention trial of statin therapy. The operating characteristics of three reaction strategies to baseline imbalances observed post-hoc were assessed. Results Unadjusted analyses produced valid p-values irrespective of chance imbalance on a prognostic covariate. Switching to an adjusted analysis introduced no bias when the decision was made without knowledge of the direction of the imbalance. When the decision was based on the direction of the imbalance, the risk of incorrectly declaring the experimental treatment superior was inflated (by up to 48% in the scenarios investigated). Conclusion Overreaction to baseline imbalances observed post-hoc is unwarranted and we support adherence to the ICH guideline recommendations on the use of covariates. A legitimate case for switching to an adjusted analysis prior to finalisation of the statistical analysis plan (SAP) could nevertheless be potentially made provided that the direction of an observed covariate imbalance is unknown. Investigators should avoid reviewing the distribution of baseline characteristics across randomised groups in an unblinded fashion, for open-label and blinded studies alike, prior to finalisation of the SAP. What is new ICH guidelines on adjustment for covariates in RCT analyses appropriately advise against overreaction to baseline imbalances observed post-hoc. CONSORT reporting guidelines nevertheless place an emphasis on comparability of baseline characteristics across randomised groups. We demonstrate through a series of simulation studies why the ICH guidance is sound, but that a switch to an adjusted analysis in reaction to an observed prognostic covariate imbalance could legitimately be made provided that, when reaching the decision, treatment allocation is masked, and the direction of the imbalance is unknown. Trialists should therefore consider preserving the masking of actual treatment assignment when assessing the distribution of baseline characteristics across randomised groups.

Background Despite careful planning, changes to some aspects of an ongoing randomised clinical trial (RCT), with a fixed design, may be warranted. We sought to elucidate the distinction between legitimate versus illegitimate changes to serve as a guide for less experienced clinical trialists and other stakeholders. Methods Using data from a large trial of statin therapy for secondary prevention, we generated a set of simulated trial datasets under the null hypothesis (H0) and a set under an alternative hypothesis (H1). Through analysis of these simulated trials, we assessed the performance of the strategy of changing aspects of the design/analysis with knowledge of treatment allocation (illegitimate) versus the strategy of making changes without knowledge of treatment allocation (legitimate). Performance was assessed using the type 1 error, as well as measures of absolute and relative bias in the treatment effect. Results Illegitimate changes led to a relative bias of 61% under H1, and a type 1 error rate under H0 of 23%—well in excess of the 5% significance level targeted. Legitimate changes produced unbiased estimates under H1 and did not inflate the type 1 error rate under H0. Conclusions Changes to pre-specified aspects of the design and analysis of an ongoing RCT may be a necessary response to unforeseen circumstances. Such changes risk introducing a bias if undertaken with knowledge of treatment allocation. Legitimate changes need to be adequately documented to provide assurance to all stakeholders of their validity.

Randomized clinical trials comparing several treatments to a common control are often reported in the medical literature. For example, multiple experimental treatments may be compared with placebo, or in combination therapy trials, a combination therapy may be compared with each of its constituent monotherapies. Such trials are typically designed using a balanced approach in which equal numbers of individuals are randomized to each arm, however, this can result in an inefficient use of resources. We provide a unified framework and new theoretical results for optimal design of such single-control multiple-comparator studies. We consider variance optimal designs based on D-, A-, and E-optimality criteria, using a general model that allows for heteroscedasticity and a range of effect measures that include both continuous and binary outcomes. We demonstrate the sensitivity of these designs to the type of optimality criterion by showing that the optimal allocation ratios are systematically ordered according to the optimality criterion. Given this sensitivity to the optimality criterion, we argue that power optimality is a more suitable approach when designing clinical trials where testing is the objective. Weighted variance optimal designs are also discussed, which, like power optimal designs, allow the treatment difference to play a major role in determining allocation ratios. We illustrate our methods using two real clinical trial examples taken from the medical literature. Some recommendations on the use of optimal designs in single-control multiple-comparator trials are also provided.

In recent years, there has been a prominent discussion in the literature about the potential for overestimation of the treatment effect when a clinical trial stops at an interim analysis due to the experimental treatment showing a benefit over the control. However, there has been much less attention paid to the converse issue, namely, that sequentially monitored clinical trials which did not stop early for benefit tend to underestimate the treatment effect. In meta-analyses of many studies, these two sources of bias will tend to balance each other to produce an unbiased estimate of the treatment effect. However, for the interpretation of a single study in isolation, underestimation due to interim analysis may be an important consideration. In this paper, we discuss the nature of this underestimation, including theoretical and simulation results demonstrating that it can be substantial in some contexts. Furthermore, we show how a conditional approach to estimation, in which we condition on the study reaching its final analysis, may be used to validly inflate the observed treatment difference from a sequentially monitored clinical trial. Expressions for the conditional bias and information are derived, and these are used in supplied R code that computes the bias-adjusted estimate and an associated confidence interval. As well as simulation results demonstrating the validity of the methods, we present a data analysis example from a pivotal clinical trial in cardiovascular disease. The methods will be most useful when an unbiased treatment effect estimate is critical, such as in cost-effectiveness analysis or risk prediction.

Supplemental material for Underestimation of treatment effects in sequentially monitored clinical trials that did not stop early for benefit by Ian C Marschner and I Manjula Schou in Statistical Methods in Medical Research

Multi-country randomised clinical trials (MRCTs) are common in the medical literature, and their interpretation has been the subject of extensive recent discussion. In many MRCTs, an evaluation of treatment effect homogeneity across countries or regions is conducted. Subgroup analysis principles require a significant test of interaction in order to claim heterogeneity of treatment effect across subgroups, such as countries in an MRCT. As clinical trials are typically underpowered for tests of interaction, overly optimistic expectations of treatment effect homogeneity can lead researchers, regulators and other stakeholders to over-interpret apparent differences between subgroups even when heterogeneity tests are insignificant. In this paper, we consider some exploratory analysis tools to address this issue. We present three measures derived using the theory of order statistics, which can be used to understand the magnitude and the nature of the variation in treatment effects that can arise merely as an artefact of chance. These measures are not intended to replace a formal test of interaction but instead provide non-inferential visual aids, which allow comparison of the observed and expected differences between regions or other subgroups and are a useful supplement to a formal test of interaction. We discuss how our methodology differs from recently published methods addressing the same issue. A case study of our approach is presented using data from the Study of Platelet Inhibition and Patient Outcomes (PLATO), which was a large cardiovascular MRCT that has been the subject of controversy in the literature. An R package is available that implements the proposed methods.

Hongyan Zhang1, Huafang Li2, Liang Shu1, Niufan Gu2, Gang Wang3, Yongzhen Weng3, Shiping Xie4, Xinbao Zhang4, Ting Li5, Cui Ma5, Wei Yu6, Bruce Parsons7, Manjula Schou81Institute of Mental Health, Peking University, Beijing, China; 2Shanghai Mental Health Center, Shanghai, China; 3Capital Medical University, Beijing An Ding Hospital, Beijing, China; 4Nanjing Brain Hospital, Nanjing, China; 5Guangzhou Brain Hospital, Guangzhou, China; 6Pfizer China, Beijing, China; 7Pfizer Inc, New York, NY, USA; 8Pfizer Australia, Sydney, AustraliaBackground: The aim of the study was to evaluate the efficacy and safety of ziprasidone versus risperidone in Chinese subjects with acute exacerbation of schizophrenia.Methods: In patients meeting the Chinese Classification of Mental Disorders criteria for schizophrenia and with a Positive and Negative Syndrome Scale (PANSS) total score ≥60 were randomly assigned to six weeks of double-blind treatment with ziprasidone 40–80 mg twice daily or risperidone 1–3 mg bid, flexibly dosed. Noninferiority was demonstrated if the upper limit of the two-sided 95% confidence interval (CI) for the difference in PANSS total score improvement from baseline in the evaluable population was smaller than the prespecified noninferiority margin of 10 units.Results: The intent-to-treat population comprised 118 ziprasidone-treated and 121 risperidone-treated subjects. Improvement (reduction) from baseline to week 6 in PANSS total score was (-35.6 [95% CI: -38.6, -32.6]) for ziprasidone and (-37.1 [95% CI: -39.9, -34.4]) for risperidone. Noninferiority was demonstrated in the evaluable population with a difference score of 1.5 [95% CI: -2.5, 5.5]. Mean prolactin levels decreased at week 6 compared with baseline for ziprasidone (-3.5 ng/mL), but significantly increased for risperidone (61.1 ng/mL; P < 0.001). More risperidone-treated subjects (14.9%) than ziprasidone-treated subjects (4.2%) reported weight gain ≥7%. Akathisia and somnolence in the ziprasidone group and akathisia and insomnia in the risperidone group were the most common side effects. Treatment-related/treatment-emergent adverse events were reported by 79.7% and 71.1% of ziprasidone-treated and risperidone-treated subjects, respectively.Conclusion: In Chinese subjects, ziprasidone was as effective as risperidone, with less weight gain and less prolactin elevation.Keywords: ziprasidone, risperidone, schizophrenia

Combinations of a third-generation cephalosporin and metronidazole, with or without an aminoglycoside, often are used for the treatment of intra-abdominal infections in surgical settings. Simpler regimens that preserve an adequate spectrum of coverage, but allow easier administration and have fewer side effects, may be a more desirable option.This randomized, open-label, active comparator study evaluated the effectiveness (non-inferiority hypothesis) of the beta-lactam/beta-lactamase inhibitor combination cefoperazone-sulbactam (2-8 g/day), compared with ceftazidime (2-6 g/day)-amikacin (15 mg/kg/day)-metronidazole (500 mg three times daily) in 154 and 152 subjects, respectively, having intra-abdominal infections. The study was conducted at 17 centers in India.Non-inferiority of cefoperazone-sulbactam (91.9%) compared with ceftazidime-amikacin-metronidazole (81.8%) was demonstrated for continued resolution of clinical signs and symptoms at the 30-day follow-up (primary endpoint) with a treatment difference of 10.1% (95% confidence interval 2.1%, 18.1%; pre-defined non-inferiority limit-12.5%). Superiority of cefoperazone-sulbactam also was demonstrated for this endpoint, with significantly more subjects achieving continued resolution at the 30-day follow-up than in the comparator group (p = 0.015). On microbiologic outcomes, cefoperazone-sulbactam had higher success rates than ceftazidime-amikacin-metronidazole (92.9% vs. 80.0%). The pathogens (202 isolated) isolated most commonly were Escherichia coli (38.6%) and Klebsiella spp. (12.9%). The incidence of treatment-related adverse events was 6.5% and 16.4% in the cefoperazone-sulbactam and ceftazidime-amikacin-metronidazole groups, respectively, with more discontinuations due to treatment-related adverse events in the comparator arm (3.2% vs. 9.9%).Empirical cefoperazone-sulbactam monotherapy could be a useful adjunct to surgical intervention for intra-abdominal infections.

Randomized clinical trials comparing several treatments to a common control are often reported in the medical literature. For example, multiple experimental treatments may be compared with placebo, or in combination therapy trials, a combination therapy may be compared with each of its constituent monotherapies. Such trials are typically designed using a balanced approach in which equal numbers of individuals are randomized to each arm, however, this can result in an inefficient use of resources. We provide a unified framework and new theoretical results for optimal design of such single-control multiple-comparator studies. We consider variance optimal designs based on D-, A-, and E-optimality criteria, using a general model that allows for heteroscedasticity and a range of effect measures that include both continuous and binary outcomes. We demonstrate the sensitivity of these designs to the type of optimality criterion by showing that the optimal allocation ratios are systematically ordered according to the optimality criterion. Given this sensitivity to the optimality criterion, we argue that power optimality is a more suitable approach when designing clinical trials where testing is the objective. Weighted variance optimal designs are also discussed, which, like power optimal designs, allow the treatment difference to play a major role in determining allocation ratios. We illustrate our methods using two real clinical trial examples taken from the medical literature. Some recommendations on the use of optimal designs in single-control multiple-comparator trials are also provided.

Multi-country randomised clinical trials (MRCTs) are common in the medical literature, and their interpretation has been the subject of extensive recent discussion. In many MRCTs, an evaluation of treatment effect homogeneity across countries or regions is conducted. Subgroup analysis principles require a significant test of interaction in order to claim heterogeneity of treatment effect across subgroups, such as countries in an MRCT. As clinical trials are typically underpowered for tests of interaction, overly optimistic expectations of treatment effect homogeneity can lead researchers, regulators and other stakeholders to over-interpret apparent differences between subgroups even when heterogeneity tests are insignificant. In this paper, we consider some exploratory analysis tools to address this issue. We present three measures derived using the theory of order statistics, which can be used to understand the magnitude and the nature of the variation in treatment effects that can arise merely as an artefact of chance. These measures are not intended to replace a formal test of interaction but instead provide non-inferential visual aids, which allow comparison of the observed and expected differences between regions or other subgroups and are a useful supplement to a formal test of interaction. We discuss how our methodology differs from recently published methods addressing the same issue. A case study of our approach is presented using data from the Study of Platelet Inhibition and Patient Outcomes (PLATO), which was a large cardiovascular MRCT that has been the subject of controversy in the literature. An R package is available that implements the proposed methods.

Clinical trials that stop early for benefit have a treatment difference that overestimates the true effect. The consequences of this fact have been extensively debated in the literature. Some researchers argue that early stopping, or truncation, is an important source of bias in treatment effect estimates, particularly when truncated studies are incorporated into meta-analyses. Such claims are bound to lead some systematic reviewers to consider excluding truncated studies from evidence synthesis. We therefore investigated the implications of this strategy by examining the properties of sequentially monitored studies conditional on reaching the final analysis. As well as estimation bias, we studied information bias measured by the difference between standard measures of statistical information, such as sample size, and the actual information based on the conditional sampling distribution. We found that excluding truncated studies leads to underestimation of treatment effects and overestimation of information. Importantly, the information bias increases with the estimation bias, meaning that greater estimation bias is accompanied by greater overweighting in a meta-analysis. Simulations of meta-analyses confirmed that the bias from excluding truncated studies can be substantial. In contrast, when meta-analyses included truncated studies, treatment effect estimates were essentially unbiased. Previous analyses comparing treatment effects in truncated and non-truncated studies are shown not to be indicative of bias in truncated studies. We conclude that early stopping of clinical trials is not a substantive source of bias in meta-analyses and recommend that all studies, both truncated and non-truncated, be included in evidence synthesis.

Rationale: New treatment strategies are needed to improve airway clearance and reduce the morbidity and the time burden associated with cystic fibrosis (CF). Objectives: To determine whether long-term treatment with inhaled mannitol, an osmotic agent, improves lung function and morbidity. Methods: Double-blind, randomized, controlled trial of inhaled mannitol, 400 mg twice a day (n = 192, "treated" group) or 50 mg twice a day (n = 126, "control" group) for 26 weeks, followed by 26 weeks of open-label treatment. Measurements and Main Results: The primary endpoint was absolute change in FEV1 from baseline in treated versus control groups, averaged over the study period. Secondary endpoints included other spirometric measurements, pulmonary exacerbations, and hospitalization. Clinical, microbiologic, and laboratory safety were assessed. The treated group had a mean improvement in FEV1 of 105 ml (8.2% above baseline). The treated group had a relative improvement in FEV1 of 3.75% (P = 0.029) versus the control group. Adverse events and sputum microbiology were similar in both treatment groups. Exacerbation rates were low, but there were fewer in the treated group (hazard ratio, 0.74; 95% confidence interval, 0.42-1.32; P = 0.31), although this was not statistically significant. In the 26-week open-label extension study, FEV1 was maintained in the original treated group, and improved in the original control group to the same degree. Conclusions: Inhaled mannitol, 400 mg twice a day, resulted in improved lung function over 26 weeks, which was sustained after an additional 26 weeks of treatment. The safety profile was also acceptable, demonstrating the potential role for this chronic therapy for CF. Clinical trial registered with www.clinicaltrials.gov (NCT 00630812)

Objectives It is well known that a sequentially monitored clinical trial that stops early for benefit has a crude treatment difference that overestimates the true treatment effect. This has led to extended debate in the literature, with some researchers arguing that early stopping is an important source of bias in meta-analyses of clinical trials. We therefore investigated the implications of excluding studies that stopped early, so-called truncated studies, from estimation of treatment effects.

Objective: Exposure to non-steroidal anti-inflammatory drugs (NSAIDs) is associated wit increased risk of serious gastrointestinal (GI) events compared with non-exposure. We investigated whether that risk is sustained over time. Data sources: Cochrane Controlled Trials Register (to 2002); MEDLINE, EMBASE, Derwent Drug File and Current Contents (1999-2002); manual searching of reviews (1999-2002). Study selection: From 479 search results reviewed and 221 articles retrieved, seven studies of patients exposed to prescription non-selective NSAIDs for more than 6 months and reporting time-dependent serious GI event rates were selected for quantitative data synthesis. These were stratified into two groups by study design. Data extraction: Incidence of GI events and number of patients at specific time points were extracted. Data synthesis: Meta-regression analyses were performed. Change in risk was evaluated by testing whether the slope of the regression line declined over time. Four randomised controlled trials (RCTs) provided evaluable data from five NSAID arms (aspirin, naproxen, two ibuprofen arms, and diclofenac). When the RCT data were combined, a small significant decline in annualised risk was seen: -0.005% (95% Cl, -0.008% to -0.001%) per month. Sensitivity analyses were conducted because there was disparity within the RCT data. The pooled estimate from three cohort studies showed no significant decline in annualised risk over periods up to 2 years: -0.003% (95% Cl, -0.008% to 0.003%) per month. Conclusions: Small decreases in risk over time were observed; these were of negligible clinical importance. For patients who need long-term (> 6 months) treatment, precautionary measures should be considered to reduce the net probability of serious GI events over the anticipated treatment duration. The effect of intermittent versus regular daily therapy on long-term risk needs further investigation.

This study was carried out to obtain information on the efficacy and safety of the controlled release formulation of the doxazosin Gastrointestinal Therapeutic System (GITS) in Taiwanese subjects with benign prostatic hyperplasia (BPH). Studies of doxazosin in Asian populations for this indication have lacked data particularly from Taiwan. This was an 8-week, post-marketing, open-label, non-comparative study. Eighty male subjects (mean age=64 years) with BPH received doxazosin GITS 4 mg once daily. At week 4, subjects who achieved an increase in maximum urinary flow rate (Qmax) of >3 mL/s and a >30% reduction in the total International Prostate Symptom Score (IPSS) continued on doxazosin GITS 4 mg for the remaining 4 weeks; all other subjects were up-titrated to 8mg once daily. Change from baseline at weeks 4 and 8 (primary endpoint) in IPSS and Qmax was evaluated using two-sided paired t tests for the intent-to-treat population. Safety was assessed throughout the study. A total of 53 (66.3%) subjects completed the study. Baseline Qmax and IPSS were 10.7+3.4 mL/s and 20.6+5.4, respectively. At week 8, a significant increase from baseline in Qmax of 3.3+4.6 mL/s (95% confidence interval = 2.2-4.4, p< 0.001) and a significant decrease in total IPSS of −8.9 + 7.0 (95% confidence interval=−10.5 to −7.3, p< 0.001) was observed. The most common treatment-related adverse event was dizziness. Doxazosin GITS 4 mg per day (with an 8-mg titration step) effectively improved symptoms of BPH. The results from this study provide further information for clinicians on the use of doxazosin GITS for the treatment of BPH, particularly in Taiwanese patients.