Search

Introduction In a five-arm randomized clinical trial (RCT) with stratified randomization across 54 sites, we encountered low primary outcome event proportions, resulting in multiple sites with zero events either overall or in one or more study arms. In this paper, we systematically evaluated different statistical methods of accounting for center in settings with low outcome event proportions. Methods We conducted a simulation study and a reanalysis of a completed RCT to compare five popular methods of estimating an odds ratio for multicenter trials with stratified randomization by center: (i) no center adjustment, (ii) random intercept model, (iii) Mantel–Haenszel model, (iv) generalized estimating equation (GEE) with an exchangeable correlation structure, and (v) GEE with small sample correction (GEE-small sample correction). We varied the number of total participants (200, 500, 1000, 5000), number of centers (5, 50, 100), control group outcome percentage (2%, 5%, 10%), true odds ratio (1, > 1), intra-class correlation coefficient (ICC) (0.025, 0.075), and distribution of participants across the centers (balanced, skewed). Results Mantel–Haenszel methods generally performed poorly in terms of power and bias and led to the exclusion of participants from the analysis because some centers had no events. Failure to account for center in the analysis generally led to lower power and type I error rates than other methods, particularly with ICC = 0.075. GEE had an inflated type I error rate except in some settings with a large number of centers. GEE-small sample correction maintained the type I error rate at the nominal level but suffered from reduced power and convergence issues in some settings when the number of centers was small. Random intercept models generally performed well in most scenarios, except with a low event rate (i.e., 2% scenario) and small total sample size (n ≤ 500), when all methods had issues. Discussion Random intercept models generally performed best across most scenarios. GEE-small sample correction performed well when the number of centers was large. We do not recommend the use of Mantel–Haenszel, GEE, or models that do not account for center. When the expected event rate is low, we suggest that the statistical analysis plan specify an alternative method in the case of non-convergence of the primary method.

Background An estimand is a precise description of the treatment effect to be estimated from a trial (the question) and is distinct from the methods of statistical analysis (how the question is to be answered). The potential use of estimands to improve trial research and reporting has been underpinned by the recent publication of the ICH E9(R1) Addendum on the use of estimands in clinical trials in 2019. We set out to assess how well estimands are described in published trial protocols. Methods We reviewed 50 trial protocols published in October 2020 in Trials and BMJ Open. For each protocol, we determined whether the estimand for the primary outcome was explicitly stated, not stated but inferable (i.e. could be constructed from the information given), or not inferable. Results None of the 50 trials explicitly described the estimand for the primary outcome, and in 74% of trials, it was impossible to infer the estimand from the information included in the protocol. The population attribute of the estimand could not be inferred in 36% of trials, the treatment condition attribute in 20%, the population-level summary measure in 34%, and the handling of intercurrent events in 60% (the strategy for handling non-adherence was not inferable in 32% of protocols, and the strategy for handling mortality was not inferable in 80% of the protocols for which it was applicable). Conversely, the outcome attribute was stated for all trials. In 28% of trials, three or more of the five estimand attributes could not be inferred. Conclusions The description of estimands in published trial protocols is poor, and in most trials, it is impossible to understand exactly what treatment effect is being estimated. Given the utility of estimands to improve clinical research and reporting, this urgently needs to change.

Background Pre-specification of outcomes is an important tool to guard against outcome switching in clinical trials. However, if the outcome is not sufficiently clearly defined, then different definitions could be applied and analysed, with only the most favourable result reported. Methods In order to assess the impact that differing outcome definitions could have on treatment effect estimates, we re-analysed data from TRIGGER, a cluster randomised trial comparing two red blood cell transfusion strategies for patients with acute upper gastrointestinal bleeding. We varied several aspects of the definition of further bleeding: (1) the criteria for what constitutes a further bleeding episode; (2) how further bleeding is assessed; and (3) the time-point at which further bleeding is measured. Results There were marked discrepancies in the estimated odds ratios (OR) (range 0.23–0.94) and corresponding P values (range

Background The cluster randomised crossover (CRXO) design is gaining popularity in trial settings where individual randomisation or parallel group cluster randomisation is not feasible or practical. Our aim is to stimulate discussion on the content of a reporting guideline for CRXO trials and to assess the reporting quality of published CRXO trials. Methods We undertook a systematic review of CRXO trials. Searches of MEDLINE, EMBASE, and CINAHL Plus as well as citation searches of CRXO methodological articles were conducted to December 2014. Reporting quality was assessed against both modified items from 2010 CONSORT and 2012 cluster trials extension and other proposed quality measures. Results Of the 3425 records identified through database searching, 83 trials met the inclusion criteria. Trials were infrequently identified as “cluster randomis(z)ed crossover” in title (n = 7, 8%) or abstract (n = 21, 25%), and a rationale for the design was infrequently provided (n = 20, 24%). Design parameters such as the number of clusters and number of periods were well reported. Discussion of carryover took place in only 17 trials (20%). Sample size methods were only reported in 58% (n = 48) of trials. A range of approaches were used to report baseline characteristics. The analysis method was not adequately reported in 23% (n = 19) of trials. The observed within-cluster within-period intracluster correlation and within-cluster between-period intracluster correlation for the primary outcome data were not reported in any trial. The potential for selection, performance, and detection bias could be evaluated in 30%, 81%, and 70% of trials, respectively. Conclusions There is a clear need to improve the quality of reporting in CRXO trials. Given the unique features of a CRXO trial, it is important to develop a CONSORT extension. Consensus amongst trialists on the content of such a guideline is essential. Electronic supplementary material The online version of this article (doi:10.1186/s13063-016-1685-6) contains supplementary material, which is available to authorized users.

Background Patient recruitment in clinical trials is often challenging, and as a result, many trials are stopped early due to insufficient recruitment. The re-randomization design allows patients to be re-enrolled and re-randomized for each new treatment episode that they experience. Because it allows multiple enrollments for each patient, this design has been proposed as a way to increase the recruitment rate in clinical trials. However, it is unknown to what extent recruitment could be increased in practice. Methods We modelled the expected recruitment rate for parallel-group and re-randomization trials in different settings based on estimates from real trials and datasets. We considered three clinical areas: in vitro fertilization, severe asthma exacerbations, and acute sickle cell pain crises. We compared the two designs in terms of the expected time to complete recruitment, and the sample size recruited over a fixed recruitment period. Results Across the different scenarios we considered, we estimated that re-randomization could reduce the expected time to complete recruitment by between 4 and 22 months (relative reductions of 19% and 45%), or increase the sample size recruited over a fixed recruitment period by between 29% and 171%. Re-randomization can increase recruitment most for trials with a short follow-up period, a long trial recruitment duration, and patients with high rates of treatment episodes. Conclusions Re-randomization has the potential to increase the recruitment rate in certain settings, and could lead to quicker and more efficient trials in these scenarios.

Patients with acute upper gastrointestinal bleeding are often given a red blood cell (RBC) transfusion when their haemoglobin (Hb) drops below a certain threshold, however the optimal threshold is unknown. TRIGGER (Transfusion in Gastrointestinal Bleeding, ISRCTN 85757829) was a cluster randomised feasibility trial which assessed the feasibility of implementing a transfusion policy on a hospital wide scale. The trial recruited 936 patients across six UK hospitals. One of the key feasibility outcomes was to assess adherence to the transfusion policy. Maintaining high adherence levels in an emergency setting where patients are typically seen by many physicians across multiple departments in a short space of time can be challenging. We therefore evaluated the reasons for non-adherence in order to inform strategies to increase adherence rates in the planned phase III trial. We separated protocol deviations according to whether a transfusion was given when it should not have been, or when a transfusion should have been given but was not. We looked at whether protocol violations were influenced by factors such as baseline characteristics of the patient, their perceived risk of adverse outcomes, clinician preference, or because the patient had already experienced an adverse outcome during the trial. Based on the results of this analysis, we provide recommendations for strategies to reduce non-adherence in the main trial and our findings may have broader implications to inform randomised trials of transfusion strategies in other therapeutic areas.

Selection bias occurs when recruiters selectively enrol patients into the trial based on what the next treatment allocation is likely to be. This can occur even if appropriate allocation concealment is used if recruiters can guess the next treatment assignment with some degree of accuracy. This typically occurs in unblinded trials when restricted randomisation is implemented to force the number of patients in each arm or within each centre to be the same. Several methods to reduce the risk of selection bias have been suggested; however, it is unclear how often these techniques are used in practice.We performed a review of published trials which were not blinded to assess whether they utilised methods for reducing the risk of selection bias. We assessed the following techniques: (a) blinding of recruiters; (b) use of simple randomisation; (c) avoidance of stratification by site when restricted randomisation is used; (d) avoidance of permuted blocks if stratification by site is used; and (e) incorporation of prognostic covariates into the randomisation procedure when restricted randomisation is used. We included parallel group, individually randomised phase III trials published in four general medical journals (BMJ, Journal of the American Medical Association, The Lancet, and New England Journal of Medicine) in 2010.We identified 152 eligible trials. Most trials (98%) provided no information on whether recruiters were blind to previous treatment allocations. Only 3% of trials used simple randomisation; 63% used some form of restricted randomisation, and 35% did not state the method of randomisation. Overall, 44% of trials were stratified by site of recruitment; 27% were not, and 29% did not report this information. Most trials that did stratify by site of recruitment used permuted blocks (58%), and only 15% reported using random block sizes. Many trials that used restricted randomisation also included prognostic covariates in the randomisation procedure (56%).The risk of selection bias could not be ascertained for most trials due to poor reporting. Many trials which did provide details on the randomisation procedure were at risk of selection bias due to a poorly chosen randomisation methods. Techniques to reduce the risk of selection bias should be more widely implemented.

Background Blinded outcome assessment is recommended in open-label trials to reduce bias, however it is not always feasible. It is therefore important to find other means of reducing bias in these scenarios. Methods We describe two randomised trials where blinded outcome assessment was not possible, and discuss the strategies used to reduce the possibility of bias. Results TRIGGER was an open-label cluster randomised trial whose primary outcome was further bleeding. Because of the cluster randomisation, all researchers in a hospital were aware of treatment allocation and so could not perform a blinded assessment. A blinded adjudication committee was also not feasible as it was impossible to compile relevant information to send to the committee in a blinded manner. Therefore, the definition of further bleeding was modified to exclude subjective aspects (such as whether symptoms like vomiting blood were severe enough to indicate the outcome had been met), leaving only objective aspects (the presence versus absence of active bleeding in the upper gastrointestinal tract confirmed by an internal examination). TAPPS was an open-label trial whose primary outcome was whether the patient was referred for a pleural drainage procedure. Allowing a blinded assessor to decide whether to refer the patient for a procedure was not feasible as many clinicians may be reluctant to enrol patients into the trial if they cannot be involved in their care during follow-up. Assessment by an adjudication committee was not possible, as the outcome either occurred or did not. Therefore, the decision pathway for procedure referral was modified. If a chest x-ray indicated that more than a third of the pleural space filled with fluid, the patient could be referred for a procedure; otherwise, the unblinded clinician was required to reach a consensus on referral with a blinded assessor. This process allowed the unblinded clinician to be involved in the patient’s care, while reducing the potential for bias. Conclusions When blinded outcome assessment is not possible, it may be useful to modify the outcome definition or method of assessment to reduce the risk of bias. Trial registration TRIGGER: ISRCTN85757829. Registered 26 July 2012. TAPPS: ISRCTN47845793. Registered 28 May 2012.

Background Adjustment for prognostic covariates can lead to increased power in the analysis of randomized trials. However, adjusted analyses are not often performed in practice. Methods We used simulation to examine the impact of covariate adjustment on 12 outcomes from 8 studies across a range of therapeutic areas. We assessed (1) how large an increase in power can be expected in practice; and (2) the impact of adjustment for covariates that are not prognostic. Results Adjustment for known prognostic covariates led to large increases in power for most outcomes. When power was set to 80% based on an unadjusted analysis, covariate adjustment led to a median increase in power to 92.6% across the 12 outcomes (range 80.6 to 99.4%). Power was increased to over 85% for 8 of 12 outcomes, and to over 95% for 5 of 12 outcomes. Conversely, the largest decrease in power from adjustment for covariates that were not prognostic was from 80% to 78.5%. Conclusions Adjustment for known prognostic covariates can lead to substantial increases in power, and should be routinely incorporated into the analysis of randomized trials. The potential benefits of adjusting for a small number of possibly prognostic covariates in trials with moderate or large sample sizes far outweigh the risks of doing so, and so should also be considered.

BACKGROUND: Previous research has suggested an association between more liberal red blood cell (RBC) transfusion and greater risk of further bleeding and mortality following acute upper gastrointestinal bleeding (AUGIB). METHODS AND DESIGN: The Transfusion in Gastrointestinal Bleeding (TRIGGER) trial is a pragmatic cluster-randomised feasibility trial which aims to evaluate the feasibility of implementing a restrictive vs. liberal RBC transfusion policy for adult patients admitted to hospital with AUGIB in the UK. This trial will help to inform the design and methodology of a phase III trial. The protocol for TRIGGER has been published in Transfusion Medicine Reviews. Recruitment began in September 2012 and was completed in March 2013. This update presents the statistical analysis plan, detailing how analysis of the TRIGGER trial will be performed. It is hoped that prospective publication of the full statistical analysis plan will increase transparency and give readers a clear overview of how TRIGGER will be analysed. TRIAL REGISTRATION: ISRCTN85757829.

Objectives Many randomised trials use stratified permuted blocks or minimisation to balance key prognostic variables between treatment groups. It is widely argued in the statistical literature that any balancing variables should be adjusted for in the analysis, however a review of major medical journals shows that this is not commonly done. Our objective was to determine the effects of an unadjusted analysis after balancing.

Patient recruitment is a major challenge for randomised trials. Reviews of publicly funded UK trials have found that 45 to 69% fail to recruit to target. This increases costs, delays results, and adversely impacts on the feasibility of conducting trials for conditions where there is a limited patient pool. For some conditions a patient requires treatment on multiple occasions. For example, patients with sickle cell disease require pain relief for each pain crisis, or women having fertility treatment undergo multiple treatment cycles until becoming pregnant. The current norm is for patients to be included for only one treatment episode. We propose a re-randomisation design, allowing patients to be randomised on multiple occasions, which could substantially increase the recruitment rate. We describe some properties of the re-randomisation design, such as the conditions required to obtain unbiased estimates of the treatment effect and control type I error rates, and offer advice on practical design and analysis issues. We show that this design can reduce the required number of patients compared with a parallel group design. It can be used in a wider variety of settings than a crossover design and may be more patient-centred as the number of treatment periods is determined by participants rather than the trial design. We also highlight situations where this design would be inappropriate.

Blinded outcome assessment is a key component of randomised trials, as unblinded assessment can lead to bias. However, in some circumstances blinded assessment may be difficult to achieve. In these situations, it may be useful to modify the outcome definition to remove the most subjective elements, thereby reducing the risk of bias. This is the approach used in TRIGGER, an open-label cluster-randomised trial in patients with acute upper gastrointestinal bleeding. The primary clinical outcome was further bleeding. Blinded outcome assessment was impossible, as all clinicians throughout a hospital were aware of the treatment allocation due to the use of cluster-randomisation, and given the emergency nature of the condition, it was not possible to compile relevant information to send to an adjudication committee in a blinded matter. We therefore modified the outcome definition to remove subjective events (e.g. if a patient vomited blood, whether it was ‘fresh’ enough to indicate a new bleed), leaving only relatively objective events (the presence vs. absence of blood in the patient's upper gastrointestinal tract, based on a visual inspection by endoscopy). We collected both outcomes (including vs. removing subjective events) during the trial, and compared the estimated treatment effects from both. Including subjective events led to an odds ratio (OR) of 0.83 (95% CI 0.50 to 1.37), compared to an OR of 0.50 (95% CI 0.32 to 0.78) after removing subjective events. The ratio of odds ratios was 1.66, indicating that including subjective events may biased the treatment effect upwards by 66%.

Background Selection bias occurs when recruiters selectively enrol patients into the trial based on what the next treatment allocation is likely to be. This can occur even if appropriate allocation concealment is used if recruiters can guess the next treatment assignment with some degree of accuracy. This typically occurs in unblinded trials when restricted randomisation is implemented to force the number of patients in each arm or within each centre to be the same. Several methods to reduce the risk of selection bias have been suggested; however, it is unclear how often these techniques are used in practice. Methods We performed a review of published trials which were not blinded to assess whether they utilised methods for reducing the risk of selection bias. We assessed the following techniques: (a) blinding of recruiters; (b) use of simple randomisation; (c) avoidance of stratification by site when restricted randomisation is used; (d) avoidance of permuted blocks if stratification by site is used; and (e) incorporation of prognostic covariates into the randomisation procedure when restricted randomisation is used. We included parallel group, individually randomised phase III trials published in four general medical journals (BMJ, Journal of the American Medical Association, The Lancet, and New England Journal of Medicine) in 2010. Results We identified 152 eligible trials. Most trials (98 %) provided no information on whether recruiters were blind to previous treatment allocations. Only 3 % of trials used simple randomisation; 63 % used some form of restricted randomisation, and 35 % did not state the method of randomisation. Overall, 44 % of trials were stratified by site of recruitment; 27 % were not, and 29 % did not report this information. Most trials that did stratify by site of recruitment used permuted blocks (58 %), and only 15 % reported using random block sizes. Many trials that used restricted randomisation also included prognostic covariates in the randomisation procedure (56 %). Conclusions The risk of selection bias could not be ascertained for most trials due to poor reporting. Many trials which did provide details on the randomisation procedure were at risk of selection bias due to a poorly chosen randomisation methods. Techniques to reduce the risk of selection bias should be more widely implemented.