Search

Abstract Background Surgical interventions are complex. Key elements of this complexity are the surgeon and their learning curve. They pose methodological challenges in the design, analysis and interpretation of surgical RCTs. We identify, summarise, and critically examine current guidance about how to incorporate learning curves in the design and analysis of RCTs in surgery. Examining current guidance Current guidance presumes that randomisation must be between levels of just one treatment component, and that the evaluation of comparative effectiveness will be made via the average treatment effect (ATE). It considers how learning effects affect the ATE, and suggests solutions which seek to define the target population such that the ATE is a meaningful quantity to guide practice. We argue that these are solutions to a flawed formulation of the problem, and are inadequate for policymaking in this setting. Reformulating the problem The premise that surgical RCTs are limited to single-component comparisons, evaluated via the ATE, has skewed the methodological discussion. Forcing a multi-component intervention, such as surgery, into the framework of the conventional RCT design ignores its factorial nature. We briefly discuss the multiphase optimisation strategy (MOST), which for a Stage 3 trial would endorse a factorial design. This would provide a wealth of information to inform nuanced policy but would likely be infeasible in this setting. We discuss in more depth the benefits of targeting the ATE conditional on operating surgeon experience (CATE). The value of estimating the CATE for exploring learning effects has been previously recognised, but with discussion limited to analysis methods only. The robustness and precision of such analyses can be ensured via the trial design, and we argue that trial designs targeting CATE represent a clear gap in current guidance. Conclusion Trial designs that facilitate robust, precise estimation of the CATE would allow for more nuanced policymaking, leading to patient benefit. No such designs are currently forthcoming. Further research in trial design to facilitate the estimation of the CATE is needed.

Abstract Background Commonly in surgical randomised controlled trials (RCT) the experimental treatment is a relatively new technique which the surgeons may still be learning, while the control is a well-established standard. This can lead to biased comparisons between treatments. In this paper we discuss the implementation of approaches for addressing this issue in the ROLARR trial, and points of consideration for future surgical trials. Methods ROLARR was an international, randomised, parallel-group trial comparing robotic vs. laparoscopic surgery for the curative treatment of rectal cancer. The primary endpoint was conversion to open surgery (binary). A surgeon inclusion criterion mandating a minimum level of experience in each technique was incorporated. Additionally, surgeon self-reported data were collected periodically throughout the trial to capture the level of experience of every participating surgeon. Multi-level logistic regression adjusting for operating surgeon as a random effect is used to estimate the odds ratio for conversion to open surgery between the treatment groups. We present and contrast the results from the primary analysis, which did not account for learning effects, and a sensitivity analysis which did. Results The primary analysis yields an estimated odds ratio (robotic/laparoscopic) of 0.614 (95% CI 0.311, 1.211; p = 0.16), providing insufficient evidence to conclude superiority of robotic surgery compared to laparoscopic in terms of the risk of conversion to open. The sensitivity analysis reveals that while participating surgeons in ROLARR were expert at laparoscopic surgery, some, if not all, were still learning robotic surgery. The treatment-effect odds ratio decreases by a factor of 0.341 (95% CI 0.121, 0.960; p = 0.042) per unit increase in log-number of previous robotic operations performed by the operating surgeon. The odds ratio for a patient whose operating surgeon has the mean experience level in ROLARR – 152.46 previous laparoscopic, 67.93 previous robotic operations – is 0.40 (95% CI 0.168, 0.953; p = 0.039). Conclusions In this paper we have demonstrated the implementation of approaches for accounting for learning in a practical example of a surgery RCT analysis. The results demonstrate the value of implementing such approaches, since we have shown that without them the ROLARR analysis would indeed have been confounded by the learning effects. Trial registration International Standard Randomised Controlled Trial Number (ISRCTN) registry, ID: ISRCTN80500123. Registered on 27 May 2010.

Abstract Background Although surgical resection has been considered the only curative option for colorectal liver metastases (CLM), thermal ablation has recently been suggested as an alternative curative treatment. A prospective randomised trial is required to define the efficacy of resection vs ablation for the treatment of colorectal liver metastases. Methods Design and setting: This is a multicentre, open, randomised controlled non-inferiority trial design with internal pilot and will be performed in tertiary liver centres in UK and The Netherlands. Participants: Eligible patients will be those with colorectal liver metastases at high surgical risk because of their age, co-morbidities or tumour burden and who would be suitable for liver resection or thermal ablation. Intervention: Thermal ablation as per local policy. Control: Surgical liver resection performed as per centre protocol. Co-interventions: Further chemotherapy will be offered to patients as per current practice. Outcomes Pilot study: Same as main study and in addition patients and clinicians’ acceptability of the trial to assist in optimisation of recruitment. Primary outcome: Disease-free survival (DFS) at two years post randomisation. Secondary outcomes: Overall survival, timing and site of recurrence, additional therapy after treatment failure, quality of life, complications, length of hospital stay, costs, trial acceptability, DFS measured from end of intervention. Follow-up: 24 months from randomisation; five-year follow-up for overall survival. Sample size: 330 patients to demonstrate non-inferiority of thermal ablation. Discussion This trial will determine the effectiveness and cost-effectiveness of thermal ablation vs surgical resection for high-risk people with colorectal liver metastases, and guide the optimal treatment for these patients. Trial registration ISRCTN Registry, ISRCTN52040363. Registered on 9 March 2016.