Where next for partial randomisation of research funding? The feasibility of RCTs and alternatives (RoRI Working Paper No.9)

In recent years, applications of partial randomisation to research funding processes have received growing attention from meta-researchers and research funders (Woods & Wilsdon, 2021a). Partial randomisation, also known as focal randomisation or random selection, is a method for allocating research funding. It is used in addition to peer review, where peer review has reached its limits and the qualities of applications are largely indistinguishable or ‘equally good’ (Bedessem, 2020). In this article we outline essential considerations for any study of partial randomisation of research funding, and consider scenarios in which RCTs would be relevant. We highlight the interdependence of target outcomes, sample availability and statistical power for determining the cost and feasibility of a trial. For many choices of target outcome RCTs may be less practical and more expensive than they first appear (in large part due to issues pertaining to sample size and statistical power). As such, we also introduce and briefly discuss alternatives to RCTs. It is worth noting that many of the considerations relevant to experiments on partial randomisation may also apply to other potential experiments on funding processes (see Bendiscioli et al, 2021).