1. Silence is golden, but my measures still see—why cheaper-but-noisier outcome measures in large simple trials can be more cost-effective than gold standards.
- Author
-
Woolf, Benjamin, Pedder, Hugo, Rodriguez-Broadbent, Henry, and Edwards, Phil
- Subjects
MEASUREMENT errors ,RANDOMIZED controlled trials ,SAMPLING errors ,JUDGMENT (Psychology) ,RESEARCH personnel ,SELECTION bias (Statistics) ,PERCENTILES - Abstract
Objective: To assess the cost-effectiveness of using cheaper-but-noisier outcome measures, such as a short questionnaire, for large simple clinical trials. Background: To detect associations reliably, trials must avoid bias and random error. To reduce random error, we can increase the size of the trial and increase the accuracy of the outcome measurement process. However, with fixed resources, there is a trade-off between the number of participants a trial can enrol and the amount of information that can be collected on each participant during data collection. Methods: To consider the effect on measurement error of using outcome scales with varying numbers of categories, we define and calculate the variance from categorisation that would be expected from using a category midpoint; define the analytic conditions under which such a measure is cost-effective; use meta-regression to estimate the impact of participant burden, defined as questionnaire length, on response rates; and develop an interactive web-app to allow researchers to explore the cost-effectiveness of using such a measure under plausible assumptions. Results: An outcome scale with only a few categories greatly reduced the variance of non-measurement. For example, a scale with five categories reduced the variance of non-measurement by 96% for a uniform distribution. We show that a simple measure will be more cost-effective than a gold-standard measure if the relative increase in variance due to using it is less than the relative increase in cost from the gold standard, assuming it does not introduce bias in the measurement. We found an inverse power law relationship between participant burden and response rates such that a doubling the burden on participants reduces the response rate by around one third. Finally, we created an interactive web-app (https://benjiwoolf.shinyapps.io/cheapbutnoisymeasures/) to allow exploration of when using a cheap-but-noisy measure will be more cost-effective using realistic parameters. Conclusion: Cheaper-but-noisier questionnaires containing just a few questions can be a cost-effective way of maximising power. However, their use requires a judgement on the trade-off between the potential increase in risk of information bias and the reduction in the potential of selection bias due to the expected higher response rates. Key messages: A cheaper-but-noisier outcome measure, like a short form questionnaire, is a more cost-effective method of maximising power in large simple clinical trials than an error free gold standard measure when the percentage increase in noise from using the cheaper-but-noisier measure is less than the relative difference in the cost of administering the two measures. We have created an R-shiny app to facilitate the exploration of when this condition is met at https://benjiwoolf.shinyapps.io/cheapbutnoisymeasures/ Cheaper-but-noisier outcome measures are more likely to introduce information bias than a gold standard but may reduce selection bias because they reduce loss-to-follow-up. Researchers therefore need to form a judgement about the relative increase or decrease in bias before using a cheap-but-noisy measure. We encourage the development and validation of short form questionnaires to enable the use of high quality cheaper-but-noisier outcome measures in randomised controlled trials. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF