Commentary on Engen et al: Risk-based, dynamic, process-oriented monitoring strategies and their burden

BACKGROUND: Evidence from prospectively designed studies to guide on-site monitoring practices for randomized trials is limited. A cluster randomized study, nested within the Strategic Timing of AntiRetroviral Treatment (START) trial, was conducted to evaluate on-site monitoring. METHODS: Sites were randomized to either annual on-site monitoring or no on-site monitoring. All sites were centrally monitored, and local monitoring was carried out twice each year. Randomization was stratified by country and projected enrollment in START. The primary outcome was a participant level composite outcome including components for eligibility errors, consent violations, use of antiretroviral treatment not recommended by protocol, late reporting of START primary and secondary clinical endpoints (defined as the event being reported more than 6 months from occurrence), and data alteration and fraud. Logistic regression fixed effect hierarchical models were used to compare on-site versus no on-site monitoring for the primary composite outcome and its components. Odds ratios (ORs) and 95% confidence intervals (CIs) comparing on-site monitoring vs no on-site monitoring are cited. RESULTS: Ninety-nine sites (2,107 participants) were randomized to receive annual on-site monitoring and 97 sites (2,264 participants) were randomized to be monitored only centrally and locally. The two monitoring groups were well-balanced at entry. In the on-site monitoring group, 469 annual on-site monitoring visits were conducted, and 134 participants (6.4%) in 56 of 99 sites (57%) had a primary monitoring outcome. In the no on-site monitoring group, 85 participants (3.8%) in 34 of 97 sites (35%) had a primary monitoring outcome (OR=1.7; 95% CI: 1.1-2.7; p=0.03). Informed consent violations accounted for most outcomes in each group (56 versus 41 participants). The largest OR was for eligibility violations (OR=12.2; 95% CI: 1.8-85.2; p=0.01). The number of participants with a late START primary endpoint was similar for each monitoring group (23 versus 16 participants). Late START grade 4 and unscheduled hospitalization events were found for 34 participants in the on-site monitoring group and 19 participants in the no on-site monitoring group (OR=2.0; 95% CI: 1.1-3.7; p=0.02). There were no cases of data alteration or fraud. Based on the travel budget for on-site monitoring and the hours spent conducting on-site monitoring, the estimated cost of on-site monitoring was over $2 million. CONCLUSIONS: On-site monitoring led to the identification of more eligibility and consent violations and START clinical events being reported more than 6 months from occurrence as compared to no on-site monitoring. Considering the nature of the excess monitoring outcomes identified at sites receiving on-site monitoring, as well as the cost of on-site monitoring, the value to the START study was limited.