The Blind Review: A quality assessment measure for review standardization of Institutional Review Boards

Background: Challenging research environment while the multinational trials are increasing, standardization of IRB review has become more important. The inconsistency is often inevitable in that opinions on the ethical part may be different, but the standardization and understanding the differences is required to assure IRB review quality. Thus, we aimed to develop and suggest quality assessment measure of IRB named “Blind review” by reviewing the same research protocols by multiple IRB panels. We further describe an analyzed result of differences to understand the mechanism of IRB standardization.Methods: We present a description of the Blind review process. Based on the HRPP (Human research Protection Program) SOPs (Standard of Procedure), eight blind review results from January 2010 to December 2018 at a single institution with multiple panels were included using Severance Hospital HRPP database. Review results were analyzed with review scores ranged 0 to 60 points including GCP requirements and protocol issues. Panel agreement was estimated by observed multiple rater agreement. Differences between review scores according to expertise member and IRB member duration were analysed with Wilcoxon rank-sum test and Kruskal-Wallis test.Results: Observed multiple raters’ agreement was increased from 0.444 (95% CI: 0.167-1.000) in 2010 to 0.479 (95% CI: 0.271-0.708) in 2014~2018 as IRB review experiences increases. In order to analyze the review mechanism, three GCP requirements and three protocol issues were scored (range 0 to 60). Mean values for GCP requirement and protocol issues were 19.25±8.21 and 18.40±9.04, respectively. Mean score of the panels where experts participated (n=16, 28.13±10.47) is higher than control group (n=32, 25.16±10.96) (p=0.93). According to career duration of IRB members’, scores of the group whose career is less than 3 years was 25.0±10.0 (n=14), the score of group whose career is 3-5 years was 26.3±9.6 (n=23). The mean score of the group whose career was more than 5 years shown 27.3±14.2 (n=11), and it is statistically significant. (p=0.09)Conclusions: We suggest Blind Review as an effective measure for overseeing and ensuring IRB’s review quality and overall GCP compliance.