1. EVALUATION OF VGC ANALYZER BY COMPARISON WITH GOLD STANDARD ROC SOFTWARE AND ANALYSIS OF SIMULATED VISUAL GRADING DATA
- Author
-
Lars Gunnar Månsson, Jonny Hansson, and Magnus Båth
- Subjects
Paper ,Spectrum analyzer ,Computer science ,030218 nuclear medicine & medical imaging ,03 medical and health sciences ,0302 clinical medicine ,Software ,Resampling ,Radiology, Nuclear Medicine and imaging ,Paired Data ,AcademicSubjects/SCI00180 ,Radiation ,Radiological and Ultrasound Technology ,Receiver operating characteristic ,business.industry ,Public Health, Environmental and Occupational Health ,Pattern recognition ,General Medicine ,Gold standard (test) ,Rejection rate ,Confidence interval ,ROC Curve ,030220 oncology & carcinogenesis ,Artificial intelligence ,business - Abstract
The purpose of the present work was to evaluate the use of resampling statistical methods for analysis of visual grading data—implemented in the software VGC Analyzer—by comparing the reanalyzed results from previously performed visual grading studies with the results calculated by gold standard receiver operating characteristic (ROC) methodology, Obuchowski-Rockette (OR)-Dorfman–Berbaum–Metz (DBM) multiple-readers and multiple-case (MRMC) and by analysis of simulated visual grading data where the true distribution was presumed to be known. The reanalysis was performed on two multiple-reader studies with non-paired data and paired data, respectively. The simulation study was performed by simulating a large number of visual grading characteristics (VGC) studies and by analyzing the statistical distribution of null hypothesis (H0) rejection rate. The comparison with OR-DBM MRMC showed good agreement when analyzing non-paired data for both fixed-reader and random-reader settings for the calculated area under the curve values and the confidence intervals (CIs). For paired data analysis, VGC Analyzer showed significantly lower CIs compared with the ROC software. This effect was also illustrated by the simulation study, where the VGC Analyzer, in general, showed good accuracy for simulated studies with stable statistical basis. For simulated studies with unstable statistics, the accuracy in the H0 rejection rate decreased. The present study has shown that resampling methodology can be used to accurately perform the statistical analysis of a VGC study, although the resampling technique used makes the method sensitive to small data sets.
- Published
- 2021