Back to Search
Start Over
A global goodness-of-fit test for receiver operating characteristic curve analysis via the bootstrap method
- Source :
- Journal of Biomedical Informatics. 38(5):395-403
- Publication Year :
- 2005
- Publisher :
- Elsevier BV, 2005.
-
Abstract
- Objective. Medical classification accuracy studies often yield continuous data based on predictive models for treatment outcomes. A popular method for evaluating the performance of diagnostic tests is the receiver operating characteristic (ROC) curve analysis. The main objective was to develop a global statistical hypothesis test for assessing the goodness-of-fit (GOF) for parametric ROC curves via the bootstrap.Design. A simple log (or logit) and a more flexible Box-Cox normality transformations were applied to untransformed or transformed data from two clinical studies to predict complications following percutaneous coronary interventions (PCIs) and for image-guided neurosurgical resection results predicted by tumor volume, respectively. We compared a non-parametric with a parametric binormal estimate of the underlying ROC curve. To construct such a GOF test, we used the non-parametric and parametric areas under the curve (AUCs) as the metrics, with a resulting p value reported.Results. In the interventional cardiology example, logit and Box-Cox transformations of the predictive probabilities led to satisfactory AUCs (AUC = 0.888; p = 0.78, and AUC = 0.888; p = 0.73, respectively), while in the brain tumor resection example, log and Box-Cox transformations of the tumor size also led to satisfactory AUCs (AUC = 0.898; p = 0.61, and AUC = 0.899; p = 0.42, respectively). In contrast, significant departures from GOF were observed without applying any transformation prior to assuming a binormal model (AUC = 0.766; p = 0.004, and AUC=0.831; p = 0.03), respectively.Conclusions. In both studies the p values suggested that transformations were important to consider before applying any binormal model to estimate the AUC. Our analyses also demonstrated and confirmed the predictive values of different classifiers for determining the interventional complications following PCIs and resection outcomes in image-guided neurosurgery.
- Subjects :
- Male
Classification accuracy
Percutaneous coronary intervention
Goodness-of-fit test
Sensitivity
Goodness of fit
Risk Factors
Outcome Assessment, Health Care
Statistics
Diagnosis, Computer-Assisted
Angioplasty, Balloon, Coronary
Normality
media_common
Mathematics
Parametric statistics
Brain Neoplasms
Incidence
Discriminant Analysis
Middle Aged
Prognosis
Computer Science Applications
Survival Rate
Receiver-operating characteristic curve analysis
Data Interpretation, Statistical
Calibration
Specificity
Female
Algorithms
Adult
Predictive analysis
Adolescent
media_common.quotation_subject
Logit
Expert Systems
Health Informatics
Risk Assessment
Humans
p-value
Statistical hypothesis testing
Receiver operating characteristic
Area under the ROC curve
business.industry
Pattern recognition
Decision Support Systems, Clinical
Linear discriminant analysis
Survival Analysis
United States
ROC Curve
Artificial intelligence
business
Subjects
Details
- ISSN :
- 15320464
- Volume :
- 38
- Issue :
- 5
- Database :
- OpenAIRE
- Journal :
- Journal of Biomedical Informatics
- Accession number :
- edsair.doi.dedup.....3c1cc261a6d85d07210e87e4caa41720
- Full Text :
- https://doi.org/10.1016/j.jbi.2005.02.004