Search

With the growing focus on comparative effectiveness research and personalized medicine, receiver-operating characteristic analysis can continue to play an important role in health care decision making. Specific applications of receiver-operating characteristic analysis include predictive model assessment and validation, biomarker diagnostics, responder analysis in patient-reported outcomes, and comparison of alternative treatment options. The authors present a survey of the potential applications of the method and briefly review several relevant extensions. Given the level of attention paid to biomarker validation, personalized medicine and comparative effectiveness research, it is highly likely that the receiver-operating characteristic analysis will remain an important visual and analytic tool for medical research and evidence-based medicine in the foreseeable future.

Rationale and Objectives When diagnostic tests are repeated and combined, a number of schemes may be adopted. Guidelines for their interpretations are required. Materials and Methods Three combination schemes, “and” (A), “or” (O), and “majority” (M), are considered. To evaluate these schemes, dependency by specifying κ values quantifying repeated test agreement was structured. In a pilot study, the combined accuracies of magnetic resonance imaging using six different pulse sequences of medial collateral ligaments of the elbows of 28 cadavers, with eight having lesions artificially created surgically, were examined. Images were evaluated simultaneously by using a five-point ordinal scale. For each pulse sequence, individuals for whom the diagnosis varied from once to three repetitions were considered. Results Scheme M improves diagnostic accuracy when sensitivity and specificity of a single test exceed 0.5, with maximal improvement at 0.79. Under scheme A, sensitivity decreases to 0.38–0.59. Under scheme O, sensitivity increases to 0.53–0.79. Scheme M yields a small improvement, reaching 0.50–0.71. Under scheme A, specificity increases to 0.95–0.98. Under scheme O, specificity decreases to 0.91–0.98. Scheme M also yields a small improvement, reaching 0.94–0.98. Conclusion Scheme A is recommended for ruling in diagnoses, scheme O is recommended for ruling out diagnoses, and scheme M is neutral. Consequently, different schemes may be used to optimize the target diagnostic accuracy.

Rationale and Objectives To determine and analyze the patterns of practice of nonvascular interventional procedures in academic centers in the United States. Materials and Methods A survey was administered via e-mail to the chief residents of 112 academic radiology departments in the United States. Recipients of the survey were asked to identify which sections within their respective radiology departments performed each of 43 types of nonvascular interventional procedures. An additional entry for performance of procedures by non-radiologists was provided. A total of 66 (59%) recipients responded. The statistical method used was the analysis of contingency tables. Results Percutaneous abdominal biopsies are performed mainly by abdominal and body imaging sections (43/66, 65%), followed by vascular/interventional sections (21/66, 32%). Percutaneous abdominal drainages are performed mostly by abdominal, body imaging, and computed tomography sections (40/66, 61%), followed by vascular/interventional sections. Fluoroscopically guided procedures were performed most commonly by vascular/interventional sections, including percutaneous gastrostomy (40/66, 61%), percutaneous nephrostomy (42/66, 64%), and biliary interventions such as percutaneous transhepatic cholangiography (47/66, 71%). Breast and musculoskeletal procedures are performed by their respective sections most frequently. Non-radiologists perform a significant portion of certain types of procedures: paracenteses, thoracenteses, biliary interventions (particularly stone extractions), enterostomies (particularly percutaneous jejunostomies and cecostomies), and certain biopsies (kidney and prostate). Conclusions Academic US radiology sections perform nonvascular interventional radiology procedures in a complex and nonuniform manner. The vascular/interventional sections and organ system and modality (especially CT) sections perform the bulk of the procedures included in the survey. Breast imaging sections predominate in procedures in their disciplines. A substantial number and amount of interventional radiology procedures are performed by non-radiologists.

Rationale and objectives Accurate and reproducible segmentations of two-dimensional images are an important prerequisite for assessing tumor ablations three dimensionally (3D). We evaluated whether supervised learning methods would improve multiperformer repeated segmentations of magnetic resonance images (MRI) obtained before and after MRI-guided cryotherapy of renal cell carcinoma. Materials and methods Three medical students independently performed five manual segmentations of a biopsy-proven renal cell carcinoma that was treated with percutaneous MRI-guided cryotherapy. Using pretreatment (T2-weighted fast recovery fast spin echo [FRFSE]) and posttreatment (T1-weighted, fat-suppressed, dynamically enhanced) MRIs, regions of tumor cryonecrosis were segmented. The same tasks were repeated after an experienced abdominal radiologist provided supervised learning. Segmentation sensitivity was compared with an estimated 3D-ground truth via voxel counts for regions of tumor, both before and after treatment, and for the regions of cryonecrosis. The sensitivity of each repeated segmentation was compared against the estimated ground truth using sensitivity, overlap index, and volume (mL). Results Supervised learning significantly improved posttreatment segmentation sensitivity ( P = .03). With supervised learning, the ranges of the performance metrics over the segmentation performers were: pretreated tumor, sensitivity 0.902–0.999, overlap index 0.935–0.961, and volume 19.15–23.71 mL; posttreated tumor, sensitivity 0.923–0.991, overlap index 0.952–0.981, and volume 20.67–22.70 mL; in the ablation zone, sensitivity 0.938–0.969, overlap index 0.940–0.962, and volume 31.79–32.36 mL. Conclusions Supervised learning improved multiperformer repeated segmentations of MRIs obtained before and after MRI-guided percutaneous cryotherapy of renal cell carcinoma. These methods may prove useful in aiding the 3D assessment of percutaneous tumor ablations.

Magnetic resonance imaging (MRI) provides indispensable information about anatomy and pathology, enabling quantitative pathologic and clinical evaluations. Segmentation is an important image-processing step by which regions of an image are classified according to the presence of relevant anatomic features. For example, segmentation of MRI of the brain assigns a unique label (eg, white matter, gray matter, lesions, cerebrospinal fluid, to each voxel in an input gray-scale image) (1). Segmentation methods typically yield binary or categoric classification results. However, continuous classification schemes (eg, volume size, distance between the volume surfaces, percentage of overlap voxels, percentage of highly discrepant voxels, and probability-based fractional segmentation) are increasingly becoming commonplace (2,3). The performance of segmentation methods has a direct impact on the detection and target definition, as well as monitoring of disease progression. Thus, the main clinical goal of surgical planning and quantitative monitoring of disease progression requires segmentation methods with high reproducibility because of the limited number of images available per patient. Several recent articles have addressed the importance of developing new automated segmentation methods in addition to binary classification, using overlap mixture intensity distributions of abnormal and normal tissues (4–6), as well as a probabilistic fractional segmentation methods on a continuous probability scale per voxel (2,3). These methods used geometric and probabilistic models to allow improved tissue volume estimates and contrast among tissue types (7–10). An overview and comparison of several existing algorithms for brain segmentation (eg, finite normal mixture histograms, genetic algorithms, and hidden Markov random field methods using percent correct identified voxels against digital phantoms) are found in the literature (7). However, it is a challenging task to evaluate the accuracy and reproducibility of MRI segmentations. To conduct a validation analysis of the quality of image segmentation, it is typically necessary to know a voxel-wise gold standard. Under a simple binary truth (here labeled, T), this gold standard is defined as an indicator of true tissue class per voxel, ie, the target class (C1) such as malignant tumor, and the background class (C0) such as the remaining healthy tissues. Unfortunately, it is often impractical to know T only based on clinical data. Various alternative methods have been sought to carry out statistical validations. A useful method is to construct phantoms, either physically or digitally, with known T, specified before building such a phantom. Because it is difficult to construct a physical phantom that can mimic the tissue properties of the human body, great efforts have been devoted to building digital phantoms that are both realistic and assessable by the radiologic community. Simulated MR digital brain phantom images of a normal subject or one with multiple sclerosis may be downloaded online from the Montreal BrainWeb (http://www.bic.mni.mcgill.ca/brainweb) (11,12). Nevertheless, even sophisticated phantoms may not yield clinical images with full range of characteristics frequently observed in practice, such as partial volume artifacts, intensity heterogeneity, noise, and normal and pathologic anatomic variability. Without a known gold standard obtained by non-imaging methods such as histology, the validation task becomes an assessment of reliability or reproducibility of segmentation. A simple spatial overlap index is the Dice similarity coefficient (DSC), first proposed by Dice (13). Dice similarity coefficient is a spatial overlap index and a reproducibility validation metric. It was also called the proportion of specific agreement by Fleiss (14). The value of a DSC ranges from 0, indicating no spatial overlap between two sets of binary segmentation results, to 1, indicating complete overlap. Dice similarity coefficient has been adopted to validate the segmentation of white matter lesions in MRIs (15) and the peripheral zone (PZ) of the prostate gland in prostate brachytherapy (16). Other validation metrics considered for statistical validation included Jaccard similarity coefficient (17), odds ratio (18), receiver operating characteristic analysis (19–22), mutual information (3,22), and distance-based statistics (23,24). In the present work, we applied and extended the DSC metric on two clinical examples analyzed previously. We aimed to validate (A) repeated binary segmentation of preoperative 1.5T and intraoperative 0.5T MRIs of the prostate’s PZ collected before and during brachytherapy for prostate cancer (16); and (B) semi-automated probabilistic fractional segmentation of MRIs of three different types of brain tumors, against a composite voxel-wise gold standard derived from repeated expert manual segmentations of the images (25). For both the prostate and brain datasets, segmentations were performed and reported previously (16,25). Here we have extended our methodology and shown a statistical validation analysis using these existing databases.

Rationale and Objectives. In this review article, we present the definition and useful concepts of evidence-based medicine (EBM). The principles of EBM are provided and major steps of practicing EBM are described. Materials and Methods. We emphasize the importance of the Cochrane Collaboration (see http://www.cochrane.org), which initiated the research and practice in this area. Because it can be difficult to systematically access and review individual research studies, it is often useful to focus on a critical overview of clinical trials by conducting a meta-analysis. Results. Useful literature and resources related to meta-analysis are provided. Conclusion. Statistical methods for evaluating radiologic diagnostic performances derived from meta-analysis are summarized, with a special focus on summary outcomes measures.

Rationale and Objectives. The accuracy of diagnostic test and imaging segmentation is important in clinical practice because it has a direct impact on therapeutic planning. Statistical validations of classification accuracy was conducted based on parametric receiver operating characteristic analysis, illustrated on three radiologic examples. Materials and Methods. Two parametric models were developed for diagnostic or imaging data. Example 1: A semi-automated fractional segmentation algorithm was applied to magnetic resonance imaging of nine cases of brain tumors. The tumor and background pixel data were assumed to have bi-beta distributions. Fractional segmentation was validated against an estimated composite pixel-wise gold standard based on multi-reader manual segmentations. Example 2: The predictive value of 100 cases of spiral computed tomography of ureteral stone sizes, distributed as bi-normal after a nonlinear transformation, under two treatment options received. Example 3: One hundred eighty cases had prostate-specific antigen levels measured in a prospective clinical trial. Radical prostatectomy was performed in all to provide a binary gold standard of local and advanced cancer stages. Prostate-specific antigen level was transformed and modeled by bi-normal distributions. In all examples, areas under the receiver operating characteristic curves were computed. Results. The areas under the receiver operating characteristic curves were: Example 1: Fractional segmentation of magnetic resonance imaging of brain tumors: meningiomas (0.924–0.984); astrocytomas (0.786–0.986); and other low-grade gliomas (0.896–0.983). Example 3: Ureteral stone size for treatment planning (0.813). Example 2: Prostate-specific antigen for staging prostate cancer (0.768). Conclusion. All clinical examples yielded fair to excellent accuracy. The validation metric area under the receiver operating characteristic curves may be generalized to evaluating the performances of several continuous classifiers related to imaging.

The authors performed this study to document the deformations that occur between pretreatment magnetic resonance (MR) imaging and intraoperative MR imaging during brachytherapy.MR images obtained at 1.5 and 0.5 T in 10 patients with prostate cancer were analyzed for changes in the shape and substructure of the prostate. Three-dimensional models of the prostate were obtained. The authors measured anteroposterior dimension; total gland, peripheral zone, and central gland volumes; transverse dimension; and superoinferior height.Gland deformations were seen at visual inspection of the three-dimensional models. The anteroposterior dimension of the total gland, central gland, and peripheral zone increased from 1.5- to 0.5-T imaging (median dimension, 4.9, 1.5, and 1.8 mm, respectively), and the increase was greatest in the peripheral zone (P.05, all comparisons). There was a decrease in the transverse dimension from 1.5- to 0.5-T imaging (median, 4.5 mm; P.005). The total gland volume and the superoinferior height did not show a statistically significant change.There were significant deformations in the shape of the prostate, especially in the peripheral zone, between the two imaging studies. The likely causes of the shape change are differences in rectal filling (endorectal coil used in 1.5-T studies vs obturator in 0.5-T studies) and/or changes in patient position (supine vs lithotomy). These findings suggest that pretreatment images alone may not be reliable for accurate therapy planning. It may be useful to integrate pre-and intraoperative data.

Rationale and Objectives The authors compared independent and side-by-side evaluation of magnetic resonance (MR) images of the medial collateral ligament (MCL) of the elbow, with regard to sensitivity, specificity, and interreader agreement. Materials and Methods Six MR imaging sequences were used to image the MCLs in 28 cadaveric specimens, eight with surgically created lesions. Two reading methods were used. For independent evaluation, the images were first evaluated independently and rated on a five-point scale by two musculoskeletal radiologists experienced in interpreting MR images and blinded to the MCL integrity. The images were then reevaluated on the same scale by both readers after at least 2 weeks, with images from all six sequences shown side by side. For each MR sequence and reading method, the sensitivity and specificity were estimated nonparametrically, and differences were tested with the McNemar test. Interreader agreement was assessed with a κ statistic, and differences were tested with Z and χ 2 tests after adjustment for the dependence structure between correlated κ statistics. Results For all sequences, side-by-side evaluation generally yielded higher specificity than independent evaluation, as well as better agreement between readers. Conclusion Observer performance is superior when multiple MR imaging pulse sequences are reviewed simultaneously rather than independently and separately. Side-by-side review of different MR pulse sequences enabled higher accuracy and lower interreader variability for evaluation of the elbow MCL. These findings have implications for the design of studies to optimize MR imaging protocols by using multiple pulse sequences and multiple readers.

Rationale and Objectives The authors evaluated two Bayesian regression models for receiver operating characteristic (ROC) curve analysis of continuous diagnostic outcome data with covariates. Materials and Methods Full and partial Bayesian regression models were applied to data from two studies (n = 180 and 100, respectively): (a) The diagnostic value of prostate-specific antigen (PSA) levels (outcome variable) for predicting disease after radical prostatectomy (gold standard) was evaluated for three risk groups (covariates) based on Gleason scores. (b) Spiral computed tomography was performed on patients with proved obstructing ureteral stones. The predictive value of stone size (outcome) was evaluated along with two treatment options (gold standard), as well as stone location (in or not in the ureterovesical junction [UVJ]) and patient age (covariates). Summary ROC measures were reported, and various prior distributions of the regression coefficients were investigated. Results (a) In the PSA example, the ROC areas under the full model were 0.667, 0.769, and 0.703, respectively, for the low-, intermediate-, and high-risk groups. Under the partial model, the area beneath the ROC curve was 0.706. (b) The ROC areas for patients with ureteral stones in the UVJ decreased dramatically with age but otherwise were close to that under the partial model (ie, 0.774). The prior distribution had greater influence in the second example. Conclusion The diagnostic tests were accurate in both examples. PSA levels were most accurate for staging prostate cancer among intermediate-risk patients. Stone size was predictive of treatment option for all patients other than those 40 years or older and with a stone in the UVJ.

Rationale and Objectives. Diagnostic studies such as spiral computed tomography (CT) in patients with obstructing ureteral calculi often necessitate the analysis of continuous test data (eg, stone sizes). The accuracy of a test can be summarized by using a receiver operating characteristic (ROC) curve. The authors developed and compared three methods for constructing a smooth ROC curve from continuous diagnostic data. Materials and Methods. Nonparametric, semiparametric, and parametric smooth ROC curve analyses were applied to data from 100 unenhanced spiral CT scans of patients with proved obstructing ureteral stones. Accuracy in using stone size to predict the need for intervention was evaluated by means of these methods. Characteristics and summary measures of the resulting ROC curves were estimated. Results. All methods fit the data well. The nonparametric method followed the details of the empiric data. The semiparametric and parametric methods yielded similar estimates of the ROC curve parameters. Areas under the ROC curves were 0.807, 0.821, and 0.814 for nonparametric, semiparametric, and parametric methods, respectively, in comparison with 0.811 for the empiric method. Conclusion. The parametric method is preferred for constructing a smooth ROC curve with available stone-size data derived from spiral CT. The analyses confirm the predictive value of stone size in determining the need for intervention.

Rationale and Objectives Based on imaging features, the optimal thresholds are typically determined as cutoff points to dichotomize the corresponding measurement scales. Materials and Methods Five metrics (ie, the Youden index, Euclidian distance, percent of correct diagnosis, kappa statistic, and mutual information) are individually maximized or minimized to derive the corresponding optimal threshold. These optimal thresholds are estimated under the parametric binormal assumption. Monte Carlo simulation studies are conducted to compare the performances of these different methods. A published radiological example on the choice of treatment outcomes following ureteral stones is used to illustrate and compare the estimated thresholds both empirically and parametrically. Results The optimal threshold can be a “moving target” because it would depend on modeling assumptions, metrics, and variability in the data. Even with large samples, disease prevalence has an impact on the robustness of the metrics. Conclusions It is recommended that researchers compare different optimal cutoff points using several metrics and select one that is most clinically relevant. The ultimate goal is to maximize diagnostic performances that are clinically meaningful to achieve improved global health.

Rationale and Objectives To perform a retrospective, quantitative assessment of the anatomic relationship between intra-axial, supratentorial, primary brain tumors, and adjacent white matter fiber tracts based on anatomic and diffusion tensor magnetic resonance imaging (MRI). We hypothesized that white matter infiltration may be common among different types of tumor. Material and Methods Preoperative, anatomic (T1- and T2-weighted), and LINESCAN diffusion tensor MRI were obtained in 12 patients harboring supratentorial gliomas (World Health Organization [WHO] Grades II and III). The two imaging modalities were rigidly registered. The tumors were manually segmented from the T1- and T2-weighted MRI, and their volume calculated. A three-dimensional tractography was performed in each case. A second segmentation and volume measurement was performed on the tumor regions intersecting adjacent white matter fiber tracts. Statistical methods included summary statistics to examine the fraction of tumor volume infiltrating adjacent white matter. Results There were five patients with low-grade oligodendroglioma (WHO Grade II), one with low-grade mixed oligoastrocytoma (WHO Grade II), one with ganglioglioma, two with low-grade astrocytoma (WHO Grade II), and three with anaplastic astrocytoma (WHO Grade III). We identified white matter tracts infiltrated by tumor in all 12 cases. The median tumor volume (± standard deviation) in our patient population was 42.5 ± 28.9 mL. The median tumor volume (± standard deviation) infiltrating white matter fiber tracts was 5.2 ± 9.9 mL. The median percentage of tumor volume infiltrating white matter fiber tracts was 21.4% ± 9.7%. Conclusions The information provided by diffusion tensor imaging combined with anatomic MRI might be useful for neurosurgical planning and intraoperative guidance. Our results confirm previous reports that extensive white matter infiltration by primary brain tumors is a common occurrence. However, prospective, large population studies are required to definitively clarify this issue, and how infiltration relates to histologic tumor type, tumor size, and location.

To determine and analyze the patterns of practice of nonvascular interventional procedures in academic centers in the United States.A survey was administered via e-mail to the chief residents of 112 academic radiology departments in the United States. Recipients of the survey were asked to identify which sections within their respective radiology departments performed each of 43 types of nonvascular interventional procedures. An additional entry for performance of procedures by non-radiologists was provided. A total of 66 (59%) recipients responded. The statistical method used was the analysis of contingency tables.Percutaneous abdominal biopsies are performed mainly by abdominal and body imaging sections (43/66, 65%), followed by vascular/interventional sections (21/66, 32%). Percutaneous abdominal drainages are performed mostly by abdominal, body imaging, and computed tomography sections (40/66, 61%), followed by vascular/interventional sections. Fluoroscopically guided procedures were performed most commonly by vascular/interventional sections, including percutaneous gastrostomy (40/66, 61%), percutaneous nephrostomy (42/66, 64%), and biliary interventions such as percutaneous transhepatic cholangiography (47/66, 71%). Breast and musculoskeletal procedures are performed by their respective sections most frequently. Non-radiologists perform a significant portion of certain types of procedures: paracenteses, thoracenteses, biliary interventions (particularly stone extractions), enterostomies (particularly percutaneous jejunostomies and cecostomies), and certain biopsies (kidney and prostate).Academic US radiology sections perform nonvascular interventional radiology procedures in a complex and nonuniform manner. The vascular/interventional sections and organ system and modality (especially CT) sections perform the bulk of the procedures included in the survey. Breast imaging sections predominate in procedures in their disciplines. A substantial number and amount of interventional radiology procedures are performed by non-radiologists.

Rationale and objectives Surgical planning now routinely uses both two-dimensional (2D) and three-dimensional (3D) models that integrate data from multiple imaging modalities, each highlighting one or more aspects of morphology or function. We performed a preliminary evaluation of the use of spherical harmonics (SH) in approximating the 3D shape and estimating the volume of brain tumors of varying characteristics. Materials and methods Magnetic resonance (MR) images from five patients with brain tumors were selected randomly from our MR-guided neurosurgical practice. Standardized mean square reconstruction errors (SMSRE) by tumor volume were measured. Validation metrics for comparing performances of the SH method against segmented contours (SC) were the dice similarity coefficient (DSC) and standardized Euclidean distance (SED) measure. Results Tumor volume range was 22413–85189 mm3, and range of number of vertices in triangulated models was 3674–6544. At SH approximations with degree of at least 30, SMSRE were within 1.66 × 10–5 mm−1. Summary measures yielded a DSC range of 0.89–0.99 (pooled median, 0.97 and significantly >0.7; P Conclusion 3D shapes of tumors may be approximated by using SH for neurosurgical applications.

Accurate and reproducible segmentations of two-dimensional images are an important prerequisite for assessing tumor ablations three dimensionally (3D). We evaluated whether supervised learning methods would improve multiperformer repeated segmentations of magnetic resonance images (MRI) obtained before and after MRI-guided cryotherapy of renal cell carcinoma.Three medical students independently performed five manual segmentations of a biopsy-proven renal cell carcinoma that was treated with percutaneous MRI-guided cryotherapy. Using pretreatment (T2-weighted fast recovery fast spin echo [FRFSE]) and posttreatment (T1-weighted, fat-suppressed, dynamically enhanced) MRIs, regions of tumor cryonecrosis were segmented. The same tasks were repeated after an experienced abdominal radiologist provided supervised learning. Segmentation sensitivity was compared with an estimated 3D-ground truth via voxel counts for regions of tumor, both before and after treatment, and for the regions of cryonecrosis. The sensitivity of each repeated segmentation was compared against the estimated ground truth using sensitivity, overlap index, and volume (mL).Supervised learning significantly improved posttreatment segmentation sensitivity (P = .03). With supervised learning, the ranges of the performance metrics over the segmentation performers were: pretreated tumor, sensitivity 0.902-0.999, overlap index 0.935-0.961, and volume 19.15-23.71 mL; posttreated tumor, sensitivity 0.923-0.991, overlap index 0.952-0.981, and volume 20.67-22.70 mL; in the ablation zone, sensitivity 0.938-0.969, overlap index 0.940-0.962, and volume 31.79-32.36 mL.Supervised learning improved multiperformer repeated segmentations of MRIs obtained before and after MRI-guided percutaneous cryotherapy of renal cell carcinoma. These methods may prove useful in aiding the 3D assessment of percutaneous tumor ablations.

In this review article, we present the definition and useful concepts of evidence-based medicine (EBM). The principles of EBM are provided and major steps of practicing EBM are described.We emphasize the importance of the Cochrane Collaboration (see http://www.cochrane.org), which initiated the research and practice in this area. Because it can be difficult to systematically access and review individual research studies, it is often useful to focus on a critical overview of clinical trials by conducting a meta-analysis.Useful literature and resources related to meta-analysis are provided.Statistical methods for evaluating radiologic diagnostic performances derived from meta-analysis are summarized, with a special focus on summary outcomes measures.

Receiver-operating characteristic (ROC) methodology was developed during World War II to evaluate new signal detection technology. Subsequently, it was used extensively in radar signal detection and psychophysical research. There have been renewed interests and an increasing number of applications in radiology and medical imaging in recent decades (1). Nowadays, with an explosion of computer-aided diagnosis, comparative effectiveness research, technology assessment, imaging biomarkers, and personalized medicine, ROC analysis continues to play important roles in informed medical decision making (2). This article series is one of twomemorial issues that will pay special tribute to Professor Charles E. Metz of the University of Chicago. As evident in both the editorial by Dr John Eng and the obituary by Dr Robert M. Nishikawa herein, Dr Metz has made significant contributions to the field of ROC analysis and was widely recognized as one of our founding giants in medical physics and diagnostic medicine. Dr Metz has left behind a rich body of ROC literature. For example, a PubMed search conducted on September 20, 2012, using the search string ‘‘Metz CE’’ yielded a staggering total of 150 articles, with his very first articles appearing in the late 1960s and early 1970s. According to the history of this journal, in 1994, the three leading academic radiology societies in the United States, along with their counterpart among radiology residents, severed a long relationship with Investigative Radiology and its publisher, J. B. Lippincott (now Lippincott-Raven), to found their own journal.Academic Radiology now serves as the official journal of the Association of University Radiologists, the Society of Chairs of Academic Radiology Departments, the Association of Program Directors in Radiology, and the American Association of Academic Chief Residents in

The accuracy of diagnostic test and imaging segmentation is important in clinical practice because it has a direct impact on therapeutic planning. Statistical validations of classification accuracy was conducted based on parametric receiver operating characteristic analysis, illustrated on three radiologic examples,Two parametric models were developed for diagnostic or imaging data. Example 1: A semi-automated fractional segmentation algorithm was applied to magnetic resonance imaging of nine cases of brain tumors. The tumor and background pixel data were assumed to have bi-beta distributions. Fractional segmentation was validated against an estimated composite pixel-wise gold standard based on multi-reader manual segmentations. Example 2: The predictive value of 100 cases of spiral computed tomography of ureteral stone sizes, distributed as bi-normal after a non-linear transformation, under two treatment options received. Example 3: One hundred eighty cases had prostate-specific antigen levels measured in a prospective clinical trial. Radical prostatectomy was performed in all to provide a binary gold standard of local and advanced cancer stages. Prostate-specific antigen level was transformed and modeled by bi-normal distributions. In all examples, areas under the receiver operating characteristic curves were computed. RESULTS. The areas under the receiver operating characteristic curves were: Example 1: Fractional segmentation of magnetic resonance imaging of brain tumors: meningiomas (0.924-0.984); astrocytomas (0.786-0.986); and other low-grade gliomas (0.896-0.983). Example 3: Ureteral stone size for treatment planning (0.813). Example 2: Prostate-specific antigen for staging prostate cancer (0.768).All clinical examples yielded fair to excellent accuracy. The validation metric area under the receiver operating characteristic curves may be generalized to evaluating the performances of several continuous classifiers related to imaging.

Declining fee schedules, decreasing operating margins, and increasingly stringent compliance regulations create a need for intense scrutiny and optimization of a radiology organization's billing and collection procedures. The authors' goal was to analyze the effectiveness of departmental professional billing procedures, identify controllable factors, and intervene when they could be improved.A detailed audit of professional claims and payments was performed for all patients who underwent abdominal-pelvic computed tomography (CT) during July 1999 (n = 717). The adequacy of indication for the CT examination as given by the referring physician and modified by the radiology staff, the time required for claim generation, and the status of reimbursement within 120 days were assessed by an interdisciplinary team. After an intervention was performed to improve adequacy of the available clinical indication, the audit was repeated in December 1999 (n = 710).Despite a significant (P.05) improvement in wording of clinical indications for billing purposes between July (68%) and December (85%), there was no significant change in reimbursement against gross charges. The vast majority of claims (97% in July, 99% in December) were generated in less than 30 days. At 120 days after the date of service, payments had been received that amounted to only 66% and 54% of discounted professional fees for July and December, respectively. For examinations performed in December, payment was delayed beyond contracted time periods in 138 cases (19%).Optimum billing and collection for imaging studies is an increasingly complex task. Even when substantial efforts are devoted to eliciting the proper indication for the study, reimbursement remains low primarily because of payer delays.

Rational and Objectives It is common to administer the same diagnostic test more than once to the same set of patients. The purpose of this study was to develop two statistical methods for estimating and comparing correlated receiver operating characteristic (ROC) curves for data derived from repeated diagnostic tests. Material and Methods Parametric and semiparametric transformation models were developed. These estimation methods were illustrated with data from 72 pigmented lesions suspected of being malignant melanoma. A diagnostic scoring system based on asymmetry, border irregularity, color variation, and diameter was used repeatedly, with or without a dermoscope. Statistical hypothesis tests were conducted to evaluate whether a dermoscope improved the clarity of the lesion features in the scoring system. The resulting ROC curves were constructed, along with characteristics and summary measures. Results The areas under the ROC curves were 0.885 (parametric method) and 0.893 (semiparametric method) without the dermoscope, and 0.916 (parametric) and 0.912 (semiparametric) with the dermoscope. The statistical hypothesis tests did not yield statistically significant differences between the underlying ROC curves for either estimation method. Conclusion The two transformation models yielded similar results for estimation and comparison of the ROC curves. Although a dermoscope did not add extra information, the scoring system was accurate for diagnosing malignant melanoma.