Search

Purpose To examine the effect that a test set from 1 region of the world has on the performance of expert breast radiologists from that region and from a distant region. Methods Forty-seven Australian and 15 Singaporean breast radiologists interpreted the same test set (ie, 60 mammograms of Australian women consisting of 40 typical and 20 atypical cases) in their respective locations. The radiologists assigned each case a confidence score of 1 to 5, where 1 was typical, 2 was benign, and 3 to 5 indicated identifiable cancer. Localization-response operating characteristic area under the curve (AUC), receiver operating characteristic AUC, location sensitivity, case sensitivity, and specificity were calculated for each radiologist and compared between Australian and Singaporean readers using nonparametric 2-tailed Mann-Whitney U tests. Results Australian readers performed better than did their Singaporean counterparts with respect to localization-response operating characteristic AUC (.738 vs .642, P = .0395) and location sensitivity (.775 vs .660, P = .0185). Discussion Technological advancement has made possible telereporting, in which radiologists report on mammograms originating from other countries. However, those readers' levels of performance might be affected by interpreting images from populations that are different from those with which they are familiar. Conclusion Radiologist performance differed between the groups when localization of lesions was taken into account. This difference might be because of the Australians' reading mammograms from populations with which they were familiar, in contrast to the Singaporeans. [ABSTRACT FROM AUTHOR]

Introduction Test sets have been increasingly utilised to augment clinical audit in breast screening programmes; however, their relationship has never been satisfactorily understood. This study examined the relationship between mammographic test set performance and clinical audit data. Methods Clinical audit data over a 2-year period was generated for each of 20 radiologists. Sixty mammographic examinations, consisting of 40 normal and 20 cancer cases, formed the test set. Readers located any identifiable cancer, and levels of confidence were scored from 2 to 5, where a score of 3 and above is considered a recall rating. Jackknifing free response operating characteristic ( JAFROC) figure-of-merit ( FOM), location sensitivity and specificity were calculated for individual readers and then compared with clinical audit values using Spearman's rho. Results JAFROC FOM showed significant correlations to: recall rate at a first round of screening ( r = 0.51; P = 0.02); rate of small invasive cancers per 10 000 reads ( r = 0.5; P = 0.02); percentage of all cancers read that were not recalled ( r = −0.51; P = 0.02); and sensitivity ( r = 0.51; P = 0.02). Location sensitivity demonstrated significant correlations with: rate of small invasive cancers per 10 000 reads ( r = 0.46; P = 0.04); rate of DCIS (ductal carcinoma in situ) per 10 000 reads ( r = 0.44; P = 0.05); detection rate of all invasive cancers and DCIS per 10 000 reads ( r = 0.54; P = 0.01); percentage of all cancers read that were not recalled ( r = −0.57; P = 0.009); and sensitivity ( r = 0.57; P = 0.009). No other significant relationships were noted. Conclusion Performance indicators from test set demonstrate significant correlations with specific aspects of clinical performance, although caution needs to be exercised when generalising test set specificity to the clinical situation. [ABSTRACT FROM AUTHOR]