Mass detection on mammograms: signal variations and performance changes for human and model observers

We studied the influence of signal variability on human and model observer performances for a detection task with mammographic backgrounds and computer generated clustered lumpy backgrounds (CLB). We used synthetic yet realistic masses and backgrounds that have been validated by radiologists during previous studies, ensuring conditions close to the clinical situation. Four trained non-physician observers participated in two-alternative forced-choice (2-AFC) experiments. They were asked to detect synthetic masses superimposed on real mammographic backgrounds or CLB. Separate experiments were conducted with sets of benign and malignant masses. Results under the signal-known-exactly (SKE) paradigm were compared with signal-known-statistically (SKS) experiments. In the latter case, the signal was chosen randomly for each of the 1,400 2-AFC trials (image pairs) among a set of 50 masses with similar dimensions, and the observers did not know which signal was present. Human observers' results were then compared with model observers (channelized Hotelling with Difference-of-Gaussian and Gabor channels) in the same experimental conditions. Results show that the performance of the human observers does not differ significantly when benign masses are superimposed on real images or on CLB with locally matched gray level mean and standard deviation. For both benign and malignant masses, the performance does not differ significantly between SKE and SKS experiments, when the signals' dimensions do not vary throughout the experiment. However, there is a performance drop when the SKS signals' dimensions vary from 5.5 to 9.5 mm in the same experiment. Noise level in the model observers can be adjusted to reproduce human observers' proportion of correct answers in the 2-AFC task within 5% accuracy for most conditions