Comparison of ROC curves for landmine detection by holographic radar with ROC data from other methods

In de-mining or UXO work, classification of subsurface targets into bins such as “mine” vs. “clutter” is critical. So is statistical evaluation of classification accuracy. For radar systems with automated target recognition involving some threshold parameter, one can plot an ROC curve showing the detection rate versus the false alarm rate across a range of threshold values. However, for visual interpretation of images, there is no parameter. Instead, the operator makes a “judgment call” based on training and experience. We propose that for visual interpretation of radargrams, differences in judgment between operators are a proxy for a variable threshold parameter. To test this, we recorded holographic radar images of a test bed containing plastic mine casings and clutter items. Each image contained between 0 and 3 mines, and 3–7 clutter items. University students with no prior training in mine or UXO recognition where given minimal training, and then asked to interpret the images. The detection and false alarm rate for each test subject across all of the images yielded a single point on an ROC curve. In addition, the false alarm rate for each clutter item was determined individually. Based on this, it appeared that rounded rocks are the most frequent false alarm. ROC curves for the “worst operators” were compared to published ROC data from other landmine detection methods, and fall within the range of performance for these other methods — even for testing by trained operators. We propose that in appropriate conditions, the holographic method will provide competitive detection metrics, even by minimally-trained lay-people such as de-miners recruited from within mined communities, and that the described method for developing ROC data can be used to quantify their performance with statistical significance.