Simultaneous segmentation and classification of 99mTc-DMSA renal scintigraphic images with a deep learning approach

Abstract Background 99mTc-DMSA scan plays an important role in assessing functional abnormalities in kidneys. As a promising network for deep learning (DL), Mask R-CNN has the capability of simultaneously segmenting and classifying objects in images. In this study, we tested the feasibility and accuracy of Mask R-CNN in diagnosing acute pyelonephritis (APN) and segmenting kidneys in 99mTc-DMSA scintigraphic images. Two hundred and sixty patients with suspected APN were recruited for DMSA scan, of which 358 kidneys were diagnosed as APN. Of the recruited patients, 210 were randomly selected for training and validating Mask R-CNN, and the other 50 patients’ images were used for model testing. Accuracy of the results was assessed by comparing against references from human experts. Results In the validation phase, the trained model provided segmentation masks with intersection over union (mask IoU, for segmentation accuracy) of 86.6%, and classifications with mean average precision at the bounding box IoU ≥ 50% (mAP50, for classification accuracy) of 86.2%. In testing with the 50 independent patients, mask IoU of the model’s segmentation was 90.3%±2.2%, and in classifying the kidneys for APN, the trained model showed accuracy of 89.0%, sensitivity of 84.8% and specificity of 97.0%. In identifying patients with any APN kidney, 3 out of 50 patients were mis-diagnosed, achieving accuracy of 94.0%. Conclusions Mask R-CNN, designed to perform both segmentation and classification for images, showed much promise in analyzing 99mTc-DMSA images for both accurate diagnosis of APN and kidney segmentation.