A fully automated landmark detection for spine surgery planning with a cascaded convolutional neural net

The quality of spine surgery planning using X-ray images depends on the accuracy of the delineating landmarks. Owing to the extensive number of landmarks, each planning needs considerable time per patient. This could lead to fatigue and inaccurate planning due to susceptibility to inter- and intra-observer variabilities. Therefore, we propose a fully automated landmark detection in spine X-ray images with a cascaded convolutional neural net (CNN). Five hundred 5K by 2K spine X-ray images from Severance Hospital were acquired. The dataset was split into training, validation, and test sets at a ratio of 8:1:1. The five landmarks for surgery planning from each image were identified by an expert neurosurgeon. To improve accuracy, a two-step cascaded CNN was used. In the first step, the regions of interest (ROIs) were predicted using RetinaNet with focal loss. At this stage, we conducted experiments in various ROI sizes to identify the optimal ROI size for each landmark. In the second step, U-Net predicted the landmarks in the ROIs precisely, and experiments were conducted on different mask sizes to make more accurate predictions; the optimal mask size is presented for each landmark. The errors (mean ± SD) of ROI detection-only model and segmentation-only model were 3.61 ± 3.67 and 7.01 ± 12.53 mm, respectively. The errors of the cascaded CNN models with fixed size and optimal size were 2.58 ± 1.92 mm and 2.08 ± 1.33 mm, respectively. The cascaded semantic segmentation was developed and evaluated for determining landmarks for surgery planning in spine X-ray images, which could be used in clinical settings to reduce the clinical burden of spine surgery planning.