A Visualization and Radiation Treatment Plan Quality Scoring Method for Triage in a Population-Based Context

Purpose: Our purpose was to develop a clinically intuitive and easily understandable scoring method using statistical metrics to visually determine the quality of a radiation treatment plan. Methods and Materials: Data from 111 patients with head and neck cancer were used to establish a percentile-based scoring system for treatment plan quality evaluation on both a plan-by-plan and objective-by-objective basis. The percentile scores for each clinical objective and the overall treatment plan score were then visualized using a daisy plot. To validate our scoring method, 6 physicians were recruited to assess 60 plans, each using a scoring table consisting of a 5-point Likert scale (with scores ≥3 considered passing). Spearman correlation analysis was conducted to assess the association between increasing treatment plan percentile rank and physician rating, with Likert scores of 1 and 2 representing clinically unacceptable plans, scores of 3 and 4 representing plans needing minor edits, and a score of 5 representing clinically acceptable plans. Receiver operating characteristic curve analysis was used to assess the scoring system's ability to quantify plan quality. Results: Of the 60 plans scored by the physicians, 8 were deemed as clinically acceptable; these plans had an 89.0th ± 14.5 percentile value using our scoring system. The plans needing minor edits or deemed unacceptable had more variation, with scores falling in the 62.6nd ± 25.1 percentile and 35.6th ± 25.7 percentile, respectively. The estimated Spearman correlation coefficient between the physician score and treatment plan percentile was 0.53 (P < .001), indicating a moderate but statistically significant correlation. Receiver operating characteristic curve analysis demonstrated discernment between acceptable and unacceptable plan quality, with an area under the curve of 0.76. Conclusions: Our scoring system correlates with physician ratings while providing intuitive visual feedback for identifying good treatment plan quality, thereby indicating its utility in the quality assurance process.