Purpose: To demonstrate an efficient method for training and validation of a knowledge-based planning (KBP) system as a radiation therapy clinical trial plan quality-control system., Methods and Materials: We analyzed 86 patients with stage IB through IVA cervical cancer treated with intensity modulated radiation therapy at 2 institutions according to the standards of the INTERTECC (International Evaluation of Radiotherapy Technology Effectiveness in Cervical Cancer, National Clinical Trials Network identifier: 01554397) protocol. The protocol used a planning target volume and 2 primary organs at risk: pelvic bone marrow (PBM) and bowel. Secondary organs at risk were rectum and bladder. Initial unfiltered dose-volume histogram (DVH) estimation models were trained using all 86 plans. Refined training sets were created by removing sub-optimal plans from the unfiltered sample, and DVH estimation models… and DVH estimation models were constructed by identifying 30 of 86 plans emphasizing PBM sparing (comparing protocol-specified dosimetric cutpoints V 10 (percentage volume of PBM receiving at least 10 Gy dose) and V 20 (percentage volume of PBM receiving at least 20 Gy dose) with unfiltered predictions) and another 30 of 86 plans emphasizing bowel sparing (comparing V 40 (absolute volume of bowel receiving at least 40 Gy dose) and V 45 (absolute volume of bowel receiving at least 45 Gy dose), 9 in common with the PBM set). To obtain deliverable KBP plans, refined models must inform patient-specific optimization objectives and/or priorities (an auto-planning "routine"). Four candidate routines emphasizing different tradeoffs were composed, and a script was developed to automatically re-plan multiple patients with each routine. After selection of the routine that best met protocol objectives in the 51-patient training sample (KBP FINAL ), protocol-specific DVH metrics and normal tissue complication probability were compared for original versus KBP FINAL plans across the 35-patient validation set. Paired t tests were used to test differences between planning sets., Results: KBP FINAL plans outperformed manual planning across the validation set in all protocol-specific DVH cutpoints. The mean normal tissue complication probability for gastrointestinal toxicity was lower for KBP FINAL versus validation-set plans (48.7% vs 53.8%, P<.001). Similarly, the estimated mean white blood cell count nadir was higher (2.77 vs 2.49 k/mL, P<.001) with KBP FINAL plans, indicating lowered probability of hematologic toxicity., Conclusions: This work demonstrates that a KBP system can be efficiently trained and refined for use in radiation therapy clinical trials with minimal effort. This patient-specific plan quality control resulted in improvements on protocol-specific dosimetric endpoints., Competing Interests: Dr. Bosch reports personal fees from Augmenix, Inc., other from ASTRO, other from AAPM, outside the submitted work; Dr. Straube reports grants from National Institutes of Health, during the conduct of the study; Dr. Mell reports grants from U.S National Institutes of Health, grants and personal fees from Varian Medical System, during the conduct of the study; Dr. Moore reports grants from National Institute of Health, grants, personal fees and non-financial support from Varian Medical Systems, during the conduct of the study; In addition, Dr. Moore has a patent US Patent 20,120,310,615 licensed to Varian Medical Systems, and a patent THREE-DIMENSIONAL RADIOTHERAPY DOSE DISTRIBUTION PREDICTION pending., (Copyright © 2016 Elsevier Inc. All rights reserved.)