Search

PURPOSE To facilitate identification of clinical trial participation candidates, we developed a machine learning tool that automates the determination of a patient's metastatic status, on the basis of unstructured electronic health record (EHR) data. METHODS This tool scans EHR documents, extracting text snippet features surrounding key words (such as metastatic, progression, and local). A regularized logistic regression model was trained and used to classify patients across five metastatic categories: highly likely and likely positive, highly likely and likely negative, and unknown. Using a real-world oncology database of patients with solid tumors with manually abstracted information as reference, we calculated sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV). We validated the performance in a real-world data set, evaluating accuracy gains upon additional user review of tool's outputs after integration into clinic workflows. RESULTS In the training data set (N = 66,532), the model sensitivity and specificity (% [95% CI]) were 82.4 [81.9 to 83.0] and 95.5 [95.3 to 96.7], respectively; the PPV was 89.3 [88.8 to 90.0], and the NPV was 94.0 [93.8 to 94.2]. In the validation sample (n = 200 from five distinct care sites), after user review of model outputs, values increased to 97.1 [85.1 to 99.9] for sensitivity, 98.2 [94.8 to 99.6] for specificity, 91.9 [78.1 to 98.3] for PPV, and 99.4 [96.6 to 100.0] for NPV. The model assigned 163 of 200 patients to the highly likely categories. The error prevalence was 4% before and 2% after user review. CONCLUSION This tool infers metastatic status from unstructured EHR data with high accuracy and high confidence in more than 75% of cases, without requiring additional manual review. By enabling efficient characterization of metastatic status, this tool could mitigate a key barrier for patient ascertainment and clinical trial participation in community clinics.

274 Background: Clinical decision support (CDS) tools facilitate value based cancer care delivery and can enable measurement of guideline concordance, but can be challenging to implement. Flatiron Assist is an EHR-embedded and customizable CDS tool which facilitates selection and documentation of NCCN Guideline concordant and NCCN Preferred treatment regimens. We assessed performance metrics after the first year of use of this tool at eleven sites of care. Methods: We reviewed all non-small cell lung cancer (NSCLC) treatment orders as entered at 11 community practice sites of care in the northeast and southeast US from launch May 15, 2020, through May 23, 2021. Use of the CDS tool is not mandatory at these practices, and is deployed via prescriber choice. The tool documents and reports various quality metrics. We describe monthly prevalence of the CDS tool use, proportion of orders documented as guideline concordant, and prescriber-reported reasons for non-concordance. Results: All 954 NSCLC treatment regimen orders by the 89 prescribers who had the option to use the CDS tool were analyzed during the 1 year observation period. 658 regimens (69%) were ordered via the tool. Table describes prescriber users and non-users over time. The tool was deployed for 60% of NSCLC orders at 2 months, 69% at 6 months and 78% at 1 year post launch. Over the observed time period, 92% of treatment regimens ordered via the tool were documented as guideline concordant (94% at 2 months, 96% at 6 months and 85% at 1 year post-launch). Prescriber-reported reasons for ordering non-concordant regimens were “physician choice” (59%), “patient status” (12%), “newly published evidence” (10%), “second opinion from outside institution” (9%), “financial burden on patient/insurance doesn’t cover” (5%), “patient choice” (2%) and “other (allergy, disease progression, atypical disease)” (3%). Conclusions: At the study sites, this EHR-embedded CDS tool was rapidly adopted by most prescribers (approximately two thirds within 6 months of launch), and used for the majority of NSCLC candidate order sets during the 1 year observation period. NCCN concordance among users was empirically high overall and declined slightly over time, perhaps due to increased usage and/or greater comfort with the optional tool with ongoing use. Prescribers most commonly self-attributed the selection of non-concordant therapy to “physician choice” as opposed to factors such as financial hardship or patient choice. Further research will characterize workflow time, predictors of non-use and non-concordant orders, and evaluate whether Flatiron Assist improves clinical outcomes.[Table: see text]

2051 Background: Efforts to facilitate patient identification for clinical trials in routine practice, such as automating electronic health record (EHR) data reviews, are hindered by the lack of information on metastatic status in structured format. We developed a machine learning tool that infers metastatic status from unstructured EHR data, and we describe its real-world implementation. Methods: This machine learning model scans EHR documents, extracting features from text snippets surrounding key words (ie, ‘Metastatic’ ‘Progression’ ‘Local’). A regularized logistic regression model was trained, and used to classify patients across 5 metastatic status inference categories: highly-likely and likely positive, highly-likely and likely negative, and unknown. The model accuracy was characterized using the Flatiron Health EHR-derived de-identified database of patients with solid tumors, where manually abstracted information served as standard accurate reference. We assessed model accuracy using sensitivity and specificity (patients in the ‘unknown’ category omitted from numerator), negative and positive predictive values (NPV, PPV; patients ‘unknown’ included in denominator), and its performance in a real-world dataset. In a separate validation, we evaluated the accuracy gained upon additional user review of the model outputs after integration of this tool into workflows. Results: This metastatic status inference model was characterized using a sample of 66,532 patients. The model sensitivity and specificity (95%CI) were 82.% (82, 83) and 95% (95, 96), respectively; PPV was 89% (89, 90) and NPV was 94% (94, 94). In the validation sample (N = 200 originated from 5 distinct care sites), and after user review of model outputs, values increased to 97% (85, 100) for sensitivity, 98% (95, 100) for specificity, 92 (78, 98) for PPV and 99% (97, 100) for NPV. The model assigned 163/200 patients to the highly-likely categories, which were deemed not to require further EHR review by users. The prevalence of errors was 4% without user review, and 2% after user review. Conclusions: This machine learning model infers metastatic status from unstructured EHR data with high accuracy. The tool assigns metastatic status with high confidence in more than 75% of cases without requiring additional manual review, allowing more efficient identification of clinical trial candidates and clinical trial matching, thus mitigating a key barrier for clinical trial participation in community clinics.

PURPOSE We have previously identified paclitaxel as an active single agent in the treatment of esophageal cancer. We performed a phase II trial of paclitaxel in combination with cisplatin and fluorouracil (5-FU), conventionally used chemotherapy for esophageal cancer. The antitumor response, toxicity, and survival of patients treated with the three-drug regimen were evaluated. PATIENTS AND METHODS Sixty-one patients with advanced, surgically unresectable, or metastatic squamous cell or adenocarcinoma of the esophagus were treated. No prior chemotherapy was allowed. Thirty patients had adenocarcinoma and 31 patients had squamous cell carcinoma. The majority (47 patients; 77%) had metastatic disease and 14 patients (23%) had an unresectable primary or locally recurrent tumor. Patients received paclitaxel 175 mg/m2 by 3-hour infusion day 1; cisplatin 20 mg/m2 daily days 1 through 5, and 5-FU by continuous infusion at a dose of 1,000 mg/m2 daily days 1 through 5. Because of toxicity observed in the first 10 patients treated, the starting dose of 5-FU was subsequently reduced to 750 mg/m2 daily in the remaining patients. A planned attenuation of cisplatin to 15 mg/m2 daily was made after the first three cycles. Granulocyte colony-stimulating factor (G-CSF) was not routinely administered unless the patient had an episode of febrile neutropenia or prolonged grade 4 neutropenia. Treatment was recycled every 28 days. RESULTS Sixty-one patients completed a median of five cycles, and 60 patients were assessable for response. Major responses were seen in 29 patients (48%; 95% confidence intervals, 35 to 61), which included seven complete responses (12%). Comparable response rates were seen for patients with adenocarcinoma (46%) and those with squamous carcinoma (50%), and for patients with metastatic disease (22 of 46 patients; 48%) and those with locally advanced disease (seven of 14 patients; 50%). A significantly higher complete response rate was observed in patients with squamous carcinoma (20%) compared with those with adenocarcinoma (3%; chi2 P=.04). The median duration of response was 5.7 months (range, 1 to 18.6 months). Median survival was 10.8 months (range, 1.5 to 25 months). Toxicity was severe but manageable with dose attenuation, and included 18% of patients with grade 3 neurologic toxicity. Twenty-eight patients (46%) required a dose attenuation for toxicity, and 42 of 275 treatment cycles (15%) required a dose attenuation. Twenty-nine patients (48%) required hospitalization for toxicity, which included 11 patients for neutropenic fever (18%). There were no treatment-related deaths. CONCLUSION The combination of paclitaxel, cisplatin, and 5-FU has substantial antitumor activity in metastatic esophageal carcinoma, with a remarkable complete response rate noted in patients with squamous carcinoma. Paclitaxel is an important new agent in the treatment of esophageal carcinoma, and further evaluation of this agent in combination chemotherapy is warranted. Given the toxicity associated with the current regimen, the optimal dose and schedule of paclitaxel in combination chemotherapy remain to be established.