Search

PURPOSE: Stage in multiple myeloma (MM) is an essential measure of disease risk, but its measurement in large databases is often lacking. We aimed to develop and validate a natural language processing (NLP) algorithm to extract oncologists' documentation of stage in the national Veterans Affairs (VA) Healthcare System. METHODS: Using nationwide electronic health record (EHR) and cancer registry data from the VA Corporate Data Warehouse, we developed and validated a rule-based NLP algorithm to extract oncologist-determined MM stage. To that end, a clinician annotated MM stage within over 5,000 short snippets of clinical notes, and annotated MM stage at MM treatment initiation for 200 patients. These were allocated into snippet- and patient-level development and validation sets. We developed MM stage extraction and roll-up algorithms within the development sets. After the algorithms were finalized, we validated them using standard measures in held-out validation sets. RESULTS: We developed algorithms for three different MM staging systems that have been in widespread use (Revised International Staging System [R-ISS], International Staging System [ISS], and Durie-Salmon [DS]) and for stage reported without a clearly defined system. Precision and recall were uniformly high for MM stage at the snippet level, ranging from 0.92 to 0.99 for the different MM staging systems. Performance in identifying for MM stage at treatment initiation at the patient level was also excellent, with precision of 0.92, 0.96, 0.90, and 0.86 and recall of 0.99, 0.98, 0.94, and 0.92 for R-ISS, ISS, DS, and unclear stage, respectively. CONCLUSION: Our MM stage extraction algorithm uses rule-based NLP and data aggregation to accurately measure MM stage documented in oncology notes and pathology reports in VA's national EHR system. It may be adapted to other systems where MM stage is recorded in clinical notes. Stage is a key factor in assessment of multiple myeloma (MM) risk, but large databases lack the necessary laboratory and pathology data for staging. We developed a rules-based NLP algorithm to extract oncologist-determined MM stage from VA EHR data. The algorithm demonstrated accurate assignment of MM stage when validated against clinician annotation. [ABSTRACT FROM AUTHOR]

Background: Electronic frailty indices (eFIs) can expand measurement of frailty in research and practice and have demonstrated predictive validity in associations with clinical outcomes. However, their construct validity is less well studied. We aimed to assess the construct validity of the VA‐FI, an eFI developed for use in the U.S. Veterans Affairs Healthcare System. Methods: Veterans who underwent comprehensive geriatric assessments between January 31, 2019 and June 6, 2022 at VA Boston and had sufficient data documented for a comprehensive geriatric assessment‐frailty index (CGA‐FI) were included. The VA‐FI, based on diagnostic and procedural codes, and the CGA‐FI, based on geriatrician‐measured deficits, were calculated for each patient. Geriatricians also assessed the Clinical Frailty Scale (CFS), functional status (ADLs and IADLs), and 4‐meter gait speed (4MGS). Results: A total of 132 veterans were included, with median age 81.4 years (IQR 75.8–88.7). Across increasing levels of VA‐FI (<0.2; 0.2–0.4; >0.4), mean CGA‐FI increased (0.24; 0.30; 0.40). The VA‐FI was moderately correlated with the CGA‐FI (r 0.45, p < 0.001). Every 0.1‐unit increase in the VA‐FI was associated with an increase in the CGA‐FI (linear regression beta 0.05; 95% confidence interval [CI] 0.03–0.06), higher CFS category (ordinal regression OR 1.69; 95% CI 1.24–2.30), higher odds of ADL dependency (logistic regression OR 1.59; 95% CI 1.20–2.11), IADL dependency (logistic regression OR 1.68; 95% CI 1.23–2.30), and a decrease in 4MGS (linear regression beta −0.07, 95% CI −0.12 to −0.02). All models were adjusted for age and race, and associations held after further adjustment for the Charlson Comorbidity Index. Conclusion: Our results demonstrate the construct validity of the VA‐FI through its associations with clinical measures of frailty, including summary frailty measures, functional status, and objective physical performance. Our findings complement others' in showing that eFIs can capture functional and mobility domains of frailty beyond just comorbidity and may be useful to measure frailty among populations and individuals. [ABSTRACT FROM AUTHOR]

Background: We aimed to evaluate and compare the performance of multiple myeloma (MM) selection algorithms for use in Veterans Affairs (VA) research. Methods: Using the VA Corporate Data Warehouse (CDW), the VA Cancer Registry (VACR), and VA pharmacy data, we randomly selected 500 patients from 01/01/1999 to 06/01/2021 who had (1) either one MM diagnostic code OR were listed in the VACR as having MM AND (2) at least one MM treatment code. A team reviewed oncology notes for each veteran to annotate details regarding MM diagnosis and initial treatment within VA. We evaluated inter‐annotator agreement and compared the performance of four published algorithms (two developed and validated external to VA data and two used in VA data). Results: A total of 859 patients were reviewed to obtain 500 patients who were annotated as having MM and initiating MM treatment in VA. Agreement was high among annotators for all variables: MM diagnosis (98.3% agreement, Kappa = 0.93); initial treatment in VA (91.8% agreement; Kappa = 0.77); and initial treatment classification (87.6% agreement; Kappa = 0.86). VA Algorithms were more specific and had higher PPVs than non‐VA algorithms for both MM diagnosis and initial treatment in VA. We developed the "VA Recommended Algorithm," which had the highest PPV among all algorithms in identifying patients diagnosed with MM (PPV = 0.98, 95% CI = 0.95–0.99) and in identifying patients who initiated their MM treatment in VA (PPV = 0.93, 95% CI = 0.90–0.96). Conclusion: Our VA Recommended Algorithm optimizes sensitivity and PPV for cohort selection and treatment classification. [ABSTRACT FROM AUTHOR]