Search

Objective To evaluate the feasibility, accuracy, and interoperability of a natural language processing (NLP) system that extracts diagnostic assertions of pneumonia in different clinical notes and institutions. Materials and Methods A rule-based NLP system was designed to identify assertions of pneumonia in 3 types of clinical notes from electronic health records (EHRs): emergency department notes, radiology reports, and discharge summaries. The lexicon and classification logic were tailored for each note type. The system was first developed and evaluated using annotated notes from the Department of Veterans Affairs (VA). Interoperability was assessed using data from the University of Utah (UU). Results The NLP system was comprised of 782 rules and achieved moderate-to-high performance in all 3 note types in VA (precision/recall/f1: emergency = 88.1/86.0/87.1; radiology = 71.4/96.2/82.0; discharge = 88.3/93.0/90.1). When applied to UU data, performance was maintained in emergency and radiology but decreased in discharge summaries (emergency = 84.7/94.3/89.3; radiology = 79.7/100.0/87.9; discharge = 65.5/92.7/76.8). Customization with 34 additional rules increased performance for all note types (emergency = 89.3/94.3/91.7; radiology = 87.0/100.0/93.1; discharge = 75.0/95.1/83.4). Conclusion NLP can be used to accurately identify the diagnosis of pneumonia across different clinical settings and institutions. A limited amount of customization to account for differences in lexicon, clinical definition of pneumonia, and EHR structure can achieve high accuracy without substantial modification.

ObjectiveTo evaluate the feasibility, accuracy, and interoperability of a natural language processing (NLP) system which extracts diagnostic assertions of pneumonia in different clinical notes and institutions.Materials and MethodsAn NLP system was designed to identify assertions of pneumonia in three types of clinical notes from electronic health records (EHRs): emergency department notes, radiology reports, and discharge summaries. The lexicon and classification logic were tailored for each note type. The system was first developed and evaluated using annotated notes from the Department of Veterans Affairs. Interoperability was assessed using data from the University of Utah.ResultsThe NLP system was comprised of 782 rules and achieved moderate-to-high performance in all three note types in VA (precision/recall/f1: emergency=88.1/86.0/87.1; radiology=71.4/96.2/82.0; discharge=88.3/93.0/90.1). When applied to UU data, performance was maintained in emergency and radiology but decreased in discharge summaries (emergency=84.7/94.3/89.3; radiology=79.7/100.0/87.9; discharge=65.5/92.7/76.8). Customization with 34 additional rules increased performance for all note types (emergency=89.3/94.3/91.7; radiology=87.0/100.0/93.1; discharge=75.0/95.1/83.4).ConclusionNLP can be used to accurately identify the diagnosis of pneumonia in different clinical settings and institutions. A limited amount of customization to account for differences in lexicon, clinical definition of pneumonia, and EHR structure can achieve high accuracy without substantial modification.

Deaths from pneumonia were decreasing globally prior to the COVID-19 pandemic, but it is unclear whether this was due to changes in patient populations, illness severity, diagnosis, hospitalization thresholds, or treatment. Using clinical data from the electronic health record among a national cohort of patients initially diagnosed with pneumonia, we examined temporal trends in severity of illness, hospitalization, and short- and long-term deaths.Retrospective cohort PARTICIPANTS: All patients18 years presenting to emergency departments (EDs) at 118 VA Medical Centers between 1/1/2006 and 12/31/2016 with an initial clinical diagnosis of pneumonia and confirmed by chest imaging report.Year of encounter.Hospitalization and 30-day and 90-day mortality. Illness severity was defined as the probability of each outcome predicted by machine learning predictive models using age, sex, comorbidities, vital signs, and laboratory data from encounters during years 2006-2007, and similar models trained on encounters from years 2015 to 2016. We estimated the changes in hospitalizations and 30-day and 90-day mortality between the first and the last 2 years of the study period accounted for by illness severity using time covariate decompositions with model estimates.Among 196,899 encounters across the study period, hospitalization decreased from 71 to 63%, 30-day mortality 10 to 7%, 90-day mortality 16 to 12%, and 1-year mortality 29 to 24%. Comorbidity risk increased, but illness severity decreased. Decreases in illness severity accounted for 21-31% of the decrease in hospitalizations, and 45-47%, 32-24%, and 17-19% of the decrease in 30-day, 90-day, and 1-year mortality. Findings were similar among underrepresented patients and those with only hospital discharge diagnosis codes.Outcomes for community-onset pneumonia have improved across the VA healthcare system after accounting for illness severity, despite an increase in cases and comorbidity burden.