Search

OBJECTIVE: Analyze responses to a national request for information (RFI) to uncover gaps in policy, practice, and understanding of veteran suicide to inform federal research strategy. DATA SOURCE: An RFI with 21 open‐ended questions generated from Presidential Executive Order #1386, administered nationally from July 3 to August 5, 2019. STUDY DESIGN: Semi‐structured, open‐ended responses analyzed using a collaborative qualitative and text‐mining data process. DATA EXTRACTION METHODS: We aligned traditional qualitative methods with natural language processing (NLP) text‐mining techniques to analyze 9040 open‐ended question responses from 722 respondents to provide results within 3 months. Narrative inquiry and the medical explanatory model guided the data extraction and analytic process. RESULTS: Five major themes were identified: risk factors, risk assessment, prevention and intervention, barriers to care, and data/research. Individuals and organizations mentioned different concepts within the same themes. In responses about risk factors, individuals frequently mentioned generic terms like “illness” while organizations mentioned specific terms like “traumatic brain injury.” Organizations and individuals described unique barriers to care and emphasized ways to integrate data and research to improve points of care. Organizations often identified lack of funding as barriers while individuals often identified key moments for prevention such as military transitions and ensuring care providers have military cultural understanding. CONCLUSIONS: This study provides an example of a rapid, adaptive analysis of a large body of qualitative, public response data about veteran suicide to support a federal strategy for an important public health topic. Combining qualitative and text‐mining methods allowed a representation of voices and perspectives including the lived experiences of individuals who described stories of military transition, treatments that worked or did not, and the perspective of organizations treating veterans for suicide. The results supported the development of a national strategy to reduce suicide risks for veterans as well as civilians.

Objective To evaluate the feasibility, accuracy, and interoperability of a natural language processing (NLP) system that extracts diagnostic assertions of pneumonia in different clinical notes and institutions. Materials and Methods A rule-based NLP system was designed to identify assertions of pneumonia in 3 types of clinical notes from electronic health records (EHRs): emergency department notes, radiology reports, and discharge summaries. The lexicon and classification logic were tailored for each note type. The system was first developed and evaluated using annotated notes from the Department of Veterans Affairs (VA). Interoperability was assessed using data from the University of Utah (UU). Results The NLP system was comprised of 782 rules and achieved moderate-to-high performance in all 3 note types in VA (precision/recall/f1: emergency = 88.1/86.0/87.1; radiology = 71.4/96.2/82.0; discharge = 88.3/93.0/90.1). When applied to UU data, performance was maintained in emergency and radiology but decreased in discharge summaries (emergency = 84.7/94.3/89.3; radiology = 79.7/100.0/87.9; discharge = 65.5/92.7/76.8). Customization with 34 additional rules increased performance for all note types (emergency = 89.3/94.3/91.7; radiology = 87.0/100.0/93.1; discharge = 75.0/95.1/83.4). Conclusion NLP can be used to accurately identify the diagnosis of pneumonia across different clinical settings and institutions. A limited amount of customization to account for differences in lexicon, clinical definition of pneumonia, and EHR structure can achieve high accuracy without substantial modification.

ObjectiveTo evaluate the feasibility, accuracy, and interoperability of a natural language processing (NLP) system which extracts diagnostic assertions of pneumonia in different clinical notes and institutions.Materials and MethodsAn NLP system was designed to identify assertions of pneumonia in three types of clinical notes from electronic health records (EHRs): emergency department notes, radiology reports, and discharge summaries. The lexicon and classification logic were tailored for each note type. The system was first developed and evaluated using annotated notes from the Department of Veterans Affairs. Interoperability was assessed using data from the University of Utah.ResultsThe NLP system was comprised of 782 rules and achieved moderate-to-high performance in all three note types in VA (precision/recall/f1: emergency=88.1/86.0/87.1; radiology=71.4/96.2/82.0; discharge=88.3/93.0/90.1). When applied to UU data, performance was maintained in emergency and radiology but decreased in discharge summaries (emergency=84.7/94.3/89.3; radiology=79.7/100.0/87.9; discharge=65.5/92.7/76.8). Customization with 34 additional rules increased performance for all note types (emergency=89.3/94.3/91.7; radiology=87.0/100.0/93.1; discharge=75.0/95.1/83.4).ConclusionNLP can be used to accurately identify the diagnosis of pneumonia in different clinical settings and institutions. A limited amount of customization to account for differences in lexicon, clinical definition of pneumonia, and EHR structure can achieve high accuracy without substantial modification.

Despite impressive success of machine learning algorithms in clinical natural language processing (cNLP), rule-based approaches still have a prominent role. In this paper, we introduce medspaCy, an extensible, open-source cNLP library based on spaCy framework that allows flexible integration of rule-based and machine learning-based algorithms adapted to clinical text. MedspaCy includes a variety of components that meet common cNLP needs such as context analysis and mapping to standard terminologies. By utilizing spaCy’s clear and easy-to-use conventions, medspaCy enables development of custom pipelines that integrate easily with other spaCy-based modules. Our toolkit includes several core components and facilitates rapid development of pipelines for clinical text.

Deaths from pneumonia were decreasing globally prior to the COVID-19 pandemic, but it is unclear whether this was due to changes in patient populations, illness severity, diagnosis, hospitalization thresholds, or treatment. Using clinical data from the electronic health record among a national cohort of patients initially diagnosed with pneumonia, we examined temporal trends in severity of illness, hospitalization, and short- and long-term deaths.Retrospective cohort PARTICIPANTS: All patients18 years presenting to emergency departments (EDs) at 118 VA Medical Centers between 1/1/2006 and 12/31/2016 with an initial clinical diagnosis of pneumonia and confirmed by chest imaging report.Year of encounter.Hospitalization and 30-day and 90-day mortality. Illness severity was defined as the probability of each outcome predicted by machine learning predictive models using age, sex, comorbidities, vital signs, and laboratory data from encounters during years 2006-2007, and similar models trained on encounters from years 2015 to 2016. We estimated the changes in hospitalizations and 30-day and 90-day mortality between the first and the last 2 years of the study period accounted for by illness severity using time covariate decompositions with model estimates.Among 196,899 encounters across the study period, hospitalization decreased from 71 to 63%, 30-day mortality 10 to 7%, 90-day mortality 16 to 12%, and 1-year mortality 29 to 24%. Comorbidity risk increased, but illness severity decreased. Decreases in illness severity accounted for 21-31% of the decrease in hospitalizations, and 45-47%, 32-24%, and 17-19% of the decrease in 30-day, 90-day, and 1-year mortality. Findings were similar among underrepresented patients and those with only hospital discharge diagnosis codes.Outcomes for community-onset pneumonia have improved across the VA healthcare system after accounting for illness severity, despite an increase in cases and comorbidity burden.

The early prediction of deterioration could have an important role in supporting healthcare professionals, as an estimated 11% of deaths in hospital follow a failure to promptly recognize and treat deteriorating patients1. To achieve this goal requires predictions of patient risk that are continuously updated and accurate, and delivered at an individual level with sufficient context and enough time to act. Here we develop a deep learning approach for the continuous risk prediction of future deterioration in patients, building on recent work that models adverse events from electronic health records2–17 and using acute kidney injury—a common and potentially life-threatening condition18—as an exemplar. Our model was developed on a large, longitudinal dataset of electronic health records that cover diverse clinical environments, comprising 703,782 adult patients across 172 inpatient and 1,062 outpatient sites. Our model predicts 55.8% of all inpatient episodes of acute kidney injury, and 90.2% of all acute kidney injuries that required subsequent administration of dialysis, with a lead time of up to 48 h and a ratio of 2 false alerts for every true alert. In addition to predicting future acute kidney injury, our model provides confidence assessments and a list of the clinical features that are most salient to each prediction, alongside predicted future trajectories for clinically relevant blood tests9. Although the recognition and prompt treatment of acute kidney injury is known to be challenging, our approach may offer opportunities for identifying patients at risk within a time window that enables early treatment. A deep learning approach that predicts the risk of acute kidney injury may help to identify patients at risk of health deterioration within a time window that enables early treatment.

Rationale: Computerized severity assessment for community-acquired pneumonia could improve consistency and reduce clinician burden. Objectives: To develop and compare 30-day mortality-prediction mo...

BACKGROUND Patient travel history can be crucial in evaluating evolving infectious disease events. Such information can be challenging to acquire in electronic health records, as it is often available only in unstructured text. OBJECTIVE This study aims to assess the feasibility of annotating and automatically extracting travel history mentions from unstructured clinical documents in the Department of Veterans Affairs across disparate health care facilities and among millions of patients. Information about travel exposure augments existing surveillance applications for increased preparedness in responding quickly to public health threats. METHODS Clinical documents related to arboviral disease were annotated following selection using a semiautomated bootstrapping process. Using annotated instances as training data, models were developed to extract from unstructured clinical text any mention of affirmed travel locations outside of the continental United States. Automated text processing models were evaluated, involving machine learning and neural language models for extraction accuracy. RESULTS Among 4584 annotated instances, 2659 (58%) contained an affirmed mention of travel history, while 347 (7.6%) were negated. Interannotator agreement resulted in a document-level Cohen kappa of 0.776. Automated text processing accuracy (F1 85.6, 95% CI 82.5-87.9) and computational burden were acceptable such that the system can provide a rapid screen for public health events. CONCLUSIONS Automated extraction of patient travel history from clinical documents is feasible for enhanced passive surveillance public health systems. Without such a system, it would usually be necessary to manually review charts to identify recent travel or lack of travel, use an electronic health record that enforces travel history documentation, or ignore this potential source of information altogether. The development of this tool was initially motivated by emergent arboviral diseases. More recently, this system was used in the early phases of response to COVID-19 in the United States, although its utility was limited to a relatively brief window due to the rapid domestic spread of the virus. Such systems may aid future efforts to prevent and contain the spread of infectious diseases.

Identifying occurrences of medication side effects and adverse drug events (ADEs) is an important and challenging task because they are frequently only mentioned in clinical narrative and are not formally reported. We developed a natural language processing (NLP) system that aims to identify mentions of symptoms and drugs in clinical notes and label the relationship between the mentions as indications or ADEs. The system leverages an existing word embeddings model with induced word clusters for dimensionality reduction. It employs a conditional random field (CRF) model for named entity recognition (NER) and a random forest model for relation extraction (RE). Final performance of each model was evaluated separately and then combined on a manually annotated evaluation set. The micro-averaged F1 score was 80.9% for NER, 88.1% for RE, and 61.2% for the integrated systems. Outputs from our systems were submitted to the NLP Challenges for Detecting Medication and Adverse Drug Events from Electronic Health Records (MADE 1.0) competition (Yu et al. in http://bio-nlp.org/index.php/projects/39-nlp-challenges , 2018). System performance was evaluated in three tasks (NER, RE, and complete system) with multiple teams submitting output from their systems for each task. Our RE system placed first in Task 2 of the challenge and our integrated system achieved third place in Task 3. Adding to the growing number of publications that utilize NLP to detect occurrences of ADEs, our study illustrates the benefits of employing innovative feature engineering.

Early detection of patient deterioration is key to unlocking the potential for targeted preventative care and improving patient outcomes. This protocol describes a workflow for developing deep learning continuous risk models for early prediction of future acute adverse events from electronic health records (EHR), taking the prediction of the risk of future acute kidney injury (AKI) as an exemplar. The protocol consists of 34 steps grouped into the following stages: formal problem definition, data processing, model architecture selection, risk calibration and uncertainty, and evaluating model generalisability. For the protocol to be applicable to modelling the future risk of a particular condition, the problem formulation should be clinically and physiologically plausible and there needs to be sufficient associated predictive signal in routinely collected EHR data. Prospective validation is key in evaluating whether retrospective models developed by following the proposed protocol are clinically applicable and useful.

BackgroundPatient travel history can be crucial in evaluating evolving infectious disease events. Such information can be challenging to acquire in electronic health records, as it is often available only in unstructured text.ObjectiveThis study aims to assess the feasibility of annotating and automatically extracting travel history mentions from unstructured clinical documents in the Department of Veterans Affairs across disparate health care facilities and among millions of patients. Information about travel exposure augments existing surveillance applications for increased preparedness in responding quickly to public health threats.MethodsClinical documents related to arboviral disease were annotated following selection using a semiautomated bootstrapping process. Using annotated instances as training data, models were developed to extract from unstructured clinical text any mention of affirmed travel locations outside of the continental United States. Automated text processing models were evaluated, involving machine learning and neural language models for extraction accuracy.ResultsAmong 4584 annotated instances, 2659 (58%) contained an affirmed mention of travel history, while 347 (7.6%) were negated. Interannotator agreement resulted in a document-level Cohen kappa of 0.776. Automated text processing accuracy (F1 85.6, 95% CI 82.5-87.9) and computational burden were acceptable such that the system can provide a rapid screen for public health events.ConclusionsAutomated extraction of patient travel history from clinical documents is feasible for enhanced passive surveillance public health systems. Without such a system, it would usually be necessary to manually review charts to identify recent travel or lack of travel, use an electronic health record that enforces travel history documentation, or ignore this potential source of information altogether. The development of this tool was initially motivated by emergent arboviral diseases. More recently, this system was used in the early phases of response to COVID-19 in the United States, although its utility was limited to a relatively brief window due to the rapid domestic spread of the virus. Such systems may aid future efforts to prevent and contain the spread of infectious diseases.

Background Antimicrobial inappropriateness is highly contextual and dynamic, depending not only on the patient’s disease condition but also the information available at the time. To estimate the extent to which antimicrobials could theoretically be decreased with antimicrobial stewardship, we sought to capture unnecessary inpatient antimicrobial use in context over time as manifested in the electronic health record in Veterans Affairs (VA). Methods We extracted antimicrobial use, administrative, admission, and laboratory data from all acute care VA medical centers between 2010 and 2016. Information present during Choice (hospital day [HD] 1–3), Change (HD 4–5), Completion (HD 6–7), and Post-completion (thereafter) was used to determine context. All antimicrobial use without any documented infection was considered unnecessary (admission, discharge, or otherwise). Choice Anti-MRSA agents were considered unnecessary in cellulitis without history of or current positive culture for MRSA. Choice HOMDR agents were unnecessary in cellulitis without history of positive culture for ceftriaxone-resistant Gram-negative rods. Also unnecessary were broad-spectrum antimicrobials (anti-methicillin-resistant Staphylococcus aureus [MRSA] and hospital-onset multidrug-resistant [HOMDR] organisms antimicrobials as defined by the National Healthcare Safety Network) administered without evidence of multidrug-resistant organisms existed during Change and Completion time frames. Results Figure 1 demonstrates the distribution of facility proportions of unnecessary antimicrobials of different classes over time. Table 1 illustrates the percentage of unnecessary antimicrobials administered during choice, change, completion, and post-completion time-frames. Conclusion By this measure, unnecessary anti-MRSA and HOMDR use has been decreasing in VA over time. The bulk of unnecessary use is empiric but there is a substantial proportion that is used for longer stays, during which time more information was likely present. More research is necessary to determine how well these simple rules correlate with clinical determinations of appropriateness. Also ICD-10-CM was implemented in October 2015, which may have introduced an ascertainment bias. Disclosures V. Stevens, Pfizer, Inc.: Grant Investigator, Research grant.

Background Antimicrobial regimens evolve with changing recommendations and emerging practice patterns. We sought to explore the diversity of these patterns and to identify which inpatient regimens may be emerging in US Veterans Affairs medical centers (VAMC). Methods We extracted antimicrobial use and admission data from all acute care VA medical centers between 2005 and 2016. A regimen was defined as all unique antimicrobials and their routes given in a day to a single patient. We applied smoothing to account for intended discontinuation and intermittent dosing due to clearance. We described the distribution of regimens among VAMCs using the Gini index (a Gini index of 0 would mean all regimens were equally frequent and 1 would mean that one regimen dominated all others). We calculated the rank percentile of all regimens. We also used the absolute change in rank percentile between years 2005 and 2016 of the regimen used to describe emerging and declining regimens. Results There were 55,767 distinct regimens. Table 1 describes the Gini index and its decomposition among VAMCs. Overlap accounts for most of the inequality present because regimens are shared between VAMCs. Approximately 20% of the inequality present can be accounted for by variation between VAMCs. Table 2 describes the top 10 rising and the top 10 declining regimens. Conclusion While there was a large number of distinct regimens, there was a relative handful of antimicrobial regimens dominated—most of which were commonly present among VAMCs (as manifest by the Gini “overlap” percent). Most regimens in the top 10 were broad-spectrum IV agents, with PO levofloxacin and doxycycline being notable standouts. IV vancomycin, which was the single most common regimen in 2005, decreased markedly. Linezolid and mixed PO metronidazole agents appear to be on the decline. Disclosures V. Stevens, Pfizer, Inc.: Grant Investigator, Research grant.

Background Medical research publications on sepsis have increased at an exponential rate, whereas our capacity to absorb and understand them has remained limited. We used topic modeling, a method that allows machines to distill large amounts of information into its elemental themes, to help us infer the discourse that led us to the present model/understanding of sepsis. Using this model to augment our understanding of sepsis, an evolving, networked and complex disease, we aimed to recognize connections that could be further explored and aid in knowledge discovery. Methods We extracted all abstracts from PubMed containing the terms “sepsis”, “septic shock”, and “septicemia” between 1890 and 2017 and retained the most informative words. Using topic modeling approaches based on Latent Dirichlet Allocation, we trained dynamic models to five topics from the corpus. We conducted a thematic analysis of topics across publication periods by examining the 30 most frequent words in each topic for each decade. We then fit a static topic model to the last 5 years. We compared the respective themes and their relatedness, and compared the frequency of each topic over the first and second halves of the century. Results Five themes emerged overall: surgery, physiology, microbiology, neonatal/maternal health, and cellular and endothelial responses to infection. When limited to the last 5 years, topics were: acute organ failure and ICU management, early sepsis management and cost, cellular and endothelial response, biomarkers and viruses, and neonatal infection. For the first half of the twentieth century, the bulk of research focused on microbiology while in the latter half of the century there was increased attention on the host response. Conclusion When visualizing the frequency of each topic over the last 100 years we found that the focus has shifted from the pathogen to the host response both from a cellular and physiologic perspective. In the last 5 years, biomarkers, early recognition and system management emerged as new themes. Reasons for this may include: evolution of scientific tools, treatments and statistical abilities, an increasing focus on healthcare cost, and ultimately an incorporation of the individual host response into the disease model. Disclosures All authors: No reported disclosures.

Background Historically, Zika virus infection presented as a mild disease. However, more severe disease was reported during recent outbreaks in French Polynesia and more recently within southern regions of North America as well as Central and South America. It is still unclear what predicts more severe manifestations. Here we report on potential predictors of severe Zika virus infection within VA, selected for their role in immunological status. Methods We extracted the first positive Zika visit for a patient between February 9, 2016 and April 1, 2017. Each visit was classified by acuity (no ED visit, ED only, observation, ward, ICU [in this order of severity]). Diagnoses were extracted by ICD-9-CM and ICD-10-CM codes. Predictors included history of hepatitis C virus (HCV; a flavivirus) by laboratories, dengue diagnosis, immunocompromising condition diagnosis, gender, age, and history of exposure to dengue endemic region (either through birth, travel, or residency). These predictors were used in a generalized ordered logit model, relaxing the proportional odds assumption, to estimate odds ratios for a higher level of visit acuity over the current or lower levels of acuity. Robust covariance estimates were used. Results There were 748 unique patient visits meeting criteria. Distribution of predictors among the patient sample are shown in Table 1. As expected, most were males with a majority only visiting the ED. Wards and ICU were combined due to the small number of ICU visits. Table 2 shows results of model for predictors of higher acuity visits. Age was generally associated with higher levels of acuity. Odds ratios could not be computed for HCV and immunocompromised predictors. Conclusion There may be an increased risk of Zika disease severity based on age. We could not rule out associations with other predictors due to the size of our study. Further larger studies are needed to investigate these and other predictors. Disclosures All Authors: No reported disclosures.