Search

Objectives: This study evaluates the accuracy and portability of a natural language processing (NLP) tool for extracting clinical findings of influenza from clinical notes across two large healthcare systems. Effectiveness is evaluated on how well NLP supports downstream influenza case-detection for disease surveillance., Methods: We independently developed two NLP parsers, one at Intermountain Healthcare (IH) in Utah and the other at University of Pittsburgh Medical Center (UPMC) using local clinical notes from emergency department (ED) encounters of influenza. We measured NLP parser performance for the presence and absence of 70 clinical findings indicative of influenza. We then developed Bayesian network models from NLP processed reports and tested their ability to discriminate among cases of (1) influenza, (2) non-influenza influenza-like illness (NI-ILI), and (3) 'other' diagnosis., Results: On Intermountain Healthcare reports, recall and precision of the IH NLP parser were 0.71 and 0.75, respectively, and UPMC NLP parser, 0.67 and 0.79. On University of Pittsburgh Medical Center reports, recall and precision of the UPMC NLP parser were 0.73 and 0.80, respectively, and IH NLP parser, 0.53 and 0.80. Bayesian case-detection performance measured by AUROC for influenza versus non-influenza on Intermountain Healthcare cases was 0.93 (using IH NLP parser) and 0.93 (using UPMC NLP parser). Case-detection on University of Pittsburgh Medical Center cases was 0.95 (using UPMC NLP parser) and 0.83 (using IH NLP parser). For influenza versus NI-ILI on Intermountain Healthcare cases performance was 0.70 (using IH NLP parser) and 0.76 (using UPMC NLP parser). On University of Pisstburgh Medical Center cases, 0.76 (using UPMC NLP parser) and 0.65 (using IH NLP parser)., Conclusion: In all but one instance (influenza versus NI-ILI using IH cases), local parsers were more effective at supporting case-detection although performances of non-local parsers were reasonable.

We asked two physicians to review the medical records (electronic and paper) of 100 patients on antithrombotics. The physicians used published classification criteria to identify all of the bleeding events that the patients experienced. The goal of the review was to investigate whether the physicians would identify the same antithrombotic related major bleeding events (ARMBEs) for each patient. The correct identification and classification of multiple bleeding events is a prerequisite for studies of antithrombotic treatment practices during hospitalization that predispose patients to ARMBEs. In addition, we were interested in the reasons for disagreement between the physicians, so that we could find ways of improving their agreement. The reviewers identified 299 bleeding events for the 100 patients. They disagreed on whether 29 of the events represented an ARMBE occurring during hospitalization. With a kappa statistic of 0.49 (95% confidence interval, 0.31 to 0.66) the agreement was moderate. The reviewers most often disagreed either because they misinterpreted the data (12 events) or because the classification criteria for ARMBEs were not explicit enough (9 events). Disagreement took two main forms: either the reviewers disagreed on ARMBEs by not identifying the same bleeds (11 events) or by not applying the severity criteria appropriately (7 events). Because the main type of disagreement was not identifying the same bleeds, a study investigating the antithrombotic treatment practices that predispose patients to ARMBEs would be threatened. We therefore proposed supplementing the existing classification criteria with additional rules to avoid ambiguities in patients with multiple events.

Objectives: To discuss the advantages and disadvantages of an interfaced approach to clinical information systems architecture., Methods: After many years of internally building almost all components of a hospital clinical information system (HELP) at Intermountain Health Care, we changed our architectural approach as we chose to encompass ambulatory as well as acute care. We now seek to interface applications from a variety of sources (including some that we build ourselves) to a clinical data repository that contains a longitudinal electronic patient record., Results: We have a total of 820 instances of interfaces to 51 different applications. We process nearly 2 million transactions per day via our interface engine and feel that the reliability of the approach is acceptable. Interface costs constitute about four percent of our total information systems budget. The clinical database currently contains records for 1.45 m patients and the response time for a query is 0.19 sec., Discussion: Based upon our experience with both integrated (monolithic) and interfaced approaches, we conclude that for those with the expertise and resources to do so, the interfaced approach offers an attractive alternative to systems provided by a single vendor. We expect the advantages of this approach to increase as the costs of interfaces are reduced in the future as standards for vocabulary and messaging become increasingly mature and functional.

An essential step toward the effective processing of the medical language is the development of representational models that formalize the language semantics. These models, also known as semantic data models, help to unlock the meaning of descriptive expressions, making them accessible to computer systems. The present study tries to determine the quality of a semantic data model created to encode chest radiology findings. The evaluation methodology relied on the ability of physicians to extract information from textual and encoded representations of chest X-ray reports, whilst answering questions associated with each report. The evaluation demonstrated that the encoded reports seemed to have the same information content of the original textual reports. The methodology generated useful data regarding the quality of the data model, demonstrating that certain segments were creating ambiguous representations and that some details were not being represented.