Search

Background: Clinical knowledge bases of problem-medication pairs are necessary for many informatics solutions that improve patient safety, such as clinical summarization. However, developing these knowledge bases can be challenging., Objective: We sought to validate a previously developed crowdsourcing approach for generating a knowledge base of problem-medication pairs in a large, non-university health care system with a widely used, commercially available electronic health record., Methods: We first retrieved medications and problems entered in the electronic health record by clinicians during routine care during a six month study period. Following the previously published approach, we calculated the link frequency and link ratio for each pair then identified a threshold cutoff for estimated problem-medication pair appropriateness through clinician review; problem-medication pairs meeting the threshold were included in the resulting knowledge base. We selected 50 medications and their gold standard indications to compare the resulting knowledge base to the pilot knowledge base developed previously and determine its recall and precision., Results: The resulting knowledge base contained 26,912 pairs, had a recall of 62.3% and a precision of 87.5%, and outperformed the pilot knowledge base containing 11,167 pairs from the previous study, which had a recall of 46.9% and a precision of 83.3%., Conclusions: We validated the crowdsourcing approach for generating a knowledge base of problem-medication pairs in a large non-university health care system with a widely used, commercially available electronic health record, indicating that the approach may be generalizable across healthcare settings and clinical systems. Further research is necessary to better evaluate the knowledge, to compare crowdsourcing with other approaches, and to evaluate if incorporating the knowledge into electronic health records improves patient outcomes.

Background: Correlation of data within electronic health records is necessary for implementation of various clinical decision support functions, including patient summarization. A key type of correlation is linking medications to clinical problems; while some databases of problem-medication links are available, they are not robust and depend on problems and medications being encoded in particular terminologies. Crowdsourcing represents one approach to generating robust knowledge bases across a variety of terminologies, but more sophisticated approaches are necessary to improve accuracy and reduce manual data review requirements., Objective: We sought to develop and evaluate a clinician reputation metric to facilitate the identification of appropriate problem-medication pairs through crowdsourcing without requiring extensive manual review., Approach: We retrieved medications from our clinical data warehouse that had been prescribed and manually linked to one or more problems by clinicians during e-prescribing between June 1, 2010 and May 31, 2011. We identified measures likely to be associated with the percentage of accurate problem-medication links made by clinicians. Using logistic regression, we created a metric for identifying clinicians who had made greater than or equal to 95% appropriate links. We evaluated the accuracy of the approach by comparing links made by those physicians identified as having appropriate links to a previously manually validated subset of problem-medication pairs., Results: Of 867 clinicians who asserted a total of 237,748 problem-medication links during the study period, 125 had a reputation metric that predicted the percentage of appropriate links greater than or equal to 95%. These clinicians asserted a total of 2464 linked problem-medication pairs (983 distinct pairs). Compared to a previously validated set of problem-medication pairs, the reputation metric achieved a specificity of 99.5% and marginally improved the sensitivity of previously described knowledge bases., Conclusion: A reputation metric may be a valuable measure for identifying high quality clinician-entered, crowdsourced data., (Copyright © 2013 Elsevier Inc. All rights reserved.)

Objective: We describe a novel, crowdsourcing method for generating a knowledge base of problem-medication pairs that takes advantage of manually asserted links between medications and problems., Methods: Through iterative review, we developed metrics to estimate the appropriateness of manually entered problem-medication links for inclusion in a knowledge base that can be used to infer previously unasserted links between problems and medications., Results: Clinicians manually linked 231,223 medications (55.30% of prescribed medications) to problems within the electronic health record, generating 41,203 distinct problem-medication pairs, although not all were accurate. We developed methods to evaluate the accuracy of the pairs, and after limiting the pairs to those meeting an estimated 95% appropriateness threshold, 11,166 pairs remained. The pairs in the knowledge base accounted for 183,127 total links asserted (76.47% of all links). Retrospective application of the knowledge base linked 68,316 medications not previously linked by a clinician to an indicated problem (36.53% of unlinked medications). Expert review of the combined knowledge base, including inferred and manually linked problem-medication pairs, found a sensitivity of 65.8% and a specificity of 97.9%., Conclusion: Crowdsourcing is an effective, inexpensive method for generating a knowledge base of problem-medication pairs that is automatically mapped to local terminologies, up-to-date, and reflective of local prescribing practices and trends.

Increasing use of electronic health records requires comprehensive patient-centered views of clinical data. We describe a prototype knowledge base and SMART app that facilitates organization of patient medications by clinical problems, comprising a preliminary step in building such patient-centered views. The knowledge base includes 7,164,444 distinct problem-medication links, generated from RxNorm, SNOMED CT, and NDF-RT within the UMLS Metathesaurus. In an evaluation of the knowledge base applied to 5000 de-identified patient records, 22.4% of medications linked to an entry in the patient's active problem list, compared to 32.6% of medications manually linked by providers; 46.5% of total links were unique to the knowledge base, not added by providers. Expert review of a random patient subset estimated a sensitivity of 37.1% and specificity of 98.9%. The SMART API successfully utilized the knowledge base to generate problem-medication links for test patients. Future work is necessary to improve knowledge base sensitivity and efficiency.