Generative artificial intelligence responses to patient messages in the electronic health record: early lessons learned.

Background: Electronic health record (EHR)-based patient messages can contribute to burnout. Messages with a negative tone are particularly challenging to address. In this perspective, we describe our initial evaluation of large language model (LLM)-generated responses to negative EHR patient messages and contend that using LLMs to generate initial drafts may be feasible, although refinement will be needed.
Methods: A retrospective sample ( n  = 50) of negative patient messages was extracted from a health system EHR, de-identified, and inputted into an LLM (ChatGPT). Qualitative analyses were conducted to compare LLM responses to actual care team responses.
Results: Some LLM-generated draft responses varied from human responses in relational connection, informational content, and recommendations for next steps. Occasionally, the LLM draft responses could have potentially escalated emotionally charged conversations.
Conclusion: Further work is needed to optimize the use of LLMs for responding to negative patient messages in the EHR.
Competing Interests: S.L.B. reports equipment support from Topcon and Optomed, and consulting fees from Topcon, outside the submitted work. C.A.L. reports equity from consulting with Doximity. M.T.-S., M.M., and A.M.S. report no relevant conflicts.
(© The Author(s) 2024. Published by Oxford University Press on behalf of the American Medical Informatics Association.)