101. Automated rating of patient and physician emotion in primary care visits
- Author
-
Jihyun Park, Zac E. Imel, Abhishek Jindal, Padhraic Smyth, Jennifer Elston Lafata, David C. Atkins, Ming Tai-Seale, Patty Kuo, and Michael J. Tanana
- Subjects
Primary Health Care ,Office Visits ,030503 health policy & services ,Emotion classification ,Communication ,Sentiment analysis ,Emotions ,Reproducibility of Results ,General Medicine ,Primary care ,Logistic regression ,Correlation ,03 medical and health sciences ,0302 clinical medicine ,Recurrent neural network ,Physicians ,Humans ,030212 general & internal medicine ,Emotion recognition ,0305 other medical science ,Psychology ,Recurrent neural network model ,Clinical psychology - Abstract
Objective Train machine learning models that automatically predict emotional valence of patient and physician in primary care visits. Methods Using transcripts from 353 primary care office visits with 350 patients and 84 physicians (Cook, 2002 [ 1 ], Tai-Seale et al., 2015 [ 2 ]), we developed two machine learning models (a recurrent neural network with a hierarchical structure and a logistic regression classifier) to recognize the emotional valence (positive, negative, neutral) (Posner et al., 2005 [ 3 ]) of each utterance. We examined the agreement of human-generated ratings of emotional valence with machine learning model ratings of emotion. Results The agreement of emotion ratings from the recurrent neural network model with human ratings was comparable to that of human-human inter-rater agreement. The weighted-average of the correlation coefficients for the recurrent neural network model with human raters was 0.60, and the human rater agreement was also 0.60. Conclusions The recurrent neural network model predicted the emotional valence of patients and physicians in primary care visits with similar reliability as human raters. Practice implications As the first machine learning-based evaluation of emotion recognition in primary care visit conversations, our work provides valuable baselines for future applications that might help monitor patient emotional signals, supporting physicians in empathic communication, or examining the role of emotion in patient-centered care.
- Published
- 2020