Back to Search Start Over

The Illusion of Empathy: How AI Chatbots Shape Conversation Perception

Authors :
Liu, Tingting
Giorgi, Salvatore
Aich, Ankit
Lahnala, Allison
Curtis, Brenda
Ungar, Lyle
Sedoc, João
Publication Year :
2024

Abstract

As AI chatbots become more human-like by incorporating empathy, understanding user-centered perceptions of chatbot empathy and its impact on conversation quality remains essential yet under-explored. This study examines how chatbot identity and perceived empathy influence users' overall conversation experience. Analyzing 155 conversations from two datasets, we found that while GPT-based chatbots were rated significantly higher in conversational quality, they were consistently perceived as less empathetic than human conversational partners. Empathy ratings from GPT-4o annotations aligned with users' ratings, reinforcing the perception of lower empathy in chatbots. In contrast, 3 out of 5 empathy models trained on human-human conversations detected no significant differences in empathy language between chatbots and humans. Our findings underscore the critical role of perceived empathy in shaping conversation quality, revealing that achieving high-quality human-AI interactions requires more than simply embedding empathetic language; it necessitates addressing the nuanced ways users interpret and experience empathy in conversations with chatbots.

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2411.12877
Document Type :
Working Paper