ChatGPT’s advice is perceived as better than that of professional advice columnists.

ChatGPT is a high-performance large language model that has the potential to significantly improve human-computer interactions. It can provide advice on a range of topics, but it is unclear how good this advice is relative to that provided by competent humans, especially in situations where empathy is required. Here, we report the first investigation of whether ChatGPT’s responses are perceived as better than those of humans in a task where humans were attempting to be empathetic. Fifty social dilemma questions were randomly selected from 10 well-known advice columns. In a pre-registered survey, participants (N  =  404) were each shown one question, along with the corresponding response by an advice columnist and by Chat GPT. Chat GPT’s advice was perceived as more balanced, complete, empathetic, helpful, and better than the advice provided by professional advice columnists (all values of p  <  0.001). Although participants could not determine which response was written by Chat GPT (54%, p  =  0.29), most participants preferred that their own social dilemma questions be answered by a human than by a computer (77%, p  <  0.001). Chat GPT’s responses were longer than those produced by the advice columnists (mean 280.9 words vs. 142.2 words, p  <  0.001). In a second pre-registered survey, each Chat GPT answer was constrained to be approximately the same length as that of the advice columnist (mean 143.2 vs. 142.2 words, p  =  0.95). This survey (N  =  401) replicated the above findings, showing that the benefit of Chat GPT was not solely due to it writing longer answers. [ABSTRACT FROM AUTHOR]