Search

Abstract Large language models (LLM) have been a catalyst for the public interest in artificial intelligence (AI). These technologies perform some knowledge-based tasks better and faster than human beings. However, whether AIs can correctly assess social situations and devise socially appropriate behavior, is still unclear. We conducted an established Situational Judgment Test (SJT) with five different chatbots and compared their results with responses of human participants (N = 276). Claude, Copilot and you.com’s smart assistant performed significantly better than humans in proposing suitable behaviors in social situations. Moreover, their effectiveness rating of different behavior options aligned well with expert ratings. These results indicate that LLMs are capable of producing adept social judgments. While this constitutes an important requirement for the use as virtual social assistants, challenges and risks are still associated with their wide-spread use in social contexts.

IntroductionDespite significant challenges, personnel selection procedures had to continue as on-site testing in the Covid-19 pandemic. Health and safety measures and specifically the use of face masks threaten to limit the fairness of cognitive testing and behavioral observation in the assessment center.MethodsIn this study, we compare the performance and pass rates of pilot selection under three different conditions in the selection campaigns of 2019 (pre-pandemic), 2020 (health and safety measures without mask), and 2021 (health and safety measures with mask).ResultsMask wearing and other health and safety measures had no influence on the objective parameters of pilot selection. However, for some of the areas of competence in the assessment center subjective observability was rated lower for the condition with face masks.DiscussionWe conclude that the fairness and precision of selection processes are not compromised by wearing face masks and that a high degree of standardization in diagnostic instruments prevents a partially reduced subjective observability from affecting the selection’s outcome.