Search

BackgroundEmpathy is a driving force in our connection to others, our mental well-being, and resilience to challenges. With the rise of generative artificial intelligence (AI) systems, mental health chatbots, and AI social support companions, it is important to understand how empathy unfolds toward stories from human versus AI narrators and how transparency plays a role in user emotions. ObjectiveWe aim to understand how empathy shifts across human-written versus AI-written stories, and how these findings inform ethical implications and human-centered design of using mental health chatbots as objects of empathy. MethodsWe conducted crowd-sourced studies with 985 participants who each wrote a personal story and then rated empathy toward 2 retrieved stories, where one was written by a language model, and another was written by a human. Our studies varied disclosing whether a story was written by a human or an AI system to see how transparent author information affects empathy toward the narrator. We conducted mixed methods analyses: through statistical tests, we compared user’s self-reported state empathy toward the stories across different conditions. In addition, we qualitatively coded open-ended feedback about reactions to the stories to understand how and why transparency affects empathy toward human versus AI storytellers. ResultsWe found that participants significantly empathized with human-written over AI-written stories in almost all conditions, regardless of whether they are aware (t196=7.07, P

Introduction: Older adults are engaging more and more with voice-based agent and social robot technologies, and roboticists are increasingly designing interactions for these systems with older adults in mind. Older adults are often not included in these design processes, yet there are many opportunities for older adults to collaborate with design teams to design future robot interactions and help guide directions for robot development.Methods: Through a year-long co-design project, we collaborated with 28 older adults to understand the key focus areas that older adults see promise in for older adult-robot interaction in their everyday lives and how they would like these interactions to be designed. This paper describes and explores the robot-interaction guidelines and future directions identified by older adults, specifically investigating the change and trajectory of these guidelines through the course of the co-design process from the initial interview to the design guideline generation session to the final interview. Results were analyzed through an adapted ethnographic decision tree modeling approach to understand older adults’ decision making surrounding the various focus areas and guidelines for social robots.Results: Overall, over the course of the co-design process between the beginning and end, older adults developed a better understanding of the robot that translated to them being more certain of their attitudes of how they would like a robot to engage with them in their lives. Older adults were more accepting of transactional functions such as reminders and scheduling and less open to functions that would involve sharing sensitive information and tracking and/or monitoring of them, expressing concerns around surveillance. There was some promise in robot interactions for connecting with others, body signal monitoring, and emotional wellness, though older adults brought up concerns around autonomy, privacy, and naturalness of the interaction with a robot that need to be further explored.Discussion: This work provides guidance for future interaction development for robots that are being designed to interact with older adults and highlights areas that need to be further investigated with older adults to understand how best to design for user concerns.

IntroductionSuicide is a leading cause of death around the world, interpolating a huge suffering to the families and communities of the individuals. Such pain and suffering are preventable with early screening and monitoring. However, current suicide risk identification relies on self-disclosure and/or the clinician's judgment.Research question/statmentTherefore, we investigate acoustic and nonverbal behavioral markers that are associated with different levels of suicide risks through a multimodal approach for suicide risk detection.Given the differences in the behavioral dynamics between subregions of facial expressions and body gestures in terms of timespans, we propose a novel region-based multimodal fusion.MethodsWe used a newly collected video interview dataset of young Japanese who are at risk of suicide to extract engineered features and deep representations from the speech, regions of the face (i.e., eyes, nose, mouth), regions of the body (i.e., shoulders, arms, legs), as well as the overall combined regions of face and body.ResultsThe results confirmed that behavioral dynamics differs between regions, where some regions benefit from a shorter timespans, while other regions benefit from longer ones. Therefore, a region-based multimodal approach is more informative in terms of behavioral markers and accounts for both subtle and strong behaviors. Our region-based multimodal results outperformed the single modality, reaching a sample-level accuracy of 96% compared with the highest single modality that reached sample-level accuracy of 80%. Interpretation of the behavioral markers, showed the higher the suicide risk levels, the lower the expressivity, movement and energy observed from the subject. Moreover, the high-risk suicide group express more disgust and contact avoidance, while the low-risk suicide group express self-soothing and anxiety behaviors.DiscussionEven though multimodal analysis is a powerful tool to enhance the model performance and its reliability, it is important to ensure through a careful selection that a strong behavioral modality (e.g., body movement) does not dominate another subtle modality (e.g., eye blink). Despite the small sample size, our unique dataset and the current results adds a new cultural dimension to the research on nonverbal markers of suicidal risks. Given a larger dataset, future work on this method can be useful in helping psychiatrists with the assessment of suicide risk and could have several applications to identify those at risk.

As voice-user interfaces (VUIs), such as smart speakers like Amazon Alexa or social robots like Jibo, enter multi-user environments like our homes, it is critical to understand how group members perceive and interact with these devices. VUIs engage socially with users, leveraging multi-modal cues including speech, graphics, expressive sounds, and movement. The combination of these cues can affect how users perceive and interact with these devices. Through a set of three elicitation studies, we explore family interactions (N = 34 families, 92 participants, ages 4–69) with three commercially available VUIs with varying levels of social embodiment. The motivation for these three studies began when researchers noticed that families interacted differently with three agents when familiarizing themselves with the agents and, therefore, we sought to further investigate this trend in three subsequent studies designed as a conceptional replication study. Each study included three activities to examine participants’ interactions with and perceptions of the three VUIS in each study, including an agent exploration activity, perceived personality activity, and user experience ranking activity. Consistent for each study, participants interacted significantly more with an agent with a higher degree of social embodiment, i.e., a social robot such as Jibo, and perceived the agent as more trustworthy, having higher emotional engagement, and having higher companionship. There were some nuances in interaction and perception with different brands and types of smart speakers, i.e., Google Home versus Amazon Echo, or Amazon Show versus Amazon Echo Spot between the studies. In the last study, a behavioral analysis was conducted to investigate interactions between family members and with the VUIs, revealing that participants interacted more with the social robot and interacted more with their family members around the interactions with the social robot. This paper explores these findings and elaborates upon how these findings can direct future VUI development for group settings, especially in familial settings.

Can robots help children be more creative? In this work, we posit social robots as creativity support tools for children in collaborative interactions. Children learn creative expressions and behaviors through social interactions with others during playful and collaborative tasks, and socially emulate their peers’ and teachers’ creativity. Social robots have a unique ability to engage in social and emotional interactions with children that can be leveraged to foster creative expression. We focus on two types of social interactions: creativity demonstration, where the robot exhibits creative behaviors, and creativity scaffolding, where the robot poses challenges, suggests ideas, provides positive reinforcement, and asks questions to scaffold children’s creativity. We situate our research in three playful and collaborative tasks - the Droodle Creativity game (that affords verbal creativity), the MagicDraw game (that affords figural creativity), and the WeDo construction task (that affords constructional creativity), that children play with Jibo, a social robot. To evaluate the efficacy of the robot’s social behaviors in enhancing creative behavior and expression in children, we ran three randomized controlled trials with 169 children in the 5–10 yr old age group. In the first two tasks, the robot exhibited creativity demonstration behaviors. We found that children who interacted with the robot exhibiting high verbal creativity in the Droodle game and high figural creativity in the MagicDraw game also exhibited significantly higher creativity than a control group of participants who interacted with a robot that did not express creativity (p < 0.05*). In the WeDo construction task, children who interacted with the robot that expressed creative scaffolding behaviors (asking reflective questions, generating ideas and challenges, and providing positive reinforcement) demonstrated higher creativity than participants in the control group by expressing a greater number of ideas, more original ideas, and more varied use of available materials (p < 0.05*). We found that both creativity demonstration and creativity scaffolding can be leveraged as social mechanisms for eliciting creativity in children using a social robot. From our findings, we suggest design guidelines for pedagogical tools and social agent interactions to better support children’s creativity.

The storytelling lens in human-computer interaction has primarily focused on personas, design fiction, and other stories crafted by designers, yet informal personal narratives from everyday people have not been considered meaningful data, such as storytelling from older adults. Storytelling may provide a clear path to conceptualize how technologies such as social robots can support the lives of older or disabled individuals. To explore this, we engaged 28 older adults in a year-long co-design process, examining informal stories told by older adults as a means of generating and expressing technology ideas and needs. This paper presents an analysis of participants’ stories around their prior experience with technology, stories shaped by social context, and speculative scenarios for the future of social robots. From this analysis, we present suggestions for social robot design, considerations of older adults’ values around technology design, and promotion of participant stories as sources for design knowledge and shifting perspectives of older adults and technology.

Across a wide variety of domains, artificial agents that can adapt and personalize to users have potential to improve and transform how social services are provided. Because of the need for personalized interaction data to drive this process, long-term (or longitudinal) interactions between users and agents, which unfold over a series of distinct interaction sessions, have attracted substantial research interest. In recognition of the expanded scope and structure of a long-term interaction, researchers are also adjusting the personalization models and algorithms used, orienting toward “continual learning” methods, which do not assume a stationary modeling target and explicitly account for the temporal context of training data. In parallel, researchers have also studied the effect of “multitask personalization,” an approach in which an agent interacts with users over multiple different tasks contexts throughout the course of a long-term interaction and learns personalized models of a user that are transferrable across these tasks. In this paper, we unite these two paradigms under the framework of “Lifelong Personalization,” analyzing the effect of multitask personalization applied to dynamic, non-stationary targets. We extend the multi-task personalization approach to the more complex and realistic scenario of modeling dynamic learners over time, focusing in particular on interactive scenarios in which the modeling agent plays an active role in teaching the student whose knowledge the agent is simultaneously attempting to model. Inspired by the way in which agents use active learning to select new training data based on domain context, we augment a Gaussian Process-based multitask personalization model with a mechanism to actively and continually manage its own training data, allowing a modeling agent to remove or reduce the weight of observed data from its training set, based on interactive context cues. We evaluate this method in a series of simulation experiments comparing different approaches to continual and multitask learning on simulated student data. We expect this method to substantially improve learning in Gaussian Process models in dynamic domains, establishing Gaussian Processes as another flexible modeling tool for Long-term Human-Robot Interaction (HRI) Studies.

Generative Artificial Intelligence (AI) approaches open up new avenues of digital creation, and are simultaneously accompanied by societal and ethical implications such as the creation of Deepfakes and spread of misinformation, renewing our understanding of technical AI systems as socio-technical systems. Applications of, and media generated by generative AI techniques are abundantly present on social media platforms frequented by children, who are not yet aware of the existence of AI-manipulated media. Previous work has highlighted the importance of digital media literacy and AI literacy for children. In this work, we introduce middle school students to generative AI techniques as a tool for creation, while also focusing on critical discussion about their societal and ethical implications, and encouraging pro-activeness in being responsible consumers, creators and stakeholders of technology. We present learning activities that introduce 38 middle-school students to generative modeling, how it is used to generate Deepfakes, cues that help to recognize Deepfakes, and the spread and effects of misinformation. Students demonstrated an understanding that generative media may be believable, but not necessarily true, and can contribute to the spread of misinformation. They were also able to identify why misinformation may be harmful or lasting, drawing specific examples to social settings that indicate human-centered implications. Finally, students expressed opinions about policies surrounding the presence of Deepfakes on social media. This approach can be adopted to introduce students to other technical systems that constitute both productive applications and potential negative implications of technology. CCS concepts: ⋅Applied computing → Interactive learning environments; ⋅Human-centered computing → Social media; Social networks; ⋅Social and professional topics → Computing literacy; K-12 education; Additional key words and phrases: Misinformation, Deepfakes, digital literacy, media literacy, social media.

Prior research has demonstrated the importance of children's peers for their learning and development. In particular, peer interaction, especially with more advanced peers, can enhance preschool children's language growth. In this paper, we explore one factor that may modulate children's language learning with a peer-like social robot: rapport. We explore connections between preschool children's learning, rapport, and emulation of the robot's language during a storytelling intervention. We performed a long-term field study in a preschool with 17 children aged 4–6 years. Children played a storytelling game with a social robot for 8 sessions over two months. For some children, the robot matched the level of its stories to the children's language ability, acting as a slightly more advanced peer (Matched condition); for the others, the robot did not match the story level (Unmatched condition). We examined children's use of target vocabulary words and key phrases used by the robot, children's emulation of the robot's stories during their own storytelling, and children's language style matching (LSM—a measure of overlap in function word use and speaking style associated with rapport and relationship) to see whether they mirrored the robot more over time. We found that not only did children emulate the robot more over time, but also, children who emulated more of the robot's phrases during storytelling scored higher on the vocabulary posttest. Children with higher LSM scores were more likely to emulate the robot's content words in their stories. Furthermore, the robot's personalization in the Matched condition led to increases in both children's emulation and their LSM scores. Together, these results suggest first, that interacting with a more advanced peer is beneficial for children, and second, that children's emulation of the robot's language may be related to their rapport and their learning. This is the first study to empirically support that rapport may be a modulating factor in children's peer learning, and furthermore, that a social robot can serve as an effective intervention for language development by leveraging this insight.

In positive human-human relationships, people frequently mirror or mimic each other's behavior. This mimicry, also called entrainment, is associated with rapport and smoother social interaction. Because rapport in learning scenarios has been shown to lead to improved learning outcomes, we examined whether enabling a social robotic learning companion to perform rapport-building behaviors could improve children's learning and engagement during a storytelling activity. We enabled the social robot to perform two specific rapport and relationship-building behaviors: speech entrainment and self-disclosure (shared personal information in the form of a backstory about the robot's poor speech and hearing abilities). We recruited 86 children aged 3–8 years to interact with the robot in a 2 × 2 between-subjects experimental study testing the effects of robot entrainment Entrainment vs. No entrainment and backstory about abilities Backstory vs. No Backstory. The robot engaged the children one-on-one in conversation, told a story embedded with key vocabulary words, and asked children to retell the story. We measured children's recall of the key words and their emotions during the interaction, examined their story retellings, and asked children questions about their relationship with the robot. We found that the robot's entrainment led children to show more positive emotions and fewer negative emotions. Children who heard the robot's backstory were more likely to accept the robot's poor hearing abilities. Entrainment paired with backstory led children to use more of the key words and match more of the robot's phrases in their story retells. Furthermore, these children were more likely to consider the robot more human-like and were more likely to comply with one of the robot's requests. These results suggest that the robot's speech entrainment and backstory increased children's engagement and enjoyment in the interaction, improved their perception of the relationship, and contributed to children's success at retelling the story.

Current state-of-the-art approaches to emotion recognition primarily focus on modeling the nonverbal expressions of the sole individual without reference to contextual elements such as the co-presence of the partner. In this paper, we demonstrate that the accurate inference of listeners’ social-emotional state of attention depends on accounting for the nonverbal behaviors of their storytelling partner, namely their speaker cues. To gain a deeper understanding of the role of speaker cues in attention inference, we conduct investigations into real-world interactions of children (5–6 years old) storytelling with their peers. Through in-depth analysis of human–human interaction data, we first identify nonverbal speaker cues (i.e., backchannel-inviting cues) and listener responses (i.e., backchannel feedback). We then demonstrate how speaker cues can modify the interpretation of attention-related backchannels as well as serve as a means to regulate the responsiveness of listeners. We discuss the design implications of our findings toward our primary goal of developing attention recognition models for storytelling robots, and we argue that social robots can proactively use speaker cues to form more accurate inferences about the attentive state of their human partners.