Search

International audience; Emotional contagion, in particular of happiness, is essential to creating social bonds. The somatic marker hypothesis posits that embodied physiological changes associated with emotions and relayed to the brain by the autonomous nervous system influence behavior. Perceiving others’ positive emotions should thus be associated with activity in brain regions relaying information from and to the autonomic nervous system. Here, we address this question using a unique corpus of brain activity recorded during unconstrained conversations between participants and a human or a humanoid robot. fMRI recordings are used to test whether activity in key brain regions of the autonomic system, the amygdala, hypothalamus, and insula, is differentially affected by the level of happiness expressed by the human and robot agents. Results indicate that for the hypothalamus and the insula, in particular the anterior agranular region strongly involved in processing social emotions, activity in the right hemisphere increases with the level of happiness expressed by the human but not the robot. Perceiving positive emotions in social interactions induces local brain responses predicted by the contagion of somatic markers of emotions only when the interacting agent is a fellow human.

International audience; Characterizing precisely neurophysiological activity involved in natural conversations remains a major challenge. We explore in this paper the relationship between multimodal conversational behavior and brain activity during natural conversations. This is challenging due to Functional Magnetic Resonance Imaging (fMRI) time resolution and to the diversity of the recorded multimodal signals.We use a unique corpus including localized brain activity and behavior recorded during a fMRI experiment when several participants had natural conversations alternatively with a human and a conversational robot. The corpus includes fMRI responses as well as conversational signals that consist of synchronized raw audio and their transcripts, video and eye-tracking recordings.The proposed approach includes a first step to extract discrete neurophysiological time-series from functionally well defined brain areas, as well as behavioral time-series describing specific behaviors. Then, machine learning models are applied to predict neurophysiological time-series based on the extracted behavioral features.The results show promising prediction scores, and specific causal relationships are found between behaviors and the activity in functional brain areas for both conditions, i.e., human-human and human-robot conversations.

International audience; Dans cet article, nous présentons un outil permettant la prédiction et la visualisation de l’activité cérébrale locale d’un individu au cours d’une conversation. Cet outil a été utilisé en particulier pour comparer l’activité cérébrale de conversation d’individu suivant que l’interlocuteur est un humain ou une tête parlante. Le module de prédiction de cet outil est construit à partir de classifieurs entraînés sur un corpus de conversations humain-humain et humain-machine incluant des données comportementales et neurophysiologiques. Le module prend en entrée des variables comportementales calculées à partir de données brutes, principalement la parole du participant et de l’interlocuteur, la vidéo de l’interlocuteur et les mouvements oculaires du participant, et permet de prédire l’activité cérébrale de l’individu. Le module de visualisation montre en temps réel la dynamique des zones actives cérébrales synchronisées avec les données comportementales brutes, permettant ainsi de faire un lien entre les comportements et l’activité cérébrale. De plus, il montre quelles variables comportementales permettent de prédire l’activité des zones cérébrales.

International audience; Whole-brain neuroimaging using functional Magnetic Resonance Imaging (fMRI) provides valuable data to localize brain activity in space and time. Here, we use a unique corpus including fMRI and behavior recorded when participants discussed with a human or a conversational robot. Temporal dynamic is crucial when studying conversation, yet identifying relationship between the participants' behavior and their brain activity is technically challenging given the time resolution of fMRI. We propose here an approach developed to extract neurophysiological and behavioral time-series from the corpus and analyse their causal relationships. Preprocessing entails the construction of discrete neurophysiological time-series from functionally well defined brain areas, as well as behavioral time-series describing higher-order behaviors extracted from synchronized raw audio, video and eyetracking recordings. The second step consists in applying machine learning models to predict brain activity on the basis of various aspects of behavior given knowledge about the functional role of the areas under scrutiny. Results demonstrate the specificity of the behaviors allowing the predictions of the activity in functional brain areas.

We present a novel functional magnetic resonance imaging paradigm for second-person neuroscience. The paradigm compares a human social interaction (human–human interaction, HHI) to an interaction with a conversational robot (human–robot interaction, HRI). The social interaction consists of 1 min blocks of live bidirectional discussion between the scanned participant and the human or robot agent. A final sample of 21 participants is included in the corpus comprising physiological (blood oxygen level-dependent, respiration and peripheral blood flow) and behavioural (recorded speech from all interlocutors, eye tracking from the scanned participant, face recording of the human and robot agents) data. Here, we present the first analysis of this corpus, contrasting neural activity between HHI and HRI. We hypothesized that independently of differences in behaviour between interactions with the human and robot agent, neural markers of mentalizing (temporoparietal junction (TPJ) and medial prefrontal cortex) and social motivation (hypothalamus and amygdala) would only be active in HHI. Results confirmed significantly increased response associated with HHI in the TPJ, hypothalamus and amygdala, but not in the medial prefrontal cortex. Future analysis of this corpus will include fine-grained characterization of verbal and non-verbal behaviours recorded during the interaction to investigate their neural correlates.This article is part of the theme issue ‘From social brains to social robots: applying neurocognitive insights to human–robot interaction'.

Paleolithic stone tools provide concrete evidence of major developments in human behavioural and cognitive evolution. Of particular interest are evolving cognitive mechanisms implied by the cultural transmission of increasingly complex prehistoric technologies, hypothetically including motor resonance, causal reasoning and mentalizing. To test the relevance of these mechanisms to specific Paleolithic technologies, we conducted a functional magnetic resonance imaging study of Naı¨ve, Trained and Expert subjects observing two toolmaking methods of differing complexity and antiquity: the simple ‘Oldowan’ method documented by the earliest tools 2.5 million years ago; and the more complex ‘Acheulean’ method used to produce refined tools 0.5 million years ago. Subjects observed 20-s video clips of an expert demonstrator, followed by behavioural tasks designed to maintain attention. Results show that observational understanding of Acheulean toolmaking involves increased demands for the recognition of abstract technological intentions. Across subject groups, Acheulean compared with Oldowan toolmaking was associated with activation of left anterior intraparietal and inferior frontal sulci, indicating the relevance of resonance mechanisms. Between groups, Naïve subjects relied on bottom-up kinematic simulation in the premotor cortex to reconstruct unfamiliar intentions, and Experts employed a combination of familiarity-based sensorimotor matching in the posterior parietal cortex and top-down mentalizing involving the medial prefrontal cortex. While no specific differences between toolmaking technologies were found for Trained subjects, both produced frontal activation relative to Control, suggesting focused engagement with toolmaking stimuli. These findings support motor resonance hypotheses for the evolutionary origins of human social cognition and cumulative culture, directly linking these hypotheses with archaeologically observable behaviours in prehistory.

International audience; This paper presents a unique corpus intended to be shared with the community at the crossroad of linguistics and human-robot interactions (HRI). It is the result of an interdisciplinary collaborations within Cognitive Sciences combining social neu-rosciences, computational sciences, robot-ics and linguistics. It was recorded when brain activity of 25 participants was scanned using functional Magnetic Resonance Imaging (fMRI) while having uncon-strained conversations with a fellow human or a conversational robotic head. Behaviors from the participant and the conversant were recorded synchronously (speech, eye-tracking, head and face movements). Manual and automatic analysis of these behaviors provide rich sets of data for the analysis of both behaviors and neurophysiological responses. Examples of results obtained with this corpus are provided.

International audience; In this paper, we present a tool allowing dynamic prediction and visualization of an individual's local brain activity during a conversation. The prediction module of this tool is based on classifiers trained using a corpus of human-human and human-robot conversations including fMRI recordings. More precisely, the module takes as input behavioral features computed from raw data, mainly the participant and the interlocutor speech but also the participant's visual input and eye movements. The visualisation module shows in real-time the dynamics of brain active areas synchronised with the behavioral raw data. In addition, it shows which integrated behavioral features are used to predict the activity in individual brain areas.

International audience; We present an approach to objectify the social competence of artificial agents using human brain neurophysiology. Whole brain activity is recorded with functional Magnetic Resonance Imaging (fMRI) while participants discuss either with a human confederate or an artificial agent. This allows a direct comparison of local brain responses, including deep brain structures invisible to other neuroimaging techniques, as a function of the nature of the interlocutor. The present data (9 participants, artificial agent is the robotic conversational head Furhat controlled with a Wizard of Oz procedure) demonstrates the feasibility of this approach, and results confirm an increased activity in the hypothalamic region when interacting with a human compared to an artificial agent. 1

International audience; Assessing the social competence of anthropomorphic artificial agents developed to produce engaging social interactions with humans has become of primary importance to effectively compare various appearances and/or behaviours. Here we attempt to objectify the social competence of artificial agents, across different dimensions, using human brain neurophysiology. Whole brain activity is recorded with functional Magnetic Resonance Imaging (fMRI) while participants, naïve to the real purpose of the experiment, discuss either with a human confederate or with an artificial agent, presently the robotic conversational head Furhat controlled with a Wizard of Oz procedure. This allows a direct comparison of local brain responses, not only at the cortical level but also in subcortical structures associated with motivational drive and impossible to investigate with non-invasive neurophysiology techniques such as surface recordings. The present data (n=21 participants) demonstrates the feasibility of this approach, and results confirm an increased activity in subcortical structures, in particular the amygdala involved in emotional processing and the hypothalamus, known to secrete, among others, the Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org.

International audience; In this paper, we present a brief overview of our ongoing work about artificial interactive agents and their adaptation to users. Several possibilities to introduce humorous productions in a spoken dialog system are investigated in order to enhance naturalness during social interactions between the agent and the user. We finally describe our plan on how neuroscience will help to better evaluate the proposed systems, both objectively and subjectively.

International audience; Sibling and friend relationships have significant impact on individuals' socio-emotional development. Hypothalamic supraoptic nucleus (SON) and paraventricular nucleus (PVN) synthesize and secrete neuropeptides, including oxytocin, associated with attachment behaviors. Here, using 10 fMRI, we investigate the implication of these two hypothalamic nuclei in the visual processing of personally known faces. Faces of same-sex sibling, best friend, celebrity, and unknown person appear in the middle of the screen while participants perform a task requiring a button click each time a central white dot turns red. Ratings of familiarity (time spent together) and emotionality (feelings toward individual) toward the four individuals are recorded. Local activation within the 15 hypothalamus is assessed via two complementary methods: (1) voxel-based analyses within inclusive mask of the hypothalamus; (2) region-of-interest (ROI) analysis of partial hypothalamic volumes using SON and PVN as center of mass coordinates, with percent signal change extracted and analyzed within these ROIs. Results suggest that the SON responds to all familiar individuals while the PVN has increased response to sibling compared to friend faces and is correlated to 20 familiarity but not emotionality. These findings support differential involvement of local hypotha-lamic substructures SON and PVN in response to faces of individuals with different social relationships.

The classical experimental methodology is ill-suited for the investigation of the behavioral and physiological correlates of natural social interactions. A new experimental approach combining a natural conversation between two persons with control conditions is proposed in this paper. Behavior, including gaze direction and speech, and physiology, including electrodermal activity, are recorded during a discussion between two participants through videoconferencing. Control for the social aspect of the interaction is provided by the use of an artificial agent and of videoed conditions. A cover story provides spurious explanations for the purpose of the experiment and for the recordings, as well as a controlled and engaging topic of discussion. Preprocessing entails transforming raw measurements into boxcar and delta functions time series indicating when a certain behaviour or physiological event is present. The preliminary analysis presented here consists in finding statistically significant differences between experimental conditions in the temporal associations between behavioral and physiological time series. Significant results validate the experimental approach and further developments including more elaborate analysis and adaptation of the paradigm to functional MRI are discussed.

International audience; The hypothalamus is a brain structure containing multiple nuclei that mediate essential behavioral, autonomic, and endocrine functions including oxytocin synthesis. Oxytocin is a neuropeptide linked to complex social cognition and behaviors necessary for an effective social interaction. Oxytocinergic system dysfunction has been linked to social deficits in autism spectrum disorders (ASD). Limited studies have been carried out on the hypothalamus because of its small size and methodological constraints in current technologies. This neuroimaging study examines hypothalamic atrophy in ASD in comparison with a typically developing population (a) by directly measuring gray matter (GM) density with a region-of-interest analysis using voxel-based morphometry in a homogenous sample of participants controlled for age and intelligence quotient; (b) for generalization, by measuring third ventricular volume, on the basis of its position bilaterally surrounded by the hypothalamus, using Freesurfer in a heterogeneous sample of participants. A voxel-based morphometry analysis of cerebrospinal fluid density on the first sample provides a link between GM density and third ventricle volume. Our results show decreased hypothalamic GM density and increased third ventricle volume in ASD compared with typically developing patients. Our findings provide neuroanatomical insights into social deficits in ASD within the hypothalamus that might be relevant for other psychiatric conditions.

International audience; In this paper we present investigation of real-life, bi-directional conversations. We introduce the multimodal corpus derived from these natural conversations alternating between human-human and human-robot interactions. The human-robot interactions were used as a control condition for the social nature of the human-human conversations. The experimental set up consisted of conversations between the participant in a functional magnetic resonance imaging (fMRI) scanner and a human confederate or conversational robot outside the scanner room, connected via bidirectional audio and unidirectional videoconferencing (from the outside to inside the scanner). A cover story provided a framework for natural, real-life conversations about images of an advertisement campaign. During the conversations we collected a multimodal corpus for a comprehensive characterization of bi-directional conversations. In this paper we introduce this multimodal corpus which includes neural data from functional magnetic resonance imaging (fMRI), physiological data (blood flow pulse and respiration), transcribed conversational data, as well as face and eye-tracking recordings. Thus, we present a unique corpus to study human conversations including neural, physiological and behavioral data.