Search

Social signal recognition techniques based on nonverbal behavioral sensing allow conversational robots to understand the user’s social signals, thereby enabling them to adopt interaction strategies based on internal states inferred from the social signals. This research investigates how the online social signal recognition and adaptive dialog strategy influences the dynamic change in a user’s inner state. For this purpose, we develop a semiautonomous interview robot system with an online speaker’s willingness recognition module and an adaptive question selection module based on the willingness level. The online recognition model of speaker willingness is trained from multimodal nonverbal features extracted using a novel interview corpus, which allows appropriate interview questions to be chosen based on the estimated willingness level of the user. We conduct the experiment using the system to evaluate the effectiveness of adaptive question selection based on the willingness recognition model. First, the multimodal willingness recognition model is evaluated using the interview corpus. The best recognition accuracy of willingness level (high or low) was $72. 8\%$ 72. 8 p e r c n t ; with the random forest classifier. Second, 27 interviewees were interviewed with the two interview robot systems: (I) with the adaptive question selection module based on willingness recognition and (II) with a random question selection strategy. The proposed adaptive question strategy significantly increased the number of utterances with high willingness compared with the baseline system (II); thus, adaptive question selection with online willingness recognition elicited the speaker’s willingness even though the model cannot be estimated with near-perfect accuracy. [ABSTRACT FROM AUTHOR]

Prior research on affect elicitation indicates that stimuli with social content (pictures or videos) are more arousing than nonsocial stimuli. In particular, they elicit stronger physiological arousal as measured by electrodermal activity (EDA; i.e., social EDA effect). However, it is unclear how this effect applies to virtual reality (VR), which enables an enhanced sense of presence (SoP) and ecological validity. The study here approached this question from a social–emotional VR framework. A sample of N = 72 participants (55 percent women) experienced a set of six virtual environments (VEs) in the form of emotional parks specifically designed to elicit positive, negative, or neutral affectivity. Half of these VEs included human-shaped agents (social context) and the other half omitted these agents (nonsocial context). The results supported the social EDA effect, which in addition was amplified by the reported SoP. Importantly, the VE featuring a social negative content qualified this observed social EDA effect. The finding is discussed in the light of a negativity bias reported in affect literature, through which negative stimuli typically mobilize attention and bodily activation as a mechanism linked to stress responses. The study's implications extend to the use of VR in both research and practical applications, emphasizing the role of social content in influencing affective and physiological responses. [ABSTRACT FROM AUTHOR]

Current methods for assessing individual well-being in team collaboration at the workplace often rely on manually collected surveys. This limits continuous real-world data collection and proactive measures to improve team member workplace satisfaction. We propose a method to automatically derive social signals related to individual well-being in team collaboration from raw audio and video data collected in teamwork contexts. The goal was to develop computational methods and measurements to facilitate the mirroring of individuals' well-being to themselves. We focus on how speech behavior is perceived by team members to improve their well-being. Our main contribution is the assembly of an integrated toolchain to perform multi-modal extraction of robust speech features in noisy field settings and to explore which features are predictors of self-reported satisfaction scores. We applied the toolchain to a case study, where we collected videos of 20 teams with 56 participants collaborating over a four-day period in a team project in an educational environment. Our audiovisual speaker diarization extracted individual speech features from a noisy environment. As the dependent variable, team members filled out a daily PERMA (positive emotion, engagement, relationships, meaning, and accomplishment) survey. These well-being scores were predicted using speech features extracted from the videos using machine learning. The results suggest that the proposed toolchain was able to automatically predict individual well-being in teams, leading to better teamwork and happier team members. [ABSTRACT FROM AUTHOR]

Current methods for assessing individual well-being in team collaboration at the workplace often rely on manually collected surveys. This limits continuous real-world data collection and proactive measures to improve team member workplace satisfaction. We propose a method to automatically derive social signals related to individual well-being in team collaboration from raw audio and video data collected in teamwork contexts. The goal was to develop computational methods and measurements to facilitate the mirroring of individuals’ well-being to themselves. We focus on how speech behavior is perceived by team members to improve their well-being. Our main contribution is the assembly of an integrated toolchain to perform multi-modal extraction of robust speech features in noisy field settings and to explore which features are predictors of self-reported satisfaction scores. We applied the toolchain to a case study, where we collected videos of 20 teams with 56 participants collaborating over a four-day period in a team project in an educational environment. Our audiovisual speaker diarization extracted individual speech features from a noisy environment. As the dependent variable, team members filled out a daily PERMA (positive emotion, engagement, relationships, meaning, and accomplishment) survey. These well-being scores were predicted using speech features extracted from the videos using machine learning. The results suggest that the proposed toolchain was able to automatically predict individual well-being in teams, leading to better teamwork and happier team members.

Charisma is considered as one's ability to attract and potentially influence others. Clearly, there can be considerable interest from an artificial intelligence's (AI) perspective to provide it with such skill. Beyond, a plethora of use cases opens up for computational measurement of human charisma, such as for tutoring humans in the acquisition of charisma, mediating human-to-human conversation, or identifying charismatic individuals in big social data. While charisma is a subject of research in its own right, a number of models exist that base it on various “pillars,” that is, dimensions, often following the idea that charisma is given if someone could and would help others. Examples of such pillars, therefore, include influence (could help) and affability (would help) in scientific studies, or power (could help), presence, and warmth (both would help) as a popular concept. Modeling high levels in these dimensions, i. e., high influence and high affability, or high power, presence, and warmth for charismatic AI of the future, e. g., for humanoid robots or virtual agents, seems accomplishable. Beyond, also automatic measurement appears quite feasible with the recent advances in the related fields of Affective Computing and Social Signal Processing. Here, we therefore present a brick by brick blueprint for building machines that can appear charismatic, but also analyse the charisma of others. We first approach the topic very broadly and discuss how the foundation of charisma is defined from a psychological perspective. Throughout the manuscript, the building blocks (bricks) then become more specific and provide concrete groundwork for capturing charisma through artificial intelligence (AI). Following the introduction of the concept of charisma, we switch to charisma in spoken language as an exemplary modality that is essential for human-human and human-computer conversations. The computational perspective then deals with the recognition and generation of charismatic behavior by AI. This includes an overview of the state of play in the field and the aforementioned blueprint. We then list exemplary use cases of computational charismatic skills. The building blocks of application domains and ethics conclude the article.

The attachment system is an innate human instinct to gain a sense of security as a form of self-defense from threats. Adults with secure attachment can maintain the balance of their relationships with themselves and significant others such as parents, romantic partners, and close friends. Generally, the adult attachment assessment data are collected primarily from subjective responses through questionnaires or interviews, which are closed to the research community. Attachment assessment from behavioral traits has also not been studied in depth because attachment-related behavioral data are still not openly available for research. This limits the scope of attachment assessment to new alternative innovations, such as the application of machine learning and deep learning-based approaches. This paper presents the Indonesian Young Adult Attachment (IYAA) dataset, a facial expression and speech audio dataset of Indonesian young adults in attachment projective-based assessment. The assessment contains two stages: exposure and response of 14 attachment-based stimuli. IYAA consists of audio-video data from age groups between 18-29 years old, with 20 male and 67 female subjects. It contains 1216 exposure videos, 1217 response videos, and 1217 speech response audios. Each data has a varying duration; the duration for exposure video ranges from 25 seconds to 1 minute 39 seconds, while for response video and speech response audio ranges from 40 seconds to 8 minutes and 25 seconds. The IYAA dataset is annotated into two kinds of labels: emotion and attachment. First, emotion labeling is annotated on each stimulus for all subject data (exposure videos, response videos, speech response audios). Each data is annotated into one or more labels among eight basic emotion categories (neutral, happy, sad, contempt, anger, disgust, surprised, fear) since each attachment-related event involves unconscious mental processes characterized by emotional changes. Second, each subject is annotated into one among three attachment style labels: secure, insecure-anxious, and insecure-avoidance. Given these two kinds of labeling, the IYAA dataset supports several research purposes, either using one kind of label separately or using them together for attachment classification research. It also supports innovative approaches to build automatic attachment classification through collaboration between the study of Behavioral, Developmental, and Social Psychology with Social Signal Processing.

Recent technological advances have made the remote recording of interactions and the automatic extraction of nonverbal, paraverbal, and verbal behavioral cues from the videos possible. The development of computer programs that can replace human observational coders is imminent. The increasing availability of such new technologies, accompanied by their lower costs and greater convenience, is likely to advance behavioral observation research, giving new insight into the fine-grained moment-to-moment interactional processes. We illustrate how couple researchers can use recent technological advances to bring behavioral observation research into the 21st century. (PsycInfo Database Record (c) 2022 APA, all rights reserved). [ABSTRACT FROM AUTHOR]

Socio-conversational systems are dialogue systems, including what are sometimes referred to as chatbots, vocal assistants, social robots, and embodied conversational agents, that are capable of interacting with humans in a way that treats both the specifically social nature of the interaction and the content of a task. The aim of this paper is twofold: 1) to uncover some places where the compartmentalized nature of research conducted around socio-conversational systems creates problems for the field as a whole, and 2) to propose a way to overcome this compartmentalization and thus strengthen the capabilities of socio-conversational systems by defining common challenges. Specifically, we examine research carried out by the signal processing, natural language processing and dialogue, machine/deep learning, social/affective computing and social sciences communities. We focus on three major challenges for the development of effective socio-conversational systems, and describe ways to tackle them.

Over the past 10 years, psychiatry, psychology and artificial intelligence (AI) have opened a fruitful dialogue. In the field of social interactions and social signal processing, many multidisciplinary works have contributed to a better understanding of the complexity of certain psychological phenomena and to the progress of human-machine interfaces. They allow the computational consideration of the dyadic dimension, the possibility to take into account the multimodal nature of turn taking and the development of specific tools to accurately and automatically extract the social signals of partners. We propose here a brief survey of our work illustrating the interest of not only associating AI and psychopathology to explore certain specific contexts (effect of oxytocin, early maternal neglect, developmental robotics) and certain psychopathologies (autism, borderline states) but also to design therapeutic applications such as serious games or intelligent companion robots. [ABSTRACT FROM AUTHOR]

Eating is a fundamental part of human life and is, more than anything, a social activity. A new field, known as Computational Commensality has been created to computationally address various social aspects of food and eating. This paper illustrates a study on remote dining we conducted online in May 2021. To better understand this phenomenon, known as Digital Commensality, we recorded 11 pairs of friends sharing a meal online through a videoconferencing app. In the videos, participants consume a plate of pasta while chatting with a friend or a family member. After the remote dinner, participants were asked to fill in the Digital Commensality questionnaire, a validated questionnaire assessing the eects of remote commensal experiences, and provide their opinions on the shortcomings of currently available technologies. Besides presenting the study, the paper introduces the first Digital Commensality Data-set, containing videos, facial landmarks, quantitative and qualitative responses. After surveying multimodal data-sets and corpora that we could exploit to understand commensal behavior, we comment on the feasibility of using remote meals as a source to build data-sets to investigate commensal behavior. Finally, we explore possible future research directions emerging from our results. [ABSTRACT FROM AUTHOR]

Ambient Assisted Living (AAL) technologies are being developed which could assist elderly people to live healthy and active lives. These technologies have been used to monitor people’s daily exercises, consumption of calories and sleep patterns, and to provide coaching interventions to foster positive behaviour. Speech and audio processing can be used to complement such AAL technologies to inform interventions for healthy ageing by analyzing speech data captured in the user’s home. However, collection of data in home settings presents challenges. One of the most pressing challenges concerns how to manage privacy and data protection. To address this issue, we proposed a low cost system for recording disguised speech signals which can protect user identity by using pitch shifting. The disguised speech so recorded can then be used for training machine learning models for affective behaviour monitoring. Affective behaviour could provide an indicator of the onset of mental health issues such as depression and cognitive impairment, and help develop clinical tools for automatically detecting and monitoring disease progression. In this article, acoustic features extracted from the non-disguised and disguised speech are evaluated in an affect recognition task using six different machine learning classification methods. The results of transfer learning from non-disguised to disguised speech are also demonstrated. We have identified sets of acoustic features which are not affected by the pitch shifting algorithm and also evaluated them in affect recognition. We found that, while the non-disguised speech signal gives the best Unweighted Average Recall (UAR) of 80.01%, the disguised speech signal only causes a slight degradation of performance, reaching 76.29%. The transfer learning from non-disguised to disguised speech results in a reduction of UAR (65.13%). However, feature selection improves the UAR (68.32%). This approach forms part of a large project which includes health and wellbeing monitoring and coaching.

Eating is a fundamental part of human life and is, more than anything, a social activity. A new field, known as Computational Commensality has been created to computationally address various social aspects of food and eating. This paper illustrates a study on remote dining we conducted online in May 2021. To better understand this phenomenon, known as Digital Commensality, we recorded 11 pairs of friends sharing a meal online through a videoconferencing app. In the videos, participants consume a plate of pasta while chatting with a friend or a family member. After the remote dinner, participants were asked to fill in the Digital Commensality questionnaire, a validated questionnaire assessing the effects of remote commensal experiences, and provide their opinions on the shortcomings of currently available technologies. Besides presenting the study, the paper introduces the first Digital Commensality Data-set, containing videos, facial landmarks, quantitative and qualitative responses. After surveying multimodal data-sets and corpora that we could exploit to understand commensal behavior, we comment on the feasibility of using remote meals as a source to build data-sets to investigate commensal behavior. Finally, we explore possible future research directions emerging from our results.

Alzheimer’s dementia is a progressive neurodegenerative disease that causes cognitive and physical impairment. It severely deteriorates the quality of life in affected individuals. An early diagnosis can assist immensely in better management of their healthcare needs. In recent years, there has been a renewed impetus in development of automated methods for recognition of various disorders by leveraging advancements in artificial intelligence. Here, we propose a multimodal system that can identify linguistic and paralinguistic traits of dementia using an automated screening tool. We show that bag-of-deep-neural-embeddings and ensemble learning offer a viable approach to objective assessment of dementia. The developed system is tested on the Alzheimer’s Dementia Recognition Challenge dataset, where it achieved a new state-of-the-art (SOTA) performance for the classification task and matched the current SOTA for the regression task. These results highlight the efficacy of our proposed system for facilitating an early diagnosis of dementia.

This work shows that fillers - short utterances like “ehm” and “uhm” - allow one to predict whether someone is above median along the Big-Five personality traits. The experiments have been performed over a corpus of 2,988 fillers uttered by 120 different speakers in spontaneous conversations. The results show that the prediction accuracies range between 74 and 82 percent depending on the particular trait. The proposed approach includes a feature selection step - based on Quantum Evolutionary Algorithms - that has been used to detect the personality markers, i.e., the subset of the features that better account for the prediction outcomes and, indirectly, for the personality of the speakers. The results show that only a relatively few features tend to be consistently selected, thus acting as reliable personality markers. [ABSTRACT FROM AUTHOR]

Ambient Assisted Living (AAL) technologies are being developed which could assist elderly people to live healthy and active lives. These technologies have been used to monitor people's daily exercises, consumption of calories and sleep patterns, and to provide coaching interventions to foster positive behaviour. Speech and audio processing can be used to complement such AAL technologies to inform interventions for healthy ageing by analyzing speech data captured in the user's home. However, collection of data in home settings presents challenges. One of the most pressing challenges concerns how to manage privacy and data protection. To address this issue, we proposed a low cost system for recording disguised speech signals which can protect user identity by using pitch shifting. The disguised speech so recorded can then be used for training machine learning models for affective behaviour monitoring. Affective behaviour could provide an indicator of the onset of mental health issues such as depression and cognitive impairment, and help develop clinical tools for automatically detecting and monitoring disease progression. In this article, acoustic features extracted from the non-disguised and disguised speech are evaluated in an affect recognition task using six different machine learning classification methods. The results of transfer learning from non-disguised to disguised speech are also demonstrated. We have identified sets of acoustic features which are not affected by the pitch shifting algorithm and also evaluated them in affect recognition. We found that, while the non-disguised speech signal gives the best Unweighted Average Recall (UAR) of 80.01%, the disguised speech signal only causes a slight degradation of performance, reaching 76.29%. The transfer learning from non-disguised to disguised speech results in a reduction of UAR (65.13%). However, feature selection improves the UAR (68.32%). This approach forms part of a large project which includes health and wellbeing monitoring and coaching. [ABSTRACT FROM AUTHOR]

Abstract Microphone leakage or crosstalk is a common problem in multichannel close-talk audio recordings (e.g., meetings or live music performances), which occurs when a target signal does not only couple into its dedicated microphone, but also in all other microphone channels. For further signal processing such as automatic transcription of a meeting, a multichannel speaker interference reduction is required in order to eliminate the interfering speech signals in the microphone channels. The contribution of this paper is twofold: First, we consider multichannel close-talk recordings of a three-person meeting scenario with various different crosstalk levels. In order to eliminate the crosstalk in the target microphone channel, we extend a multichannel Wiener filter approach, which considers all individual microphone channels. Therefore, we integrate an adaptive filter method, which was originally proposed for acoustic echo cancellation (AEC), in order to obtain a well-performing interferer (noise) component estimation. This results in an improved speech-to-interferer ratio by up to 2.7 dB at constant or even better speech component quality. Second, since an AEC method requires typically clean reference channels, we investigate and report findings why the AEC algorithm is able to successfully estimate the interfering signals and the room impulse responses between the microphones of the interferer and the target speakers even though the reference signals are themselves disturbed by crosstalk in the considered meeting scenario.

This paper presents GAME-ON (Group Analysis of Multimodal Expression of cohesiON), a multimodal dataset specifically designed for studying group cohesion and for explicitly controlling its variation over time. Cohesion is here addressed according to the Severt and Estrada's theoretical multidimensional integrative framework. More specifically, GAME-ON focuses on the social and task dimensions of the instrumental function of cohesion. The dataset consists of over 11 hours of synchronized multimodal recordings (audio, video, and motion capture data) of 17 small groups (3 persons) playing a social game, i.e., an escape game. The game consists of several tasks designed to manipulate the variation of cohesion over time. GAME-ON includes annotations consisting of self-assessment of cohesion and other constructs such as emotions, leadership, and warmth and competence. A first statistical analysis of these annotations shows that we successfully manipulated all the relative variations of cohesion (between tasks) over time. This holds for all tasks except for one where we observed a significant variation of cohesion in the opposite direction than expected. The dataset will be publicly available for research purposes. The motivation of our work is to provide the scientific community with an asset for studying cohesion and other group phenomena.

One of the main and most effective measures to contain the recent viral outbreak is the maintenance of the so-called Social Distancing (SD). To comply with this constraint, governments are adopting restrictions over the minimum inter-personal distance between people. Given this actual scenario, it is crucial to massively measure the compliance to such physical constraint in our life, in order to figure out the reasons of the possible breaks of such distance limitations, and understand if this implies a potential threat. To this end, we introduce the Visual Social Distancing (VSD) problem, defined as the automatic estimation of the inter-personal distance from an image, and the characterization of related people aggregations. VSD is pivotal for a non-invasive analysis to whether people comply with the SD restriction, and to provide statistics about the level of safety of specific areas whenever this constraint is violated. We first point out that measuring VSD is not only a geometrical problem, but it also implies a deeper understanding of the social behaviour in the scene. The aim is to truly detect potentially dangerous situations while avoiding false alarms (e.g., a family with children or relatives, an elder with their caregivers), all of this by complying with current privacy policies. We then discuss how VSD relates with previous literature in Social Signal Processing and indicate a path to research new Computer Vision methods that can possibly provide a solution to such problem. We conclude with future challenges related to the effectiveness of VSD systems, ethical implications and future application scenarios.

A functional head-movement corpus and convolutional neural networks (CNNs) for detecting head-movement functions are presented for analyzing the multiple communicative functions of head movements in multiparty face-to-face conversations. First, focusing on the multifunctionality of head movements, i.e., that a single head movement can simultaneously perform multiple functions, this paper defines 32 non-mutually-exclusive function categories, whose genres are speech production, eliciting and giving feedback, turn management, and cognitive and affect display. To represent and capture arbitrary multifunctional structures, our corpus employs multiple binary codes and logical-sum-based aggregations of multiple coders’ judgments. A corpus analysis targeting four-party Japanese conversations revealed multifunctional patterns in which the speaker modulates multiple functions, such as emphasis and eliciting listeners’ responses, through rhythmic head movements, and listeners express various attitudes and responses through continuous back-channel head movements. This paper proposes CNN-based binary classifiers for detecting each of the functions from the angular velocity of the head pose and the presence or absence of utterances. The experimental results showed that the recognition performance varies greatly, from approximately 30% to 90% in terms of the F-score, depending on the function category, and the performance was positively correlated with the amount of data and inter-coder agreement. In addition, we noted a tendency toward overdetection that added more functions to those originally in the corpus. The analyses and experiments confirm that our approach is promising for studying the multifunctionality of head movements.

In this paper, we discuss the development of artificial theory of mind as foundational to an agent's ability to collaborate with human team members. Agents imbued with artificial social intelligence will require various capabilities to gather the social data needed to inform an artificial theory of mind of their human counterparts. We draw from social signals theorizing and discuss a framework to guide consideration of core features of artificial social intelligence. We discuss how human social intelligence, and the development of theory of mind, can contribute to the development of artificial social intelligence by forming a foundation on which to help agents model, interpret and predict the behaviors and mental states of humans to support human-agent interaction. Artificial social intelligence will need the processing capabilities to perceive, interpret, and generate combinations of social cues to operate within a human-agent team. Artificial Theory of Mind affords a structure by which a socially intelligent agent could be imbued with the ability to model their human counterparts and engage in effective human-agent interaction. Further, modeling Artificial Theory of Mind can be used by an ASI to support transparent communication with humans, improving trust in agents, so that they may better predict future system behavior based on their understanding of and support trust in artificial socially intelligent agents.

The affective motion of humans conveys messages that other humans perceive and understand without conventional linguistic processing. This ability to classify human movement into meaningful gestures or segments plays also a critical role in creating social interaction between humans and robots. In the research presented here, grasping and social gesture recognition by humans and four machine learning techniques (k-Nearest Neighbor, Locality-Sensitive Hashing Forest, Random Forest and Support Vector Machine) is assessed by using human classification data as a reference for evaluating the classification performance of machine learning techniques for thirty hand/arm gestures. The gestures are rated according to the extent of grasping motion on one task and the extent to which the same gestures are perceived as social according to another task. The results indicate that humans clearly rate differently according to the two different tasks. The machine learning techniques provide a similar classification of the actions according to grasping kinematics and social quality. Furthermore, there is a strong association between gesture kinematics and judgments of grasping and the social quality of the hand/arm gestures. Our results support previous research on intention-from-movement understanding that demonstrates the reliance on kinematic information for perceiving the social aspects and intentions in different grasping actions as well as communicative point-light actions.

In face-to-face interaction, speakers establish common ground incrementally, the mutual belief of understanding. Instead of constructing “one-shot” complete utterances, speakers tend to package pieces of information in smaller fragments (what Clark calls “installments”). The aim of this paper was to investigate how speakers' fragmented construction of utterances affect the cognitive load of the conversational partners during utterance production and comprehension. In a collaborative furniture assembly, participants instructed each other how to build an IKEA stool. Pupil diameter was measured as an outcome of effort and cognitive processing in the collaborative task. Pupillometry data and eye-gaze behaviour indicated that more cognitive resources were required by speakers to construct fragmented rather than non-fragmented utterances. Such construction of utterances by audience design was associated with higher cognitive load for speakers. We also found that listeners' cognitive resources were decreased in each new speaker utterance, suggesting that speakers' efforts in the fragmented construction of utterances were successful to resolve ambiguities. The results indicated that speaking in fragments is beneficial for minimising collaboration load, however, adapting to listeners is a demanding task. We discuss implications for future empirical research on the design of task-oriented human-robot interactions, and how assistive social robots may benefit from the production of fragmented instructions.

In face-to-face interaction, speakers establish common ground incrementally, the mutual belief of understanding. Instead of constructing "one-shot" complete utterances, speakers tend to package pieces of information in smaller fragments (what Clark calls "installments"). The aim of this paper was to investigate how speakers' fragmented construction of utterances affect the cognitive load of the conversational partners during utterance production and comprehension. In a collaborative furniture assembly, participants instructed each other how to build an IKEA stool. Pupil diameter was measured as an outcome of effort and cognitive processing in the collaborative task. Pupillometry data and eye-gaze behaviour indicated that more cognitive resources were required by speakers to construct fragmented rather than non-fragmented utterances. Such construction of utterances by audience design was associated with higher cognitive load for speakers. We also found that listeners' cognitive resources were decreased in each new speaker utterance, suggesting that speakers' efforts in the fragmented construction of utterances were successful to resolve ambiguities. The results indicated that speaking in fragments is beneficial for minimising collaboration load, however, adapting to listeners is a demanding task. We discuss implications for future empirical research on the design of task-oriented human-robot interactions, and how assistive social robots may benefit from the production of fragmented instructions. [ABSTRACT FROM AUTHOR]

Introduction: Motherese, or emotional infant directed speech (IDS), is the specific form of speech used by parents to address their infants. The prosody of IDS has affective properties, expresses caregiver involvement, is a marker of caregiver-infant interaction quality. IDS prosodic characteristics can be detected with automatic analysis. We aimed to explore whether pregnant women “speak” to their unborn baby, whether they use motherese while speaking and whether anxio-depressive or obstetrical status impacts speaking to the fetus.Participants and Methods: We conducted an observational study of pregnant women with gestational ages from 26 to 38 weeks. Women were recruited in a university hospital department of obstetrics. Thirty-five women agreed to participate in the study, and 26 audio records were exploitable. We collected obstetrical and sociodemographic data, pregnancy outcomes, anxiety and depressive status using the Covy and Raskin Scales, and life events using the Sensations During Pregnancy and Life Event Questionnaire. Each participant was left alone with an audio recorder with a recommendation to feel free to speak to her fetus as she would have done at home. The recording was stopped after 3 min. Audio recordings were analyzed by two methods: psycholinguist experts' annotation and computational objective automatic analyses.Results: Most mothers (89%) reported speaking to their fetuses. We found a correlation between maternal first perceptions of fetal movements and the start of mother's speaking to fetus. Motherese prosody was detected with both annotation and automatic analysis with a significant correlation between the two methods. In this exploratory study, motherese use was not associated with maternal anxiodepressive or obstetrical status. However, the more future mothers were depressed, the less they spoke with their fetuses during the recording.Conclusion: Fetal directed speech (FDS) can be detected during pregnancy, and it contains a period of prosody that shares the same characteristics of motherese that can be described as prenatal motherese or emotional fetal-directed speech (e-FDS). This means that pregnant women start using motherese much earlier than expected. FDS seems to be correlated with maternal first perceptions of fetal movements and depression scores. However, more research is needed to confirm these exploratory results.

Introduction: Motherese, or emotional infant directed speech (IDS), is the specific form of speech used by parents to address their infants. The prosody of IDS has affective properties, expresses caregiver involvement, is a marker of caregiver-infant interaction quality. IDS prosodic characteristics can be detected with automatic analysis. We aimed to explore whether pregnant women "speak" to their unborn baby, whether they use motherese while speaking and whether anxio-depressive or obstetrical status impacts speaking to the fetus. Participants and Methods: We conducted an observational study of pregnant women with gestational ages from 26 to 38 weeks. Women were recruited in a university hospital department of obstetrics. Thirty-five women agreed to participate in the study, and 26 audio records were exploitable. We collected obstetrical and sociodemographic data, pregnancy outcomes, anxiety and depressive status using the Covy and Raskin Scales, and life events using the Sensations During Pregnancy and Life Event Questionnaire. Each participant was left alone with an audio recorder with a recommendation to feel free to speak to her fetus as she would have done at home. The recording was stopped after 3 min. Audio recordings were analyzed by two methods: psycholinguist experts' annotation and computational objective automatic analyses. Results: Most mothers (89%) reported speaking to their fetuses. We found a correlation between maternal first perceptions of fetal movements and the start of mother's speaking to fetus. Motherese prosody was detected with both annotation and automatic analysis with a significant correlation between the two methods. In this exploratory study, motherese use was not associated with maternal anxiodepressive or obstetrical status. However, the more future mothers were depressed, the less they spoke with their fetuses during the recording. Conclusion: Fetal directed speech (FDS) can be detected during pregnancy, and it contains a period of prosody that shares the same characteristics of motherese that can be described as prenatal motherese or emotional fetal-directed speech (e-FDS). This means that pregnant women start using motherese much earlier than expected. FDS seems to be correlated with maternal first perceptions of fetal movements and depression scores. However, more research is needed to confirm these exploratory results. [ABSTRACT FROM AUTHOR]

Public speaking is an important skill, the acquisition of which requires dedicated and time consuming training. In recent years, researchers have started to investigate automatic methods to support public speaking skills training. These methods include assessment of the trainee's oral presentation delivery skills which may be accomplished through automatic understanding and processing of social and behavioral cues displayed by the presenter. In this study, we propose an automatic scoring system for presentation delivery skills using a novel active data representation method to automatically rate segments of a full video presentation. While most approaches have employed a two step strategy consisting of detecting multiple events followed by classification, which involve the annotation of data for building the different event detectors and generating a data representation based on their output for classification, our method does not require event detectors. The proposed data representation is generated unsupervised using low-level audiovisual descriptors and self-organizing mapping and used for video classification. This representation is also used to analyse video segments within a full video presentation in terms of several characteristics of the presenter's performance. The audio representation provides the best prediction results for self-confidence and enthusiasm, posture and body language, structure and connection of ideas, and overall presentation delivery. The video data representation provides the best results for presentation of relevant information with good pronunciation, usage of language according to audience, and maintenance of adequate voice volume for the audience. The fusion of audio and video data provides the best results for eye contact. Applications of the method to provision of feedback to teachers and trainees are discussed.