Search

Studies in human–human interaction have introduced the concept of F-formation to describe the spatial arrangement of participants during social interactions. This paper has two objectives. It aims at detecting F-formations in video sequences and at predicting the next speaker in a group conversation. The proposed approach exploits time information and multimodal signals of humans in video sequences. In particular, we rely on measuring the engagement level of people as a feature of group belonging. Our approach makes use of a recursive neural network, the Long Short Term Memory (LSTM), to predict who will take the speaker's turn in a conversation group. Experiments on the MatchNMingle dataset led to 85% true positives in group detection and 98% accuracy in predicting the next speaker. • New method for analyzing videos of groups of persons in interaction. • New method for detecting groups (F-formations) using temporal information and multimodal signals. • Consider engagement level between persons as a feature of group belonging. • New method for predicting the next speaker based on LSTM. • Experiments on the MatchNMingle dataset: 85% true positives in group detection, 98% accuracy in predicting next speaker. [ABSTRACT FROM AUTHOR]

Analysing the dynamics in social interactions in indoor spaces entails evaluating spatial–temporal variables from the event, such as location and time. Additionally, social interactions include invisible spaces that we unconsciously acknowledge due to social constraints, e.g., space between people having a conversation with each other. Nevertheless, current sensor arrays focus on detecting the physically occupied spaces from social interactions, i.e., areas inhabited by physically measurable objects. Our goal is to detect the socially occupied spaces, i.e., spaces not physically occupied by subjects and objects but inhabited by the interaction they sustain. We evaluate the social representation of the space structure between two or more active participants, so-called F-Formation for small gatherings. We propose calculating body orientation and location from skeleton joint data sets by integrating depth cameras. The body orientation is derived by integrating the shoulders and spine joint data with head/face rotation data and spatial–temporal information from trajectories. From the physically occupied measurements, we can detect socially occupied spaces. In our user study implementing the system, we compared the capabilities and skeleton tracking datasets from three depth camera sensors, the Kinect v2, Azure Kinect, and Zed 2i. We collected 32 walking patterns for individual and dyad configurations and evaluated the system's accuracy regarding the intended and socially accepted orientations. Experimental results show accuracy above 90% for the Kinect v2, 96% for the Azure Kinect, and 89% for the Zed 2i for assessing socially relevant body orientation. Our algorithm contributes to the anonymous and automated assessment of socially occupied spaces. The depth sensor system is promising in detecting more complex social structures. These findings impact research areas that study group interactions within complex indoor settings. [ABSTRACT FROM AUTHOR]

Analysing the dynamics in social interactions in indoor spaces entails evaluating spatial–temporal variables from the event, such as location and time. Additionally, social interactions include invisible spaces that we unconsciously acknowledge due to social constraints, e.g., space between people having a conversation with each other. Nevertheless, current sensor arrays focus on detecting the physically occupied spaces from social interactions, i.e., areas inhabited by physically measurable objects. Our goal is to detect the socially occupied spaces, i.e., spaces not physically occupied by subjects and objects but inhabited by the interaction they sustain. We evaluate the social representation of the space structure between two or more active participants, so-called F-Formation for small gatherings. We propose calculating body orientation and location from skeleton joint data sets by integrating depth cameras. The body orientation is derived by integrating the shoulders and spine joint data with head/face rotation data and spatial–temporal information from trajectories. From the physically occupied measurements, we can detect socially occupied spaces. In our user study implementing the system, we compared the capabilities and skeleton tracking datasets from three depth camera sensors, the Kinect v2, Azure Kinect, and Zed 2i. We collected 32 walking patterns for individual and dyad configurations and evaluated the system’s accuracy regarding the intended and socially accepted orientations. Experimental results show accuracy above 90% for the Kinect v2, 96% for the Azure Kinect, and 89% for the Zed 2i for assessing socially relevant body orientation. Our algorithm contributes to the anonymous and automated assessment of socially occupied spaces. The depth sensor system is promising in detecting more complex social structures. These findings impact research areas that study group interactions within complex indoor settings.

We present MatchNMingle, a novel multimodal/multisensor dataset for the analysis of free-standing conversational groups and speed-dates in-the-wild. MatchNMingle leverages the use of wearable devices and overhead cameras to record social interactions of 92 people during real-life speed-dates, followed by a cocktail party. To our knowledge, MatchNMingle has the largest number of participants, longest recording time and largest set of manual annotations for social actions available in this context in a real-life scenario. It consists of 2 hours of data from wearable acceleration, binary proximity, video, audio, personality surveys, frontal pictures and speed-date responses. Participants’ positions and group formations were manually annotated; as were social actions (eg. speaking, hand gesture) for 30 minutes at 20 FPS making it the first dataset to incorporate the annotation of such cues in this context. We present an empirical analysis of the performance of crowdsourcing workers against trained annotators in simple and complex annotation tasks, founding that although efficient for simple tasks, using crowdsourcing workers for more complex tasks like social action annotation led to additional overhead and poor inter-annotator agreement compared to trained annotators (differences up to 0.4 in Fleiss’ Kappa coefficients). We also provide example experiments of how MatchNMingle can be used. [ABSTRACT FROM AUTHOR]

Face-to-face interaction is a primordial site for human activity and intersubjectivity. Empirical studies have shown how people reflexively exhibit a face orientation and work to establish a formation in which everyone is facing each other in local participation frameworks. The Face has also been described by, e.g., Levinas as the basis for a first ethical philosophy. Humans have established these Face-formations when interacting since time immemorial, but what happens when one of the participants is present through a telepresence robot? Based on ethnomethodology, Peircean/Goodwinian semiotics, multimodal conversation analysis and video data from a Danish residential rehabilitation center, the article shows the ways in which participants manage to interactively, cooperatively, and moment by moment achieve an F-formation in situ. The article contributes a detailed analysis and discussion of the kind of participant a telepresence robot is, in and through situated interactions: I propose that we term this participant the RoboDoc, given that it is an assemblage of a doctor who controls a robot. By focusing on the affordances of mobility, the article contributes to a renewed understanding of the importance and relevance of establishing Face-orientations in an increasingly technofied telepresence world. [ABSTRACT FROM AUTHOR]

This article studies how co-present workers can join in a co-present interaction they were not previously involved in, thus challenging initial participants' interactional preserve. It is based on ethnographically-informed analyses of video-recorded interactions in workplaces, in English as a lingua franca and in French. Potential joiners' recurrent embodied and verbal practices are identified and analyzed, showing regular methods associated with potential joiners' position relative to the F-formation, and different layouts typical of workplaces. Another set of findings bears on how potential joiners shape their move so as to account for joining in at that moment, to project a more or less extended participation, and to implement a collaborative project. Beyond the variety of projects the practice can serve, potential joiners' moves are systematically designed so as to demonstrate their contribution to the progression of work. [ABSTRACT FROM AUTHOR]

Guided tours are a perspicuous setting for the study of mobile formations. Guided visits are characterized by mobile phases in which the group moves forward, alternating with moments in which participants adopt a more stationary, object-focused positioning. In this article, we pay attention to specific ways of walking from one point to another as a mobile formation: We focus on mobile reorientations of the group changing the initially projected trajectory. This particular movement allows us to observe key features of mobile formations: how they are initiated, by whom, with which resources. We sketch a systematic study of multimodal practices through which various kinds of participants initiate a reorientation of the group, with a particular focus on the category of the participants initiating the reorientation (the “guide” vs. the “guided”), on the action they achieve at the beginning of a sequence in order to do so (questions, noticings, comments), and on the multimodal resources they use. [ABSTRACT FROM AUTHOR]

Objective: The present study aims to investigate how facial kinemorphs influence F-formation shapes occurred in a virtual environment social interaction setting. Methods: The problem statement relies on two aspects of this investigation, more precisely, two comparative analysis. The first comparison is between the effect of the gaze and the lip movement on F-formation shapes. The importance of this aspect in HRI studies depends on the reason being that it is not clear whether the shape of a social group's spatial arrangement is affected by the kinemorphs, while it is known that kinemorphs have an influence on the proximity between humans and robots. Through the comparative analysis between the gaze and the lip movement, the effect of the kinemorphs on F-formation shapes can be clearly observed. The second comparison is between the interactions of human participants with robots in virtual reality and the interaction between humans in physical settings. It is known that human behavior has different patterns when it is towards a human and when it is towards a robot. Therefore, F-formation, as a behavior of the decision on location and rotation, might also have a difference in forming interactions with humans and robots. This difference is revealed by investigating the role of F-formation shapes on joining dyadic and triadic interaction scenarios. Results: The results evince systematic findings that address likely postures in dyadic and triadic interaction settings for the robot agents in a virtual environment. Conclusion: Both kinemorphs (gaze and lip movement) seem to determine the focus of the participants in a social interaction with robots; however, the effect of gaze is higher than lip movement. H and V formations are the most frequently formed shapes in virtual triadic and dyadic interactions. I formation does not even have a single instance neither in triadic nor in dyadic interactions. [ABSTRACT FROM AUTHOR]