Search

Human activity recognition using wearable and mobile devices is used for decades to monitor humans’ daily behaviours. In recent years as smartphones being widely integrated into our daily lives, the use of smartphone’s built-in sensors in human activity recognition has been receiving more attention, in which smartphone accelerometer plays the main role. However, in comparison to the standard machine, when developing human activity recognition using a smartphone, the limitations such as processing capability and energy consumption should be taken into consideration, and therefore, a trade-off between performance and computational complexity should be considered. In this paper, we shed light on the importance of feature selection and its impact on simplifying the activity classification process, which enhances the computational complexity of the system. The novelty of this work is related to identifying the most efficient features for the detection of each individual activity uniquely. In an experimental study with human users and using different smartphones, we investigated how to achieve an optimal feature set, using which the system complexity can be decreased while the activity recognition accuracy remains high. For that, in the considered scenario, we instructed the participants to perform different activities, including static, dynamic, going up and down the stairs, and walking fast and slow while freely holding a smartphone in their hands. To evaluate the obtained optimal feature set implementing two major classification algorithms, the decision tree and the Bayesian network, we investigated activity recognition accuracy for different activities. We further evaluated the optimal feature set by comparing the performance of the activity recognition system using the optimal feature set and three feature sets taken from the state-of-the-art. The experimental results demonstrated that replacing a large number of conventional features with an optimal feature set has only a negligible impact on the overall activity recognition system performance while it can significantly decrease the system’s complexity, which is essential for smartphone-based systems.

In this work, we tested a recently developed novel methodology to assist children with Autism Spectrum Disorder (ASD) improve their Visual Perspective Taking (VPT) and Theory of Mind (ToM) skills using the humanoid robot Kaspar. VPT is the ability to see the world from another person’s perspective, drawing upon both social and spatial information. Children with ASD often find it difficult to understand that others might have perspectives, viewpoints and beliefs that are different from their own, which is a fundamental aspect of both VPT and ToM. The games we designed were implemented as the first attempt to study if these skills can be improved in children with ASD through interacting with a humanoid robot in a series of trials. The games involved a number of different actions with the common goal of helping the children to see the world from the robot’s perspective. Children with ASD were recruited to the study according to specific inclusion criteria that were determined in a previous pilot study. In order to measure the potential impact of the games on the children, three pre- and post-tests (Smarties, Sally–Anne and Charlie tests) were conducted with the children. Our findings suggest that children with ASD can indeed benefit from this approach of robot-assisted therapy.

Nowadays, smartphones are gradually being integrated in our daily lives, and they can be considered powerful tools for monitoring human activities. However, due to the limitations of processing capability and energy consumption of smartphones compared to standard machines, a trade-off between performance and computational complexity must be considered when developing smartphone-based systems. In this paper, we shed light on the importance of feature selection and its impact on simplifying the activity classification process which enhances the computational complexity of the system. Through an in-depth survey on the features that are widely used in state-of-the-art studies, we selected the most common features for sensor-based activity classification, namely conventional features. Then, in an experimental study with 10 participants and using 2 different smartphones, we investigated how to reduce system complexity while maintaining classification performance by replacing the conventional feature set with an optimal set. For this reason, in the considered scenario, the users were instructed to perform different static and dynamic activities, while freely holding a smartphone in their hands. In our comparison to the state-of-the-art approaches, we implemented and evaluated major classification algorithms, including the decision tree and Bayesian network. We demonstrated that replacing the conventional feature set with an optimal set can significantly reduce the complexity of the activity recognition system with only a negligible impact on the overall system performance.

This paper presents a contribution aiming at testing novel child–robot teaching schemes that could be used in future studies to support the development of social and collaborative skills of children with autism spectrum disorders (ASD). We present a novel experiment where the classical roles are reversed: in this scenario the children are the teachers providing positive or negative reinforcement to the Kaspar robot in order for it to learn arbitrary associations between different toy names and the locations where they are positioned. The objective is to stimulate interaction and collaboration between children while teaching the robot, and also provide them tangible examples to understand that sometimes learning requires several repetitions. To facilitate this game, we developed a reinforcement learning algorithm enabling Kaspar to verbally convey its level of uncertainty during the learning process, so as to better inform the children about the reasons behind its successes and failures. Overall, 30 typically developing (TD) children aged between 7 and 8 (19 girls, 11 boys) and 9 children with ASD performed 25 sessions (16 for TD; 9 for ASD) of the experiment in groups, and managed to teach Kaspar all associations in 2 to 7 trials. During the course of study Kaspar only made rare unexpected associations (2 perseverative errors and 2 win-shifts, within a total of 314 trials), primarily due to exploratory choices, and eventually reached minimal uncertainty. Thus, the robot’s behaviour was clear and consistent for the children, who all expressed enthusiasm in the experiment.

We used a new method called “Ghost-in-the-Machine” (GiM) to investigate social interactions with a robotic bartender taking orders for drinks and serving them. Using the GiM paradigm allowed us to identify how human participants recognize the intentions of customers on the basis of the output of the robotic recognizers. Specifically, we measured which recognizer modalities (e.g., speech, the distance to the bar) were relevant at different stages of the interaction. This provided insights into human social behavior necessary for the development of socially competent robots. When initiating the drink-order interaction, the most important recognizers were those based on computer vision. When drink orders were being placed, however, the most important information source was the speech recognition. Interestingly, the participants used only a subset of the available information, focussing only on a few relevant recognizers while ignoring others. This reduced the risk of acting on erroneous sensor data and enabled them to complete service interactions more swiftly than a robot using all available sensor data. We also investigated socially appropriate response strategies. In their responses, the participants preferred to use the same modality as the customer’s requests, e.g., they tended to respond verbally to verbal requests. Also, they added redundancy to their responses, for instance by using echo questions. We argue that incorporating the social strategies discovered with the GiM paradigm in multimodal grammars of human–robot interactions improves the robustness and the ease-of-use of these interactions, and therefore provides a smoother user experience.

Human gaze and blinking behaviours have been recently considered, to empower humanlike robots to convey a realistic behaviour in a social human-robot interaction. This paper reports the findings of our investigation on human eye-blinking behaviour in relation to human gaze behaviour, in a human-human interaction. These findings then can be used to design a humanlike eye-blinking model for a social humanlike robot. In an experimental eye-tracking study, we showed to 11 participants, a 7-minute video of social interactions of two people, and collected their eye-blinking and gaze behaviours with an eye-tracker. Analysing the collected data, we measured information such as participants’ blinking rate, maximum and minimum blinking duration, number of frequent (multiple) blinking, as well as the participants’ gaze directions on environment. The results revealed that participants’ blinking rate in a social interaction are qualitatively correlated to the gaze behaviour, as higher number of gaze shift increased the blinking rate. Based on the findings of this study, we can propose a context-dependent blinking model as an important component of the robot’s gaze control system that can empower our robot to mimic human blinking behaviour in a multiparty social interaction.

Sensing and interpreting the interlocutor’s social behaviours is a core challenge in the development of social robots. Social robots require both an innovative sensory apparatus able to perceive the “social and emotional world” in which they act and a cognitive system able to manage this incoming sensory information and plan an organized and pondered response. In order to allow scientists to design cognitive models for this new generation of social machines, it is necessary to develop control architectures that can be easily used also by researchers without technical skills of programming such as psychologists and neuroscientists. In this work an innovative hybrid deliberative/reactive cognitive architecture for controlling a social humanoid robot is presented. Design and implementation of the overall architecture take inspiration from the human nervous system. In particular, the cognitive system is based on the Damasio’s thesis. The architecture has been preliminary tested with the FACE robot. A social behaviour has been modeled to make FACE able to properly follow a human subject during a basic social interaction task and perform facial expressions as a reaction to the social context.

Two perspectives define a human being in his social sphere: appearance and behaviour. The aesthetic aspect is the first significant element that impacts a communication while the behavioural aspect is a crucial factor in evaluating the ongoing interaction. In particular, we have more expectations when interacting with anthropomorphic robots and we tend to define them believable if they respect human social conventions. Therefore researchers are focused both on increasingly anthropomorphizing the embodiment of the robots and on giving the robots a realistic behaviour. This paper describes our research on making a humanoid robot socially interacting with human beings in a believable way.

Autism Spectrum Disorder (ASD) is a neural development disorder characterized by specific patterns of behavioral and social difficulties. Beyond these core symptoms, additional problems such as absence of gender differences identification, interactional distortions of environmental and family responses are often present. Taking into account these emotional and behavioral problems researchers and clinicians are focusing on the design of innovative therapeutic approaches aimed to improve social capabilities of subjects with ASD. Thanks to the technological and scientific progresses of the last years, nowadays it is possible to create human-like robots with social and emotional capabilities. Furthermore it is also possible to analyze physiological signals inferring subjects' psycho-physiological state which can be compared with a behavioral analysis in order to obtain a deeper understanding of subjects reactions to treatments. In this work a preliminary evaluation of an innovative social robot-based treatment for subjects with ASD is described. The treatment consists in a complex stimulation and acquisition platform composed of a social robot, a multi-parametric acquisition system and a therapeutic protocol. During the preliminary tests of the treatment the subject's physiological signals and behavioral parameters have been recorded and used together with the therapists' annotations to infer the subjects' induced reactions. Physiological signals were analyzed and statistically evaluated demonstrating the possibility to correctly discern the two groups (ASD and normally developing subjects) with a classification percentage higher than 92%. Statistical analysis also highlighted the treatment capability to induce different affective states in subjects with ASDs more than in control subjects, demonstrating that the treatment is well designed and tuned on ASDs deficits and behavioral lacks.

Visual Perspective Taking (VPT) is the ability to see the world from another person’s perspective, taking into account what they see and how they see it, drawing upon both spatial and social information. Children with autism often find it difficult to understand that other people might have perspectives, viewpoints, beliefs and knowledge that are different from their own which is a fundamental aspect VPT. In this paper, we present the piloting of scenarios for our first large scale pilot-study using a humanoid robot to assist children with autism develop their VPT skills. The games were implemented with the Kaspar robot and to our knowledge this is the first attempt to improve the VPT skills of children with autism through playing and interacting with a humanoid robot.

This paper presents a contribution to the active field of robotics research to support the development of social skills and capabilities in children with Autism Spectrum Disorders as well as Typically Developing children. We present preliminary results of a novel experiment where classical roles are reversed: children are here the teachers giving positive or negative reinforcement to the Kaspar robot to make it learn arbitrary associations between toys and locations where to tidy them. The goal is to help children change perspective, and understand that sometimes a learning agent needs several repetitions before correctly learning something. We developed a reinforcement learning algorithm enabling Kaspar to verbally convey its uncertainty along learning, so as to better inform the interacting child of the reasons behind successes and failures made by the robot. Overall, 30 children aged between 7 and 8 (19 girls, 11 boys) performed 16 sessions of the experiment in groups, and managed to teach Kaspar all associations in 2 to 7 trials. Kaspar only made a few unexpected associations, mostly due to exploratory choices, and eventually reached minimal uncertainty. All children expressed enthusiasm in the experiment.