Search

Collaborative robotic manipulators can stop in case of a collision, according to the ISO/TS 15066 and ISO 10218-1 standards. However, in a human-robot collaboration scenario where the robot and the human share the workspace, a better solution with concerns to production and operator safety would be to perceive in advance the presence of an obstacle and be able to avoid it, thereby completing the task without halting the robot. In this paper, an obstacle avoidance algorithm is tested using a sensor system for real-time human detection; the operator represents a potential dynamic obstacle that can interfere with the robot motion. The sensor system consists of three RGB-D cameras. A custom software framework has been developed in Python exploiting machine learning tools for human skeleton detection. The coordinates of the human body joints relative to the manipulator base are used as input to the obstacle avoidance algorithm. The use of multiple sensors makes it possible to limit the occlusion problem; in addition, the choice of non-wearable sensors goes in the direction of better operator comfort. A series of experimental tests were performed to verify the accuracy of skeleton detection and the ability of the system to avoid obstacles in real time. The human motion caption system, in particular, was validated through a comparison with a commercial system based on wearable IMU sensors widely used and validated in motion capture. There is an NRMSE of 3.23 % for the RGB-D camera-based skeleton detection system, against an NRMSE of 12.23 % for the IMU wearable sensor system. Test results confirm that the system is able to avoid collisions with the human body under various conditions, static or dynamic, ensuring a minimum safety distance to any part of the manipulator. In 44 tests in which the operator moves around the robot with possible collisions (at a speed typical of manufacturing operations), the minimum operator-robot distance averaged 208 mm , being 200 mm the limit safety distance set by algorithm. [ABSTRACT FROM AUTHOR]

Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration. [ABSTRACT FROM AUTHOR]

Reminding is often identified as a central function of socially assistive robots in the healthcare sector. The robotic reminders are supposed to help people with memory impairments to remember to take their medicine, to drink and eat, or to attend appointments. Such standalone reminding technologies can, however, be too demanding for people with memory injuries. In a co-creation process, we developed an individual reminder robot together with a person with traumatic brain injury and her care personnel. During this process, we learned that while current research describe reminding as a prototypical task for socially assistive robots, there is no clear definition of what constitutes a reminder nor that it is based on complex sequences of interactions that evolve over time and space, across different actions, actors and technologies. Based on our data from the co-creation process and the first deployment, we argue for a shift towards a sequential and socially distributed character of reminding. Understanding socially assistive robots as rehabilitative tools for people with memory impairment, they need to be reconsidered as interconnected elements in institutional care practices instead of isolated events for the remindee. [ABSTRACT FROM AUTHOR]

Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.

3D Skeleton-based human motion prediction, a classic task in computer vision, aims to forecast subsequent motions based on historical motion observations. In particular, precise short-term motion prediction is crucial for the effectiveness of machines designed for real-time human-computer interaction. This study aims to achieve accurate predictions of human motion within timeframes of less than 400 milliseconds, with the goal of improving machine responsiveness and efficiency. Previous research has predominantly relied on sequence models like Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM) or Transformers. Transformer-based methods have been relied on temporal forecasting ability of Transformer. In this study, instead, we introduce an alternative perspective for Transformer-based models to comprehend the structure of skeletons, named Joint-Aware Transformer (JAT). Within our model, the attention mechanism is employed to encode inter-joint correlation instead of temporal dependencies. Our approach outperformed the state-of-the-art (SOTA) model in short-term prediction on three types of human motion datasets.

Distinguishable vibrotactile feedback can convey information non-verbally and complete the sensory loop when using assistive devices. Feedback can increase acceptance of assistive devices but could require personalization as these devices need to adapt to user capabilities and preferences that may affect the location for inducing feedback. Developing personalized feedback for each user may be ideal, but impractical if demand for these devices increases. In this research, we evaluate the hypothesis that the ability to define, generate, and use personalized feedback is preferred and should be provided. The hypothesis is evaluated using a system capable of capturing and recognizing non-verbal inputs, providing vibrotactile feedback to verify the identified action, and executing the verified action. Evaluation focuses on determining the ability to initiate actions using the system architecture, and define, generate, and use personalized feedback. In addition to recording performance metrics (success or failure), the participants also completed the NASA TLX and post-evaluation questionnaires. Based on results, the actions can be initiated using the system architecture, and defining, generating, and using personalized feedback was found to be preferable with the success rate reaching 100% after only three successive repetitions, and could decrease mental demand and effort while interacting with the system.

Introduction: Underlying mechanisms of speech perception masked by background speakers, a common daily listening condition, are often investigated using various and lengthy psychophysical tests. The presence of a social agent, such as an interactive humanoid NAO robot, may help maintain engagement and attention. However, such robots potentially have limited sound quality or processing speed. Methods: As a first step toward the use of NAO in psychophysical testing of speech-in-speech perception, we compared normal-hearing young adults' performance when using the standard computer interface to that when using a NAO robot to introduce the test and present all corresponding stimuli. Target sentences were presented with colour and number keywords in the presence of competing masker speech at varying target-to-masker ratios. Sentences were produced by the same speaker, but voice differences between the target and masker were introduced using speech synthesis methods. To assess test performance, speech intelligibility and data collection duration were compared between the computer and NAO setups. Human-robot interaction was assessed using the Negative Attitude Toward Robot Scale (NARS) and quantification of behavioural cues (backchannels). Results: Speech intelligibility results showed functional similarity between the computer and NAO setups. Data collection durations were longer when using NAO. NARS results showed participants had a relatively positive attitude toward "situations of interactions" with robots prior to the experiment, but otherwise showed neutral attitudes toward the "social influence" of and "emotions in interaction" with robots. The presence of more positive backchannels when using NAO suggest higher engagement with the robot in comparison to the computer. Discussion: Overall, the study presents the potential of the NAO for presenting speech materials and collecting psychophysical measurements for speech-inspeech perception. [ABSTRACT FROM AUTHOR]

The mindBot robot is a new educational robot toy that can be controlled by brain signals and voice commands. It was evaluated with children with disabilities as well as healthy children as the potential users. The most significant challenge was the size of the used Emotiv Insight electroencephalogram headset when adjusting it on the children's' heads. Despite all the challenges, the mindBot robot is a promising technology that could be fun and educational for disabled children. The 11 participants took 36 minutes to finish all tasks on average. This includes the time they spent setting up the robot for the first time, putting on the headset, learning how to use the robot, and using the main educational features. The System Usability Scale usability score for the robot is 71.13, which is considered to be the score of good. The future stages of improving the mindBot includes adding more mobility capabilities and adding the feature of educational assessment. [ABSTRACT FROM AUTHOR]

Reminding is often identified as a central function of socially assistive robots in the healthcare sector. The robotic reminders are supposed to help people with memory impairments to remember to take their medicine, to drink and eat, or to attend appointments. Such standalone reminding technologies can, however, be too demanding for people with memory injuries. In a co-creation process, we developed an individual reminder robot together with a person with traumatic brain injury and her care personnel. During this process, we learned that while current research describe reminding as a prototypical task for socially assistive robots, there is no clear definition of what constitutes a reminder nor that it is based on complex sequences of interactions that evolve over time and space, across different actions, actors and technologies. Based on our data from the co-creation process and the first deployment, we argue for a shift towards a sequential and socially distributed character of reminding. Understanding socially assistive robots as rehabilitative tools for people with memory impairment, they need to be reconsidered as interconnected elements in institutional care practices instead of isolated events for the remindee.

IntroductionUnderlying mechanisms of speech perception masked by background speakers, a common daily listening condition, are often investigated using various and lengthy psychophysical tests. The presence of a social agent, such as an interactive humanoid NAO robot, may help maintain engagement and attention. However, such robots potentially have limited sound quality or processing speed.MethodsAs a first step toward the use of NAO in psychophysical testing of speech- in-speech perception, we compared normal-hearing young adults’ performance when using the standard computer interface to that when using a NAO robot to introduce the test and present all corresponding stimuli. Target sentences were presented with colour and number keywords in the presence of competing masker speech at varying target-to-masker ratios. Sentences were produced by the same speaker, but voice differences between the target and masker were introduced using speech synthesis methods. To assess test performance, speech intelligibility and data collection duration were compared between the computer and NAO setups. Human-robot interaction was assessed using the Negative Attitude Toward Robot Scale (NARS) and quantification of behavioural cues (backchannels).ResultsSpeech intelligibility results showed functional similarity between the computer and NAO setups. Data collection durations were longer when using NAO. NARS results showed participants had a relatively positive attitude toward “situations of interactions” with robots prior to the experiment, but otherwise showed neutral attitudes toward the “social influence” of and “emotions in interaction” with robots. The presence of more positive backchannels when using NAO suggest higher engagement with the robot in comparison to the computer.DiscussionOverall, the study presents the potential of the NAO for presenting speech materials and collecting psychophysical measurements for speech-in-speech perception.

This systematic literature review presents the latest advancements and insights about digital twin technology and robotics interfaced with extended reality in the context of Industry 4.0. As the extended reality technologies emerge, it results in an increasing overlap between digital twins and human-robot interactions in industrial settings, promoting collaboration between operators and cobots in manufacturing environments. The objective of this study is to serve as a valuable resource for researchers and practitioners working in the field of Industry 4.0. It aims to highlight the latest developments and innovations in the application of digital twins and robotics interfaced with extended reality technologies in manufacturing. By extracting data from relevant articles, it provides a comprehensive understanding of the current state-of-the-art in this field by: analyzing the favored extended reality interfaces for digital twin and robotics interactions; analyzing the digital twin and physical twin interaction; evaluating the digital twin application levels and pillars through extended reality interfacing; and introducing a new concept called augmented perception for creating new physical-digital interactions. [ABSTRACT FROM AUTHOR]

Abstract Nuclear facilities have a regulatory requirement to measure radiation levels within Post Operational Clean Out (POCO) around nuclear facilities each year, resulting in a trend towards robotic deployments to gain an improved understanding during nuclear decommissioning phases. The UK Nuclear Decommissioning Authority supports the view that human‐in‐the‐loop (HITL) robotic deployments are a solution to improve procedures and reduce risks within radiation characterisation of nuclear sites. The authors present a novel implementation of a Cyber‐Physical System (CPS) deployed in an analogue nuclear environment, comprised of a multi‐robot (MR) team coordinated by a HITL operator through a digital twin interface. The development of the CPS created efficient partnerships across systems including robots, digital systems and human. This was presented as a multi‐staged mission within an inspection scenario for the heterogeneous Symbiotic Multi‐Robot Fleet (SMuRF). Symbiotic interactions were achieved across the SMuRF where robots utilised automated collaborative governance to work together, where a single robot would face challenges in full characterisation of radiation. Key contributions include the demonstration of symbiotic autonomy and query‐based learning of an autonomous mission supporting scalable autonomy and autonomy as a service. The coordination of the CPS was a success and displayed further challenges and improvements related to future MR fleets.

Robotics is a cross-disciplinary branch of science and technology, and lays foundation on mechanics, control, computer science, artificial intelligence, and so on. With the developments of both softwares and hardwares, especially in the artificial intelligence technologies, robots have been widely applied in multiple areas in the society, and become more and more interactive in our daily life, such as the service robots in the museums, shopping malls, restaurants, etc. Though the ultimate goal for a service robot to behave like a human is not easy to be achieved, significant processes have been made during the past decades. Considering that it is universal that service robots are triggered to execute tasks specified by human users via commands (Comm-TET), and it is essential to process and understand human users' commands correctly, we comprehensively overview the developments of the task understanding (TU) sub-process of Comm-TET for service robots. In order to organize the related literature in a reasonable manner, we abstracted the pipeline of Comm-TET and the generic framework of TU based on the existing researches. Following the abstracted framework, we present in-depth discussions on each of its building blocks over the past decades, and give some insights on the future research directions. Compared to other reviews on TU, this review emphasizes more on the technical developments and organizes the existing researches as an integrality. [ABSTRACT FROM AUTHOR]

The uncanny valley hypothesis states that users might experience eerie when interacting with almost but not fully human-like artificial characters. The advancements in artificial intelligence, robotics, and computer graphics have led to life-like virtual humans and humanoid robots. It is necessary to revisit the hypothesis to check if they positively or negatively affect the current population, who are much more accustomed to the latest technologies. In this paper, we study and present a unique evaluation of the uncanny valley hypothesis by allowing participants to interact live with four different humanoid robots (of varying levels of humanlikeness). To evaluate the affinity of each robot, each participant needs to fill a survey questionnaire. Apart from this, we also use deep learning methods to quantify the participants’ emotional states using multi-modalcues, including visual, audio, and text, by recording the participant-robot interaction. The multi-modal analysis and surveys provide interesting results and insights into the uncanny valley hypothesis.

Abstract Speech interaction is an important means of robot interaction. With the rapid development of deep learning, end‐to‐end speech synthesis methods based on this technique have gradually become mainstream. Chinese deep learning‐based speech synthesis techniques suffer from problems such as unstable synthesised speech, poor naturalness and poor personalised speech synthesis, which do not satisfy some practical application scenarios. Hence, an F‐MelGAN model is adopted to improve the performance of Chinese speech synthesis. A post‐processing network is used to refine the Mel‐spectrum predicted by the decoder and alleviate the Mel‐spectrum distortion phenomenon. A phoneme‐level and sentence‐level combined module is proposed to model the personalised style of speakers. A combination of an acoustic conditioning network, speaker encoder network GCNet and feedback‐constrained training is proposed to solve the problem of poor personalised speech synthesis and achieve personalised speech customisation in Chinese. Experimental results show that the whole model can generate high‐quality speech with high speaker similarity for both speakers that appear in the training process and speakers that never appear in the training process.

In robot-assisted surgeries, the surgeon focuses on the surgical tool and its pose, and not on the complete robot’s shape. However, the joints of redundant robots (robots that have more degrees of freedom (DoF) than needed for the positioning of surgical tools) might move in unexpected/undesired ways. Joint motions that lead to patient or collisions are safety critical. We assume that the medical personnel in the operating room can best decide if a planned robot motions come too close to the patient or not. Therefore, we propose an augmented reality-based solution to interact with the robot during surgery planning, and intervention. The tool can be used to command a robot by drawing a trajectory in augmented reality (AR), visualizing the robot movement to check if it is safe before execution. The proposed solution allows surgeons to plan safe robot motion paths before-hand and adapt them when necessary in situ. As a proof-of-concept, we implemented and demonstrated the proposed solution on a 7-DoF redundant robot by commanding different trajectories. The control architecture to plan and execute motion for a surgical robot using AR is a key result of this work.

In this research, a new remote rehabilitation system was developed that integrates an IoT-based connected robot intended for wrist and forearm rehabilitation. In fact, the mathematical model of the wrist and forearm joints was developed and integrated into the main controller. The proposed new rehabilitation protocol consists of three main sessions: the first is dedicated to the extraction of the passive components of the dynamic model of wrist–forearm biomechanics while the active components are extracted in the second session. The third session consists of performing continuous exercises using the determined dynamic model of the forearm–wrist joints, taking into account the torque generated by muscle fatigue. The main objective of this protocol is to determine the state level of the affected wrist and above all to provide a dynamic model in which the torque generated by the robot and the torque supplied by the patient are combined, taking into account the constraints of fatigue. A Support Vector Machine (SVM) classifier is designed for the estimation of muscle fatigue based on the features extracted from the electromyography (EMG) signal acquired from the patient. The results show that the developed rehabilitation system allows a good progression of the joint's range of motion as well as the resistive-active torques. [ABSTRACT FROM AUTHOR]

Recognising intent in collaborative human robot tasks can improve team performance and human perception of robots. Intent can differ from the observed outcome in the presence of mistakes which are likely in physically dynamic tasks. We created a dataset of 1227 throws of a ball at a target from 10 participants and observed that 47% of throws were mistakes with 16% completely missing the target. Our research leverages facial images capturing the person's reaction to the outcome of a throw to predict when the resulting throw is a mistake and then we determine the actual intent of the throw. The approach we propose for outcome prediction performs 38% better than the two-stream architecture used previously for this task on front-on videos. In addition, we propose a 1D-CNN model which is used in conjunction with priors learned from the frequency of mistakes to provide an end-to-end pipeline for outcome and intent recognition in this throwing task. [ABSTRACT FROM AUTHOR]

Consider a mobile robot exploring an office building with the aim of observing as much human activity as possible over several days. It must learn where and when people are to be found, count the observed activities, and revisit popular places at the right time. In this paper we present a series of Bayesian estimators for the levels of human activity that improve on simple counting. We then show how these estimators can be used to drive efficient exploration for human activities. The estimators arise from modelling the human activity counts as a partially observable Poisson process (POPP). This paper presents novel extensions to POPP for the following cases: (i) the robot's sensors are correlated, (ii) the robot's sensor model, itself built from data, is also unreliable, (iii) both are combined. It also combines the resulting Bayesian estimators with a simple, but effective solution to the exploration-exploitation trade-off faced by the robot in a real deployment. A series of 15 day robot deployments show how our approach boosts the number of human activities observed by 70% relative to a baseline and produces more accurate estimates of the level of human activity in each place and time. [ABSTRACT FROM AUTHOR]

Integrating Artificial Intelligence (AI) and drone technology into bridge inspections offers numerous advantages, including increased efficiency and enhanced safety. However, it is essential to recognize that this integration changes the cognitive ergonomics of the inspection task. Gaining a deeper understanding of how humans process information and behave when collaborating with drones and AI systems is necessary for designing and implementing effective AI-assisted inspection drones. To further understand human-drone-AI intricate dynamics, an experiment was conducted in which participants' biometric and behavioral data were collected during a simulated drone-enabled bridge inspection under two conditions: with an 80% accurate AI assistance and with no AI assistance. Results indicate that cognitive and behavioral factors, including vigilance, cognitive processing intensity, gaze patterns, and visual scanning efficiency can influence inspectors' performance respectively in either condition. This highlights the importance of designing inspection protocols, drones and AI systems based on a comprehensive understanding of the cognitive processes required in each condition to prevent cognitive overload and minimize errors. We also remark on the visual scanning and gaze patterns associated with a higher chance of missing critical information in each condition, insights that inspectors can use to enhance their inspection performance. [ABSTRACT FROM AUTHOR]

When humans and robots operate in and occupy the same local space, proximity detection and proactive collision avoidance is of high importance. As legged robots, such as the Boston Dynamics (BD) Spot, start to appear in real-world application environments, ensuring safe robot-human interactions while operating in full autonomy mode becomes a critical gate-keeping technology for trust in robotic workers. Towards that problem, this article proposes a track-and-avoid architecture for legged robots that combines a visual object detection and estimation pipeline with a Nonlinear Model Predictive Controller (NMPC) based on the Optimization Engine, capable of generating trajectories that satisfy the avoidance and tracking problems in real-time operations where the computation time never exceeded 40 ms. The system is experimentally evaluated using the BD Spot, in a custom sensor and computational suite, and in fully autonomous operational conditions, for the robot-human safety scenario of quickly moving noncooperative obstacles. The results demonstrate the efficacy of the scheme in multiple scenarios where the maximum safety distance violation was only 9 cm for an obstacle moving at 2.5 m/s while affected by both state estimation and object detection uncertainty and noise.

Abstract Robotics has a special place in AI as robots are connected to the real world and robots increasingly appear in humans everyday environment, from home to industry. Apart from cases were robots are expected to completely replace them, humans will largely benefit from real interactions with such robots. This is not only true for complex interaction scenarios like robots serving as guides, companions or members in a team, but also for more predefined functions like autonomous transport of people or goods. More and more, robots need suitable interfaces to interact with humans in a way that humans feel comfortable and that takes into account the need for a certain transparency about actions taken. The paper describes the requirements and state-of-the-art for a human-centered robotics research and development, including verbal and non-verbal interaction, understanding and learning from each other, as well as ethical questions that have to be dealt with if robots will be included in our everyday environment, influencing human life and societies.

In human–robot interaction (HRI), sharing emotions between the human and robot is one of the most important elements. However, market trends suggest that being able to perform productive tasks is more important than being able to express emotions in order for robots to be more accepted by society. In this study, we introduce a method of conveying emotions through a robot arm while it simultaneously executes main tasks. This method utilizes the null space control scheme to exploit the kinematic redundancy of a robot manipulator. In addition, the concept of manipulability ellipsoid is used to maximize the motion in the kinematic redundancy. The "Nextage-Open" robot was used to implement the proposed method, and HRI was recorded on video. Using these videos, a questionnaire with Pleasure–Arousal–Dominance (PAD) scale was conducted via the internet to evaluate people's impressions of the robot's emotions. The results suggested that even when industrial machines perform emotional behaviors within the safety standards set by the ISO/TS 15066, it is difficult to provide enough variety for each emotion to be perceived differently. However, people's reactions to the unclear movements yielded useful and interesting results, showing the complementary roles of motion features, interaction content, prejudice toward robots, and facial expressions in understanding emotion. [ABSTRACT FROM AUTHOR]

This research presents the technical considerations for implementing the CeCi (Computer Electronic Communication Interface) social robot. In this case, this robot responds to the need to achieve technological development in an emerging country with the aim of social impact and social interaction. There are two problems with the social robots currently on the market, which are the main focus of this research. First, their costs are not affordable for companies, universities, or individuals in emerging countries. The second is that their design is exclusively oriented to the functional part with a vision inherent to the engineers who create them without considering the vision, preferences, or requirements of the end users, especially for their social interaction. This last reason ends causing an aversion to the use of this type of robot. In response to the issues raised, a low-cost prototype is proposed, starting from a commercial platform for research development and using open source code. The robot design presented here is centered on the criteria and preferences of the end user, prioritizing acceptability for social interaction. This article details the selection process and hardware capabilities of the robot. Moreover, a programming section is provided to introduce the different software packages used and adapted for the social interaction, the main functions implemented, as well as the new and original part of the proposal. Finally, a list of applications currently developed with the robot and possible applications for future research are discussed. [ABSTRACT FROM AUTHOR]

Virtual Reality (VR) is being increasingly used to provide a more intuitive and embodied approach to robotic teleoperation, giving operators a sense of presence in the remote environment. Prior research has shown that presence can be enhanced when additional sensory cues such as sound are introduced. Data sonification is the use of non-speech audio to convey information and, in the context of VR robot teleoperation, it has the potential to 1) improve task performance by enhancing an operator’s sense of presence and 2) reduce task load by spreading data between sensory modalities. Here we present a novel study methodology to investigate how the design of data sonification impacts on these important metrics and other key measures of user experience, such as stress. We examine a nuclear decommissioning application of robotic teleoperation where the benefits of VR in terms of spatial reasoning and task performance are desirable. However, as the operational environment is hazardous, a sense of presence may not be desirable as it can lead to heightened operator stress. We conduct a study in which we compare the effects of diegetic sounds (literal and established sonifications) with abstract sounds (non-established sonifications). Our findings show that the diegetic sounds decrease workload, whereas abstract sounds increase workload, and are more stressful. Additionally, and contrary to expectations, sonification does not impact presence. These findings have implications for the design of sonic environments in virtual reality [ABSTRACT FROM AUTHOR]

Background: Although robotic manipulators have great potential in promoting motor independence of people with motor impairments, only few systems are currently commercially available. In addition to technical, economic, and normative barriers, a key challenge for their distribution is the current lack of evidence regarding their usefulness, acceptance, and user-specific requirements. Objective: Against this background, a semiautonomous robot system was developed in the research and development project, robot-assisted services for individual and resource-oriented intensive and palliative care of people with amyotrophic lateral sclerosis (ROBINA), to support people with amyotrophic lateral sclerosis (ALS) in various everyday activities. Methods: The developed early-stage demonstrator was evaluated in a task-based laboratory study of 11 patients with ALS. On the basis of a multimethod design consisting of standardized questionnaires, open-ended questions, and observation protocols, participants were asked about its relevance to everyday life, usability, and design requirements. Results: Most participants considered the system to provide relevant support within the test scenarios and for their everyday life. On the basis of the System Usability Scale, the overall usability of the robot-assisted services for individual and resource-oriented intensive and palliative care of people with ALS system was rated as excellent, with a median of 90 (IQR 75-95) points. Moreover, 3 central areas of requirements for the development of semiautonomous robotic manipulators were identified and discussed: requirements for semiautonomous human-robot collaboration, requirements for user interfaces, and requirements for the adaptation of robotic capabilities regarding everyday life. Conclusions: Robotic manipulators can contribute to increase the autonomy of people with ALS. A key issue for future studies is how the existing ability level and the required robotic capabilities can be balanced to ensure both high user satisfaction and effective and efficient task performance. [ABSTRACT FROM AUTHOR]

In this article, we present a low cost open source robotic solution for remote rehabilitation. In a pandemic context, where the trend is to manage our services remotely, the proposed system guarantees that the physiotherapist remotely controls and supervises the rehabilitation process. In the developed solution, it is not necessary to directly touch either the subject or the system, since the proposed architecture ensures control of the device remotely and using only a simple gesture. According to experts, pain is considered one of the major constraints that prevent patients from easily accepting rehabilitation sessions. To our knowledge, there is no method or tool presented in the literature to measure pain. The proposed solution aims to estimate the pain then to manage the control of the robot according to this estimate. In fact, the idea is to estimate pain through a decision support system based on cascading fuzzy logic. The first compartment of this fuzzy system was used to estimate muscle contact based on the EMG signal. The second compartment was set up to estimate pain according to three parameters: muscle contraction, the patient's resistance force and the last angle reached. A user-friendly human machine interface (HMI) has been developed to ensure communication between the robot and the physiotherapist. The control is done by gestures taking advantage of the Kinect technology. Sensory information about strength, range of motion (RoM) and estimated pain level is communicated to the physiotherapist's board. Patient information, health changes and tracked activities are all recorded, providing an electronic health record. Experimental studies carried out on a set of patients in a clinical environment have shown the efficiency and reliability of the developed robotic solution.

This paper suggests guidelines of social behavior for students using robots to interact with their classmates in group settings. The guidelines allow for the establishment of some common expectations for both the remote student and classmates. It is important to have clear expectations for robot behavior in the classroom in order for the social dynamics of the classroom to be productive. When remote students and their groups (i.e., classmates) all have a clear understanding of the technology and expected robot behavior, they may experience improved group dynamics and performance. In addition, if technological difficulties occur, they may be able to engage in group-led problem solving to both adapt the current technology and advocate for improved design of future technologies to meet their needs.

Encountered-Type Haptic Displays (ETHDs) enable users to touch virtual surfaces by using robotic actuators capable of co-locating real and virtual surfaces without encumbering users with actuators. One of the main challenges of ETHDs is to ensure that the robotic actuators do not interfere with the VR experience by avoiding unexpected collisions with users. This paper presents a design space for safety techniques using visual feedback to make users aware of the robot’s state and thus reduce unintended potential collisions. The blocks that compose this design space focus on what and when the feedback is displayed and how it protects the user. Using this design space, a set of 18 techniques was developed exploring variations of the three dimensions. An evaluation questionnaire focusing on immersion and perceived safety was designed and evaluated by a group of experts, which was used to provide a first assessment of the proposed techniques.

In this study, we present an IoT-based robot for wrist rehabilitation with a new protocol for determining the state of injured muscles as well as providing dynamic model parameters. In this model, the torque produced by the robot and the torque provided by the patient are determined and updated taking into consideration the constraints of fatigue. Indeed, in the proposed control architecture based on the EMG signal extraction, a fuzzy classifier was designed and implemented to estimate muscle fatigue. Based on this estimation, the patient's torque is updated during the rehabilitation session. The first step of this protocol consists of calculating the subject-related parameters. This concerns axis offset, inertial parameters, passive stiffness, and passive damping. The second step is to determine the remaining component of the wrist model, including the interaction torque. The subject must perform the desired movements providing the torque necessary to move the robot in the desired direction. In this case, the robot applies a resistive torque to calculate the torque produced by the patient. After that, the protocol considers the patient and the robot as active and all exercises are performed accordingly. The developed robotics-based solution, including the proposed protocol, was tested on three subjects and showed promising results. [ABSTRACT FROM AUTHOR]

Corona Virus Disease 2019 (COVID-19) pandemic has become a global challenge faced by people all over the world. Social distancing has been proved to be an effective practice to reduce the spread of COVID-19. Against this backdrop, we propose that the surveillance robots can not only monitor but also promote social distancing. Robots can be flexibly deployed and they can take precautionary actions to remind people of practicing social distancing. In this paper, we introduce a fully autonomous surveillance robot based on a quadruped platform that can promote social distancing in complex urban environments. Specifically, to achieve autonomy, we mount multiple cameras and a three dimensional light detection and ranging sensor (3D LiDAR) on the legged robot. The robot then uses an onboard real-time social distancing detection system to track nearby pedestrian groups. Next, the robot uses a crowd-aware navigation algorithm to move freely in highly dynamic scenarios. The robot finally uses a crowd-aware routing algorithm to effectively promote social distancing by using human-friendly verbal cues to send suggestions to over-crowded pedestrians. We demonstrate and validate that our robot can be operated autonomously by conducting several experiments in various urban scenarios.

This paper reports a soft fingertip-mountable tactile actuator based on a Dielectric Elastomer Actuator (DEA), which exhibits high output force over a wide frequency range with a lightweight and soft structure. DEA is a soft actuator characterized by its large area strain, fast response speed, and high specific energy density. The proposed soft tactile actuator is constructed of a multi-layered conical DEA structure. This design has safety benefits because it isolates the high voltage components from the contact point. In this paper, the resonance frequency of the tactile actuator was designed to be at 250 Hz to maximize vibrotactile stimulation. In addition, the geometric design parameters of the soft tactile actuator were optimized by conducting the simulations and the experiments. Based on these efforts, the proposed actuator produces a high output force of 8.48 N at the resonance frequency, with a maximum displacement of 0.46 mm. Our wearable prototype was an entirely soft haptic system, which exhibits high output force, as well as flexibility and conformity with a total weight of 2.6 g.

Learning from demonstrations with Probabilistic Movement Primitives (ProMPs) has been widely used in robot skill learning, especially in human-robot collaboration. Although ProMP has been extended to multi-task situations inspired by the Gaussian mixture model, it still treats each task independently. ProMP ignores the common scenario that robots conduct adaptive switching of the collaborative tasks in order to align with the instantaneous change of human intention. To solve this problem, we proposed an alternate learning-based parameter estimation method and an empirical minimum variation-based decomposition strategy with projection points, combining with linear interpolation strategy for weights, based on a Gaussian mixture model framework. Alternate learning of weights and parameters in multi-task ProMP (MTProMP) allows the robot to obtain a smooth composite trajectory planning which crosses expected via points. Decomposition strategy reflects how the desired via point state is projected onto the individual ProMP component, rendering the minimum total sum of deviations between each projection point with the respective prior. Linear interpolation is used to adjust the weights among sequential via points automatically. The proposed method and strategy are successfully extended to multi-task interaction ProMPs (MTiProMP). With MTProMP and MTiProMP, the robot can be applied to multiple tasks in industrial factories and collaborate with the worker to switch from one task to another according to changing intentions of the human. Classical via points trajectory planning experiments and human-robot collaboration experiments are performed on the Sawyer robot. The results of experiments show that MTProMP and MTiProMP with the proposed method and strategy perform better.

For robots that can provide physical assistance, maintaining synchronicity of the robot and human movement is a precursor for interaction safety. Existing research on collaborative HRI does not consider how synchronicity can be affected if humans are subjected to cognitive overloading and distractions during close physical interaction. Cognitive neuroscience has shown that unexpected events during interactions not only affect action cognition but also human motor control Gentsch et al. (Cognition, 2016, 146, 81–89). If the robot is to safely adapt its trajectory to distracted human motion, quantitative changes in the human movement should be evaluated. The main contribution of this study is the analysis and quantification of disrupted human movement during a physical collaborative task that involves robot-assisted dressing. Quantifying disrupted movement is the first step in maintaining the synchronicity of the human-robot interaction. The human movement data collected from a series of experiments where participants are subjected to cognitive loading and distractions during the human-robot interaction, are projected in a 2-D latent space that efficiently represents the high-dimensionality and non-linearity of the data. The quantitative data analysis is supported by a qualitative study of user experience, using the NASA Task Load Index to measure perceived workload, and the PeRDITA questionnaire to represent the human psychological state during these interactions. In addition, we present an experimental methodology to collect interaction data in this type of human-robot collaboration that provides realism, experimental rigour and high fidelity of the human-robot interaction in the scenarios.

In recent years research in manufacturing has been shifting towards flexible and intuitive Human-Robot Collaborative (HRC) solutions, allowing operators to coexist and interact safely with robots of various payloads. Previous research provided proof that HRC systems can improve the quality and cycle time of processes, including welding, grinding, polishing, assembly and picking. Applying such solutions to shipbuilding and ship maintenance, repair and conversion (SMRC) industry though is not straight forward. Lack of 3D-CAD models, variety of processes that have to be performed, manipulation of dissimilar parts, confined spaces, unpredictable external conditions are some of the challenges that arise. Nevertheless, the shipbuilding and SMRC industry is a dynamic and competitive sector, which could highly benefit from leveraging technology advancements, emerged from Industry 4.0 revolution. These advancements can not only improve the work life of the operators, supporting them in dangerous working environments, but also increase the quality of the final product as well as productivity rate. In this direction, this paper aims to present the challenges that derive from the shipbuilding and SMRC industry related with assembly operations, as well as a novel user-centric solution that aspires to overcome these challenges. The solution consists of a reconfigurable HRC cell, involving a high payload collaborative manipulator able to perform lifting and positioning of large heavy parts, while the operator performs the welding task. Intuitive interaction solutions allow precise parts handling by workers. Moreover, multi-layer safety systems allow the monitoring of the workplace and ensure the safety of the operators. Finally, hand guiding, and intuitive offline programming techniques enable the manipulation and programming of the high-payload robot by not experts. [ABSTRACT FROM AUTHOR]

Wearable robotic upper limb orthoses (ULO) are promising tools to assist or enhance the upper-limb function of their users. While the functionality of these devices has continuously increased, the robust and reliable detection of the user's intention to control the available degrees of freedom remains a major challenge and a barrier for acceptance. As the information interface between device and user, the intention detection strategy (IDS) has a crucial impact on the usability of the overall device. Yet, this aspect and the impact it has on the device usability is only rarely evaluated with respect to the context of use of ULO. A scoping literature review was conducted to identify non-invasive IDS applied to ULO that have been evaluated with human participants, with a specific focus on evaluation methods and findings related to functionality and usability and their appropriateness for specific contexts of use in daily life. A total of 93 studies were identified, describing 29 different IDS that are summarized and classified according to a four-level classification scheme. The predominant user input signal associated with the described IDS was electromyography (35.6%), followed by manual triggers such as buttons, touchscreens or joysticks (16.7%), as well as isometric force generated by residual movement in upper-limb segments (15.1%). We identify and discuss the strengths and weaknesses of IDS with respect to specific contexts of use and highlight a trade-off between performance and complexity in selecting an optimal IDS. Investigating evaluation practices to study the usability of IDS, the included studies revealed that, primarily, objective and quantitative usability attributes related to effectiveness or efficiency were assessed. Further, it underlined the lack of a systematic way to determine whether the usability of an IDS is sufficiently high to be appropriate for use in daily life applications. This work highlights the importance of a user- and application-specific selection and evaluation of non-invasive IDS for ULO. For technology developers in the field, it further provides recommendations on the selection process of IDS as well as to the design of corresponding evaluation protocols. [ABSTRACT FROM AUTHOR]

Wearable robotic upper limb orthoses (ULO) are promising tools to assist or enhance the upper-limb function of their users. While the functionality of these devices has continuously increased, the robust and reliable detection of the user's intention to control the available degrees of freedom remains a major challenge and a barrier for acceptance. As the information interface between device and user, the intention detection strategy (IDS) has a crucial impact on the usability of the overall device. Yet, this aspect and the impact it has on the device usability is only rarely evaluated with respect to the context of use of ULO. A scoping literature review was conducted to identify non-invasive IDS applied to ULO that have been evaluated with human participants, with a specific focus on evaluation methods and findings related to functionality and usability and their appropriateness for specific contexts of use in daily life. A total of 93 studies were identified, describing 29 different IDS that are summarized and classified according to a four-level classification scheme. The predominant user input signal associated with the described IDS was electromyography (35.6%), followed by manual triggers such as buttons, touchscreens or joysticks (16.7%), as well as isometric force generated by residual movement in upper-limb segments (15.1%). We identify and discuss the strengths and weaknesses of IDS with respect to specific contexts of use and highlight a trade-off between performance and complexity in selecting an optimal IDS. Investigating evaluation practices to study the usability of IDS, the included studies revealed that, primarily, objective and quantitative usability attributes related to effectiveness or efficiency were assessed. Further, it underlined the lack of a systematic way to determine whether the usability of an IDS is sufficiently high to be appropriate for use in daily life applications. This work highlights the importance of a user- and application-specific selection and evaluation of non-invasive IDS for ULO. For technology developers in the field, it further provides recommendations on the selection process of IDS as well as to the design of corresponding evaluation protocols.

As assistive robotics has expanded to many task domains, comparing assistive strategies among the varieties of research becomes increasingly difficult. To begin to unify the disparate domains into a more general theory of assistance, we present a definition of assistance, a survey of existing work, and three key design axes that occur in many domains and benefit from the examination of assistance as a whole. We first define an assistance perspective that focuses on understanding a robot that is in control of its actions but subordinate to a user’s goals. Next, we use this perspective to explore design axes that arise from the problem of assistance more generally and explore how these axes have comparable trade-offs across many domains. We investigate how the assistive robot handles other people in the interaction, how the robot design can operate in a variety of action spaces to enact similar goals, and how assistive robots can vary the timing of their actions relative to the user’s behavior. While these axes are by no means comprehensive, we propose them as useful tools for unifying assistance research across domains and as examples of how taking a broader perspective on assistance enables more cross-domain theorizing about assistance.

In this research, a new remote rehabilitation system was developed that integrates an IoT-based connected robot intended for wrist and forearm rehabilitation. In fact, the mathematical model of the wrist and forearm joints was developed and integrated into the main controller. The proposed new rehabilitation protocol consists of three main sessions: the first is dedicated to the extraction of the passive components of the dynamic model of wrist–forearm biomechanics while the active components are extracted in the second session. The third session consists of performing continuous exercises using the determined dynamic model of the forearm–wrist joints, taking into account the torque generated by muscle fatigue. The main objective of this protocol is to determine the state level of the affected wrist and above all to provide a dynamic model in which the torque generated by the robot and the torque supplied by the patient are combined, taking into account the constraints of fatigue. A Support Vector Machine (SVM) classifier is designed for the estimation of muscle fatigue based on the features extracted from the electromyography (EMG) signal acquired from the patient. The results show that the developed rehabilitation system allows a good progression of the joint’s range of motion as well as the resistive-active torques.