Search

Collaborative robotic manipulators can stop in case of a collision, according to the ISO/TS 15066 and ISO 10218-1 standards. However, in a human-robot collaboration scenario where the robot and the human share the workspace, a better solution with concerns to production and operator safety would be to perceive in advance the presence of an obstacle and be able to avoid it, thereby completing the task without halting the robot. In this paper, an obstacle avoidance algorithm is tested using a sensor system for real-time human detection; the operator represents a potential dynamic obstacle that can interfere with the robot motion. The sensor system consists of three RGB-D cameras. A custom software framework has been developed in Python exploiting machine learning tools for human skeleton detection. The coordinates of the human body joints relative to the manipulator base are used as input to the obstacle avoidance algorithm. The use of multiple sensors makes it possible to limit the occlusion problem; in addition, the choice of non-wearable sensors goes in the direction of better operator comfort. A series of experimental tests were performed to verify the accuracy of skeleton detection and the ability of the system to avoid obstacles in real time. The human motion caption system, in particular, was validated through a comparison with a commercial system based on wearable IMU sensors widely used and validated in motion capture. There is an NRMSE of 3.23 % for the RGB-D camera-based skeleton detection system, against an NRMSE of 12.23 % for the IMU wearable sensor system. Test results confirm that the system is able to avoid collisions with the human body under various conditions, static or dynamic, ensuring a minimum safety distance to any part of the manipulator. In 44 tests in which the operator moves around the robot with possible collisions (at a speed typical of manufacturing operations), the minimum operator-robot distance averaged 208 mm , being 200 mm the limit safety distance set by algorithm. [ABSTRACT FROM AUTHOR]

Teleoperation systems find many applications from earlier search-and-rescue to more recent daily tasks. It is widely acknowledged that using external sensors can decouple the view of the remote scene from the motion of the robot arm during manipulation, facilitating the control task. However, this design requires the coordination of multiple operators or may exhaust a single operator as s/he needs to control both the manipulator arm and the external sensors. To address this challenge, our work introduces a viewpoint prediction model, the first data-driven approach that autonomously adjusts the viewpoint of a dynamic camera to assist in telemanipulation tasks. This model is parameterized by a deep neural network and trained on a set of human demonstrations. We propose a contrastive learning scheme that leverages viewpoints in a camera trajectory as contrastive data for network training. We demonstrated the effectiveness of the proposed viewpoint prediction model by integrating it into a real-world robotic system for telemanipulation. User studies reveal that our model outperforms several camera control methods in terms of control experience and reduces the perceived task load compared to manual camera control. As an assistive module of a telemanipulation system, our method significantly reduces task completion time for users who choose to adopt its recommendation. [ABSTRACT FROM AUTHOR]

Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration. [ABSTRACT FROM AUTHOR]

Social signal recognition techniques based on nonverbal behavioral sensing allow conversational robots to understand the user’s social signals, thereby enabling them to adopt interaction strategies based on internal states inferred from the social signals. This research investigates how the online social signal recognition and adaptive dialog strategy influences the dynamic change in a user’s inner state. For this purpose, we develop a semiautonomous interview robot system with an online speaker’s willingness recognition module and an adaptive question selection module based on the willingness level. The online recognition model of speaker willingness is trained from multimodal nonverbal features extracted using a novel interview corpus, which allows appropriate interview questions to be chosen based on the estimated willingness level of the user. We conduct the experiment using the system to evaluate the effectiveness of adaptive question selection based on the willingness recognition model. First, the multimodal willingness recognition model is evaluated using the interview corpus. The best recognition accuracy of willingness level (high or low) was $72. 8\%$ 72. 8 p e r c n t ; with the random forest classifier. Second, 27 interviewees were interviewed with the two interview robot systems: (I) with the adaptive question selection module based on willingness recognition and (II) with a random question selection strategy. The proposed adaptive question strategy significantly increased the number of utterances with high willingness compared with the baseline system (II); thus, adaptive question selection with online willingness recognition elicited the speaker’s willingness even though the model cannot be estimated with near-perfect accuracy. [ABSTRACT FROM AUTHOR]

Assistive robots have the potential to support independence, enhance safety, and lower healthcare costs for older adults, as well as alleviate the demands of their care partners. However, ensuring that these robots will effectively and reliably address end-user needs in the long term requires user-specific design factors to be considered during the robot development process. To identify these design factors, we embedded Stretch, a mobile manipulator created by Hello Robot Inc., in the home of an older adult with motor impairments and his care partner for four weeks to support them with everyday activities. An occupational therapist and a robotics engineer lived with them during this period, employing an immersive participatory design approach to co-design and customise the robot with them. We highlight the benefits of this immersive participatory design experience and provide insights into robot design that can be applied broadly to other assistive technologies. [ABSTRACT FROM AUTHOR]

Reminding is often identified as a central function of socially assistive robots in the healthcare sector. The robotic reminders are supposed to help people with memory impairments to remember to take their medicine, to drink and eat, or to attend appointments. Such standalone reminding technologies can, however, be too demanding for people with memory injuries. In a co-creation process, we developed an individual reminder robot together with a person with traumatic brain injury and her care personnel. During this process, we learned that while current research describe reminding as a prototypical task for socially assistive robots, there is no clear definition of what constitutes a reminder nor that it is based on complex sequences of interactions that evolve over time and space, across different actions, actors and technologies. Based on our data from the co-creation process and the first deployment, we argue for a shift towards a sequential and socially distributed character of reminding. Understanding socially assistive robots as rehabilitative tools for people with memory impairment, they need to be reconsidered as interconnected elements in institutional care practices instead of isolated events for the remindee. [ABSTRACT FROM AUTHOR]

In a human–robot conversation, it is difficult for the robot to start the conversation just when the addressee is ready to listen to the robot, due to recognition technology issues. This paper proposes and evaluates a method to reduce the sense of that the timing of starting the conversation is bad. In this method, two robots perform a cooperative behavior during a waiting time from the call for attracting addressee's attention to the main utterance the robot want to deliver. To evaluate the effectiveness of this method, we conducted an experiment that compared three conversation initiation approaches: early timing and late timing by one robot, and the proposed approach involving two robots. The results revealed that the proposed method mitigates the bad timing of main utterances as perceived by the participants. [ABSTRACT FROM AUTHOR]

Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.

3D Skeleton-based human motion prediction, a classic task in computer vision, aims to forecast subsequent motions based on historical motion observations. In particular, precise short-term motion prediction is crucial for the effectiveness of machines designed for real-time human-computer interaction. This study aims to achieve accurate predictions of human motion within timeframes of less than 400 milliseconds, with the goal of improving machine responsiveness and efficiency. Previous research has predominantly relied on sequence models like Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM) or Transformers. Transformer-based methods have been relied on temporal forecasting ability of Transformer. In this study, instead, we introduce an alternative perspective for Transformer-based models to comprehend the structure of skeletons, named Joint-Aware Transformer (JAT). Within our model, the attention mechanism is employed to encode inter-joint correlation instead of temporal dependencies. Our approach outperformed the state-of-the-art (SOTA) model in short-term prediction on three types of human motion datasets.

Distinguishable vibrotactile feedback can convey information non-verbally and complete the sensory loop when using assistive devices. Feedback can increase acceptance of assistive devices but could require personalization as these devices need to adapt to user capabilities and preferences that may affect the location for inducing feedback. Developing personalized feedback for each user may be ideal, but impractical if demand for these devices increases. In this research, we evaluate the hypothesis that the ability to define, generate, and use personalized feedback is preferred and should be provided. The hypothesis is evaluated using a system capable of capturing and recognizing non-verbal inputs, providing vibrotactile feedback to verify the identified action, and executing the verified action. Evaluation focuses on determining the ability to initiate actions using the system architecture, and define, generate, and use personalized feedback. In addition to recording performance metrics (success or failure), the participants also completed the NASA TLX and post-evaluation questionnaires. Based on results, the actions can be initiated using the system architecture, and defining, generating, and using personalized feedback was found to be preferable with the success rate reaching 100% after only three successive repetitions, and could decrease mental demand and effort while interacting with the system.

The perception of agency in human robot interaction has become increasingly important as robots become more capable and more social. There are, however, no accepted or consistent methods of measuring perceived agency; researchers currently use a wide range of techniques and surveys. We provide a definition of perceived agency, and from that definition we create and psychometrically validate a scale to measure perceived agency. We then perform a scale evaluation by comparing the PA scale constructed in experiment 1 to two other existing scales. We find that our PA and PA-R (Perceived Agency–Rasch) scales provide a better fit to empirical data than existing measures. We also perform scale validation by showing that our scale shows the hypothesized relationship between perceived agency and morality. [ABSTRACT FROM AUTHOR]

Introduction: Underlying mechanisms of speech perception masked by background speakers, a common daily listening condition, are often investigated using various and lengthy psychophysical tests. The presence of a social agent, such as an interactive humanoid NAO robot, may help maintain engagement and attention. However, such robots potentially have limited sound quality or processing speed. Methods: As a first step toward the use of NAO in psychophysical testing of speech-in-speech perception, we compared normal-hearing young adults' performance when using the standard computer interface to that when using a NAO robot to introduce the test and present all corresponding stimuli. Target sentences were presented with colour and number keywords in the presence of competing masker speech at varying target-to-masker ratios. Sentences were produced by the same speaker, but voice differences between the target and masker were introduced using speech synthesis methods. To assess test performance, speech intelligibility and data collection duration were compared between the computer and NAO setups. Human-robot interaction was assessed using the Negative Attitude Toward Robot Scale (NARS) and quantification of behavioural cues (backchannels). Results: Speech intelligibility results showed functional similarity between the computer and NAO setups. Data collection durations were longer when using NAO. NARS results showed participants had a relatively positive attitude toward "situations of interactions" with robots prior to the experiment, but otherwise showed neutral attitudes toward the "social influence" of and "emotions in interaction" with robots. The presence of more positive backchannels when using NAO suggest higher engagement with the robot in comparison to the computer. Discussion: Overall, the study presents the potential of the NAO for presenting speech materials and collecting psychophysical measurements for speech-inspeech perception. [ABSTRACT FROM AUTHOR]

Human-Robot interaction plays a vital role in the service robotics field, especially to support old age-dependent people for socio-economic reasons. In this paper, an indigenous 3D printed 6 DoF robotic arm is proposed to support specially-abled people in their independent-feeding process. The objective of the present paper is to find the combination of optimal positional controllers such as CPID, FC, FPID and FOPID, which can handle the cubic reference input signal and can produce an output signal with minimal overshoot and lesser positional error. The reduced positional error helps the robotic arm to accurately reach the destination with minimal oscillations. This would reduce the wastage of food in the middle of the trajectory as well as at the destination. The technical challenge of the paper is to synchronize machine vision, robot kinematics and trajectory planning with robot control for multiple intermediate points, subjected to the cubic input signal. The feeding robotic arm is equipped with Intel© visual depth camera, which is in synchronization with the microcontroller, servo controller, and servo actuators. Here, six intermediate points were identified in the C-space using FK, among which, fuzzy controller was selected for the first two IPs, the GA optimized FOPID was selected for IP3 to IP5 and FPID was deployed for the last IP. The GA: FOPID produced a negligible overshoot of 0.67%, whereas FC and FPID produced an overshoot of 1.4% and 0.8% respectively. The positional error gap between GA: FOPID, FC and FPID was 0.9, and 0.6 Deg/sec respectively. The selection of combination of optimal controllers helps the manipulator to successfully deliver the food without wasting it. The repeatability of the feeding process is ensured by successfully conducting user testing on 20 users for 25 cycles. [ABSTRACT FROM AUTHOR]

The mindBot robot is a new educational robot toy that can be controlled by brain signals and voice commands. It was evaluated with children with disabilities as well as healthy children as the potential users. The most significant challenge was the size of the used Emotiv Insight electroencephalogram headset when adjusting it on the children's' heads. Despite all the challenges, the mindBot robot is a promising technology that could be fun and educational for disabled children. The 11 participants took 36 minutes to finish all tasks on average. This includes the time they spent setting up the robot for the first time, putting on the headset, learning how to use the robot, and using the main educational features. The System Usability Scale usability score for the robot is 71.13, which is considered to be the score of good. The future stages of improving the mindBot includes adding more mobility capabilities and adding the feature of educational assessment. [ABSTRACT FROM AUTHOR]

Reminding is often identified as a central function of socially assistive robots in the healthcare sector. The robotic reminders are supposed to help people with memory impairments to remember to take their medicine, to drink and eat, or to attend appointments. Such standalone reminding technologies can, however, be too demanding for people with memory injuries. In a co-creation process, we developed an individual reminder robot together with a person with traumatic brain injury and her care personnel. During this process, we learned that while current research describe reminding as a prototypical task for socially assistive robots, there is no clear definition of what constitutes a reminder nor that it is based on complex sequences of interactions that evolve over time and space, across different actions, actors and technologies. Based on our data from the co-creation process and the first deployment, we argue for a shift towards a sequential and socially distributed character of reminding. Understanding socially assistive robots as rehabilitative tools for people with memory impairment, they need to be reconsidered as interconnected elements in institutional care practices instead of isolated events for the remindee.

IntroductionUnderlying mechanisms of speech perception masked by background speakers, a common daily listening condition, are often investigated using various and lengthy psychophysical tests. The presence of a social agent, such as an interactive humanoid NAO robot, may help maintain engagement and attention. However, such robots potentially have limited sound quality or processing speed.MethodsAs a first step toward the use of NAO in psychophysical testing of speech- in-speech perception, we compared normal-hearing young adults’ performance when using the standard computer interface to that when using a NAO robot to introduce the test and present all corresponding stimuli. Target sentences were presented with colour and number keywords in the presence of competing masker speech at varying target-to-masker ratios. Sentences were produced by the same speaker, but voice differences between the target and masker were introduced using speech synthesis methods. To assess test performance, speech intelligibility and data collection duration were compared between the computer and NAO setups. Human-robot interaction was assessed using the Negative Attitude Toward Robot Scale (NARS) and quantification of behavioural cues (backchannels).ResultsSpeech intelligibility results showed functional similarity between the computer and NAO setups. Data collection durations were longer when using NAO. NARS results showed participants had a relatively positive attitude toward “situations of interactions” with robots prior to the experiment, but otherwise showed neutral attitudes toward the “social influence” of and “emotions in interaction” with robots. The presence of more positive backchannels when using NAO suggest higher engagement with the robot in comparison to the computer.DiscussionOverall, the study presents the potential of the NAO for presenting speech materials and collecting psychophysical measurements for speech-in-speech perception.

This systematic literature review presents the latest advancements and insights about digital twin technology and robotics interfaced with extended reality in the context of Industry 4.0. As the extended reality technologies emerge, it results in an increasing overlap between digital twins and human-robot interactions in industrial settings, promoting collaboration between operators and cobots in manufacturing environments. The objective of this study is to serve as a valuable resource for researchers and practitioners working in the field of Industry 4.0. It aims to highlight the latest developments and innovations in the application of digital twins and robotics interfaced with extended reality technologies in manufacturing. By extracting data from relevant articles, it provides a comprehensive understanding of the current state-of-the-art in this field by: analyzing the favored extended reality interfaces for digital twin and robotics interactions; analyzing the digital twin and physical twin interaction; evaluating the digital twin application levels and pillars through extended reality interfacing; and introducing a new concept called augmented perception for creating new physical-digital interactions. [ABSTRACT FROM AUTHOR]

Abstract Nuclear facilities have a regulatory requirement to measure radiation levels within Post Operational Clean Out (POCO) around nuclear facilities each year, resulting in a trend towards robotic deployments to gain an improved understanding during nuclear decommissioning phases. The UK Nuclear Decommissioning Authority supports the view that human‐in‐the‐loop (HITL) robotic deployments are a solution to improve procedures and reduce risks within radiation characterisation of nuclear sites. The authors present a novel implementation of a Cyber‐Physical System (CPS) deployed in an analogue nuclear environment, comprised of a multi‐robot (MR) team coordinated by a HITL operator through a digital twin interface. The development of the CPS created efficient partnerships across systems including robots, digital systems and human. This was presented as a multi‐staged mission within an inspection scenario for the heterogeneous Symbiotic Multi‐Robot Fleet (SMuRF). Symbiotic interactions were achieved across the SMuRF where robots utilised automated collaborative governance to work together, where a single robot would face challenges in full characterisation of radiation. Key contributions include the demonstration of symbiotic autonomy and query‐based learning of an autonomous mission supporting scalable autonomy and autonomy as a service. The coordination of the CPS was a success and displayed further challenges and improvements related to future MR fleets.

Robotics is a cross-disciplinary branch of science and technology, and lays foundation on mechanics, control, computer science, artificial intelligence, and so on. With the developments of both softwares and hardwares, especially in the artificial intelligence technologies, robots have been widely applied in multiple areas in the society, and become more and more interactive in our daily life, such as the service robots in the museums, shopping malls, restaurants, etc. Though the ultimate goal for a service robot to behave like a human is not easy to be achieved, significant processes have been made during the past decades. Considering that it is universal that service robots are triggered to execute tasks specified by human users via commands (Comm-TET), and it is essential to process and understand human users' commands correctly, we comprehensively overview the developments of the task understanding (TU) sub-process of Comm-TET for service robots. In order to organize the related literature in a reasonable manner, we abstracted the pipeline of Comm-TET and the generic framework of TU based on the existing researches. Following the abstracted framework, we present in-depth discussions on each of its building blocks over the past decades, and give some insights on the future research directions. Compared to other reviews on TU, this review emphasizes more on the technical developments and organizes the existing researches as an integrality. [ABSTRACT FROM AUTHOR]

The uncanny valley hypothesis states that users might experience eerie when interacting with almost but not fully human-like artificial characters. The advancements in artificial intelligence, robotics, and computer graphics have led to life-like virtual humans and humanoid robots. It is necessary to revisit the hypothesis to check if they positively or negatively affect the current population, who are much more accustomed to the latest technologies. In this paper, we study and present a unique evaluation of the uncanny valley hypothesis by allowing participants to interact live with four different humanoid robots (of varying levels of humanlikeness). To evaluate the affinity of each robot, each participant needs to fill a survey questionnaire. Apart from this, we also use deep learning methods to quantify the participants’ emotional states using multi-modalcues, including visual, audio, and text, by recording the participant-robot interaction. The multi-modal analysis and surveys provide interesting results and insights into the uncanny valley hypothesis.

Robot vision provides the most important information to robots so that they can read the context and interact with human partners successfully. Moreover, to allow humans recognize the robot's visual understanding during human-robot interaction (HRI), the best way is for the robot to provide an explanation of its understanding in natural language. In this paper, we propose a new approach by which to interpret robot vision from an egocentric standpoint and generate descriptions to explain egocentric videos particularly for HRI. Because robot vision equals to egocentric video on the robot's side, it contains as much egocentric view information as exocentric view information. Thus, we propose a new dataset, referred to as the global, action, and interaction (GAI) dataset, which consists of egocentric video clips and GAI descriptions in natural language to represent both egocentric and exocentric information. The encoder-decoder based deep learning model is trained based on the GAI dataset and its performance on description generation assessments is evaluated. We also conduct experiments in actual environments to verify whether the GAI dataset and the trained deep learning model can improve a robot vision system [ABSTRACT FROM AUTHOR]

In order to navigate safely and effectively with humans in close proximity, robots must be capable of predicting the future motions of humans. This study first consolidates human studies in motion, intention, and preference into a discretized human model that can readily be used in robotics decision making algorithms. Cooperative Markov Decision Process (Co-MDP), a novel framework that improves upon Multiagent MDPs, is then proposed for enabling socially aware robot obstacle avoidance. Utilizing the consolidated and discretized human model, Co-MDP allows the system to (1) approximate rational human behavior and intention, (2) generate socially-aware robotic obstacle avoidance behavior, and (3) remain robust to the uncertainty of human intention and motion variance. Simulations of a human-robot co-populated environment verify Co-MDP as a feasible obstacle avoidance algorithm. In addition, the anthropomorphic behavior of Co-MDP was assessed and confirmed with a human-in-the-loop experiment. Results reveal that participants can not directly differentiate agents that were controlled by human operators from Co-MDP, and the reported confidences of their choices indicates that the predictions from participants were backed by behavioral evidence rather than random guesses. Thus the main contributions for this paper are: consolidating past human studies of rational human behavior and intention into a simple, discretized model; the development of Co-MDP: a robotic decision framework that can utilize this human model and maximize the joint utility between the human and robot; and an experimental design for evaluation of the human acceptance of obstacle avoidance algorithms. [ABSTRACT FROM AUTHOR]

Abstract Speech interaction is an important means of robot interaction. With the rapid development of deep learning, end‐to‐end speech synthesis methods based on this technique have gradually become mainstream. Chinese deep learning‐based speech synthesis techniques suffer from problems such as unstable synthesised speech, poor naturalness and poor personalised speech synthesis, which do not satisfy some practical application scenarios. Hence, an F‐MelGAN model is adopted to improve the performance of Chinese speech synthesis. A post‐processing network is used to refine the Mel‐spectrum predicted by the decoder and alleviate the Mel‐spectrum distortion phenomenon. A phoneme‐level and sentence‐level combined module is proposed to model the personalised style of speakers. A combination of an acoustic conditioning network, speaker encoder network GCNet and feedback‐constrained training is proposed to solve the problem of poor personalised speech synthesis and achieve personalised speech customisation in Chinese. Experimental results show that the whole model can generate high‐quality speech with high speaker similarity for both speakers that appear in the training process and speakers that never appear in the training process.

In robot-assisted surgeries, the surgeon focuses on the surgical tool and its pose, and not on the complete robot’s shape. However, the joints of redundant robots (robots that have more degrees of freedom (DoF) than needed for the positioning of surgical tools) might move in unexpected/undesired ways. Joint motions that lead to patient or collisions are safety critical. We assume that the medical personnel in the operating room can best decide if a planned robot motions come too close to the patient or not. Therefore, we propose an augmented reality-based solution to interact with the robot during surgery planning, and intervention. The tool can be used to command a robot by drawing a trajectory in augmented reality (AR), visualizing the robot movement to check if it is safe before execution. The proposed solution allows surgeons to plan safe robot motion paths before-hand and adapt them when necessary in situ. As a proof-of-concept, we implemented and demonstrated the proposed solution on a 7-DoF redundant robot by commanding different trajectories. The control architecture to plan and execute motion for a surgical robot using AR is a key result of this work.

In this research, a new remote rehabilitation system was developed that integrates an IoT-based connected robot intended for wrist and forearm rehabilitation. In fact, the mathematical model of the wrist and forearm joints was developed and integrated into the main controller. The proposed new rehabilitation protocol consists of three main sessions: the first is dedicated to the extraction of the passive components of the dynamic model of wrist–forearm biomechanics while the active components are extracted in the second session. The third session consists of performing continuous exercises using the determined dynamic model of the forearm–wrist joints, taking into account the torque generated by muscle fatigue. The main objective of this protocol is to determine the state level of the affected wrist and above all to provide a dynamic model in which the torque generated by the robot and the torque supplied by the patient are combined, taking into account the constraints of fatigue. A Support Vector Machine (SVM) classifier is designed for the estimation of muscle fatigue based on the features extracted from the electromyography (EMG) signal acquired from the patient. The results show that the developed rehabilitation system allows a good progression of the joint's range of motion as well as the resistive-active torques. [ABSTRACT FROM AUTHOR]

Recognising intent in collaborative human robot tasks can improve team performance and human perception of robots. Intent can differ from the observed outcome in the presence of mistakes which are likely in physically dynamic tasks. We created a dataset of 1227 throws of a ball at a target from 10 participants and observed that 47% of throws were mistakes with 16% completely missing the target. Our research leverages facial images capturing the person's reaction to the outcome of a throw to predict when the resulting throw is a mistake and then we determine the actual intent of the throw. The approach we propose for outcome prediction performs 38% better than the two-stream architecture used previously for this task on front-on videos. In addition, we propose a 1D-CNN model which is used in conjunction with priors learned from the frequency of mistakes to provide an end-to-end pipeline for outcome and intent recognition in this throwing task. [ABSTRACT FROM AUTHOR]

With rapid developments in biometric recognition, a great deal of attention is being paid to robots which interact smartly with humans and communicate certain types of biometrical information. Such human–machine interaction (HMI), also well-known as human–robot interaction (HRI), will, in the future, prove an important development when it comes to automotive manufacturing applications. Currently, hand gesture recognition-based HRI designs are being practically used in various areas of automotive manufacturing, assembly lines, supply chains, and collaborative inspection. However, very few studies are focused on material-handling robot interactions combined with hand gesture communication of the operator. The current work develops a depth sensor-based dynamic hand gesture recognition scheme for continuous-time operations with material-handling robots. The proposed approach properly employs the Kinect depth sensor to extract features of Hu moment invariants from depth data, through which feature-based template match hand gesture recognition is developed. In order to construct continuous-time robot operations using dynamic hand gestures with concatenations of a series of hand gesture actions, the wake-up reminder scheme using fingertip detection calculations is established to accurately denote the starting, ending, and switching timestamps of a series of gesture actions. To be able to perform typical template match on continuous-time dynamic hand gesture recognition with the ability of real-time recognition, representative frame estimates using centroid, middle, and middle-region voting approaches are also presented and combined with template match computations. Experimental results show that, in certain continuous-time periods, the proposed complete hand gesture recognition framework can provide a smooth operation for the material-handling robot when compared with robots controlled using only extractions of full frames; presented representative frames estimated by middle-region voting will maintain fast computations and still reach the competitive recognition accuracy of 90.8%. The method proposed in this study can facilitate the smart assembly line and human–robot collaborations in automotive manufacturing. [ABSTRACT FROM AUTHOR]

Consider a mobile robot exploring an office building with the aim of observing as much human activity as possible over several days. It must learn where and when people are to be found, count the observed activities, and revisit popular places at the right time. In this paper we present a series of Bayesian estimators for the levels of human activity that improve on simple counting. We then show how these estimators can be used to drive efficient exploration for human activities. The estimators arise from modelling the human activity counts as a partially observable Poisson process (POPP). This paper presents novel extensions to POPP for the following cases: (i) the robot's sensors are correlated, (ii) the robot's sensor model, itself built from data, is also unreliable, (iii) both are combined. It also combines the resulting Bayesian estimators with a simple, but effective solution to the exploration-exploitation trade-off faced by the robot in a real deployment. A series of 15 day robot deployments show how our approach boosts the number of human activities observed by 70% relative to a baseline and produces more accurate estimates of the level of human activity in each place and time. [ABSTRACT FROM AUTHOR]

Consistency of tracking filter such as Interactive Multiple Model Probabilistic Data Association Filter (IMMPDAF) is the most important factor in targets tracking. Inaccurate tracking capability will lead to poor tracking performance when dealing with multiple people's interactions and occlusions. In order to validate the consistency, Normalized Estimation Error Squared (NEES) and Normalized Innovation Squared (NIS) were evaluated and tested using Monte Carlo experiments for 50 runs. These simulations has proven that the tracker is conditionally consistent on targets tracking despite the fact that it has difficulties on handling occlusions and maneuvering people. NEES requires ground truth of tracking data and predicted data, whereas NIS requires observation and predicted data for Monte Carlo simulations. In NEES simulations, the result emphasizes that state estimation errors of IMMPDAF tracker are inconsistent with filter-calculated covariances especially when dealing with sudden turns in zig-zag motion where quite a large number of points fall outside 95% probability region. In NIS simulations, IMMPDAF tracker is confirmed to have difficulties to handle multiple targets with a short period of occlusion although a small number of points falls outside of 95% probability region. Filter tracker is considered mismatched when dealing with zig-zag motion; however, it deemed to be optimistic when dealing with occlusions. As a result, the IMMPDAF tracker has limited capability in monitoring sharp turns under occlusion conditions, although it is acceptable when dealing with occlusions only. [ABSTRACT FROM AUTHOR]

Integrating Artificial Intelligence (AI) and drone technology into bridge inspections offers numerous advantages, including increased efficiency and enhanced safety. However, it is essential to recognize that this integration changes the cognitive ergonomics of the inspection task. Gaining a deeper understanding of how humans process information and behave when collaborating with drones and AI systems is necessary for designing and implementing effective AI-assisted inspection drones. To further understand human-drone-AI intricate dynamics, an experiment was conducted in which participants' biometric and behavioral data were collected during a simulated drone-enabled bridge inspection under two conditions: with an 80% accurate AI assistance and with no AI assistance. Results indicate that cognitive and behavioral factors, including vigilance, cognitive processing intensity, gaze patterns, and visual scanning efficiency can influence inspectors' performance respectively in either condition. This highlights the importance of designing inspection protocols, drones and AI systems based on a comprehensive understanding of the cognitive processes required in each condition to prevent cognitive overload and minimize errors. We also remark on the visual scanning and gaze patterns associated with a higher chance of missing critical information in each condition, insights that inspectors can use to enhance their inspection performance. [ABSTRACT FROM AUTHOR]

When humans and robots operate in and occupy the same local space, proximity detection and proactive collision avoidance is of high importance. As legged robots, such as the Boston Dynamics (BD) Spot, start to appear in real-world application environments, ensuring safe robot-human interactions while operating in full autonomy mode becomes a critical gate-keeping technology for trust in robotic workers. Towards that problem, this article proposes a track-and-avoid architecture for legged robots that combines a visual object detection and estimation pipeline with a Nonlinear Model Predictive Controller (NMPC) based on the Optimization Engine, capable of generating trajectories that satisfy the avoidance and tracking problems in real-time operations where the computation time never exceeded 40 ms. The system is experimentally evaluated using the BD Spot, in a custom sensor and computational suite, and in fully autonomous operational conditions, for the robot-human safety scenario of quickly moving noncooperative obstacles. The results demonstrate the efficacy of the scheme in multiple scenarios where the maximum safety distance violation was only 9 cm for an obstacle moving at 2.5 m/s while affected by both state estimation and object detection uncertainty and noise.

Abstract Robotics has a special place in AI as robots are connected to the real world and robots increasingly appear in humans everyday environment, from home to industry. Apart from cases were robots are expected to completely replace them, humans will largely benefit from real interactions with such robots. This is not only true for complex interaction scenarios like robots serving as guides, companions or members in a team, but also for more predefined functions like autonomous transport of people or goods. More and more, robots need suitable interfaces to interact with humans in a way that humans feel comfortable and that takes into account the need for a certain transparency about actions taken. The paper describes the requirements and state-of-the-art for a human-centered robotics research and development, including verbal and non-verbal interaction, understanding and learning from each other, as well as ethical questions that have to be dealt with if robots will be included in our everyday environment, influencing human life and societies.

In human–robot interaction (HRI), sharing emotions between the human and robot is one of the most important elements. However, market trends suggest that being able to perform productive tasks is more important than being able to express emotions in order for robots to be more accepted by society. In this study, we introduce a method of conveying emotions through a robot arm while it simultaneously executes main tasks. This method utilizes the null space control scheme to exploit the kinematic redundancy of a robot manipulator. In addition, the concept of manipulability ellipsoid is used to maximize the motion in the kinematic redundancy. The "Nextage-Open" robot was used to implement the proposed method, and HRI was recorded on video. Using these videos, a questionnaire with Pleasure–Arousal–Dominance (PAD) scale was conducted via the internet to evaluate people's impressions of the robot's emotions. The results suggested that even when industrial machines perform emotional behaviors within the safety standards set by the ISO/TS 15066, it is difficult to provide enough variety for each emotion to be perceived differently. However, people's reactions to the unclear movements yielded useful and interesting results, showing the complementary roles of motion features, interaction content, prejudice toward robots, and facial expressions in understanding emotion. [ABSTRACT FROM AUTHOR]

In this work, we propose the use of Multimodal Human-Computer Interfaces (MHCI) through body poses to command a drone in an easy and intuitive way. First, the human-user pose is recovered from a video stream, with the help of the open source library Open Pose. Then, a Support Vector Classifier (SVC), trained to distinguish between different body poses, is used to interpret eleven different human poses as the most important high level commands to the drone. The proposed strategy was successfully implemented to remotely control a drone through a web interface, where the user can interact with a drone in a remote location using only a web interface. Real-time experiments were carried out with fourteen different volunteers, selected to represent different segments of the population, with different ages, gender, experience with technology, socioeconomic class, etc., in order to evaluate the user experience with the help of a User Experience Questionnaire (UEQ), demonstrating satisfactory results. The study suggests that the use of the proposed MHCI received good acceptance between the participants, even in users without previous experience with drones, and received excellent scores in terms of attractiveness, stimulation and novelty from most of the volunteers. [ABSTRACT FROM AUTHOR]

This paper explores a framework for defining artificial intelligence (AI) that adapts to individuals within a group, and discusses the technical challenges for collaborative AI systems that must work with different human partners. Collaborative AI is not one‐size‐fits‐all, and thus AI systems must tune their output based on each human partner's needs and abilities. For example, when communicating with a partner, an AI should consider how prepared their partner is to receive and correctly interpret the information they are receiving. Forgoing such individual considerations may adversely impact the partner's mental state and proficiency. On the other hand, successfully adapting to each person's (or team member's) behavior and abilities can yield performance benefits for the human–AI team. Under this framework, an AI teammate adapts to human partners by first learning components of the human's decision‐making process and then updating its own behaviors to positively influence the ongoing collaboration. This paper explains the role of this AI adaptation formalism in dyadic human–AI interactions and examines its application through a case study in a simulated navigation domain. [ABSTRACT FROM AUTHOR]

The role of social robots as advisors for decision-making is investigated. We examined how a robot advisor with logical reasoning and one with cognitive fallacies affected participants' decision-making in different contexts. The participants were asked to make multiple decisions while receiving advice from both robots during the decision-making process. Participants had to choose which robot they agreed with and, at the end of the scenario, rank the possible options presented to them. After the interaction, participants were asked to assign jobs to the robots, e.g. jury or bartender. Based on the 'like-me' hypothesis and previous research of social mitigation of fallacious judgmental decisions, we have compared participants' agreement with the two robots for each scenario to random choice using t-tests, as well as analysed the dynamical nature of the interaction, e.g. whether participants changed their choices based on the robots' verbal opinion using Pearson correlations. Our results show that the robots had an effect on the participants' responses, regardless of the robots' fallaciousness, wherein participants changed their decisions based on the robot they agreed with more. Moreover, the context, presented as two different scenarios, also had an effect on the preferred robots, wherein an art auction scenario resulted in significantly increased agreement with the fallacious robot, whereas a detective scenario did not. Finally, an exploratory analysis showed that personality traits, e.g. agreeableness and neuroticism, and attitudes towards robots had an impact on which robot was assigned to these jobs. Taken together, the results presented here show that social robots' effects on participants' decision-making involve complex interactions between the context, the cognitive fallacies of the robot and the attitudes and personalities of the participants and should not be considered a single psychological construct. [ABSTRACT FROM AUTHOR]

This research presents the technical considerations for implementing the CeCi (Computer Electronic Communication Interface) social robot. In this case, this robot responds to the need to achieve technological development in an emerging country with the aim of social impact and social interaction. There are two problems with the social robots currently on the market, which are the main focus of this research. First, their costs are not affordable for companies, universities, or individuals in emerging countries. The second is that their design is exclusively oriented to the functional part with a vision inherent to the engineers who create them without considering the vision, preferences, or requirements of the end users, especially for their social interaction. This last reason ends causing an aversion to the use of this type of robot. In response to the issues raised, a low-cost prototype is proposed, starting from a commercial platform for research development and using open source code. The robot design presented here is centered on the criteria and preferences of the end user, prioritizing acceptability for social interaction. This article details the selection process and hardware capabilities of the robot. Moreover, a programming section is provided to introduce the different software packages used and adapted for the social interaction, the main functions implemented, as well as the new and original part of the proposal. Finally, a list of applications currently developed with the robot and possible applications for future research are discussed. [ABSTRACT FROM AUTHOR]

We studied politeness in human–robot interaction based on Lakoff's politeness theory. In a series of eight studies, we manipulated three different levels of politeness of non-humanoid robots and evaluated their effects. A table-setting task was developed for two different types of robots (a robotic manipulator and a mobile robot). The studies included two different populations (old and young adults) and were conducted in two conditions (video and live). Results revealed that polite robot behavior positively affected users' perceptions of the interaction with the robots and that participants were able to differentiate between the designed politeness levels. Participants reported higher levels of enjoyment, satisfaction, and trust when they interacted with the politest behavior of the robot. A smaller number of young adults trusted the politest behavior of the robot compared to old adults. Enjoyment and trust of the interaction with the robot were higher when study participants were subjected to the live condition compared to video and participants were more satisfied when they interacted with a mobile robot compared to a manipulator. [ABSTRACT FROM AUTHOR]

Virtual Reality (VR) is being increasingly used to provide a more intuitive and embodied approach to robotic teleoperation, giving operators a sense of presence in the remote environment. Prior research has shown that presence can be enhanced when additional sensory cues such as sound are introduced. Data sonification is the use of non-speech audio to convey information and, in the context of VR robot teleoperation, it has the potential to 1) improve task performance by enhancing an operator’s sense of presence and 2) reduce task load by spreading data between sensory modalities. Here we present a novel study methodology to investigate how the design of data sonification impacts on these important metrics and other key measures of user experience, such as stress. We examine a nuclear decommissioning application of robotic teleoperation where the benefits of VR in terms of spatial reasoning and task performance are desirable. However, as the operational environment is hazardous, a sense of presence may not be desirable as it can lead to heightened operator stress. We conduct a study in which we compare the effects of diegetic sounds (literal and established sonifications) with abstract sounds (non-established sonifications). Our findings show that the diegetic sounds decrease workload, whereas abstract sounds increase workload, and are more stressful. Additionally, and contrary to expectations, sonification does not impact presence. These findings have implications for the design of sonic environments in virtual reality [ABSTRACT FROM AUTHOR]

Social robots have great potential for the therapy of children with autism spectrum disorder, but the practical use of them is challenging. In this article, we presented a social robot YANG, which can interact with children with autism in their daily training. We propose a therapist‐robot interactive (TRI) model, which integrates with the practice of discrete trial training (DTT), a basic method utilized in autism training. To evaluate the TRI model, we implemented the model in YANG and conducted a single‐subject experiment in a rehabilitation training center. Data were collected on three children (ages 3–4) and their three therapists as they interacted with YANG in training sessions. Results showed that the children's learning ability significantly improved. YANG formed natural and friendly relationships with the children and delivered substantial support to therapists. Our research brings insight into using social robots for children with autism. [ABSTRACT FROM AUTHOR]

The human-robot interaction community has developed many methods for robots to navigate safely and socially alongside humans. However, experimental procedures to evaluate these works are usually constructed on a per-method basis. Such disparate evaluations make it difficult to compare the performance of such methods across the literature. To bridge this gap, we introduce SocNavBench, a simulation framework for evaluating social navigation algorithms. SocNavBench comprises a simulator with photo-realistic capabilities and curated social navigation scenarios grounded in real-world pedestrian data. We also provide an implementation of a suite of metrics to quantify the performance of navigation algorithms on these scenarios. Altogether, SocNavBench provides a test framework for evaluating disparate social navigation methods in a consistent and interpretable manner. To illustrate its use, we demonstrate testing three existing social navigation methods and a baseline method on SocNavBench, showing how the suite of metrics helps infer their performance trade-offs. Our code is open-source, allowing the addition of new scenarios and metrics by the community to help evolve SocNavBench to reflect advancements in our understanding of social navigation. [ABSTRACT FROM AUTHOR]

This paper presents the results of a three-week in-the-wild deployment of a wizarded service robot in a shared campus workplace. The study introduces robot-centric ethnography, a concept in which a wizarded robot acts as a mediated anthropologist, used in this case, to further our understandings of how service robots impact and integrate into everyday workplace experiences. Our research site included participants familiar with robots, recruited from 90+ students and faculty working in a shared lab space. Our wizarding team visited these participants each workday they were there for three weeks, navigating open office and lab spaces to remind participants to be aware of their mental, physical, and nutritional health needs. Using a semi-structured format, the wizards adapted the standard interaction flow to the situation. This interaction sequence was guided via pre-populated buttons on our health coach interface, with human wizards triggering the timing and adding extra responses as felt natural. Our ethnography-informed approach used the social knowledge of both participants and wizards, blending the robot into the cultural environment in which it was operating. Our data supports the positive impact of fluent service robot experience on participant mood and overall workplace experience. This suggests that effectively designed service robots can benefit workplace environments above and beyond their intended functions. [ABSTRACT FROM AUTHOR]

This article describes a small unmanned aerial system (sUAS) interface that allows untrained responders acting in a Mission Specialist role to effectively coordinate with a Pilot and directly control the UAS payload. Currently, responders view the same interface as the Pilot and give verbal directions for navigation and payload control. Field experiments with 26 experts in chemical, biological, radiological and nuclear response evaluated two interfaces for a chemical train derailment incident staged at Disaster City®. Responder participants had their own role-specific tablet display: one interface passively filtered Pilot-only artifacts from the payload camera video feed; the second interface added responder-specific indicators and information and allowed the participants to actively control the payload pan, tilt, and zoom and to take pictures. A majority of responders reported greater role empowerment for similar tasks using the responder-specific interface. This article demonstrates that a responder-specific interface is preferred by untrained responders instead of viewing the same interface as the Pilot in sUAS. [ABSTRACT FROM AUTHOR]

Background: Although robotic manipulators have great potential in promoting motor independence of people with motor impairments, only few systems are currently commercially available. In addition to technical, economic, and normative barriers, a key challenge for their distribution is the current lack of evidence regarding their usefulness, acceptance, and user-specific requirements. Objective: Against this background, a semiautonomous robot system was developed in the research and development project, robot-assisted services for individual and resource-oriented intensive and palliative care of people with amyotrophic lateral sclerosis (ROBINA), to support people with amyotrophic lateral sclerosis (ALS) in various everyday activities. Methods: The developed early-stage demonstrator was evaluated in a task-based laboratory study of 11 patients with ALS. On the basis of a multimethod design consisting of standardized questionnaires, open-ended questions, and observation protocols, participants were asked about its relevance to everyday life, usability, and design requirements. Results: Most participants considered the system to provide relevant support within the test scenarios and for their everyday life. On the basis of the System Usability Scale, the overall usability of the robot-assisted services for individual and resource-oriented intensive and palliative care of people with ALS system was rated as excellent, with a median of 90 (IQR 75-95) points. Moreover, 3 central areas of requirements for the development of semiautonomous robotic manipulators were identified and discussed: requirements for semiautonomous human-robot collaboration, requirements for user interfaces, and requirements for the adaptation of robotic capabilities regarding everyday life. Conclusions: Robotic manipulators can contribute to increase the autonomy of people with ALS. A key issue for future studies is how the existing ability level and the required robotic capabilities can be balanced to ensure both high user satisfaction and effective and efficient task performance. [ABSTRACT FROM AUTHOR]