Search

Collaborative robotic manipulators can stop in case of a collision, according to the ISO/TS 15066 and ISO 10218-1 standards. However, in a human-robot collaboration scenario where the robot and the human share the workspace, a better solution with concerns to production and operator safety would be to perceive in advance the presence of an obstacle and be able to avoid it, thereby completing the task without halting the robot. In this paper, an obstacle avoidance algorithm is tested using a sensor system for real-time human detection; the operator represents a potential dynamic obstacle that can interfere with the robot motion. The sensor system consists of three RGB-D cameras. A custom software framework has been developed in Python exploiting machine learning tools for human skeleton detection. The coordinates of the human body joints relative to the manipulator base are used as input to the obstacle avoidance algorithm. The use of multiple sensors makes it possible to limit the occlusion problem; in addition, the choice of non-wearable sensors goes in the direction of better operator comfort. A series of experimental tests were performed to verify the accuracy of skeleton detection and the ability of the system to avoid obstacles in real time. The human motion caption system, in particular, was validated through a comparison with a commercial system based on wearable IMU sensors widely used and validated in motion capture. There is an NRMSE of 3.23 % for the RGB-D camera-based skeleton detection system, against an NRMSE of 12.23 % for the IMU wearable sensor system. Test results confirm that the system is able to avoid collisions with the human body under various conditions, static or dynamic, ensuring a minimum safety distance to any part of the manipulator. In 44 tests in which the operator moves around the robot with possible collisions (at a speed typical of manufacturing operations), the minimum operator-robot distance averaged 208 mm , being 200 mm the limit safety distance set by algorithm. [ABSTRACT FROM AUTHOR]

Teleoperation systems find many applications from earlier search-and-rescue to more recent daily tasks. It is widely acknowledged that using external sensors can decouple the view of the remote scene from the motion of the robot arm during manipulation, facilitating the control task. However, this design requires the coordination of multiple operators or may exhaust a single operator as s/he needs to control both the manipulator arm and the external sensors. To address this challenge, our work introduces a viewpoint prediction model, the first data-driven approach that autonomously adjusts the viewpoint of a dynamic camera to assist in telemanipulation tasks. This model is parameterized by a deep neural network and trained on a set of human demonstrations. We propose a contrastive learning scheme that leverages viewpoints in a camera trajectory as contrastive data for network training. We demonstrated the effectiveness of the proposed viewpoint prediction model by integrating it into a real-world robotic system for telemanipulation. User studies reveal that our model outperforms several camera control methods in terms of control experience and reduces the perceived task load compared to manual camera control. As an assistive module of a telemanipulation system, our method significantly reduces task completion time for users who choose to adopt its recommendation. [ABSTRACT FROM AUTHOR]

Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration. [ABSTRACT FROM AUTHOR]

Social signal recognition techniques based on nonverbal behavioral sensing allow conversational robots to understand the user’s social signals, thereby enabling them to adopt interaction strategies based on internal states inferred from the social signals. This research investigates how the online social signal recognition and adaptive dialog strategy influences the dynamic change in a user’s inner state. For this purpose, we develop a semiautonomous interview robot system with an online speaker’s willingness recognition module and an adaptive question selection module based on the willingness level. The online recognition model of speaker willingness is trained from multimodal nonverbal features extracted using a novel interview corpus, which allows appropriate interview questions to be chosen based on the estimated willingness level of the user. We conduct the experiment using the system to evaluate the effectiveness of adaptive question selection based on the willingness recognition model. First, the multimodal willingness recognition model is evaluated using the interview corpus. The best recognition accuracy of willingness level (high or low) was $72. 8\%$ 72. 8 p e r c n t ; with the random forest classifier. Second, 27 interviewees were interviewed with the two interview robot systems: (I) with the adaptive question selection module based on willingness recognition and (II) with a random question selection strategy. The proposed adaptive question strategy significantly increased the number of utterances with high willingness compared with the baseline system (II); thus, adaptive question selection with online willingness recognition elicited the speaker’s willingness even though the model cannot be estimated with near-perfect accuracy. [ABSTRACT FROM AUTHOR]

Assistive robots have the potential to support independence, enhance safety, and lower healthcare costs for older adults, as well as alleviate the demands of their care partners. However, ensuring that these robots will effectively and reliably address end-user needs in the long term requires user-specific design factors to be considered during the robot development process. To identify these design factors, we embedded Stretch, a mobile manipulator created by Hello Robot Inc., in the home of an older adult with motor impairments and his care partner for four weeks to support them with everyday activities. An occupational therapist and a robotics engineer lived with them during this period, employing an immersive participatory design approach to co-design and customise the robot with them. We highlight the benefits of this immersive participatory design experience and provide insights into robot design that can be applied broadly to other assistive technologies. [ABSTRACT FROM AUTHOR]

Reminding is often identified as a central function of socially assistive robots in the healthcare sector. The robotic reminders are supposed to help people with memory impairments to remember to take their medicine, to drink and eat, or to attend appointments. Such standalone reminding technologies can, however, be too demanding for people with memory injuries. In a co-creation process, we developed an individual reminder robot together with a person with traumatic brain injury and her care personnel. During this process, we learned that while current research describe reminding as a prototypical task for socially assistive robots, there is no clear definition of what constitutes a reminder nor that it is based on complex sequences of interactions that evolve over time and space, across different actions, actors and technologies. Based on our data from the co-creation process and the first deployment, we argue for a shift towards a sequential and socially distributed character of reminding. Understanding socially assistive robots as rehabilitative tools for people with memory impairment, they need to be reconsidered as interconnected elements in institutional care practices instead of isolated events for the remindee. [ABSTRACT FROM AUTHOR]

In a human–robot conversation, it is difficult for the robot to start the conversation just when the addressee is ready to listen to the robot, due to recognition technology issues. This paper proposes and evaluates a method to reduce the sense of that the timing of starting the conversation is bad. In this method, two robots perform a cooperative behavior during a waiting time from the call for attracting addressee's attention to the main utterance the robot want to deliver. To evaluate the effectiveness of this method, we conducted an experiment that compared three conversation initiation approaches: early timing and late timing by one robot, and the proposed approach involving two robots. The results revealed that the proposed method mitigates the bad timing of main utterances as perceived by the participants. [ABSTRACT FROM AUTHOR]

Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.

3D Skeleton-based human motion prediction, a classic task in computer vision, aims to forecast subsequent motions based on historical motion observations. In particular, precise short-term motion prediction is crucial for the effectiveness of machines designed for real-time human-computer interaction. This study aims to achieve accurate predictions of human motion within timeframes of less than 400 milliseconds, with the goal of improving machine responsiveness and efficiency. Previous research has predominantly relied on sequence models like Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM) or Transformers. Transformer-based methods have been relied on temporal forecasting ability of Transformer. In this study, instead, we introduce an alternative perspective for Transformer-based models to comprehend the structure of skeletons, named Joint-Aware Transformer (JAT). Within our model, the attention mechanism is employed to encode inter-joint correlation instead of temporal dependencies. Our approach outperformed the state-of-the-art (SOTA) model in short-term prediction on three types of human motion datasets.

Distinguishable vibrotactile feedback can convey information non-verbally and complete the sensory loop when using assistive devices. Feedback can increase acceptance of assistive devices but could require personalization as these devices need to adapt to user capabilities and preferences that may affect the location for inducing feedback. Developing personalized feedback for each user may be ideal, but impractical if demand for these devices increases. In this research, we evaluate the hypothesis that the ability to define, generate, and use personalized feedback is preferred and should be provided. The hypothesis is evaluated using a system capable of capturing and recognizing non-verbal inputs, providing vibrotactile feedback to verify the identified action, and executing the verified action. Evaluation focuses on determining the ability to initiate actions using the system architecture, and define, generate, and use personalized feedback. In addition to recording performance metrics (success or failure), the participants also completed the NASA TLX and post-evaluation questionnaires. Based on results, the actions can be initiated using the system architecture, and defining, generating, and using personalized feedback was found to be preferable with the success rate reaching 100% after only three successive repetitions, and could decrease mental demand and effort while interacting with the system.

The perception of agency in human robot interaction has become increasingly important as robots become more capable and more social. There are, however, no accepted or consistent methods of measuring perceived agency; researchers currently use a wide range of techniques and surveys. We provide a definition of perceived agency, and from that definition we create and psychometrically validate a scale to measure perceived agency. We then perform a scale evaluation by comparing the PA scale constructed in experiment 1 to two other existing scales. We find that our PA and PA-R (Perceived Agency–Rasch) scales provide a better fit to empirical data than existing measures. We also perform scale validation by showing that our scale shows the hypothesized relationship between perceived agency and morality. [ABSTRACT FROM AUTHOR]

Introduction: Underlying mechanisms of speech perception masked by background speakers, a common daily listening condition, are often investigated using various and lengthy psychophysical tests. The presence of a social agent, such as an interactive humanoid NAO robot, may help maintain engagement and attention. However, such robots potentially have limited sound quality or processing speed. Methods: As a first step toward the use of NAO in psychophysical testing of speech-in-speech perception, we compared normal-hearing young adults' performance when using the standard computer interface to that when using a NAO robot to introduce the test and present all corresponding stimuli. Target sentences were presented with colour and number keywords in the presence of competing masker speech at varying target-to-masker ratios. Sentences were produced by the same speaker, but voice differences between the target and masker were introduced using speech synthesis methods. To assess test performance, speech intelligibility and data collection duration were compared between the computer and NAO setups. Human-robot interaction was assessed using the Negative Attitude Toward Robot Scale (NARS) and quantification of behavioural cues (backchannels). Results: Speech intelligibility results showed functional similarity between the computer and NAO setups. Data collection durations were longer when using NAO. NARS results showed participants had a relatively positive attitude toward "situations of interactions" with robots prior to the experiment, but otherwise showed neutral attitudes toward the "social influence" of and "emotions in interaction" with robots. The presence of more positive backchannels when using NAO suggest higher engagement with the robot in comparison to the computer. Discussion: Overall, the study presents the potential of the NAO for presenting speech materials and collecting psychophysical measurements for speech-inspeech perception. [ABSTRACT FROM AUTHOR]

Human-Robot interaction plays a vital role in the service robotics field, especially to support old age-dependent people for socio-economic reasons. In this paper, an indigenous 3D printed 6 DoF robotic arm is proposed to support specially-abled people in their independent-feeding process. The objective of the present paper is to find the combination of optimal positional controllers such as CPID, FC, FPID and FOPID, which can handle the cubic reference input signal and can produce an output signal with minimal overshoot and lesser positional error. The reduced positional error helps the robotic arm to accurately reach the destination with minimal oscillations. This would reduce the wastage of food in the middle of the trajectory as well as at the destination. The technical challenge of the paper is to synchronize machine vision, robot kinematics and trajectory planning with robot control for multiple intermediate points, subjected to the cubic input signal. The feeding robotic arm is equipped with Intel© visual depth camera, which is in synchronization with the microcontroller, servo controller, and servo actuators. Here, six intermediate points were identified in the C-space using FK, among which, fuzzy controller was selected for the first two IPs, the GA optimized FOPID was selected for IP3 to IP5 and FPID was deployed for the last IP. The GA: FOPID produced a negligible overshoot of 0.67%, whereas FC and FPID produced an overshoot of 1.4% and 0.8% respectively. The positional error gap between GA: FOPID, FC and FPID was 0.9, and 0.6 Deg/sec respectively. The selection of combination of optimal controllers helps the manipulator to successfully deliver the food without wasting it. The repeatability of the feeding process is ensured by successfully conducting user testing on 20 users for 25 cycles. [ABSTRACT FROM AUTHOR]

The mindBot robot is a new educational robot toy that can be controlled by brain signals and voice commands. It was evaluated with children with disabilities as well as healthy children as the potential users. The most significant challenge was the size of the used Emotiv Insight electroencephalogram headset when adjusting it on the children's' heads. Despite all the challenges, the mindBot robot is a promising technology that could be fun and educational for disabled children. The 11 participants took 36 minutes to finish all tasks on average. This includes the time they spent setting up the robot for the first time, putting on the headset, learning how to use the robot, and using the main educational features. The System Usability Scale usability score for the robot is 71.13, which is considered to be the score of good. The future stages of improving the mindBot includes adding more mobility capabilities and adding the feature of educational assessment. [ABSTRACT FROM AUTHOR]

Reminding is often identified as a central function of socially assistive robots in the healthcare sector. The robotic reminders are supposed to help people with memory impairments to remember to take their medicine, to drink and eat, or to attend appointments. Such standalone reminding technologies can, however, be too demanding for people with memory injuries. In a co-creation process, we developed an individual reminder robot together with a person with traumatic brain injury and her care personnel. During this process, we learned that while current research describe reminding as a prototypical task for socially assistive robots, there is no clear definition of what constitutes a reminder nor that it is based on complex sequences of interactions that evolve over time and space, across different actions, actors and technologies. Based on our data from the co-creation process and the first deployment, we argue for a shift towards a sequential and socially distributed character of reminding. Understanding socially assistive robots as rehabilitative tools for people with memory impairment, they need to be reconsidered as interconnected elements in institutional care practices instead of isolated events for the remindee.

IntroductionUnderlying mechanisms of speech perception masked by background speakers, a common daily listening condition, are often investigated using various and lengthy psychophysical tests. The presence of a social agent, such as an interactive humanoid NAO robot, may help maintain engagement and attention. However, such robots potentially have limited sound quality or processing speed.MethodsAs a first step toward the use of NAO in psychophysical testing of speech- in-speech perception, we compared normal-hearing young adults’ performance when using the standard computer interface to that when using a NAO robot to introduce the test and present all corresponding stimuli. Target sentences were presented with colour and number keywords in the presence of competing masker speech at varying target-to-masker ratios. Sentences were produced by the same speaker, but voice differences between the target and masker were introduced using speech synthesis methods. To assess test performance, speech intelligibility and data collection duration were compared between the computer and NAO setups. Human-robot interaction was assessed using the Negative Attitude Toward Robot Scale (NARS) and quantification of behavioural cues (backchannels).ResultsSpeech intelligibility results showed functional similarity between the computer and NAO setups. Data collection durations were longer when using NAO. NARS results showed participants had a relatively positive attitude toward “situations of interactions” with robots prior to the experiment, but otherwise showed neutral attitudes toward the “social influence” of and “emotions in interaction” with robots. The presence of more positive backchannels when using NAO suggest higher engagement with the robot in comparison to the computer.DiscussionOverall, the study presents the potential of the NAO for presenting speech materials and collecting psychophysical measurements for speech-in-speech perception.

This systematic literature review presents the latest advancements and insights about digital twin technology and robotics interfaced with extended reality in the context of Industry 4.0. As the extended reality technologies emerge, it results in an increasing overlap between digital twins and human-robot interactions in industrial settings, promoting collaboration between operators and cobots in manufacturing environments. The objective of this study is to serve as a valuable resource for researchers and practitioners working in the field of Industry 4.0. It aims to highlight the latest developments and innovations in the application of digital twins and robotics interfaced with extended reality technologies in manufacturing. By extracting data from relevant articles, it provides a comprehensive understanding of the current state-of-the-art in this field by: analyzing the favored extended reality interfaces for digital twin and robotics interactions; analyzing the digital twin and physical twin interaction; evaluating the digital twin application levels and pillars through extended reality interfacing; and introducing a new concept called augmented perception for creating new physical-digital interactions. [ABSTRACT FROM AUTHOR]

Abstract Nuclear facilities have a regulatory requirement to measure radiation levels within Post Operational Clean Out (POCO) around nuclear facilities each year, resulting in a trend towards robotic deployments to gain an improved understanding during nuclear decommissioning phases. The UK Nuclear Decommissioning Authority supports the view that human‐in‐the‐loop (HITL) robotic deployments are a solution to improve procedures and reduce risks within radiation characterisation of nuclear sites. The authors present a novel implementation of a Cyber‐Physical System (CPS) deployed in an analogue nuclear environment, comprised of a multi‐robot (MR) team coordinated by a HITL operator through a digital twin interface. The development of the CPS created efficient partnerships across systems including robots, digital systems and human. This was presented as a multi‐staged mission within an inspection scenario for the heterogeneous Symbiotic Multi‐Robot Fleet (SMuRF). Symbiotic interactions were achieved across the SMuRF where robots utilised automated collaborative governance to work together, where a single robot would face challenges in full characterisation of radiation. Key contributions include the demonstration of symbiotic autonomy and query‐based learning of an autonomous mission supporting scalable autonomy and autonomy as a service. The coordination of the CPS was a success and displayed further challenges and improvements related to future MR fleets.

Robotics is a cross-disciplinary branch of science and technology, and lays foundation on mechanics, control, computer science, artificial intelligence, and so on. With the developments of both softwares and hardwares, especially in the artificial intelligence technologies, robots have been widely applied in multiple areas in the society, and become more and more interactive in our daily life, such as the service robots in the museums, shopping malls, restaurants, etc. Though the ultimate goal for a service robot to behave like a human is not easy to be achieved, significant processes have been made during the past decades. Considering that it is universal that service robots are triggered to execute tasks specified by human users via commands (Comm-TET), and it is essential to process and understand human users' commands correctly, we comprehensively overview the developments of the task understanding (TU) sub-process of Comm-TET for service robots. In order to organize the related literature in a reasonable manner, we abstracted the pipeline of Comm-TET and the generic framework of TU based on the existing researches. Following the abstracted framework, we present in-depth discussions on each of its building blocks over the past decades, and give some insights on the future research directions. Compared to other reviews on TU, this review emphasizes more on the technical developments and organizes the existing researches as an integrality. [ABSTRACT FROM AUTHOR]