Search

Collaborative robotic manipulators can stop in case of a collision, according to the ISO/TS 15066 and ISO 10218-1 standards. However, in a human-robot collaboration scenario where the robot and the human share the workspace, a better solution with concerns to production and operator safety would be to perceive in advance the presence of an obstacle and be able to avoid it, thereby completing the task without halting the robot. In this paper, an obstacle avoidance algorithm is tested using a sensor system for real-time human detection; the operator represents a potential dynamic obstacle that can interfere with the robot motion. The sensor system consists of three RGB-D cameras. A custom software framework has been developed in Python exploiting machine learning tools for human skeleton detection. The coordinates of the human body joints relative to the manipulator base are used as input to the obstacle avoidance algorithm. The use of multiple sensors makes it possible to limit the occlusion problem; in addition, the choice of non-wearable sensors goes in the direction of better operator comfort. A series of experimental tests were performed to verify the accuracy of skeleton detection and the ability of the system to avoid obstacles in real time. The human motion caption system, in particular, was validated through a comparison with a commercial system based on wearable IMU sensors widely used and validated in motion capture. There is an NRMSE of 3.23 % for the RGB-D camera-based skeleton detection system, against an NRMSE of 12.23 % for the IMU wearable sensor system. Test results confirm that the system is able to avoid collisions with the human body under various conditions, static or dynamic, ensuring a minimum safety distance to any part of the manipulator. In 44 tests in which the operator moves around the robot with possible collisions (at a speed typical of manufacturing operations), the minimum operator-robot distance averaged 208 mm , being 200 mm the limit safety distance set by algorithm. [ABSTRACT FROM AUTHOR]

In a human–robot conversation, it is difficult for the robot to start the conversation just when the addressee is ready to listen to the robot, due to recognition technology issues. This paper proposes and evaluates a method to reduce the sense of that the timing of starting the conversation is bad. In this method, two robots perform a cooperative behavior during a waiting time from the call for attracting addressee's attention to the main utterance the robot want to deliver. To evaluate the effectiveness of this method, we conducted an experiment that compared three conversation initiation approaches: early timing and late timing by one robot, and the proposed approach involving two robots. The results revealed that the proposed method mitigates the bad timing of main utterances as perceived by the participants. [ABSTRACT FROM AUTHOR]

Human-Robot interaction plays a vital role in the service robotics field, especially to support old age-dependent people for socio-economic reasons. In this paper, an indigenous 3D printed 6 DoF robotic arm is proposed to support specially-abled people in their independent-feeding process. The objective of the present paper is to find the combination of optimal positional controllers such as CPID, FC, FPID and FOPID, which can handle the cubic reference input signal and can produce an output signal with minimal overshoot and lesser positional error. The reduced positional error helps the robotic arm to accurately reach the destination with minimal oscillations. This would reduce the wastage of food in the middle of the trajectory as well as at the destination. The technical challenge of the paper is to synchronize machine vision, robot kinematics and trajectory planning with robot control for multiple intermediate points, subjected to the cubic input signal. The feeding robotic arm is equipped with Intel© visual depth camera, which is in synchronization with the microcontroller, servo controller, and servo actuators. Here, six intermediate points were identified in the C-space using FK, among which, fuzzy controller was selected for the first two IPs, the GA optimized FOPID was selected for IP3 to IP5 and FPID was deployed for the last IP. The GA: FOPID produced a negligible overshoot of 0.67%, whereas FC and FPID produced an overshoot of 1.4% and 0.8% respectively. The positional error gap between GA: FOPID, FC and FPID was 0.9, and 0.6 Deg/sec respectively. The selection of combination of optimal controllers helps the manipulator to successfully deliver the food without wasting it. The repeatability of the feeding process is ensured by successfully conducting user testing on 20 users for 25 cycles. [ABSTRACT FROM AUTHOR]

This systematic literature review presents the latest advancements and insights about digital twin technology and robotics interfaced with extended reality in the context of Industry 4.0. As the extended reality technologies emerge, it results in an increasing overlap between digital twins and human-robot interactions in industrial settings, promoting collaboration between operators and cobots in manufacturing environments. The objective of this study is to serve as a valuable resource for researchers and practitioners working in the field of Industry 4.0. It aims to highlight the latest developments and innovations in the application of digital twins and robotics interfaced with extended reality technologies in manufacturing. By extracting data from relevant articles, it provides a comprehensive understanding of the current state-of-the-art in this field by: analyzing the favored extended reality interfaces for digital twin and robotics interactions; analyzing the digital twin and physical twin interaction; evaluating the digital twin application levels and pillars through extended reality interfacing; and introducing a new concept called augmented perception for creating new physical-digital interactions. [ABSTRACT FROM AUTHOR]

In this research, a new remote rehabilitation system was developed that integrates an IoT-based connected robot intended for wrist and forearm rehabilitation. In fact, the mathematical model of the wrist and forearm joints was developed and integrated into the main controller. The proposed new rehabilitation protocol consists of three main sessions: the first is dedicated to the extraction of the passive components of the dynamic model of wrist–forearm biomechanics while the active components are extracted in the second session. The third session consists of performing continuous exercises using the determined dynamic model of the forearm–wrist joints, taking into account the torque generated by muscle fatigue. The main objective of this protocol is to determine the state level of the affected wrist and above all to provide a dynamic model in which the torque generated by the robot and the torque supplied by the patient are combined, taking into account the constraints of fatigue. A Support Vector Machine (SVM) classifier is designed for the estimation of muscle fatigue based on the features extracted from the electromyography (EMG) signal acquired from the patient. The results show that the developed rehabilitation system allows a good progression of the joint's range of motion as well as the resistive-active torques. [ABSTRACT FROM AUTHOR]

Integrating Artificial Intelligence (AI) and drone technology into bridge inspections offers numerous advantages, including increased efficiency and enhanced safety. However, it is essential to recognize that this integration changes the cognitive ergonomics of the inspection task. Gaining a deeper understanding of how humans process information and behave when collaborating with drones and AI systems is necessary for designing and implementing effective AI-assisted inspection drones. To further understand human-drone-AI intricate dynamics, an experiment was conducted in which participants' biometric and behavioral data were collected during a simulated drone-enabled bridge inspection under two conditions: with an 80% accurate AI assistance and with no AI assistance. Results indicate that cognitive and behavioral factors, including vigilance, cognitive processing intensity, gaze patterns, and visual scanning efficiency can influence inspectors' performance respectively in either condition. This highlights the importance of designing inspection protocols, drones and AI systems based on a comprehensive understanding of the cognitive processes required in each condition to prevent cognitive overload and minimize errors. We also remark on the visual scanning and gaze patterns associated with a higher chance of missing critical information in each condition, insights that inspectors can use to enhance their inspection performance. [ABSTRACT FROM AUTHOR]

This paper explores a framework for defining artificial intelligence (AI) that adapts to individuals within a group, and discusses the technical challenges for collaborative AI systems that must work with different human partners. Collaborative AI is not one‐size‐fits‐all, and thus AI systems must tune their output based on each human partner's needs and abilities. For example, when communicating with a partner, an AI should consider how prepared their partner is to receive and correctly interpret the information they are receiving. Forgoing such individual considerations may adversely impact the partner's mental state and proficiency. On the other hand, successfully adapting to each person's (or team member's) behavior and abilities can yield performance benefits for the human–AI team. Under this framework, an AI teammate adapts to human partners by first learning components of the human's decision‐making process and then updating its own behaviors to positively influence the ongoing collaboration. This paper explains the role of this AI adaptation formalism in dyadic human–AI interactions and examines its application through a case study in a simulated navigation domain. [ABSTRACT FROM AUTHOR]

This research presents the technical considerations for implementing the CeCi (Computer Electronic Communication Interface) social robot. In this case, this robot responds to the need to achieve technological development in an emerging country with the aim of social impact and social interaction. There are two problems with the social robots currently on the market, which are the main focus of this research. First, their costs are not affordable for companies, universities, or individuals in emerging countries. The second is that their design is exclusively oriented to the functional part with a vision inherent to the engineers who create them without considering the vision, preferences, or requirements of the end users, especially for their social interaction. This last reason ends causing an aversion to the use of this type of robot. In response to the issues raised, a low-cost prototype is proposed, starting from a commercial platform for research development and using open source code. The robot design presented here is centered on the criteria and preferences of the end user, prioritizing acceptability for social interaction. This article details the selection process and hardware capabilities of the robot. Moreover, a programming section is provided to introduce the different software packages used and adapted for the social interaction, the main functions implemented, as well as the new and original part of the proposal. Finally, a list of applications currently developed with the robot and possible applications for future research are discussed. [ABSTRACT FROM AUTHOR]

The present research aims to model, simulate and implement a new hybrid control approach based on a combination of proportional integral derivative (PID) Controller and Model Reference Adaptive Controller (MRAC), in which Lyapunov's theory is used to ensure asymptotic stability to control a two degrees of freedom (DoF) manipulator driven by McKibben's artificial pneumatic muscles. The MRAC controller works as a nonlinearity compensator and PID controller works during the transient period, as the MRAC performs poorly in this regime. This new approach is entitled Hybrid Model Reference Adaptive Controller (H-MRAC) and it has an unprecedented topological structure based on three terms. The feedforward term acts in disturbances rejection, the derivative term in oscillations damping and the feedback term acts in error convergence to zero. In this article, a control system dedicated to pneumatic manipulators was developed. As a result, proof of asymptotic convergence was performed for the proposed topological approach, which was validated on a two DoF manipulator. The proposed mechanism satisfactorily met the ISO/TS 15066 standard, and the position tracking obtained a global error of 37.69% and 51.01% smaller than found in the literature examples, entitled MRAC and A-PID, respectively, for simulations and 37.46% and 30.25% for experiments. [ABSTRACT FROM AUTHOR]

A collaborative mobile robot has been developed for tea plucking in narrow terrain tea gardens. The robot executes stable side-by-side motions and carries most of the load of the tea harvesting machine. The robot is controlled with adjustable autonomy so that the human can provide a supervisory role and adjust the optimal height and forward speed of the cutting tool. During tea plucking, the robot can autonomously detect the tree rows and avoid colliding with the trees. The experimental results show that the proposed collaborative robot can stably co-work with humans and significantly improve working efficiency and comfort. [ABSTRACT FROM AUTHOR]

To achieve continuous frail care in the daily lives of the elderly, we propose AHOBO, a frail care robot for the elderly at home. Two types of support systems by AHOBO were implemented to support the elderly in both physical health and psychological aspects. For physical health frailty care, we focused on blood pressure and developed a support system for blood pressure measurement with AHOBO. For psychological frailty care, we implemented reminiscent coloring with the AHOBO as a recreational activity with the robot. The usability of the system was evaluated based on the assumption of continuous use in daily life. For the support system in blood pressure measurement, we performed a qualitative evaluation using a questionnaire for 16 subjects, including elderly people under blood pressure measurement by the system. The results confirmed that the proposed robot does not affect the blood pressure readings and is acceptable in terms of ease of use based on subjective evaluation. For the reminiscent coloring interaction, subjective evaluation was conducted on two elderly people under the verbal fluency task, and it has been confirmed that the interaction can be used continuously in daily life. The widespread use of the proposed robot as an interface for AI that supports daily life will lead to a society in which AI robots support people from the cradle to the grave. [ABSTRACT FROM AUTHOR]

• Novel approach to identity verification in Human-Robot interaction. • Identity verification produced from both sound (voice) and image (face) inputs. • Proposed approaches focus on identifying the parts of the signal that contains the high value information. • Through analysis of regions of interest substantial improvements are obtained over the state of the art. • Evaluation of performance is obtained in a complex dataset that portrays real life scenarios. In this paper, we tackle the task of improving biometric verification in the context of Human-Robot Interaction (HRI). A robot that wants to identify a specific person to provide a service can do so by either image verification or, if light conditions are not favourable, through voice verification. In our approach, we will take advantage of the possibility a robot has of recovering further data until it is sure of the identity of the person. The key contribution is that we select from both image and audio signals the parts that are of higher confidence. For images we use a system that looks at the face of each person and selects frames in which the confidence is high while keeping those frames separate in time to avoid using very similar facial appearance. For audio our approach tries to find the parts of the signal that contain a person talking, avoiding those in which noise is present by segmenting the signal. Once the parts of interest are found, each input is described with an independent deep learning architecture that obtains a descriptor for each kind of input (face/voice). We also present in this paper fusion methods that improve performance by combining the features from both face and voice, results to validate this are shown for each independent input and for the fusion methods. [ABSTRACT FROM AUTHOR]

Human intention prediction is of great significance in many applications, such as human-robot interaction, intelligent rehabilitation robots. This paper studies the problem of short-term next-active-object prediction in egocentric images. The short-term next-active-object refers to the object that a human is going to interact with in the short-term future, which is an embodiment of human intention. Most current methods usually use object-centered cues, such as the deviation of object appearance change and the unique shape of the egocentric object trajectory, to predict the next-active-object. In this paper, inspired by the fact that human intention is also revealed by human-centered cues, we propose a deep neural network model that integrates the cues from visual attention and hand positions to predict the next-active-object. Firstly, the probability maps of visual attention and hand positions are constructed, and then the probability distribution of next-active-object is generated. We experimentally compare our method with several baseline methods using two datasets and confirm its effectiveness. In addition, ablation experiments are conducted, and crucial points concerning the next-active-object are discussed. [ABSTRACT FROM AUTHOR]

In this paper, a robotic system dedicated to remote wrist rehabilitation is proposed as an Internet of Things (IoT) application. The system offers patients home rehabilitation. Since the physiotherapist and the patient are on different sites, the system guarantees that the physiotherapist controls and supervises the rehabilitation process and that the patient repeats the same gestures made by the physiotherapist. A human-machine interface (HMI) has been developed to allow the physiotherapist to remotely control the robot and supervise the rehabilitation process. Based on a computer vision system, physiotherapist gestures are sent to the robot in the form of control instructions. Wrist range of motion (RoM), EMG signal, sensor current measurement, and streaming from the patient's environment are returned to the control station. The various acquired data are displayed in the HMI and recorded in its database, which allows later monitoring of the patient's progress. During the rehabilitation process, the developed system makes it possible to follow the muscle contraction thanks to an extraction of the Electromyography (EMG) signal as well as the patient's resistance thanks to a feedback from a current sensor. Feature extraction algorithms are implemented to transform the EMG raw signal into a relevant data reflecting the muscle contraction. The solution incorporates a cascade fuzzy-based decision system to indicate the patient's pain. As measurement safety, when the pain exceeds a certain threshold, the robot should stop the action even if the desired angle is not yet reached. Information on the patient, the evolution of his state of health and the activities followed, are all recorded, which makes it possible to provide an electronic health record. Experiments on 3 different subjects showed the effectiveness of the developed robotic solution. [ABSTRACT FROM AUTHOR]

Recently, considerable research has focused on personal assistant robots, and robots capable of rich human-like communication are expected. Among humans, non-verbal elements contribute to effective and dynamic communication. However, people use a wide range of diverse gestures, and a robot capable of expressing various human gestures has not been realized. In this study, we address human behavior modeling during interaction using a deep generative model. In the proposed method, to consider interaction motion, three factors, i.e., interaction intensity, time evolution, and time resolution, are embedded in the network structure. Subjective evaluation results suggest that the proposed method can generate high-quality human motions. [ABSTRACT FROM AUTHOR]

• A driver assistance system based on driver cognitive parameters and driving style is proposed. • Deep neural networks-based approaches are used to detect human cognitive parameters. • Driving style assessment is done based on traffic sign detection and vehicle speed analysis. • A fuzzy-based approach is used to combine the whole information about driver behavior. Reducing the number of traffic accidents due to human errors is an urgent need in several countries around the world. In this scenario, the use of human-robot interaction (HRI) strategies has recently shown to be a feasible solution to compensate human limitations while driving. In this work we propose a HRI system which uses the driver's cognitive factors and driving style information to improve safety. To achieve this, deep neural networks based approaches are used to detect human cognitive parameters such as sleepiness, driver's age and head posture. Additionally, driving style information is also obtained through speed analysis and external traffic information. Finally, a fuzzy-based decision-making stage is proposed to manage both human cognitive information and driving style, and then limit the maximum allowed speed of a vehicle. The results showed that we were able to detect human cognitive parameters such as sleepiness –63% to 88% accuracy–, driver's age –80% accuracy– and head posture –90.42% to 97.86% accuracy– as well as driving style –87.8% average accuracy. Based on such results, the fuzzy-based architecture was able to limit the maximum allowed speed for different scenarios, reducing it from 50 km/h to 17 km/h. Moreover, the fuzzy-based method showed to be more sensitive with respect to inputs changes than a previous published weighted-based inference method. [ABSTRACT FROM AUTHOR]

Robots have the potential to facilitate the future education of all generations, particularly children. However, existing robots are limited in their ability to automatically perceive and respond to a human emotional states. We hypothesize that these sophisticated models suffer from individual differences in human personality. Therefore, we proposed a multi-characteristic model architecture that combines personalized machine learning models and utilizes the prediction score of each model. This architecture is formed with reference to an ensemble machine learning architecture. In this study, we presented a method for calculating the weighted average in a multi-characteristic architecture by using the similarities between a new sample and the trained characteristics. We estimated the degree of confidence during a communication as a human internal state. Empirical results demonstrate that using the multi-model training of each person's information to account for individual differences provides improvements over a traditional machine learning system and insight into dealing with various individual differences. [ABSTRACT FROM AUTHOR]

BACKGROUND: People with severe speech and motor impairment (SSMI) often uses a technique called eye pointing to communicate with outside world. One of their parents, caretakers or teachers hold a printed board in front of them and by analyzing their eye gaze manually, their intentions are interpreted. This technique is often error prone and time consuming and depends on a single caretaker. OBJECTIVE: We aimed to automate the eye tracking process electronically by using commercially available tablet, computer or laptop and without requiring any dedicated hardware for eye gaze tracking. The eye gaze tracker is used to develop a video see through based AR (augmented reality) display that controls a robotic device with eye gaze and deployed for a fabric printing task. METHODS: We undertook a user centred design process and separately evaluated the web cam based gaze tracker and the video see through based human robot interaction involving users with SSMI. We also reported a user study on manipulating a robotic arm with webcam based eye gaze tracker. RESULTS: Using our bespoke eye gaze controlled interface, able bodied users can select one of nine regions of screen at a median of less than 2 secs and users with SSMI can do so at a median of 4 secs. Using the eye gaze controlled human-robot AR display, users with SSMI could undertake representative pick and drop task at an average duration less than 15 secs and reach a randomly designated target within 60 secs using a COTS eye tracker and at an average time of 2 mins using the webcam based eye gaze tracker. CONCLUSION: The proposed system allows users with SSMI to manipulate physical objects without any dedicated eye gaze tracker. The novelty of the system is in terms of non-invasiveness as earlier work mostly used glass based wearable trackers or head/face tracking but no other earlier work reported use of webcam based eye tracking for controlling robotic arm by users with SSMI. [ABSTRACT FROM AUTHOR]

Humanoid robots should play an increasing role in hospitality and tourism services. Anthropomorphic – human like – characteristics seem critical component to consumers accepting robotic service (rService). This conceptual manuscript advances rService research by drawing on services marketing, Human Robot Interaction (HRI) and the Uncanny Valley Theory to explore anthropomorphic characteristics' range, role and impact on rService experiences. The paper proposes eleven robot capabilities that influence anthropomorphism and consequently shape HRI, three Uncanny Valley marketing outcomes, theoretical concepts, and a rich future research agenda. Hospitality and tourism literature and examples highlight the service context's importance when researching, adopting, implementing and marketing rServices. [ABSTRACT FROM AUTHOR]

Objective: Modeling of human motor intention plays an essential role in predictively controlling a robotic system in human–robot interaction tasks. In most machine learning techniques, human motor behavior is modeled as a generic stochastic process. However, the integration of a priori knowledge about underlying system structures can provide insights on otherwise unobservable intrinsic states that yield the superior prediction performance and increased generalization capabilities. Methods: We present a novel method for modeling human motor behavior that explicitly includes a neuroscientifically supported model of human motor control, in which the dynamics of the human arm are modeled by a mechanical impedance that tracks a latent desired trajectory. We adopt a Bayesian setting by defining Gaussian process (GP) priors for the impedance elements and the latent desired trajectory. This enables exploitation of a priori human arm impedance knowledge for regression of interaction forces through inference of a latent desired human trajectory. Results: The method is validated using simulated data, with particular focus on effects of GP prior parameterization and intention estimation capabilities. The superior prediction performance is shown with respect to a naive GP prior. An experiment with human participants evaluates generalization capabilities and effects of training data sparsity. Conclusion: We derive the correlations of an impedance-based GP model of human motor behavior that exploits a priori knowledge. Significance: The model effectively predicts interaction forces by inferring a latent desired human trajectory in previously observed as well as unobserved regions of the input space. [ABSTRACT FROM AUTHOR]

Gait assistance robots are used to improve gait performance ability or perform gait motion with an assistance for several articular motions. The sparing use of a gait assistance robot to decrease the duration of the robot's assistance is important for keeping the ability to perform a movement when the robot assists walking. In previous research, methods of ensuring a compliance mechanism and control method have been studied, and assistance for articular motions has been conducted independently using actuators corresponding to each articular motion. In this paper, we propose a wire-driven gait assistance robot to aid both hip and knee articular flexionmotions by applying just one force to assist motion in the swing phase. We focused on a force that assists hip and knee flexion motion, and designed a robot with a compensation mechanism for the wire length. We used an assistance timing detection method for the robot, conducting tensile force control based on information from the hip, knee, and ankle angles. We carried out an experiment to investigate the controlled performance of the proposed robot and the effect on hip and knee angular velocity. We confirmed that the proposed robotic system can aid both hip and knee articular motion with just one force application. [ABSTRACT FROM AUTHOR]

Due to the degeneration of musculoskeletal structure and strength, the elderly population is facing significant mobility challenges. Walkers are widely used for mobility-impaired people. In this research, a smart walker has been developed by using a hybrid motion model (HMM) and a machine learning model based on the vertical interaction force to match the velocity between the walker and its user, which can reduce the horizontal interaction force thus to help lower the operational effort. It is equipped with only force sensors on the handlebars as a human-robot interface. The human motion is modeled by using the HMM in which an inverted pendulum model is used in the single support phase and a constant velocity model is used in the double support phase. A novel gait phase detector is built based on a machine learning method to correlate the force information and the walking gait phases. Experiments with three subjects are carried out to verify the effectiveness of the gait phase detection method and the hybrid motion model. The results demonstrate that the walker can accurately identify gait phases with more than 90 % accuracy and the horizontal interaction force is reduced to around 50 % of that under the constant velocity model and power-off condition when using the HMM during walking. [ABSTRACT FROM AUTHOR]

Abstract This paper presents a multimodal human-robot interaction based on fusion of speech and gesture. In the interface, a robot control command system is designed, which can transform the speech and gesture of users into commands that the robot can execute. Microsoft speech SDK is used in this system to collect the speech of the operator. Then, a corpus-based algorithm of maximum entropy classification for natural language understanding is employed to generate commands. Leap Motion is employed to capture the gesture of operator in this system. Interval Kalman Filter (IKF) is used to estimate the measured data to reduce the inherent noise of the sensor. The advantage of the proposed method is that the combination of speech and gesture makes the human-robot interaction more convenient and direct. Finally, a series of experiments were carried out to validate our method, and proved that it performed better than the other proposed methods. [ABSTRACT FROM AUTHOR]

Social robots have been and are currently being used in many projects, research initiatives and experiments, but we know relatively little about them compared to humans when performing a social task such as teaching. Using an experiment in which a robot and a human teacher were used for teaching computational concepts to a group of K-12 students, the main goal of this paper is not to analyze the scores obtained in the post-test performed, but to focus on the students' attitudes towards robots. In order to do this, a version of the NARS and RAS questionnaires, adapted for children, was used. The analysis of the results of these questionnaires considers differences between age groups and students lectured by a robot vs. a human teacher. We conclude that age is the main factor that affects students’ attitudes towards robots, although we also found other differences between the robot and the human teacher group. [ABSTRACT FROM AUTHOR]

Cross-ratio invariant is used widely in projective transformations for eye gaze estimation. Establishing a virtual plane projection is an important step to use this property. Most of traditional cross-ratio approaches only used fixed parameters to calculate the gazing point. This paper proposes gazing point dependent eye gazing estimation approach. Our contributions are three-folded. First, we model a dynamic virtual plane projection, which is tangent to the cornea of pupil, to estimate the position of the gazing point. Second, we introduce a two stage approach consisting of rough-to-precise framework for gazing point estimation based on the gazing point dependent virtual plane projection. Third, a heuristic strategy which contains off-line and on-line parameter learning for gazing point estimation is proposed. The experiment results show that our approach can significantly improve the gazing estimation performance with an average accuracy of 0.70°. [ABSTRACT FROM AUTHOR]

In this paper, we propose a continuous hand gesture recognition method based on trajectory shape information. A key issue in recognizing continuous gestures is that performance of conventional recognition algorithms may be lowered by such factors as, unknown start and end points of a gesture or variations in gesture duration. These issues become particularly difficult for those methods that rely on temporal information. To alleviate the issues of continuous gesture recognition, we propose a framework that simultaneously performs both segmentation and recognition. Each component of the framework applies shape-based information to ensure robust performance for gestures with large temporal variation. A gesture trajectory is divided by a set of key frames by thresholding its tangential angular change. Variable-sized trajectory segments are then generated using the selected key frames. For recognition, these trajectory segments are examined to determine whether the segment belongs to a class among intended gestures or a non-gesture class based on fusion of shape information and temporal features. In order to assess performance, the proposed algorithm was evaluated with a database of digit hand gestures. The experimental results indicate that the proposed algorithm has a high recognition rate while maintaining its performance in the presence of continuous gestures. [ABSTRACT FROM AUTHOR]

Keeping a living and working spaces tidy is very important for healthy daily life. The efficiency in working rises when the office is tidy. We developed a system that tidies up through the cooperation between a robot and a human. In this study, we investigated the influence of a robot's behavior on the motivation of tidying up. For completing this system, it is necessary to investigate the effective behaviors that encourages a human. To validate what behavior effectively encourage human to tidy up, we conducted a preliminary experiment with 13 male and 5 female participants, aged 20-23. We found the statistically significant difference between the cases where the robot took actions or not. [ABSTRACT FROM AUTHOR]

The promise of robots assisting humans in everyday tasks has led to a variety of research questions and challenges in human-robot collaboration. Here, we address the question of whether and when a robot should take initiative during joint human-robot task execution. We designed a robotic system capable of autonomously performing table-top manipulation tasks while monitoring the environmental state. Our system is able to predict future environmental states and the robot’s actions to reach them using a dynamic Bayesian network. To evaluate our system, we implemented three different initiative conditions to trigger the robot’s actions. Human-initiated help gives control of the robot action timing to the user; robot-initiated reactive help triggers robot assistance when it detects that the human needs help; robot-initiated proactive help makes the robot help whenever it can. We performed a user study (N=18) to compare the trigger mechanisms in terms of quality of interaction, system performance and perceived sociality of the robot. We found that people collaborate best with a proactive robot, yielding better team fluency and high subjective ratings. However, they prefer having control of when the robot should help, rather than working with a reactive robot that only helps when needed. We also found that participants gazed at the robot’s face more during the human-initiated help compared to the other conditions. This shows that asking for the robot’s help may lead to a more “social” interaction, without improving the quality of interaction or the system performance. [ABSTRACT FROM AUTHOR]

Whilst robots are increasingly being deployed as social agents, it is still difficult to program them to interact socially. To create usable tools for programming these robots, tool developers need to know what abstraction levels are appropriate for programming social robot applications. We explore this through the iterative design and evaluation of an API for programming social robots. The results show that high level primitives, with a close mapping to social interaction, are suitable for programming social robot applications. However, the abstraction level should not be so high that it takes away too much control from programmers. This has the potential to enable programmers to produce high quality social robot applications with less programming effort. [ABSTRACT FROM AUTHOR]

The mirror-neuron-system (MNS) is involved in the perception of actions of humans and anthropomorphic robots. The current study investigates whether social interaction with a non-anthropomorphic robot is sufficient for a response of the MNS. Fifty-seven participants observed movements of a vacuum cleaning robot before and after it was handled by its owner. The robot was either humanized, being treated aggressively ( n = 30), or it was treated as an object ( n = 27). Electroencephalographic mu-activity is used as an index of MNS activity, because both are inversely correlated. Activity within the 8–13 Hz band was measured at central (mu-activity) and occipital (alpha-activity) electrodes. Further, the level of aggressiveness displayed by the robot's owner, and the participants' compassion were rated on visual analog scales. Mu-activity showed medium-sized correlations with rated aggressiveness and compassion: The more aggressive the action towards the robot was perceived ( r = −.379, p = .004), and the more compassion was felt for the robot ( r = −.339, p = .010), the less pronounced mu-activity was at electrode C3 in response to the robot's movement. Thus social interaction with a non-anthropomorphic robot might establish the robot as a social entity and is sufficient to activate the human MNS. [ABSTRACT FROM AUTHOR]

Emotional state recognition has become an important topic for human–robot interaction in the past years. By determining emotion expressions, robots can identify important variables of human behavior and use these to communicate in a more human-like fashion and thereby extend the interaction possibilities. Human emotions are multimodal and spontaneous, which makes them hard to be recognized by robots. Each modality has its own restrictions and constraints which, together with the non-structured behavior of spontaneous expressions, create several difficulties for the approaches present in the literature, which are based on several explicit feature extraction techniques and manual modality fusion. Our model uses a hierarchical feature representation to deal with spontaneous emotions, and learns how to integrate multiple modalities for non-verbal emotion recognition, making it suitable to be used in an HRI scenario. Our experiments show that a significant improvement of recognition accuracy is achieved when we use hierarchical features and multimodal information, and our model improves the accuracy of state-of-the-art approaches from 82.5% reported in the literature to 91.3% for a benchmark dataset on spontaneous emotion expressions. [ABSTRACT FROM AUTHOR]

A pet cat or dog’s ability to respond to our emotional state opens an interaction channel with high visceral impact, which social robots may also be able to access. Touch is a key but understudied element; here, we explore its emotional content in the context of a furry robot pet. We asked participants to imagine feeling nine emotions located in a 2-D arousal-valence affect space, then to express them by touching a lap-sized robot prototype equipped with pressure sensors and accelerometer. We found overall correct classification (Random Forests) within the 2-D grid of 36% (all participants combined) and 48% (average of participants classified individually); chance 11%. Rates rose to 56% in the high arousal zone. To better understand classifier performance, we defined and analyzed new metrics that better indicate closeness of the gestural expressions. We also present a method to combine direct affect recognition with affect inferred from gesture recognition. This analysis provides a unique first insight into the nature and quality of affective touch, with implications as a design tool and for incorporating unintrusive affect sensing into deployed interactions. [ABSTRACT FROM AUTHOR]

This paper presents the design and implementation of a novel joint-based magnetic actuator for robotic manipulators using variable stiffness mechanism. This new actuator prototype, namely M2-VSA, is developed as an improvement over previous actuators in its class. The robotic arm with M2-VSA joint can safely interact with human operators and work in hazardous conditions. The main novelty in this work is the use of magnetic force in accompanying with springs in a novel arrangement to adjust the joints’ stiffness and mitigate the effect of impact. The theoretical framework and advantages of the design are discussed in detail. Experimental studies through various tests show the effectiveness and efficiency of mechanical–magnetic joint as a safe mechanism in environments where human and robotics have interaction. [ABSTRACT FROM AUTHOR]

The field of Human Robot Interaction (HRI) encompasses many difficult challenges as robots need a better understanding of human actions. Human detection and tracking play a major role in such scenarios. One of the main challenges is to track them with long term occlusions due to agile nature of human navigation. However, in general humans do not make random movements. They tend to follow common motion patterns depending on their intentions and environmental/physical constraints. Therefore, knowledge of such common motion patterns could allow a robotic device to robustly track people even with long term occlusions. On the other hand, once a robust tracking is achieved, they can be used to enhance common motion pattern models allowing robots to adapt to new motion patterns that could appear in the environment. Therefore, this paper proposes to learn human motion patterns based on Sampled Hidden Markov Model (SHMM) and simultaneously track people using a particle filter tracker. The proposed simultaneous people tracking and human motion pattern learning has not only improved the tracking robustness compared to more conservative approaches, it has also proven robustness to prolonged occlusions and maintaining identity. Furthermore, the integration of people tracking and on-line SHMM learning have led to improved learning performance. These claims are supported by real world experiments carried out on a robot with suite of sensors including a laser range finder. [ABSTRACT FROM AUTHOR]

In the last decade, there have been significant developments into integration of robots and automation tools with brachytherapy delivery systems. These systems aim to improve the current paradigm by executing higher precision and accuracy in seed placement, improving calculation of optimal seed locations, minimizing surgical trauma, and reducing radiation exposure to medical staff. Most of the applications of this technology have been in the implantation of seeds in patients with early-stage prostate cancer. Nevertheless, the techniques apply to any clinical site where interstitial brachytherapy is appropriate. In consideration of the rapid developments in this area, the American Association of Physicists in Medicine (AAPM) commissioned Task Group 192 to review the state-of-the-art in the field of robotic interstitial brachytherapy. This is a joint Task Group with the Groupe Europeen de Curietherapie-European Society for Radiotherapy & Oncology (GEC-ESTRO). All developed and reported robotic brachytherapy systems were reviewed. Commissioning and quality assurance procedures for the safe and consistent use of these systems are also provided. Manual seed placement techniques with a rigid template have an estimated in vivo accuracy of 3-6 mm. In addition to the placement accuracy, factors such as tissue deformation, needle deviation, and edema may result in a delivered dose distribution that differs from the preimplant or intraoperative plan. However, real-time needle tracking and seed identification for dynamic updating of dosimetry may improve the quality of seed implantation. The AAPM and GEC-ESTRO recommend that robotic systems should demonstrate a spatial accuracy of seed placement ≤1.0 mm in a phantom. This recommendation is based on the current performance of existing robotic brachytherapy systems and propagation of uncertainties. During clinical commissioning, tests should be conducted to ensure that this level of accuracy is achieved. These tests should mimic the real operating procedure as closely as possible. Additional recommendations on robotic brachytherapy systems include display of the operational state; capability of manual override; documented policies for independent check and data verification; intuitive interface displaying the implantation plan and visualization of needle positions and seed locations relative to the target anatomy; needle insertion in a sequential order; robot-clinician and robot-patient interactions robustness, reliability, and safety while delivering the correct dose at the correct site for the correct patient; avoidance of excessive force on radioactive sources; delivery confirmation of the required number or position of seeds; incorporation of a collision avoidance system; system cleaning, decontamination, and sterilization procedures. These recommendations are applicable to end users and manufacturers of robotic brachytherapy systems. [ABSTRACT FROM AUTHOR]

This paper presents a multi-sensor humanoid robotic head for human robot interaction. The design of the robotic head, Muecas, is based on ongoing research on the mechanisms of perception and imitation of human expressions and emotions. These mechanisms allow direct interaction between the robot and its human companion through the different natural language modalities: speech, body language and facial expressions. The robotic head has 12 degrees of freedom, in a human-like configuration, including eyes, eyebrows, mouth and neck, and has been designed and built entirely by IADeX (Engineering, Automation and Design of Extremadura) and RoboLab. A detailed description of its kinematics is provided along with the design of the most complex controllers. Muecas can be directly controlled by FACS (Facial Action Coding System), the de facto standard for facial expression recognition and synthesis. This feature facilitates its use by third party platforms and encourages the development of imitation and of goal-based systems. Imitation systems learn from the user, while goal-based ones use planning techniques to drive the user towards a final desired state. To show the flexibility and reliability of the robotic head, the paper presents a software architecture that is able to detect, recognize, classify and generate facial expressions in real time using FACS. This system has been implemented using the robotics framework, RoboComp, which provides hardware-independent access to the sensors in the head. Finally, the paper presents experimental results showing the real-time functioning of the whole system, including recognition and imitation of human facial expressions. [ABSTRACT FROM AUTHOR]

We aim to achieve interaction between a robot and multiple people. For this, robots should localize people, select an interaction partner, and act appropriately for him/her. It is difficult to deal with all these problems using only the sensors installed into the robots. We focus on that people use a rough interaction distance among other people . We divide this interaction area into different spaces based on both the interaction distances and sensor abilities of robots. Our robots localize people roughly within this divided space. To select an interaction partner, they map friendliness holding the interaction history onto the divided space, and integrate the sensor information. Furthermore, we developed a method for appropriately changing the motions, sizes, and speeds based on the distance. Our robots regard the divided spaces as Q-Learning states, and learn the motion parameters. Our robot interacted with 27 visitors. It localized a partner with an F-value of 0.76 through integration, which is higher than that of a single sensor. A factor analysis was performed on the results from questionnaires. Exciting and Friendly were the representatives of the first and second factors, respectively. For both factors, a motion with friendliness provided higher impression scores than that without friendliness. [ABSTRACT FROM PUBLISHER]

Highlights: [•] The work employs anatomical truth of facial behaviour to recognise seven emotions. [•] 17 Action Units are recognised using upper and lower action analysers. [•] A facial emotion recogniser uses the derived actions to further decode emotions. [•] Latent Semantic Analysis is used to identify topics beyond linguistic constraints. [•] The overall development is integrated with a modern humanoid robot platform. [ABSTRACT FROM AUTHOR]

Sparsity has been showed to be one of the most important properties for visual recognition purposes. In this paper we show that sparse representation plays a fundamental role in achieving one-shot learning and real-time recognition of actions. We start off from RGBD images, combine motion and appearance cues and extract state-of-the-art features in a computationally efficient way. The proposed method relies on descriptors based on 3D Histograms of Scene Flow (3DHOFs) and Global Histograms of Oriented Gradient (GHOGs); adaptive sparse coding is applied to capture high-level patterns from data. We then propose a simultaneous on-line video segmentation and recognition of actions using linear SVMs. The main contribution of the paper is an effective real-time system for one-shot action modeling and recognition; the paper highlights the effectiveness of sparse coding techniques to represent 3D actions. We obtain very good results on three different data sets: a benchmark data set for one-shot action learning (the ChaLearn Gesture Data Set), an in-house data set acquired by a Kinect sensor including complex actions and gestures differing by small details, and a data set created for human-robot interaction purposes. Finally we demonstrate that our system is effective also in a human-robot interaction setting and propose a memory game, "All Gestures You Can", to be played against a humanoid robot. [ABSTRACT FROM AUTHOR]

User simultaneous detection and tracking is an issue at the core of human–robot interaction (HRI). Several methods exist and give good results; many use image processing techniques on images provided by the camera. The increasing presence in mobile robots of range-imaging cameras (such as structured light devices as Microsoft Kinects) allows us to develop image processing on depth maps. In this article, a fast and lightweight algorithm is presented for the detection and tracking of 3D clusters thanks to classic 2D techniques such as edge detection and connected components applied to the depth maps. The recognition of clusters is made using their 2D shape. An algorithm for the compression of depth maps has been specifically developed, allowing the distribution of the whole processing among several computers. The algorithm is then applied to a mobile robot for chasing an object selected by the user. The algorithm is coupled with laser-based tracking to make up for the narrow field of view of the range-imaging camera. The workload created by the method is light enough to enable its use even with processors with limited capabilities. Extensive experimental results are given for verifying the usefulness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper proposes a whole-body haptic sensing system that has multiple supporting points between the body frame and the end-effector. The system consists of an end-effector and multiple force sensors. Using this mechanism, the position of a contact force on the surface can be calculated without any sensor array. A haptic sensing system with a single supporting point structure has previously been developed by the present authors. However, the system has drawbacks such as low stiffness and low strength. Therefore, in this study, a mechanism with multiple supporting points was proposed and its performance was verified. In this paper, the basic concept of the mechanism is first introduced. Next, an evaluation of the proposed method, performed by conducting some experiments, is presented. © 2012 Wiley Periodicals, Inc. Electr Eng Jpn, 182(1): 48-56, 2013; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/eej.22310 [ABSTRACT FROM AUTHOR]

This paper proposes a direct anthropomorphization method to improve interaction between human and an agent. In this method, an artifact is converted into an agent by attaching humanoid parts to it. There have been many studies that have provided valuable information on using spoken directions and gestures via anthropomorphic agents such as computer graphics agents and communication robots. In the direct anthropomorphization method, an artifact is directly anthropomorphized by being fitted with robotic parts shaped in the form of human body parts. An anthropomorphized artifact with such robotic parts can provide information to people through spoken directions and body language. This will persuade people to pay more attention to the artifact, as compared to when using an anthropomorphic virtual or robot agent. The authors conducted experiments to investigate how the response of users to an explanation of the functions of an artifact changes using the direct anthropomorphization method. The results of pre-experiment indicated that participants paid more attention to the target artifact and memorized its functions more quickly and easily when using the direct anthropomorphization method than when using humanoid agent. In following two experiments, the authors compared human-like aspect separately and evaluate what is key element for anthropomorphization. The authors found that 'voice' was the key factor for rendering an object as an anthropomorphic agent. Furthermore, the 'eyes' were found to be more effective in interactions than the 'mouth'. [ABSTRACT FROM AUTHOR]

We present a novel and simple experimental method called physical human interactive guidance to study human-planned grasping. Instead of studying how the human uses his/her own biological hand or how a human teleoperates a robot hand in a grasping task, the method involves a human interacting physically with a robot arm and hand, carefully moving and guiding the robot into the grasping pose, while the robot’s configuration is recorded. Analysis of the grasps from this simple method has produced two interesting results. First, the grasps produced by this method perform better than grasps generated through a state-of-the-art automated grasp planner. Second, this method when combined with a detailed statistical analysis using a variety of grasp measures (physics-based heuristics considered critical for a good grasp) offered insights into how the human grasping method is similar or different from automated grasping synthesis techniques. Specifically, data from the physical human interactive guidance method showed that the human-planned grasping method provides grasps that are similar to grasps from a state-of-the-art automated grasp planner, but differed in one key aspect. The robot wrists were aligned with the object’s principal axes in the human-planned grasps (termed low skewness in this paper), while the automated grasps used arbitrary wrist orientation. Preliminary tests show that grasps with low skewness were significantly more robust than grasps with high skewness (77–93%). We conclude with a detailed discussion of how the physical human interactive guidance method relates to existing methods to extract the human principles for physical interaction. [ABSTRACT FROM AUTHOR]

Powered wheelchair users often struggle to drive safely and effectively and, in more critical cases, can only get around when accompanied by an assistant. To address these issues, we propose a collaborative control mechanism that assists users as and when they require help. The system uses a multiple-hypothesis method to predict the driver's intentions and, if necessary, adjusts the control signals to achieve the desired goal safely. The main emphasis of this paper is on a comprehensive evaluation, where we not only look at the system performance but also, perhaps more importantly, characterize the user performance in an experiment that combines eye tracking with a secondary task. Without assistance, participants experienced multiple collisions while driving around the predefined route. Conversely, when they were assisted by the collaborative controller, not only did they drive more safely but also they were able to pay less attention to their driving, resulting in a reduced cognitive workload. We discuss the importance of these results and their implications for other applications of shared control, such as brain–machine interfaces, where it could be used to compensate for both the low frequency and the low resolution of the user input. [ABSTRACT FROM AUTHOR]

This paper presents a haptic sensing foot system for humanoid robot. The two different kinds of implementations are investigated: One is an active tactile sensing technique to recognize a contacting ground slope. The other is to balance the robot body with one leg for human–robot interaction. The proposed sensors are implemented on two robotic feet. Each sensing unit on each foot consists of three thin sheets of force sensitive resistors arranged triangularly with the peripheral circuits. The research objective is to produce an artifact which can be operated in a natural and intuitive manner by utilizing the control of a foot pose to keep the direction of the foot normal to the ground surface. Throughout these works, we aim to realize the tactile sensing foot to detect the ground slope for natural foot posture control in order to assist the biped walking robot to balance its body on various types of ground surfaces. In these applications, the information about the ground floor or orientation is not required in advance. [ABSTRACT FROM PUBLISHER]