Search

In the industrial robots field, efficient and convenient programming methods have been a hot research topic. In recent years, immersive simulation technology has been developing rapidly in many fields, which provides new horizons for the development of industrial robots. This paper presents a HTC VIVE laser scan motion capture and Holohens augmented reality (AR) based interactive Programming by Demonstration (PbD) system for industrial robot. A portable Handheld Teaching Device (HTD) and its calibration algorithm are designed in the system. The portable HTD which is tracked by a laser motion capture system can be viewed as an AR robot end-effector to teach paths. Meanwhile, the AR robot can be simulated in real time during programing. In addition, the robot reproducing the operator's actions at the same position in space is the focus of programming. So, Multi-system registration methods are proposed to determine the relationship between robot systems, motion capture systems and virtual robot systems. Meanwhile, a path planning algorithm is proposed to convert the captured raw path points into robot-executable code. For unskilled operators, they can easily perform complex programming using the HTD. For skilled senior workers, their skills can be quickly learned by robots using the system. [ABSTRACT FROM AUTHOR]

Dynamic Movement Primitives (DMP) have found remarkable applicability and success in various robotic tasks, which can be mainly attributed to their generalization, modulation and robustness properties. However, the spatial generalization of DMP can be problematic in some cases, leading to excessive overscaling and in turn large velocities and accelerations. While other DMP variants have been proposed in the literature to tackle this issue, they can also exhibit excessive overscaling as we show in this work. Moreover, incorporating intermediate points (via-points) for adjusting the DMP trajectory to account for the geometry of objects related to the task, or to avoid or push aside objects that obstruct a specific task, is not addressed by the current DMP literature. In this work we tackle these unresolved so far issues by proposing an improved online spatial generalization, that remedies the shortcomings of the classical DMP generalization, and moreover allows the incorporation of dynamic via-points. This is achieved by designing an online adaptation scheme for the DMP weights which is proved to minimize the distance from the demonstrated acceleration profile to retain the shape of the demonstration, subject to dynamic via-point and initial/final state constraints. Extensive comparative simulations with the classical and other DMP variants are conducted, while experimental results validate the practical usefulness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Programming by demonstration is reaching industrial applications, which allows non-experts to teach new tasks without manual code writing. However, a certain level of complexity, such as online decision making or the definition of recovery behaviors, still requires experts that use conventional programming methods. Even though, experts cannot foresee all possible faults in a robotic application. To encounter this, we present a framework where user and robot collaboratively program a task that involves online decision making and recovery behaviors. Hereby, a task-graph is created that represents a production task and possible alternative behaviors. Nodes represent start, end or decision states and links define actions for execution. This graph can be incrementally extended by autonomous anomaly detection, which requests the user to add knowledge for a specific recovery action. Besides our proposed approach, we introduce two alternative approaches that manage recovery behavior programming and compare all approaches extensively in a user study involving 21 subjects. This study revealed the strength of our framework and analyzed how users act to add knowledge to the robot. Our findings proclaim to use a framework with a task-graph based knowledge representation and autonomous anomaly detection not only for initiating recovery actions but particularly to transfer those to a robot. [ABSTRACT FROM AUTHOR]

The great diversity of end-user tasks ranging from manufacturing environments to personal homes makes pre-programming robots for general purpose applications extremely challenging. In fact, teaching robots new actions from scratch that can be reused for previously unseen tasks remains a difficult challenge and is generally left up to robotics experts. In this work, we present iRoPro, an interactive Robot Programming framework that allows end-users with little to no technical background to teach a robot new reusable actions. We combine Programming by Demonstration and Automated Planning techniques to allow the user to construct the robot's knowledge base by teaching new actions by kinesthetic demonstration. The actions are generalised and reused with a task planner to solve previously unseen problems defined by the user. We implement iRoPro as an end-to-end system on a Baxter Research Robot to simultaneously teach low- and high-level actions by demonstration that the user can customise via a Graphical User Interface to adapt to their specific use case. To evaluate the feasibility of our approach, we first conducted pre-design experiments to better understand the user's adoption of involved concepts and the proposed robot programming process. We compare results with post-design experiments, where we conducted a user study to validate the usability of our approach with real end-users. Overall, we showed that users with different programming levels and educational backgrounds can easily learn and use iRoPro and its robot programming process. [ABSTRACT FROM AUTHOR]

Industrial robots used to assemble customized products in small batches require a lot of reprogramming. With this work we aim to reduce the programming complexity by autonomously finding the fastest assembly plans without any collisions with the environment. First, a digital twin of the robot uses a gym in simulation to learn which assembly skills (programmed by demonstration) are physically possible (i.e. no collisions with the environment). Only from this reduced solution space will the physical twin look for the fastest assembly plans. Experiments show that the system indeed converges to the fastest assembly plans. Moreover, pre-training in simulation drastically reduces the number of interactions before convergence compared to directly learning on the physical robot. This two-step procedure allows for the robot to autonomously find correct and fast assembly sequences, without any additional human input or mismanufactured products. [ABSTRACT FROM AUTHOR]

Programming robots often involves expert knowledge in both the robot itself and the task to execute. An alternative to direct programming is for a human to show examples of the task execution and have the robot perform the task based on these examples, in a scheme known as learning or programming from demonstration. We propose and study a generic and simple learning-from-demonstration framework. Our approach is to combine the demonstrated commands according to the similarity between the demonstrated sensory trajectories and the current replay trajectory. This tracking is solely performed based on sensor values and time and completely dispenses with the usually expensive step of precomputing an internal model of the task. We analyse the behaviour of the proposed model in several simulated conditions and test it on two different robotic platforms. We show that it can reproduce different capabilities with a limited number of meta parameters. [ABSTRACT FROM AUTHOR]

The evolution of production systems for smart factories foresees a tight relation between human operators and robots. Specifically, when robot task reconfiguration is needed, the operator must be provided with an easy and intuitive way to do it. A useful tool for robot task reconfiguration is Programming by Demonstration (PbD). PbD allows human operators to teach a robot new tasks by showing it a number of examples. The article presents two studies investigating the role of the robot in PbD. A preliminary study compares standard PbD with human–human teaching and suggests that a collaborative robot should actively participate in the teaching process as human practitioners typically do. The main study uses a wizard of oz approach to determine the effects of having a robot actively participating in the teaching process, specifically by controlling the end-effector. The results suggest that active behaviour inspired by humans can lead to a more intuitive PbD. [ABSTRACT FROM AUTHOR]

A handover strategy is proposed that aims at natural and fluent robot to human object handovers. For the approaching phase, a globally asymptotically stable dynamical system (DS) is utilized, trained from human demonstrations and exploiting the existence of mirroring in the human wrist motion. The DS operates in the robot task space thus achieving independence with respect to the robot platform, encapsulating the position and orientation of the human wrist within a single DS. It is proven that the motion generated by such a DS, having as target the current wrist pose of the receiver's hand, is bounded and converges to the previously unknown handover location. Haptic cues based on load estimates at the robot giver ensure full object load transfer before grip release. The proposed strategy is validated with simulations and experiments in real settings. [ABSTRACT FROM AUTHOR]

Task-parameterized skill learning aims at adaptive motion encoding to new situations. While existing approaches for task-parameterized skill learning have demonstrated good adaptation within the demonstrated region, the extrapolation problem of task-parameterized skills has not been investigated enough. In this work, with the aim of good adaptation not only within the demonstrated region but also outside of the region, we propose to combine a generative model with a dynamic movement primitive by formulating learning as a density estimation problem. Moreover, for efficient learning from relatively few demonstrations, we propose to augment training data with additional incomplete data. The proposed method is tested and compared with existing works in simulations and real robot experiments. Experimental results verified its generalization in the extrapolation region. [ABSTRACT FROM AUTHOR]

A visuo-haptic augmented reality system is presented for object manipulation and task learning from human demonstration. The proposed system consists of a desktop augmented reality setup where users operate a haptic device for object interaction. Users of the haptic device are not co-located with the environment where real objects are present. A three degrees of freedom haptic device, providing force feedback, is adopted for object interaction by pushing, selection, translation and rotation. The system also supports physics-based animation of rigid bodies. Virtual objects are simulated in a physically plausible manner and seem to coexist with real objects in the augmented reality space. Algorithms for calibration, object recognition, registration and haptic rendering have been developed. Automatic model-based object recognition and registration are performed from 3D range data acquired by a moving laser scanner mounted on a robot arm. Several experiments have been performed to evaluate the augmented reality system in both single-user and collaborative tasks. Moreover, the potential of the system for programming robot manipulation tasks by demonstration is investigated. Experiments show that a precedence graph, encoding the sequential structure of the task, can be successfully extracted from multiple user demonstrations and that the learned task can be executed by a robot system. [ABSTRACT FROM AUTHOR]

Programming by demonstration techniques facilitate the programming of robots. Some of them allow the generalization of tasks through parameters, although they require new training when trajectories different from the ones used to estimate the model need to be added. One of the ways to re-train a robot is by incremental learning, which supplies additional information of the task and does not require teaching the whole task again. The present study proposes three techniques to add trajectories to a previously estimated task-parameterized Gaussian mixture model. The first technique estimates a new model by accumulating the new trajectory and the set of trajectories generated using the previous model. The second technique permits adding to the parameters of the existent model those obtained for the new trajectories. The third one updates the model parameters by running a modified version of the Expectation-Maximization algorithm, with the information of the new trajectories. The techniques were evaluated in a simulated task and a real one, and they showed better performance than that of the existent model. [ABSTRACT FROM AUTHOR]

The research proposes a novel technological solution for marker-based human motion capture called WirelessSyncroVision (WSV). The WSV is formed by two main modules: the visual node (WSV-V) which is based on a stereoscopic vision system and the marker node (WSV-M) that is constituted by a 6-DOF active marker. The solution synchronizes the acquisition of images in remote muti-cameras with the ON period of the active marker. This increases the robustness of the stereoscopic system to illumination changes, which is extremely relevant for programming industrial robotic-arms using a human demonstrator programming by demonstration (PbD). In addition, the research presents a robust method named Adaptive and Robust Synchronization (ARS), that is designed for temporal alignment of remote devices using a wireless network. The algorithm models the phase difference as a function of time, measuring the parameters that must be known to predict the synchronization instant between the active marker and the remote cameras. Results demonstrate that the ARS creates a balance between the real-time capability and the performance estimation of the phase difference. Therefore, this research proposes an elegant solution to synchronize image acquisition systems in real-time that is easy to implement with low operational costs; however, the major advantage of the WSV is related to its high level of flexibility since it can be extended toward to other devices besides the PbD, for instance, motion capture, motion analysis, and remote sensoring systems. [ABSTRACT FROM AUTHOR]

We propose a new methodology for learning and adaption of manipulation skills that involve physical contact with the environment. Pure position control is unsuitable for such tasks because even small errors in the desired trajectory can cause significant deviations from the desired forces and torques. The proposed algorithm takes a reference Cartesian trajectory and force/torque profile as input and adapts the movement so that the resulting forces and torques match the reference profiles. The learning algorithm is based on dynamic movement primitives and quaternion representation of orientation, which provide a mathematical machinery for efficient and stable adaptation. Experimentally we show that the robot's performance can be significantly improved within a few iteration steps, compensating for vision and other errors that might arise during the execution of the task. We also show that our methodology is suitable both for robots with admittance and for robots with impedance control. [ABSTRACT FROM AUTHOR]

SimStudent is a machine-learning agent initially developed to help novice authors to create cognitive tutors without heavy programming. Integrated into an existing suite of software tools called Cognitive Tutor Authoring Tools (CTAT), SimStudent helps authors to create an expert model for a cognitive tutor by tutoring SimStudent on how to solve problems. There are two different ways to author an expert model with SimStudent. In the context of Authoring by Tutoring, the author interactively tutors SimStudent by posing problems to SimStudent, providing feedback on the steps performed by SimStudent, and also demonstrating steps as a response to SimStudent's hint requests when SimStudent cannot perform steps correctly. In the context of Authoring by Demonstration, the author demonstrates solution steps, and SimStudent attempts to induce underlying domain principles by generalizing those worked-out examples. We conducted evaluation studies to investigate which authoring strategy better facilitates authoring and found two key results. First, the expert model generated with Authoring by Tutoring is better and has higher accuracy while maintaining the same level of completeness than the one generated with Authoring by Demonstration. The reason for this better accuracy is that the expert model generated by tutoring benefits from negative feedback provided for SimStudent's incorrect production applications. Second, authoring by Tutoring requires less time than Authoring by Demonstration. This enhanced authoring efficiency is partially because (a) when Authoring by Demonstration, the author needs to test the quality of the expert model, whereas the formative assessment of the expert model is done naturally by observing SimStudent's performance when Authoring by Tutoring, and (b) the number of steps that need to be demonstrated during tutoring decreases as learning progresses. [ABSTRACT FROM AUTHOR]

This paper presents a technical approach to robot learning of motor skills which combines active intrinsically motivated learning with imitation learning. Our algorithmic architecture, called SGIM-D, allows efficient learning of high-dimensional continuous sensorimotor inverse models in robots, and in particular learns distributions of parameterised motor policies that solve a corresponding distribution of parameterised goals/tasks. This is made possible by the technical integration of imitation learning techniques within an algorithm for learning inverse models that relies on active goal babbling. After reviewing social learning and intrinsic motivation approaches to action learning, we describe the general framework of our algorithm, before detailing its architecture. In an experiment where a robot arm has to learn to use a flexible fishing line, we illustrate that SGIM-D efficiently combines the advantages of social learning and intrinsic motivation and benefits from human demonstration properties to learn how to produce varied outcomes in the environment, while developing more precise control policies in large spaces. [ABSTRACT FROM AUTHOR]

Video conferencing provides an environment for multiple users linked on a network to have meetings. Since a large quantity of audio and video data are transferred to multiple users in real time, research into reducing the quantity of data to be transferred has been drawing attention. Such methods extract and transfer only the features of a user from video data and then reconstruct a video conference using virtual humans. The disadvantage of such an approach is that only the positions and features of hands and heads are extracted and reconstructed, whilst the other virtual body parts do not follow the user. In order to enable a virtual human to accurately mimic the entire body of the user in a 3D virtual conference, we examined what features should be extracted to express a user more clearly and how they can be reproduced by a virtual human. This 3D video conferencing estimates the user's pose by comparing predefined images with a photographed user's image and generates a virtual human that takes the estimated pose. However, this requires predefining a diverse set of images for pose estimation and, moreover, it is difficult to define behaviors that can express poses correctly. This paper proposes a framework to automatically generate the pose-images used to estimate a user's pose and the behaviors required to present a user using a virtual human in a 3D video conference. The method for applying this framework to a 3D video conference on the basis of the automatically generated data is also described. In the experiment, the framework proposed in this paper was implemented in a mobile device. The generation process of poses and behaviors of virtual human was verified. Finally, by applying programming by demonstration, we developed a system that can automatically collect the various data necessary for a video conference directly without any prior knowledge of the video conference system. [ABSTRACT FROM AUTHOR]

This paper proposes an end-to-end learning from demonstration framework for teaching force-based manipulation tasks to robots. The strengths of this work are manyfold. First, we deal with the problem of learning through force perceptions exclusively. Second, we propose to exploit haptic feedback both as a means for improving teacher demonstrations and as a human-robot interaction tool, establishing a bidirectional communication channel between the teacher and the robot, in contrast to the works using kinesthetic teaching. Third, we address the well-known what to imitate? problem from a different point of view, based on the mutual information between perceptions and actions. Lastly, the teacher's demonstrations are encoded using a Hidden Markov Model, and the robot execution phase is developed by implementing a modified version of Gaussian Mixture Regression that uses implicit temporal information from the probabilistic model, needed when tackling tasks with ambiguous perceptions. Experimental results show that the robot is able to learn and reproduce two different manipulation tasks, with a performance comparable to the teacher's one. [ABSTRACT FROM AUTHOR]

In the human environment service robots have to be able to manipulate autonomously a large variety of objects in a workspace restricted by collisions with obstacles, self-collisions and task constraints. Planning enables the robot system to generalize predefined or learned manipulation knowledge to new environments. For dexterous manipulation tasks the manual definition of planning models is time-consuming and error-prone. In this work, planning models for dexterous tasks are learned based on multiple human demonstrations using a general feature space including automatically generated contact constraints, which are automatically relaxed to consider the correspondence problem. In order to execute the learned planning model with different objects, the contact location is transformed to given object geometry using morphing. The initial, overspecialized planning model is generalized using a previously described, parallelized optimization algorithm with the goal to find a maximal subset of task constraints, which admits a solution to a set of test problems. Experiments on two different, dexterous tasks show the applicability of the learning approach to dexterous manipulation tasks. [ABSTRACT FROM AUTHOR]

We present an approach for kinesthetic teaching of motion primitives for a humanoid robot. The proposed teaching method starts with observational learning and applies iterative kinesthetic motion refinement using a forgetting factor. Kinesthetic teaching is supported by introducing the motion refinement tube, which represents an area of allowed motion refinement around the nominal trajectory. On the realtime control level, the kinesthetic teaching is handled by a customized impedance controller, which combines tracking performance with compliant physical interaction and allows to implement soft boundaries for the motion refinement. A novel method for continuous generation of motions from a hidden Markov model (HMM) representation of motion primitives is proposed, which incorporates time information for each state. The proposed methods were implemented and tested using DLR's humanoid upper-body robot Justin. [ABSTRACT FROM AUTHOR]

Physics-based simulation is increasingly important in virtual manufacturing for product assembly and disassembly operations. This work explores potential benefits of physics-based modeling for automatic learning of assembly tasks and for intelligent disassembly planning in desktop virtual reality. The paper shows how realistic physical animation of manipulation tasks can be exploited for learning sequential constraints from user demonstrations. In particular, a method is proposed where information about physical interaction is used to discover task precedences and to reason about task similarities. A second contribution of the paper is the application of physics-based modeling to the problem of disassembly sequence planning. A novel approach is described to find all physically admissible subassemblies in which a set of rigid objects can be disassembled. Moreover, efficient strategies are presented aimed at reducing the computational time required for automatic disassembly planning. The proposed strategies take into account precedence relations arising from user assembly demonstrations as well as geometrical clustering. A motion planning technique has also been developed to generate non-destructive disassembly paths in a query-based approach. Experiments have been performed in an interactive virtual environment including a dataglove and motion tracker that allows realistic object manipulation and grasping. [ABSTRACT FROM AUTHOR]

One of the most important issues in developing an entertainment robot is human-robot interaction, in which the robot is expected to learn new behaviors specified by the user. In this article we present an imitation-based mechanism to support robot learning, and use evolutionary computing to learn new behavior sequences. We also propose several advanced techniques at the task level and the computational level to evolve complex sequences. To evaluate our approach, we use it to evolve different behaviors for a humanoid robot. The results show the promise of our approach. [ABSTRACT FROM AUTHOR]

An important function of an agent is to be “on the lookout” for bits of information that are interesting to its user, even if these items appear in the midst of a larger body of unstructured information. But how to tell these agents which patterns are meaningful and what to do with the result? Especially when agents are used to recognize text, they are usually driven by parsers which require input in the form of textual grammar rules. Editing grammars is difficult and error-prone for end users. Grammex [“Grammars by Example”] is the first direct manipulation interface designed to allow non-expert users to define grammars interactively. The user presents concrete examples of text that he or she would like the agent to recognize. Rules are constructed by an iterative process, where Grammex heuristically parses the example, displays a set of hypotheses, and the user critiques the system's suggestions. Actions to take upon recognition are also demonstrated by example. [ABSTRACT FROM AUTHOR]