Search

Human-human collaboration is characterized by a back-and-forth, where an action of one agent elicits the response of the other. This interaction is inherently multimodal and includes both high-level modalities such as language and low-level ones such as force exchanges. In this work, we investigate human collaborative manipulation: we show distinct patterns that can be identified in low-level physical data and that can be interpreted as primitives used by humans to negotiate about various aspects of the motion and to execute the motion. These primitives provide a high-level interpretation of the interaction and can be used to connect low-level behavior to language. We describe the human study used to collect the data, the data analysis process, and discuss how the identified primitives could be used by a robot’s interaction manager to mediate physical Human-Robot Interaction (pHRI).

In a collaborative task and the interaction that accompanies it, the participants often take on distinct roles, and dynamically switch the roles as the task requires. A domestic assistive robot thus needs to have similar capabilities. Using our previously proposed Multimodal Interaction Manager (MIM) framework, this paper investigates how role switching for a robot can be implemented. It identifies a set of primitive subtasks that encode common interaction patterns observed in our data corpus and that can be used to easily construct complex task models. It also describes an implementation on the NAO robot that, together with our original work, demonstrates that the robot can take on different roles. We provide a detailed analysis of the performance of the system and discuss the challenges that arise when switching roles in human-robot interactions.

With rapid advances in social robotics, humanoids and autonomy, robot assistants appear to be within reach. However, robots are still unable to effectively interact with humans in activities of daily living. One of the challenges is in the frequent use of multiple communication modalities when humans engage in collaborative activities. In this paper, we propose a Multimodal Interaction Manager, a framework for an assistive robot that maintains an active multimodal interaction with a human partner while performing physical collaborative tasks. The heart of our framework is a Hierarchical Bipartite Action-Transition Network (HBATN), which allows the robot to infer the state of the task and the dialogue given spoken utterances and observed pointing gestures from a human partner, and to plan its next actions. Finally, we implemented this framework on a robot to provide preliminary evidence that the robot can successfully participate in a task-oriented multimodal interaction.

Grasp is an integral part of manipulation actions in activities of daily living and programming by demonstration is a powerful paradigm for teaching the assistive robots how to perform a grasp. Since finger configuration and finger force are the fundamental features that need to be controlled during a grasp, using these variables is a natural choice for learning by demonstration. An important question then becomes whether the existing grasp taxonomies are appropriate when one considers these modalities. The goal of our paper is to answer this question by investigating grasp patterns that can be inferred from a static analysis of the grasp data, as the object is securely grasped. Human grasp data is measured using a newly developed data glove. The data includes pressure sensor measurements from eighteen areas of the hand, and measurements from bend sensors placed at finger joints. The pressure sensor measurements are calibrated and mapped into force by employing a novel data-driven approach. Unsupervised learning is used to identify patterns for different grasp types. Multiple clustering algorithms are used to partition the data. When the results are taken in aggregate, 25 human grasp types are reduced to 9 different clusters.

Human grasp has been studied extensively and many taxonomies have been developed to classify different grasp types. In this work, we collect a comprehensive list of grasp types and investigate the force distribution patterns among them. We conduct a human study and collect force data in various grasp types. Data collection is performed by utilizing a data glove that has seventeen force sensors. We analyze the measured forces and identify different patterns in force distribution. Employing voting technique over various clustering methods, we propose a robust clustering for the force patterns, namely the grasp taxonomy in force domain. The proposed grasp taxonomy can be exploited in designing robotic prosthetic hands, designing grasp controllers and action recognition in human-robot interaction.