Search

This work proposes a framework that improves the dexterous manipulation capabilities of two fingered grippers by: i) optimizing the finger link dimensions and the interfinger distance for a given object and ii) analyzing the effect of finger symmetry and the distance between the finger base frames on their manipulation workspaces. The results of the workspace analysis motivate the development of a multi-modal, adaptive robotic gripper. In particular, the finger link lengths optimization problem is solved by a parallel multi-start search algorithm. The optimal link lengths are then used for the workspace analysis. The results of the analysis demonstrate that different inter-finger distances lead to completely different workspace shapes and that the ratio defined by the area of the optimized workspace (nominator) and the union of all workspaces (denominator), is always significantly less than 1. This means that the area of the union of all workspaces is always larger than the area of the “optimized” workspace. Based on these results the proposed robotic gripper is equipped with reconfigurable finger bases that vary the inter-finger distance as well as with selectively lockable robotic finger joints, offering an increased dexterous manipulation performance without sacrificing grasping efficiency. The device is considered multi-modal as it can be used both as a parallel jaw gripper and as an adaptive robotic gripper.

The increased use of Virtual and Augmented Reality based systems necessitates the development of more intuitive and unobtrusive means of interfacing. Over the last years, Electromyography (EMG) based interfaces have been employed for interaction with robotic and computer applications, but no studies have been carried out to investigate the continuous decoding of the effects of human motion (e.g., manipulated object behavior) in simulated and virtual environments. In this work, we compare the object motion decoding accuracy of an EMG based learning framework for two different dexterous manipulation scenarios: i) for simulated objects handled by a teleoperated model of a hand within a virtual environment and ii) for real, everyday life objects manipulated by the human hand. To do that, we utilize EMG activations from 16 muscle sites (9 on the hand and 7 on the forearm). The object motion decoding is formulated as a regression problem using the Random Forests methodology. A 5-fold cross validation procedure is used for model assessment purposes and the feature variable importance values are calculated for each model. The decoding accuracy for the real world is considerably higher than the virtual world. Each of the objects examined had a single manipulation motion that offered the highest estimation accuracy across both worlds. This study also shows that it is feasible to decode the object motions using just the myoelectric activations of the muscles of the forearm and the hand. This is particularly surprising since simulations lacked haptic feedback and the ability to account for other dynamic phenomena like friction and contact rolling.

Robotic hand exoskeletons have become a popular and efficient technological solution for assisting people that suffer from neurological conditions and for enhancing the capabilities of healthy individuals. This class of devices ranges from rigid and complex structures to soft, lightweight, wearable gloves. In this work, we propose a hybrid (tendon-driven and pneumatic), lightweight, affordable, easy-to-operate exoskeleton glove equipped with variable stiffness, laminar jamming structures, abduction/adduction capabilities, and a pneumatic telescopic extra thumb that increases grasp stability. The efficiency of the proposed device is experimentally validated through five different types of experiments: i) abduction/adduction tests, ii) force exertion experiments that capture the forces that can be exerted by the proposed device under different conditions, iii) bending profile experiments that evaluate the effect of the laminar jamming structures on the way the fingers bend, iv) grasp quality assessment experiments that focus on the effect of the inflatable thumb on enhancing grasp stability, and v) grasping experiments involving everyday objects and seven subjects. The hybrid assistive, exoskeleton glove considerably improves the grasping capabilities of the user, being able to exert the forces required to execute a plethora of activities of daily living. All files that allow the replication of the device are distributed in an open-source manner.

With an increasing number of robotic and prosthetic devices, there is a need for intuitive Muscle-Machine Interfaces (MuMIs) that allow the user to have an embodied interaction with the devices they are controlling. Such MuMIs can be developed using machine learning based methods that utilize myoelectric activations from the muscles of the user to decode their intention. However, the choice of the learning method is subjective and depends on the features extracted from the raw Electromyography signals as well as on the intended application. In this work, we compare the performance of five machine learning methods and eight time-domain feature extraction techniques in discriminating between different gestures executed by the user of an EMG based MuMI. From the results, it can be seen that the Willison Amplitude performs consistently better for all the machine learning methods compared in this study, while the Zero Crossings achieves the worst results for the Decision Trees and the Random Forests and the Variance offers the worst performance for all the other learning methods. The Random Forests method is shown to achieve the best results in terms of achieved accuracies (has the lowest variance between subjects). In order to experimentally validate the efficiency of the Random Forest classifier and the Willison Amplitude technique, a series of gestures were decoded in a real-time manner from the myoelectric activations of the operator and they were used to control a robot hand.

Currently, ~ 1.5 million American deaf-blind individuals depend on the availability of interpreting services to communicate in their primary conversational language, tactile American Sign Language (ASL). In an effort to give the deaf-blind community access to a device that facilitates independent communication using tactile ASL, we developed TATUM (Tactile ASL Translational User Mechanism). TATUM employs 15 degrees of actuation in a hand-wrist system that is capable of signing the 26-letter ASL alphabet. Leveraging Interpres, an independent cloud-based service, all servo sequences that render desired fingerspelled letters and ASL words are stored in a web application programming interface (API). A validation study including both deaf and deaf-blind participants confirmed that the TATUM hand mimics a human hand both in size and feel. The current design of TATUM attained an average recognition rate of 94.7% in visual validation, indicative of the potential to support deaf and deaf-blind individuals in communicating via visual and tactile ASL.

Lecturers of Engineering courses around the world are struggling to increase the engagement of students through the introduction of appropriate hands-on activities and assignments. In Biomechatronics and Robotics courses these assignments typically focus on how certain devices are designed, modelled, fabricated, or controlled. The hardware for these assignments is usually purchased by some external vendor and the students only get the chance to analyze it or program it, so as to execute a useful task (e.g., programming mobile robots to perform path following tasks). Student engagement can be increased by instructing the students to prepare the hardware for their assignment. This also increases the sense of ownership of the project outcomes. In this paper, we present how a robotic gripper / hand design project and the introduction of a grasping and manipulation competition as a course assignment, can significantly increase the student engagement and their understanding of the taught concepts. The presented best practices have been trialed over the last four years in two different courses (one undergraduate and one postgraduate) of the Department of Mechanical Engineering at the University of Auckland in New Zealand. For the particular assignment the students were asked to fully develop a robotic gripper or hand from scratch using a single actuator (only the actuator and the power electronics were provided). The performance of the developed devices was assessed through the participation in a grasping and manipulation competition. All the details of the proposed assignment are presented, hoping that they could help other lecturers and teachers to prepare similar activities.

Electromyography (EMG) is a wearable, noninvasive, commonly used method for measuring the human muscular activations from the surface of the skin. In this work, we present a pilot study that focuses on the formulation of a shared control framework to facilitate the simplified execution of Electromyography (EMG) based telemanipulation tasks with a robotic platform. The framework combines a Random Forests (RF) regression method with a compliance controller that relies on the force measurements collected with a force-torque sensor. The RF regression efficiently maps the myoelectric activations of the human muscles to corresponding human wrist positions. Then, a teleoperation process is used to control the robot arm end-effector’s position, utilizing the human wrist position estimations. The examined application involves semi-autonomous cleaning of a whiteboard surface with the proposed framework. The compliance controller guarantees that a desired contact force will always be maintained on the whiteboard surface during task execution. This ensures that any EMG based decoding inaccuracies will not drive the robot away from the cleaning plane. Essentially, the system projects the EMG based estimation on the cleaning plane. The shared control framework offers robust performance, with minimal training and calibration required.

Robot grasping and manipulation planning in unstructured and dynamic environments is heavily dependent on the attributes of manipulated objects. Although deep learning approaches have delivered exceptional performance in robot perception, human perception and reasoning are still superior in processing novel object classes. Moreover, training such models requires large datasets that are generally expensive to obtain. This work combines crowdsourcing and gamification to leverage human intelligence, enhancing the object recognition and attribute estimation aspects of robot perception. The framework employs an attribute matching system that encodes visual information into an online puzzle game, utilizing the collective intelligence of players to expand an initial attribute database and react to real-time perception conflicts. The framework is deployed and evaluated in a proof-of-concept application for enhancing object recognition in autonomous robot grasping and a model for estimating the response time is proposed. The obtained results demonstrate that given enough players, the framework can offer near real-time labeling of novel objects, based purely on visual information and human experience.

Many studies have focused on Virtual Reality (VR) frameworks for remotely controlling robotic systems. Although VR systems have been used to teleoperate robots in simple scenarios, their effectiveness in terms of accuracy, speed, and usability has not been rigorously evaluated for complex tasks that require accurate trajectories. In this work, an Enhanced Virtual Reality (EVR) framework for robotic teleoperation is evaluated to assess if it can be efficiently used in complex tasks that require accurate control of the robotic end-effector. The environment and the employed robot are captured using RGB-D cameras, while the remote user controls the motion of the robot with VR controllers. The captured data are transmitted and reconstructed in 3D so as to allow the remote user to monitor the task execution progress in real time, using a VR headset. The EVR system is compared with two other interface alternatives: i) teleoperation in pure VR (the model of the robot is rendered with respect to its real joint states), and ii) teleoperation in EVRR (the model of the robot is superimposed on the real robot). The results show that pure point cloud interfaces suffer from visualization issues, reducing the effectiveness of the robot teleoperation. However, the accuracy and user experience can be greatly improved by including the robot model.

Despite the plethora of studies focusing on the design and development of dexterous robotic grippers and hands, researchers have not been able to quantify dexterity due to the lack of well defined measures or tests. Human dexterity on the other hand, is generally measured by employing various hand function tests. This work is inspired from these tests, proposing a new modular, affordable, accessible test for evaluating the grasping and manipulation capabilities of robotic end effectors. More precisely, the test allows for a quantification of the end-effectors' ability to perform various tasks effectively and irrespectively of the individual design parameters. The test rig proposed in this study combines the features of multiple dexterity tests originally developed for humans. The test quantifies dexterity through a weighted sum of task execution success, speed, and accuracy components that results to a dexterity score ranging from 0 (simplistic, nondexterous system) to 1 (human-like system). It should also be noted that dexterity can be evaluated assessing the efficiency of either the robotic hardware or the robotic perception system.

Over the last decades, the development of aerial robots for monitoring, grasping, and package delivery applications has gained momentum in both research and industry. However, grasping capable platforms are currently limited by the payload that they can carry and their operation time, due to the weight of the grasping mechanisms, their high energy consumption, and battery limitations. In this paper, we propose two reconfigurable, lightweight, robust grasping mechanisms that can be used in aerial robotic vehicles offering them grasping capabilities. The two solutions are: i) an ultra-lightweight, net-based design that utilises twisted string actuation to pull together eight 3D printed cylinders that are connected to a net, conforming around the object and ii) a reconfigurable, slider-based grasping mechanism that works like a supersized parallel jaw gripper, employing 3D-printed pads that slide on two carbon fibre shafts with linear bearings. The second design can also be configured as a shape-shifting, grasping capable device that can act as a flying gripper. The efficiency of the proposed grasping mechanisms has been experimentally validated with a series of experiments focusing on grasping of everyday objects and assessing their force exertion capabilities and durability.

Unstructured and uncertain environments that are encountered in search and rescue applications, require complex interactions that involve a wide range of grasps. These grasps are needed in order to handle unpredictable types of objects of various shapes, stiffnesses, and dimensions. Traditionally, the robotic end-effectors used in such situations are rigid and their operation requires sophisticated sensing elements and complicated control algorithms to manipulate delicate and fragile objects. Over the last decade, considerable research effort has been put into the development of adaptive, underactuated, and soft robots that facilitate robust interactions with dynamic environments. In this paper, we propose a three-fingered robotic gripper that combines pneumatic telescopic mechanisms and rigid claws. The gripper uses three large, rigid fingers to accomplish the execution of all the tasks required by a traditional robot gripper, while three inflatable, telescopic fingers provide soft interaction with objects. This synergistic combination of soft and rigid structures allows the gripper to cage / trap and firmly hold heavy and irregular objects. The experiments demonstrate that the gripper can successfully grasp a plethora of objects exerting up to 60 N of clench force.

Unmanned Aerial Vehicles (UAV) are typically based on rotorcraft designs and have been extensively used to perform mechanical interaction tasks that could involve inspection as well as search-and-rescue operations in critical unstructured and dynamic environments. UAV based aerial manipulation has also received an increased interest and it has attracted considerable research effort in the robotics community. However, the state-of-the-art control approaches that are being applied to traditional underactuated UAVs show critical stability issues during the execution of manipulation tasks. This is due to the fact that these tasks require the exertion of significant forces with respect to the vehicle-weight ratio. Designs of fully-actuated and overactuated systems show better suitability for such tasks but are still specific to certain functions and tasks, lacking the needed agility and flexibility. In this paper, we propose a novel design of an agile, coaxial, omnidirectional rotor module that can move and exert forces in all directions, without limitations on orientation and wiring. This independent rotor unit presents high mobility and maneuverability, high fight autonomy, and offers high potential for all terrain robotic manipulation applications. In this work, the proposed rotor module has been employed to develop an unconventional, hybrid, all terrain robotic rotorcraft that can act both as a drone and as a mobile robot.

Robots use their end-effectors to grasp and manipulate objects in unstructured and dynamic environments. Robot hands and grippers can vary from rigid and complex designs to soft, inflatable, and lightweight structures. In this paper, we focus on the development of a soft, retractable, telescopic gripper that exhibits an adaptive behaviour that allows efficient and stable grasps with a wide range of objects. The efficiency of the proposed device is experimentally validated through three different types of experiments: i) grasping experiments that involve different everyday objects, ii) force exertion experiments that capture the maximum forces that can be exerted by the proposed device, and iii) grasp resistance experiments that focus on the effect of the inflatable structure on resisting environmental uncertainties and disturbances. The proposed gripper is able to grasp a plethora of everyday life objects and it can exert more than 8 N of grasping force. The design is so low-cost that the soft fingers can be used in a disposable manner, facilitating the execution of specialized tasks (e.g., grasping in sterilized environments, handling of medical waste, etc).

Although soft robots are a good alternative to rigid, traditional robots due to their intrinsic compliance and environmental adaptability, there are several drawbacks that limit their impact, such as low force exertion capability and low resistance to deformation. For this reason, soft structures of variable stiffness have become a popular solution in the field to combine the benefits of both soft and rigid designs. In this paper, we develop laminar jamming flexure joints that facilitate the development of adaptive robot grippers with variable stiffness. Initially, we propose a mathematical model of the laminar jamming structures. Then, the model is experimentally validated through bending tests using different materials, pressures, and number of layers. Finally, the soft, laminar jamming structured are employed to develop variable stiffness flexure joints for two different adaptive robot grippers. Bending profile analysis and grasping tests have demonstrated the benefits of the proposed jamming structures and the capabilities of the designed grippers.

Robots and intelligent industrial systems that focus on sorting or inspection of products require end-effectors that can grasp and manipulate the objects surrounding them. The capability of such systems largely depends on their ability to efficiently identify the objects and estimate the forces exerted on them. This paper presents an underactuated, compliant, and lightweight hyper-adaptive robot gripper that can efficiently discriminate between different everyday life objects and estimate the contact forces exerted on them during a single grasp, using vision-based techniques. The hyper-adaptive mechanism consists of an array of movable steel rods that get reconfigured conforming to the geometry of the grasped object. The proposed object identification and force estimation techniques are model-free and do not rely on time consuming object exploration. A series of experiments have been carried out to discriminate between 12 different everyday life objects and estimate the forces exerted on a dynamometer. During each grasp, a series of images are captured that detect the reconfiguration of the hyper-adaptive grasping mechanism. These images are then used by an image processing algorithm to extract the required information about the gripper reconfiguration, classify the object grasped using a Random Forests (RF) classifier, and estimate the amount of force being exerted. The employed RF classifier gives a prediction accuracy of 100%, while the results of the force estimation techniques (Neural Networks, Random Forests, and 3rd order polynomial) range from 94.7% to 99.1%.

To facilitate the development of a new generation of Virtual Reality systems and their introduction in everyday life applications, new intuitive, immersive methods of interfacing have to be developed. Over the years, Electromyography (EMG) based interfaces have been utilized for unobtrusive interaction with computer systems. However, previous EMG studies have not explored the continuous decoding of the effects of human motion (e.g., manipulated object behavior) in simulated and virtual environments. In this work, we present an EMG based learning framework that can allow for an immersive interaction with Virtual Reality environments. To do that, EMG activations from the muscles of the forearm and the hand were acquired during the execution of object manipulation tasks in a virtual world along with the motion of the object. The virtual world was visualized using an HTC Vive VR headset, while the hand motions were tracked with a dataglove equipped with magnetic motion capture sensors. The object motion decoding was formulated as a regression problem using the Random Forests methodology. The study shows that the object motion can be successfully decoded using the EMG activations, despite the lack of haptic feedback.

With the emergence of agile manufacturing in highly automated industrial environments, the demand for efficient robot adaptation to dynamic task requirements is increasing. For assembly tasks in particular, classic robot programming methods tend to be rather time intensive. Thus, effectively responding to rapid production changes requires faster and more intuitive robot teaching approaches. This work focuses on combining programming by demonstration with path optimization and local replanning methods to allow for fast and intuitive programming of assembly tasks that requires minimal user expertise. Two demonstration approaches have been developed and integrated in the framework, one that relies on human to robot motion mapping (teleoperation based approach) and a kinesthetic teaching method. The two approaches have been compared with the classic, pendant based teaching. The framework optimizes the demonstrated robot trajectories with respect to the detected obstacle space and the provided task specifications and goals. The framework has also been designed to employ a local replanning scheme that adjusts the optimized robot path based on online feedback from the camera-based perception system, ensuring collision-free navigation and the execution of critical assembly motions. The efficiency of the methods has been validated through a series of experiments involving the execution of assembly tasks. Extensive comparisons of the different demonstration methods have been performed and the approaches have been evaluated in terms of teaching time, ease of use, and path length.