Search

In robotic assembly of flexible flat cables (FFCs), a unique challenge is the inherent difficulty in manipulating such flexible objects compared to their rigid counterparts and the precise estimation of the cable pose. This work proposes a framework that combines object pose estimation using computer-aided design (CAD) models and multiview fusion to perform precise FFC assembly. Our key insight is that a multiview fusion combined with pretrained 6-D pose estimation models offers a more flexible and precise object pose estimation. In a series of experiments involving FFC insertion tasks requiring assembly tolerances down to 0.1 mm, our approach achieves an insertion success rate of 399 out of 400 total attempts. Furthermore, the assembly tasks include the releasing and securing of FFCs from cable connectors, where the system is successful in 200 out of 200 trials. We have also demonstrated the generalization capability of the methodology by successfully completing insertion tasks for common electronic cables like DisplayPort and USB-A, achieving 199 successes in 200 trials. The results not only validate the feasibility of the proposed approach, but also demonstrate its robustness for real-world industrial applications.

Soft robotic devices have gained popularity for their ability perform intricate grasping and dexterous manipulation tasks, providing an alternative to traditional rigid robotic end-effectors. These devices are known for their simplicity, lightweight design, and cost-effectiveness. In recent developments, kirigami-inspired structures have been employed to fabricate affordable and disposable soft robotic grippers and hands. These grippers exhibit a complex post-contact reconfiguration process, adapting to the shape and size of objects they grasp. In this paper, we explore this new class of soft robotic grippers by proposing new designs and investigating their post-contact reconfiguration behaviour in a series of experiments covering grasping experiments and grasping force exertion measurement experiments. Moreover, we leverage their post-contact reconfiguration and further investigate their use in single-grasp object classification. Two classes of kirigami grippers are investigated, extension-based and compression-based. While the former was previously studied in the literature, the later is a novel class of kirigami grippers. We evaluate them and present a performance comparison between the two classes. The results demonstrate the outstanding and varied capabilities of the grippers, ranging from autonomous gripper mounting and disposal, to being able to pick-and-place delicate food items, raw egg yolks, human hair, and even liquids. Furthermore, the single-grasp object classification system exhibits a high accuracy in discriminating objects of various shapes, food items and transparent objects. These research outcomes demonstrate the kirigami-based robotic grippers’ potential in offering robust and intelligent grasping and manipulation solutions.

Over the last decade several machine learning (ML) based data-driven approaches have been used for Electromyography (EMG) based control of prosthetic hands. However, the performance of EMG-based frameworks can be affected by: i) the onset of fatigue due to long data collection sessions, ii) musculoskeletal differences between individuals, and iii) sensor position drifting between different sessions with the same user. To evaluate these aspects, in this work, we compare the performance of EMG-based hand gesture decoding models developed using three approaches. This comparison allows for future works in EMG-based Human-Machine Interfaces development to make more informed ML decisions. First, we trained from scratch a Transformer-based architecture, called Temporal Multi-Channel Vision Transformer (TMC-ViT). For our second approach, we utilized a pre-trained and fine-tuned TMC-ViT model (a transfer learning approach). Finally, for our third approach, we developed a Prototypical Network (a few-shot learning approach). The models are trained in a subject-specific and subject-generic manner for eight subjects and validated employing the 10-fold cross-validation procedure. This study shows that training a deep learning decoding model from scratch in a subject-specific manner leads to higher decoding accuracies when a larger dataset is available. For smaller datasets, subject-generic models, or inter-session models, the few-shot learning approach produces more robust results with better performance, and is more suited to applications where long data collection scenarios are not possible, or where multiple users are intended for the interface. Our findings show that the few-shot learning approach can outperform training a model from scratch in different scenarios.

Mobile and aerial robots offer many potential applications, including warehouse logistics, surveillance, cinematography, and search and rescue. However, most such robots are task-specific and generally lack the versatility to tackle multiple scenarios, terrains, and unstructured, dynamic environments. This paper presents the Omnirotor platform, a versatile, multi-modal, coaxial, tilt-rotor, all-terrain vehicle that combines an Unmanned Aerial Vehicle (UAV) and an Unmanned Ground Vehicle (UGV) into a hybrid, all-terrain vehicle. The Omnirotor has two locomotion modes of operation (aerial and ground vehicle) and five operational configurations, as it can fly both in the Normal and Inverted configurations and drive on the ground in the Normal and Inverted configurations. It can also recover from any non-operational state to its Normal, upside-down configuration. Moreover, in addition to the locomotion modes, the continuous omnidirectional thrust vectoring enables the Omnirotor platform to perform complex manipulation of objects. This work introduces the concept and discusses in detail the design, development, and experimental validation of the Omnirotor platform. In particular, it discusses the modeling and control schemes required by the different operation modes and configurations. It experimentally validates the platform’s capabilities with experiments focusing on traversing challenging environments and unstructured, uneven terrains (e.g., a public park). Finally, the platform’s ground, pushing-based manipulation capabilities are demonstrated through the execution of a puzzle-solving experiment where the solved puzzle serves as a landing platform for the all-terrain vehicle. The versatility of the Omnirotor offers exciting prospects for use in challenging search-and-rescue scenarios, surveillance, and aerial and ground manipulation applications.

Abstract Conventional muscle-machine interfaces like Electromyography (EMG), have significant drawbacks, such as crosstalk, a non-linear relationship between the signal and the corresponding motion, and increased signal processing requirements. In this work, we introduce a new muscle-machine interfacing technique called lightmyography (LMG), that can be used to efficiently decode human hand gestures, motion, and forces from the detected contractions of the human muscles. LMG utilizes light propagation through elastic media and human tissue, measuring changes in light luminosity to detect muscle movement. Similar to forcemyography, LMG infers muscular contractions through tissue deformation and skin displacements. In this study, we look at how different characteristics of the light source and silicone medium affect the performance of LMG and we compare LMG and EMG based gesture decoding using various machine learning techniques. To do that, we design an armband equipped with five LMG modules, and we use it to collect the required LMG data. Three different machine learning methods are employed: Random Forests, Convolutional Neural Networks, and Temporal Multi-Channel Vision Transformers. The system has also been efficiently used in decoding the forces exerted during power grasping. The results demonstrate that LMG outperforms EMG for most methods and subjects.

Electromyography (EMG) signals have been used in designing muscle-machine interfaces (MuMIs) for various applications, ranging from entertainment (EMG controlled games) to human assistance and human augmentation (EMG controlled prostheses and exoskeletons). For this, classical machine learning methods such as Random Forest (RF) models have been used to decode EMG signals. However, these methods depend on several stages of signal pre-processing and extraction of hand-crafted features so as to obtain the desired output. In this work, we propose EMG based frameworks for the decoding of object motions in the execution of dexterous, in-hand manipulation tasks using raw EMG signals input and two novel deep learning (DL) techniques called Temporal Multi-Channel Transformers and Vision Transformers. The results obtained are compared, in terms of accuracy and speed of decoding the motion, with RF-based models and Convolutional Neural Networks as a benchmark. The models are trained for 11 subjects in a motion-object specific and motion-object generic way, using the 10-fold cross-validation procedure. This study shows that the performance of MuMIs can be improved by employing DL-based models with raw myoelectric activations instead of developing DL or classic machine learning models with hand-crafted features.

Electromyography (EMG) signals are commonly used for the development of Muscle Machine Interfaces. EMG-based solutions provide intuitive and often hand-free control in a wide range of applications that range from the decoding of human intention in classification tasks to the continuous decoding of human motion employing regression models. In this work, we compare various machine learning and feature extraction methods for the creation of EMG based control frameworks for dexterous robotic telemanipulation. Various models are needed that can decode dexterous, in-hand manipulation motions and perform hand gesture classification in real-time. Three different machine learning methods and eight different time-domain features were evaluated and compared. The performance of the models was evaluated in terms of accuracy and time required to predict a data sample. The model that presented the best performance and prediction time trade-off was used for executing in real-time a telemanipulation task with the New Dexterity Autonomous Robotic Assistance (ARoA) platform (a humanoid robot). Various experiments have been conducted to experimentally validate the efficiency of the proposed methods. The robotic system is shown to successfully complete a series of tasks autonomously as well as to efficiently execute tasks in a shared control manner.

Technological imaginaries have been increasingly shaping the future perceptions of cities. From artificial intelligence and distributed ledger technology to three-dimensional printing, high-tech artifacts are very often the premises of such imaginaries. However, technology does not only refer to artifacts. Technology also encompasses the processes around the artifacts: how the artifacts are designed, manufactured, used, maintained, and disposed. From this perspective, high-tech visions often disregard problems that pertain to resource extraction, labor exploitation, energy use, and material flows. On the contrary, low-tech and localized alternatives incite lower impact and higher resilience visions. However, they fail to offer solutions of the desired scale and intensity. To address this tension, we provide an alternative vision for mid-tech: a balance between the opposite extreme qualities of low-tech and high-tech. Through a case of open-source prosthetics, we illustrate how to synergistically combine the efficiency and versatility of high-tech solutions with the potential for autonomy and resilience that low-tech offers. Then we discuss a mid-tech approach for distributed ledger technology from a city as a license lens. We provide connections with existing or conceptual applications to show how distributed ledger technology could support more socially and ecologically responsible data practices for city governance.

The increased use of Virtual and Augmented Reality based systems necessitates the development of more intuitive and unobtrusive means of interfacing. Over the last years, Electromyography (EMG) based interfaces have been employed for interaction with robotic and computer applications, but no studies have been carried out to investigate the continuous decoding of the effects of human motion (e.g., manipulated object behavior) in simulated and virtual environments. In this work, we compare the object motion decoding accuracy of an EMG based learning framework for two different dexterous manipulation scenarios: i) for simulated objects handled by a teleoperated model of a hand within a virtual environment and ii) for real, everyday life objects manipulated by the human hand. To do that, we utilize EMG activations from 16 muscle sites (9 on the hand and 7 on the forearm). The object motion decoding is formulated as a regression problem using the Random Forests methodology. A 5-fold cross validation procedure is used for model assessment purposes and the feature variable importance values are calculated for each model. The decoding accuracy for the real world is considerably higher than the virtual world. Each of the objects examined had a single manipulation motion that offered the highest estimation accuracy across both worlds. This study also shows that it is feasible to decode the object motions using just the myoelectric activations of the muscles of the forearm and the hand. This is particularly surprising since simulations lacked haptic feedback and the ability to account for other dynamic phenomena like friction and contact rolling.

This work proposes a framework that improves the dexterous manipulation capabilities of two fingered grippers by: i) optimizing the finger link dimensions and the interfinger distance for a given object and ii) analyzing the effect of finger symmetry and the distance between the finger base frames on their manipulation workspaces. The results of the workspace analysis motivate the development of a multi-modal, adaptive robotic gripper. In particular, the finger link lengths optimization problem is solved by a parallel multi-start search algorithm. The optimal link lengths are then used for the workspace analysis. The results of the analysis demonstrate that different inter-finger distances lead to completely different workspace shapes and that the ratio defined by the area of the optimized workspace (nominator) and the union of all workspaces (denominator), is always significantly less than 1. This means that the area of the union of all workspaces is always larger than the area of the “optimized” workspace. Based on these results the proposed robotic gripper is equipped with reconfigurable finger bases that vary the inter-finger distance as well as with selectively lockable robotic finger joints, offering an increased dexterous manipulation performance without sacrificing grasping efficiency. The device is considered multi-modal as it can be used both as a parallel jaw gripper and as an adaptive robotic gripper.

Evaluating the dexterity of human and robotic hands through appropriate benchmarks, scores, and metrics is of paramount importance for determining how skillful humans are and for designing and developing new bioinspired or even biomimetic end-effectors (e.g., robotic grippers and hands). Dexterity tests have been used in industrial and medical settings to assess how dexterous the hands of workers and surgeons are as well as in robotic rehabilitation settings to determine the improvement or deterioration of the hand function after a stroke or a surgery. In robotics, having a comprehensive dexterity test can allow us to evaluate and compare grippers and hands irrespectively of their design characteristics. However, there is a lack of well defined metrics, benchmarks, and tests that quantify robot dexterity. Previous work has focused on a number of widely accepted functional tests that are used for the evaluation of manual dexterity and human hand function improvement post injury. Each of these tests focuses on a different set of specific tasks and objects. Deriving from these tests, this work proposes a new modular, affordable, accessible, open-source dexterity test for both humans and robots. This test evaluates the grasping and manipulation capabilities by combining the features and best practices of the aforementioned tests, as well as new task categories specifically designed to evaluate dexterous manipulation capabilities. The dexterity test and the accompanying benchmarks allow us to determine the overall hand function recovery and dexterity of robotic end-effectors with ease. More precisely, a dexterity score that ranges from 0 (simplistic, non-dexterous system) to 1 (human-like system) is calculated using the weighted sum of the accuracy and task execution speed subscores. It should also be noted that the dexterity of a robotic system can be evaluated assessing the efficiency of either the robotic hardware, or the robotic perception system, or both. The test and the benchmarks proposed in the study have been validated using extensive human and robot trials. The human trials have been used to determine the baseline scores for the evaluation system. The results show that the time required to complete the tasks reduces significantly with trials indicating a clear learning curve in mastering the dexterous manipulation capabilities associated with the imposed tasks. Finally, the time required to complete the tasks with restricted tactile feedback is significantly higher indicating its importance.

Miniature indoor robotic airship platforms offer high mobility, safety, and extended flight times. This paper focuses on the feasibility, design, development, and evaluation of such a platform for robotics education and research. Selected commercially available envelope materials were considered and tested in terms of their helium retention capability and mechanical properties. The obtained envelope properties were used in a feasibility study, demonstrating that indoor airships are environmentally and financially viable, given an appropriate material choice. The platform's mechanical design was studied in terms of gondola placement and rotor angle positioning, resulting in an unconventional, asymmetric arrangement. The developed system was finally tested in a simple path following experiment for proof-of-concept purposes, proving its efficiency in attaining the desired heading and altitude configuration. The proposed robotic airship platform can be used for a variety of education and research oriented applications. Its design is open-source, facilitating replication by others.

Robotic hand exoskeletons have become a popular and efficient technological solution for assisting people that suffer from neurological conditions and for enhancing the capabilities of healthy individuals. This class of devices ranges from rigid and complex structures to soft, lightweight, wearable gloves. In this work, we propose a hybrid (tendon-driven and pneumatic), lightweight, affordable, easy-to-operate exoskeleton glove equipped with variable stiffness, laminar jamming structures, abduction/adduction capabilities, and a pneumatic telescopic extra thumb that increases grasp stability. The efficiency of the proposed device is experimentally validated through five different types of experiments: i) abduction/adduction tests, ii) force exertion experiments that capture the forces that can be exerted by the proposed device under different conditions, iii) bending profile experiments that evaluate the effect of the laminar jamming structures on the way the fingers bend, iv) grasp quality assessment experiments that focus on the effect of the inflatable thumb on enhancing grasp stability, and v) grasping experiments involving everyday objects and seven subjects. The hybrid assistive, exoskeleton glove considerably improves the grasping capabilities of the user, being able to exert the forces required to execute a plethora of activities of daily living. All files that allow the replication of the device are distributed in an open-source manner.

Over the last decade, there has been an increased interest in developing aerial robotic platforms that exhibit grasping and perching capabilities not only within the research community but also in companies across different industry sectors. Aerial robots range from standard multicopter vehicles/drones, to autonomous helicopters, and fixed-wing or hybrid devices. Such devices rely on a range of different solutions for achieving grasping and perching. These solutions can be classified as: 1) simple gripper systems, 2) arm-gripper systems, 3) tethered gripping mechanisms, 4) reconfigurable robot frames, 5) adhesion solutions, and 6) embedment solutions. Grasping and perching are two crucial capabilities that allow aerial robots to interact with the environment and execute a plethora of complex tasks, facilitating new applications that range from autonomous package delivery and search and rescue to autonomous inspection of dangerous or remote environments. In this review paper, we present the state-of-the-art in aerial grasping and perching mechanisms and we provide a comprehensive comparison of their characteristics. Furthermore, we analyze these mechanisms by comparing the advantages and disadvantages of the proposed technologies and we summarize the significant achievements in these two research topics. Finally, we conclude the review by suggesting a series of potential future research directions that we believe that are promising.

Over the last decade underactuated, adaptive robot grippers and hands have received an increased interest from the robotics research community. This class of robotic end-effectors can be used in many different fields and scenarios with a very promising application being the development of prosthetic devices. Their suitability for the development of such devices is attributed to the utilization of underactuation that provides increased functionality and dexterity with reduced weight, cost, and control complexity. The most critical components of underactuated, adaptive hands that allow them to perform a broad set of grasp poses are appropriate differential mechanisms that facilitate the actuation of multiple degrees of freedom using a single motor. In this work, we focus on the design, analysis, and experimental validation of a four output geared differential, a series elastic differential, and a whiffletree differential that can incorporate a series of manual and automated locking mechanisms. The locking mechanisms have been developed so as to enhance the control of the differential outputs, allowing for efficient grasp selection with a minimal set of actuators. The differential mechanisms are applied to prosthetic hands, comparing them and describing the benefits and the disadvantages of each.

Robot grasping in unstructured and dynamic environments is heavily dependent on the object attributes. Although Deep Learning approaches have delivered exceptional performance in robot perception, human perception and reasoning are still superior in processing novel object classes. Furthermore, training such models requires large, difficult to obtain datasets. This work combines crowdsourcing and gamification to leverage human intelligence, enhancing the object recognition and attribute estimation processes of robot grasping. The framework employs an attribute matching system that encodes visual information into an online puzzle game, utilizing the collective intelligence of players to expand the attribute database and react to real-time perception conflicts. The framework is deployed and evaluated in two proof-of-concept applications: enhancing the control of a robotic exoskeleton glove and improving object identification for autonomous robot grasping. In addition, a model for estimating the framework response time is proposed. The obtained results demonstrate that the framework is capable of rapid adaptation to novel object classes, based purely on visual information and human experience.

Traditionally, the robotic end-effectors that are employed in unstructured and dynamic environments are rigid and their operation requires sophisticated sensing elements and complicated control algorithms in order to handle and manipulate delicate and fragile objects. Over the last decade, considerable research effort has been put into the development of adaptive, under-actuated, soft robots that facilitate robust interactions with dynamic environments. In this paper, we present soft, retractable, pneumatically actuated, telescopic actuators that facilitate the efficient execution of stable grasps involving a plethora of everyday life objects. The efficiency of the proposed actuators is validated by employing them in two different soft and hybrid robotic grippers. The hybrid gripper uses three rigid fingers to accomplish the execution of all the tasks required by a traditional robotic gripper, while three inflatable, telescopic fingers provide soft interaction with objects. This synergistic combination of soft and rigid structures allows the gripper to cage/trap and firmly hold heavy and irregular objects. The second, simplistic and highly affordable robotic gripper employs just the telescopic actuators, exhibiting an adaptive behavior during the execution of stable grasps of fragile and delicate objects. The experiments demonstrate that both grippers can successfully and stably grasp a wide range of objects, being able to exert significantly high contact forces.

Adaptive robot hands are typically created by introducing structural compliance either in their joints (e.g., implementation of flexures joints) or in their finger-pads. In this paper, we present a series of alternative uses of structural compliance for the development of simple, adaptive, compliant and/or under-actuated robot grippers and hands that can efficiently and robustly execute a variety of grasping and dexterous, in-hand manipulation tasks. The proposed designs utilize only one actuator per finger to control multiple degrees of freedom and they retain the superior grasping capabilities of the adaptive grasping mechanisms even under significant object pose or other environmental uncertainties. More specifically, in this work, we introduce, discuss, and evaluate: (a) a design of pre-shaped, compliant robot fingers that adapts/conforms to the object geometry, (b) a hyper-adaptive finger-pad design that maximizes the area of the contact patches between the hand and the object, maximizing also grasp stability, and (c) a design that executes compliance adjustable manipulation tasks that can be predetermined by tuning the in-series compliance of the tendon routing system and by appropriately selecting the imposed tendon loads. The grippers are experimentally tested and their efficiency is validated using three different types of tests: (i) grasping tests that involve different everyday objects, (ii) grasp quality tests that estimate the contact area between the grippers and the objects grasped, and (iii) dexterous, in-hand manipulation experiments to evaluate the manipulation capabilities of the Compliance Adjustable Manipulation (CAM) hand. The devices employ mechanical adaptability to facilitate and simplify the efficient execution of robust grasping and dexterous, in-hand manipulation tasks.

This paper presents an adaptive actuation mechanism that can be employed for the development of anthropomorphic, dexterous robot hands. The tendon-driven actuation mechanism achieves both flexion/extension and adduction/abduction on the finger's metacarpophalangeal joint using two actuators. Moment arm pulleys are employed to drive the tendon laterally and achieve a simultaneous execution of abduction and flexion motion. Particular emphasis has been given to the modeling and analysis of the actuation mechanism. More specifically, the analysis determines specific values for the design parameters for desired abduction angles. Also, a model for spatial motion is provided that relates the actuation modes with the finger motions. A static balance analysis is performed for the computation of the tendon force at each joint. A model is employed for the computation of the stiffness of the rotational flexure joints. The proposed mechanism has been designed and fabricated with the hybrid deposition manufacturing technique. The efficiency of the mechanism has been validated with experiments that include the assessment of the role of friction, the computation of the reachable workspace, the assessment of the force exertion capabilities, the demonstration of the feasible motions, and the evaluation of the grasping and manipulation capabilities. An anthropomorphic robot hand equipped with the proposed actuation mechanism was also fabricated to evaluate its performance. The proposed mechanism facilitates the collaboration of actuators to increase the exerted forces, improving hand dexterity and allowing the execution of dexterous manipulation tasks.