Search

Human manual dexterity relies critically on touch. Robotic and prosthetic hands are much less dexterous and make little use of the many tactile sensors available. We propose a framework modeled on the hierarchical sensorimotor controllers of the nervous system to link sensing to action in human-in-the-loop, haptically enabled, artificial hands.

State-of-the-art myoelectric hand prostheses provide multi-functional control but lack somatosensory feedback. To accommodate the full functionality of a dexterous prosthesis, the artificial sensory feedback needs to convey several degrees of freedom (DoF) simultaneously. However, this is a challenge with current methods as they are characterized by a low information bandwidth. In this study, we leverage the flexibility of a recently developed system for simultaneous electrotactile stimulation and electromyography (EMG) recording to present the first solution for closed-loop myoelectric control of a multifunctional prosthesis with full-state anatomically congruent electrotactile feedback. The novel feedback scheme (coupled encoding) conveyed proprioceptive (hand aperture, wrist rotation) and exteroceptive information (grasping force). The coupled encoding was compared to the conventional approach (sectorized encoding) and incidental feedback in 10 non-disabled and one amputee participant who used the system to perform a functional task. The results showed that both feedback approaches increased the accuracy of position control compared to incidental feedback. However, the feedback increased completion time, and it did not significantly improve grasping force control. Importantly, the performance of the coupled feedback was not significantly different compared to the conventional scheme, despite the latter being easier to learn during training. Overall, the results indicate that the developed feedback can improve prosthesis control across multiple DoFs but they also highlight the subjects' ability to exploit minimal incidental information. Importantly, the current setup is the first to convey three feedback variables simultaneously using electrotactile stimulation while providing multi-DoF myoelectric control with all hardware components mounted on the same forearm.

Reliable force control is especially important when using myoelectric upper-limb prostheses as the force defines whether an object will be firmly grasped, damaged, or dropped. It is known from human motor control that the grasping of non-disabled subjects is based on a combination of anticipation and feedback correction. Inspired by this insight, the present study proposes a novel approach to provide artificial sensory feedback to the user of a myoelectric prosthesis using vibrotactile stimulation to facilitate both predictive and corrective processes characteristic of grasping in non-disabled people. Specifically, the level of EMG was conveyed to the subjects while closing the prosthesis (predictive strategy), whereas the actual grasping force was transmitted when the prosthesis closed (corrective strategy). To investigate if this combined EMG and force feedback is indeed an effective method to explicitly close the control loop, 16 non-disabled and 3 transradial amputee subjects performed a set of functional tasks, inspired by the "Box and Block" test, with six target force levels, in three conditions: no feedback, only EMG feedback, and combined feedback. The highest overall performance in non-disabled subjects was obtained with combined feedback (79.6±9.9%), whereas the lowest was achieved with no feedback (53±11.5%). The combined feedback, however, increased the task completion time compared to the other two conditions. A similar trend was obtained also in three amputee subjects. The results, therefore, indicate that the feedback inspired by human motor control is indeed an effective approach to improve prosthesis grasping in realistic conditions when other sources of feedback (vision and audition) are not blocked.

Just-noticeable difference (JND), indicating the ability to accurately identify small differences in stimulation parameters, can be used to choose more sensitive stimulation methods as well as to calibrate tactile feedback in closed-loop human-machine interfacing. The JND is typically estimated using a forced-choice-discrimination task, in which two stimuli with different intensities are delivered separated by a brief pause. In the applications of tactile feedback, however, the stimulation parameters are typically modulated continuously. It is unclear if the discriminability of stimuli separated in time characterizes the ability to distinguish continuous changes in stimulation intensity. The present study compared the JND when pairs of frequency-modulated electrotactile stimuli were separated in time and presented continuously at two different baseline frequencies (20 and 60 Hz). The results showed that the JND was significantly smaller with time-separation between stimuli, but that the JND obtained with different types of transitions were in most cases linearly associated. In conclusion, the discriminability of time-separated stimuli is systematically better compared to that of the stimuli presented continuously. This can have an impact when calibrating the tactile feedback where the conventional method of the JND assessment might lead to an overly optimistic estimate of detectable changes.

Closing the prosthesis control loop by providing artificial somatosensory feedback can improve utility and user experience. Additionally, closed-loop control should be more robust with respect to disturbance, but this might depend on the type of feedback provided. Thus, the present study investigates and compares the performance of EMG and force feedback in the presence of control disturbances. Twenty able-bodied subjects and one transradial amputee performed delicate and power grasps with a prosthesis in a functional task, while the control signal gain was temporarily increased (high-gain disturbance) or decreased (low-gain disturbance) without their knowledge. Three outcome measures were considered: the percentage of trials successful in the first attempt (reaction to disturbance), the average number of attempts in trials where the wrong force was initially applied (adaptation to disturbance), and the average completion time of the last attempt in every trial. EMG feedback was shown to offer significantly better performance compared to force feedback during power grasping in terms of reaction to disturbance and completion time. During power grasping with high-gain disturbance, the median first-attempt success rate was significantly higher with EMG feedback (73.3%) compared to that achieved with force feedback (60%). Moreover, the median completion time for power grasps with low-gain disturbance was significantly longer with force feedback than with EMG feedback (3.64 against 2.48 s, an increase of 32%). Contrary to our expectations, there was no significant difference between feedback types with regards to adaptation to disturbances and the two feedback types performed similarly in delicate grasps. The results indicated that EMG feedback displayed better performance than force feedback in the presence of control disturbances, further demonstrating the potential of this approach to provide a reliable prosthesis-user interaction., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Tchimino, Dideriksen and Dosen.)

Objective . Closed-loop prosthesis interfaces, which combine electromyography (EMG)-based control with supplementary feedback, represent a promising direction for developing the next generation of bionic limbs. However, we still lack an understanding of how users utilize these interfaces and how to evaluate competing solutions. In this study, we used the framework of speed-accuracy trade-off functions (SAF) to understand, evaluate, and compare the performance of two closed-loop user-prosthesis interfaces. Approach . Ten able-bodied participants and an amputee performed a force-matching task in a functional box-and-block setup at three different speeds. All participants were subjected to both interfaces in a crossover study design with a 1 week washout period. Importantly, both interfaces used direct proportional control but differed in the feedback provided to the participant (EMG feedback vs. Force feedback). We estimated the SAFs afforded by the two interfaces and sought to understand how the participants planned and executed the task under the various conditions. Main results . We found that execution speed significantly influenced performance, and that EMG feedback afforded better overall performance, especially at medium speeds. Notably, we found that there was a difference in the SAF between the two interfaces, with EMG feedback enabling participants to attain higher accuracies faster than Force feedback. Furthermore, both interfaces enabled participants to develop flexible control policies, while EMG feedback also afforded participants the ability to generate smoother, more repeatable EMG commands. Significance . Overall, the results indicate that the performance of closed-loop prosthesis interfaces depends critically on the feedback approach and execution speed. This study showed that the SAF framework could be used to reveal the differences between feedback approaches, which might not have been detected if the assessment was performed at a single speed. Therefore, we argue that it is important to consider the speed-accuracy trade-offs to rigorously evaluate and compare user-prosthesis interfaces., (© 2022 IOP Publishing Ltd.)

Tactile feedback is relevant in a broad range of human-machine interaction systems (e.g. teleoperation, virtual reality and prosthetics). The available tactile feedback interfaces comprise few sensing and stimulation units, which limits the amount of information conveyed to the user. The present study describes a novel technology that relies on distributed sensing and stimulation to convey comprehensive tactile feedback to the user of a robotic end effector. The system comprises six flexible sensing arrays (57 sensors) integrated on the fingers and palm of a robotic hand, embedded electronics (64 recording channels), a multichannel stimulator and seven flexible electrodes (64 stimulation pads) placed on the volar side of the subject's hand. The system was tested in seven subjects asked to recognize contact positions and identify contact sliding on the electronic skin, using distributed anode configuration (DAC) and single dedicated anode configuration. The experiments demonstrated that DAC resulted in substantially better performance. Using DAC, the system successfully translated the contact patterns into electrotactile profiles that the subjects could recognize with satisfactory accuracy ([Formula: see text] for static and [Formula: see text] for dynamic patterns). The proposed system is an important step towards the development of a high-density human-machine interfacing between the user and a robotic hand. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Early rehabilitation is beneficial for stroke patients, but it is often delayed since the patients are often bedbound due to their general condition. New robotic rehabilitation devices such as ROBERT® enable patients to exercise even while bedbound. During pilot testing of an automated FES-delivery system combined with ROBERT®, we observed both increased and decreased exerted velocity and interaction force during repetitive exercising with the system. The goal of the current study was thus to investigate the extent of both potentiation and fatigue, as assessed in the velocity and interaction force produced in response to repetitive robotic-FES exercising. Eight healthy subjects completed 50 repetitions of leg-press exercises using the hybrid robotic-FES system. For individual subjects, significant changes were found for both mean and maximal velocities and interaction forces exerted during the exercise. Roughly half of the subjects had an increase in maximal velocity and interaction force during the exercise, and half exhibited an increase in mean velocity, whereas three subjects had an increase in mean interaction force during the exercise. The changes in mean velocity were in the range of -40.6 to 30.9% and for the maximal velocity they were in the range of -21.9 to 22.0%. The changes for mean interaction force were in the range of -5.8 to 11.0%, while for the maximal interaction force, they were in the range of -7.8 to 14.4%. These changes might pose significant challenges for future developments of hybrid robotic-FES rehabilitation systems, as the system must be able to comply with the observed changes, and appropriately adapt to them in order to maintain efficacy and safety.

Development of haptic interfaces to enrich augmented and virtual reality with the sense of touch is the next frontier for technological advancement of these systems. Among available technologies, electrotactile stimulation enables design of high-density interfaces that can provide natural-like sensation of touch in interaction with virtual objects. The present study investigates the human perception of electrotactile sensations on fingertips, focusing on the sensation localization in function of the size and position of reference electrode. Ten healthy subjects participated in the study, with the task to mark the sensations elicited by stimulating the index fingertip using an 8-pad electrode. The test systematically explored several configurations of the active (position) and reference (position and size) electrode pads. The results indicated that there was a spreading of perceived sensations across the fingertip, but that they were mostly localized below the active pad. The position and size of the reference electrode were shown to affect the location of the perceived sensations, which can potentially be exploited as an additional parameter to modulate the feedback. The present study demonstrates that the fingertip is a promising target for the delivery of high-resolution feedback.

Modern myoelectric prostheses can perform multiple functions (e.g., several grasp types and wrist rotation) but their intuitive control by the user is still an open challenge. It has been recently demonstrated that semi-autonomous control can allow the subjects to operate complex prostheses effectively; however, this approach often requires placing sensors on the user. The present study proposes a system for semi-autonomous control of a myoelectric prosthesis that requires a single depth sensor placed on the dorsal side of the hand. The system automatically pre-shapes the hand (grasp type, size, and wrist rotation) and allows the user to grasp objects of different shapes, sizes and orientations, placed individually or within cluttered scenes. The system "reacts" to the side from which the object is approached, and enables the user to target not only the whole object but also an object part. Another unique aspect of the system is that it relies on online interaction between the user and the prosthesis; the system reacts continuously on the targets that are in its focus, while the user interprets the movement of the prosthesis to adjust aiming. Experimental assessment was conducted in ten able-bodied participants to evaluate the feasibility and the impact of training on prosthesis-user interaction. The subjects used the system to grasp a set of objects individually (Phase I) and in cluttered scenarios (Phase II), while the time to accomplish the task (TAT) was used as the performance metric. In both phases, the TAT improved significantly across blocks. Some targets (objects and/or their parts) were more challenging, requiring thus significantly more time to handle, but all objects and scenes were successfully accomplished by all subjects. The assessment therefore demonstrated that the system is indeed robust and effective, and that the subjects could successfully learn how to aim with the system after a brief training. This is an important step toward the development of a self-contained semi-autonomous system convenient for clinical applications., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Castro and Dosen.)

Electro- and vibro-tactile stimulation are commonly employed for feedback in closed-loop human-machine interfacing. Although these feedback systems have been extensively investigated individually, they are rarely objectively compared. In this study, two state-of-the-art stimulation units (concentric electrode and C2-tactor) similar in shape and size were compared in psychometric and online control tests. The just noticeable difference and number of discriminable levels for intensity and frequency modulation were determined across values of carrier frequency and intensity, respectively. Next, subjects performed a compensatory tracking task, in which the feedback encoded the momentary tracking error. In the psychometric tests, intensity modulation outperformed frequency modulation and electrotactile stimulation enabled significantly higher resolution than vibrotactile stimulation, for the same carrier frequency. However, for the best-case settings (eletro-tactile: 100 Hz; vibro-tactile: 200 Hz), the two stimulation modalities were equivalent in the psychometric tests and in the online control tests, where the two stimulation methods resulted in similar correlation and deviation between the target and the generated trajectory. Time delay was slightly but significantly lower for the vibrotactile modality. Overall, the present assessment shows that despite psychometric differences between the two stimulation methods, they enable similar online control performance when parameters are optimally selected for each modality.

Semi-autonomous (SA) control of upper-limb prostheses can improve the performance and decrease the cognitive burden of a user. In this approach, a prosthesis is equipped with additional sensors (e.g., computer vision) that provide contextual information and enable the system to accomplish some tasks automatically. Autonomous control is fused with a volitional input of a user to compute the commands that are sent to the prosthesis. Although several promising prototypes demonstrating the potential of this approach have been presented, methods to integrate the two control streams (i.e., autonomous and volitional) have not been systematically investigated. In the present study, we implemented three shared control modalities (i.e., sequential, simultaneous , and continuous ) and compared their performance, as well as the cognitive and physical burdens imposed on the user. In the sequential approach, the volitional input disabled the autonomous control. In the simultaneous approach, the volitional input to a specific degree of freedom (DoF) activated autonomous control of other DoFs, whereas in the continuous approach, autonomous control was always active except for the DoFs controlled by the user. The experiment was conducted in ten able-bodied subjects, and these subjects used an SA prosthesis to perform reach-and-grasp tasks while reacting to audio cues (dual tasking). The results demonstrated that, compared to the manual baseline (volitional control only), all three SA modalities accomplished the task in a shorter time and resulted in less volitional control input. The simultaneous SA modality performed worse than the sequential and continuous SA approaches. When systematic errors were introduced in the autonomous controller to generate a mismatch between the goals of the user and controller, the performance of SA modalities substantially decreased, even below the manual baseline. The sequential SA scheme was the least impacted one in terms of errors. The present study demonstrates that a specific approach for integrating volitional and autonomous control is indeed an important factor that significantly affects the performance and physical and cognitive load, and therefore these should be considered when designing SA prostheses., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2021 Mouchoux, Bravo-Cabrera, Dosen, Schilling and Markovic.)

As the technology moves towards more human-like bionic limbs, it is necessary to develop a feedback system that provides active touch feedback to a user of a prosthetic hand. Most of the contemporary sensory substitution methods comprise simple position and force sensors combined with few discrete stimulation units, and hence they are characterized with a limited amount of information that can be transmitted by the feedback. The present study describes a novel system for tactile feedback integrating advanced multipoint sensing (electronic skin) and stimulation (matrix electrodes). The system comprises a flexible sensing array (16 sensors) integrated on the index finger of a Michelangelo prosthetic hand mockup, embedded interface electronics and multichannel stimulator connected to a flexible matrix electrode (24 pads). The developed system conveys contact information (binary detections) to the user. To demonstrate the feasibility, the system was tested in six able-bodied subjects who were asked to recognize static patterns (contact position) with two different spatial resolutions and dynamic movement patterns (i.e., sliding along and/or across the finger) presented on the electronic skin. The experiments demonstrated that the system successfully translated the mechanical interaction into electrotactile profiles, which the subjects could recognize with good performance. The success rates (mean ± standard deviation) for the static patterns were 91 ± 4% and 58 ± 10% for low and high spatial resolution, respectively, while the success rate for sliding touch was 94 ± 4%. These results demonstrate that the developed system is an important step towards a new generation of tactile feedback interfaces that can provide high-bandwidth connection between the user and his/her bionic limb. Such systems would allow mimicking spatially distributed natural feedback, thereby facilitating the control and embodiment of the artificial device into the user body scheme.

Objective. Supplemental sensory feedback for myoelectric prostheses can provide both psychosocial and functional benefits during prosthesis control. However, the impact of feedback depends on multiple factors and there is insufficient understanding about the fundamental role of such feedback in prosthesis use. The framework of human motor control enables us to systematically investigate the user-prosthesis control loop. In this study, we explore how different task objectives such as speed and accuracy shape the control policy developed by participants in a prosthesis force-matching task. Approach. Participants were randomly assigned to two groups that both used identical electromyography control interface and prosthesis force feedback, through vibrotactile stimulation, to perform a prosthesis force-matching task. However, the groups received different task objectives specifying speed and accuracy demands. We then investigated the control policies developed by the participants. To this end, we not only evaluated how successful or fast participants were but also analyzed the behavioral strategies adopted by the participants to obtain such performance gains. Main results. First, we observed that participants successfully integrated supplemental prosthesis force feedback to develop both feedforward and feedback control policies, as demanded by the task objectives. We then observed that participants who first developed a (slow) feedback policy were quickly able to adapt their policy to more stringent speed demands, by switching to a combined feedforward-feedback control strategy. However, the participants who first developed a (fast) feedforward policy were not able to change their control policy and adjust to greater accuracy demands. Significance. Overall, the results signify how the framework of human motor control can be applied to study the role of feedback in user-prosthesis interaction. The results also reveal the utility of training prosthesis users to integrate supplemental feedback into their state estimation by designing training protocols that encourage the development of combined feedforward and feedback policy., (© 2021 IOP Publishing Ltd.)

Objective. The nervous system subconsciously estimates the state of the body as a weighted average of the information from various sensory sources, where the weights reflect the perceived reliability of each source. Loss of motor functions can be partially compensated using assistive systems (e.g. prostheses), which may also restore somatosensory feedback through tactile stimulation. Whether such artificial feedback is integrated in the neural state estimation process is not known. Approach. In this study, able-bodied subjects performed a grasp force matching task with supplementary non-invasive electrotactile stimulation with a frequency proportional to grasp force magnitude. Before the task, a brief training session taught the subjects to associate the sensation of electrotactile stimulation with the generated grasp force. In some trials, the force-frequency mapping was biased to introduce an unnoticeable mismatch between natural and electrotactile force feedback, thereby provoking the subject to subconsciously estimate the force as a compromise between the two sources of information. Main results. The outcome of this compromise revealed the weights assigned to each feedback type. The grasp forces were significantly affected by the biased mappings, as indicated by the average estimated relative weights (electrotactile: 0.69 ± 0.29; natural: 0.31 ± 0.29). Across subjects, this weight was correlatedr2=0.75) with the improvement in force matching precision when adding the unbiased electrotactile feedback to the natural force feedback, as predicted by maximum likelihood estimation. This shows that even after minimal training the nervous system adopts electrotactile stimulation as a highly reliable source of information that can improve the precision in the estimation of the grip force. Significance. This result has important implications for the restoration of sensory feedback in upper limb prostheses as it indicates that even non-invasive stimulation can be integrated naturally (i.e. subconsciously and effectively) in the motor controlloop., (© 2021 IOP Publishing Ltd.)

Objective. Brain-computer interface (BCI) systems can be employed to provide motor and communication assistance to patients suffering from neuromuscular diseases, such as amyotrophic lateral sclerosis (ALS). Movement related cortical potentials (MRCPs), which are naturally generated during movement execution, can be used to implement a BCI triggered by motor attempts. Such BCI could assist impaired motor functions of ALS patients during disease progression, and facilitate the training for the generation of reliable MRCPs. The training aspect is relevant to establish a communication channel in the late stage of the disease. Therefore, the aim of this study was to investigate the possibility of detecting MRCPs associated to movement intention in ALS patients with different levels of disease progression from slight to complete paralysis. Approach. Electroencephalography signals were recorded from nine channels in 30 ALS patients at various stages of the disease while they performed or attempted to perform hand movements timed to a visual cue. The movement detection was implemented using offline classification between movement and rest phase. Temporal and spectral features were extracted using 500 ms sliding windows with 50% overlap. The detection was tested for each individual channel and two surrogate channels by performing feature selection followed by classification using linear and non-linear support vector machine and linear discriminant analysis. Main results. The results demonstrated that the detection performance was high in all patients (accuracy 80.5 ± 5.6%) but that the classification parameters (channel, features and classifier) leading to the best performance varied greatly across patients. When the same channel and classifier were used for all patients (participant-generic analysis), the performance significantly decreased (accuracy 74 ± 8.3%). Significance. The present study demonstrates that to maximize the detection of brain waves across ALS patients at different stages of the disease, the classification pipeline should be tuned to each patient individually., (© 2021 IOP Publishing Ltd.)

The performance of a classifier in a brain-computer interface (BCI) system is highly dependent on the quality and quantity of training data. Typically, the training data are collected in a laboratory where the users perform tasks in a controlled environment. However, users' attention may be diverted in real-life BCI applications and this may decrease the performance of the classifier. To improve the robustness of the classifier, additional data can be acquired in such conditions, but it is not practical to record electroencephalogram (EEG) data over several long calibration sessions. A potentially time- and cost-efficient solution is artificial data generation. Hence, in this study, we proposed a framework based on the deep convolutional generative adversarial networks (DCGANs) for generating artificial EEG to augment the training set in order to improve the performance of a BCI classifier. To make a comparative investigation, we designed a motor task experiment with diverted and focused attention conditions. We used an end-to-end deep convolutional neural network for classification between movement intention and rest using the data from 14 subjects. The results from the leave-one subject-out (LOO) classification yielded baseline accuracies of 73.04% for diverted attention and 80.09% for focused attention without data augmentation. Using the proposed DCGANs-based framework for augmentation, the results yielded a significant improvement of 7.32% for diverted attention ( ) and 5.45% for focused attention ( ). In addition, we implemented the method on the data set IVa from BCI competition III to distinguish different motor imagery tasks. The proposed method increased the accuracy by 3.57% ( ). This study shows that using GANs for EEG augmentation can significantly improve BCI performance, especially in real-life applications, whereby users' attention may be diverted.

Objective. The implementation of somatosensory feedback in upper limb myoelectric prostheses is an important step towards the restoration of lost sensory-motor functions. EMG feedback is a recently proposed method for closing the control loop wherein the myoelectric signal that drives the prosthesis is also used to generate the feedback provided to the user. Therefore, the characteristics of the myoelectric signal (variability and sensitivity) are likely to significantly affect the ability of the subject to utilize this feedback for online control of the prosthesis. Approach. In the present study, we investigated how the cutoff frequency of the low-pass filter (0.5, 1 and 1.5 Hz) and normalization value (20%, 40% and 60% of the maximum voluntary contraction (MVC)), that are used for the generation of the myoelectric signal, affect the quality of closed-loop control with EMG feedback. Lower cutoff and normalization decrease the intrinsic variability of the EMG but also increase the time lag between the contraction and the feedback (cutoff) as well as the sensitivity of the myoelectric signal (normalization). Ten participants were asked to generate three grasp force levels with a myoelectric prosthetic hand, while receiving five-level vibrotactile EMG feedback, over nine experimental runs (all parameter combinations). Main results. The outcome measure was the success rate (SR) in achieving the appropriate level of myoelectric signal (primary outcome) and grasping force (secondary outcome). Overall, the experiments demonstrated that EMG feedback provided robust control across conditions. Nevertheless, the performance was significantly better for the lowest cutoff (0.5 Hz) and higher normalization (40% and 60%). The highest SR for the EMG was 71.9%, achieved in the condition (40% MVC and 0.5 Hz), and this was 24.1% higher than that in the condition (20% MVC and 1.5 Hz), which resulted in the lowest performance. The SR for the force followed a similar trend. Significance. This is the first study that systematically explored the parameter space for the calibration of EMG feedback, which is a critical step for the future clinical application of this approach., (© 2021 IOP Publishing Ltd.)

A new tendon driven mechanism, embedded into a soft hand exoskeleton for rehabilitation and assistance, was proposed in this study. The proposed solution was a pulley flexion mechanism inspired by the human musculoskeletal system to enable a natural and comfortable finger flexion. A biomechanical constraint for the finger flexion motion states that the relation between the proximal interphalangeal joint angle of the finger should always be flexed around 1.5 times the distal interphalangeal joint angle. The study aimed to comply with this constraint, by simultaneously distributing the forces over the distal and middle finger phalanges. For evaluation, the voluntary and exoskeleton flexions were compared based on the relation between the proximal and distal interphalangeal joint angles. The results showed that during the exoskeleton flexion the relation between the interphalangeal joints complied with the biomechanical constraint, where the proximal interphalangeal joint angle was 1.5 times larger than the distal interphalangeal joint. This ensures that the mechanism flexes the finger comfortably. The proposed solution is therefore a promising design for a novel soft exoskeleton that will be used for training and assistance of patients with hand paralysis.

Individuals suffering from quadriplegia can achieve increased independence by using an assistive robotic manipulator (ARM). However, due to their disability, the interfaces that can be used to operate such devices become limited. A versatile intraoral tongue control interface (ITCI) has previously been develop for this user group, as the tongue is usually spared from disability. A previous study has shown that the ITCI can provide direct and continuous control of 6-7 degrees of freedom (DoF) of an ARM, due to a high number of provided inputs (18). In the present pilot study we investigated whether semi-automation might further improve the efficiency of the ITCI, when controlling an ARM. This was achieved by adding a camera to the end effector of the ARM and using computer vision algorithms to guide the ARM to grasp a target object. Three ITCI and one joystick control scheme were tested and compared: 1) manual Cartesian control with a base frame reference point, 2) manual Cartesian control with an end effector reference point 3) manual Cartesian control with an end effector reference point and an autonomous grasp function 4) regular JACO 2 joystick control. The results indicated that end effector control was superior to the base frame control in total task time, number of commands issued and path efficiency. The addition of the automatic grasp function did not improve the performance, but resulted in fewer collisions/displacements of the target object when grasping.

Grasping is a complex task routinely performed in an anticipatory (feedforward) manner, where sensory feedback is responsible for learning and updating the internal model of grasp dynamics. This study aims at evaluating whether providing a proportional tactile force feedback during the myoelectric control of a prosthesis facilitates learning a stable internal model of the prosthesis force control. Ten able-bodied subjects controlled a sensorized myoelectric prosthesis performing four blocks of consecutive grasps at three levels of target force (30, 50, and 70%), repeatedly closing the fully opened hand. In the first and third block, the subjects received tactile and visual feedback, respectively, while during the second and fourth block, the feedback was removed. The subjects also performed an additional block with no feedback 1 day after the training (Retest). The median and interquartile range of the generated forces was computed to assess the accuracy and precision of force control. The results demonstrated that the feedback was indeed an effective instrument for the training of prosthesis control. After the training, the subjects were still able to accurately generate the desired force for the low and medium target (30 and 50% of maximum force available in a prosthesis), despite the feedback being removed within the session and during the retest (low target force). However, the training was substantially less successful for high forces (70% of prosthesis maximum force), where subjects exhibited a substantial loss of accuracy as soon as the feedback was removed. The precision of control decreased with higher forces and it was consistent across conditions, determined by an intrinsic variability of repeated myoelectric grasping. This study demonstrated that the subject could rely on the tactile feedback to adjust the motor command to the prosthesis across trials. The subjects adjusted the mean level of muscle activation (accuracy), whereas the precision could not be modulated as it depends on the intrinsic myoelectric variability. They were also able to maintain the feedforward command even after the feedback was removed, demonstrating thereby a stable learning, but the retention depended on the level of the target force. This is an important insight into the role of feedback as an instrument for learning of anticipatory prosthesis force control.

Prosthesis users usually agree that myoelectric prostheses should be equipped with somatosensory feedback. However, the exact role of feedback and potential benefits are still elusive. The current study investigates the nature of human control processes within a specific context of routine grasping. Although the latter includes a fast feedforward control of the grasping force, the assumption was that the feedback would still be useful; it would communicate the outcome of the grasping trial, which the subjects could use to learn an internal model of feedforward control. Nine able-bodied subjects produced repeatedly a desired level of grasping force using different control configurations: feedback versus no-feedback, virtual versus real prosthetic hand, and joystick versus myocontrol. The outcome measures were the median and dispersion of the relative force errors. The results demonstrated that the feedback was successful in limiting the variability of the routine grasping due to uncertainties in the system and/or the command interface. The internal models of feedforward control could be employed by the subjects to control the prosthesis without the loss of performance even after the force feedback was removed. The models were, however, unstable over time, especially with myocontrol. Overall, the study demonstrates that the prosthesis system can be learned by the subjects using feedback. The feedback is also essential to maintain the model, and it could be delivered intermittently. This approach has practical advantages, but the level to which this mechanism can be truly exploited in practice depends directly on the consistency of the prosthesis control interface.

The aim of this work was to minimize the number of channels, determining acceptable electrode locations and optimizing electrode-recording configurations to decode isometric flexion and extension of individual fingers. Nine healthy subjects performed cyclical isometric contractions activating individual fingers. During the experiment they tracked a moving visual marker indicating the contraction type (flexion/extension), desired activation level and the finger that should be employed. Surface electromyography (sEMG) signals were detected from the forearm muscles using a matrix of 192 channels (24 longitudinal columns and 8 transversal rows, 10 mm inter-electrode distance). The classification was evaluated in the context of a linear discriminant analysis (LDA) with different sets of EMG electrodes: A) one linear array of 8 electrodes, B) two arrays of 8 electrodes each, C) a set with one electrode on the barycenter of each sEMG activity area, D) all the recorded channels. The results showed that the classification accuracy depended on the electrode set (F=14.67, p<;0.001). The best reduction approaches were the barycenter calculation and the use of two linear arrays of electrodes, which performed similarly to each other (both > 82% of average success rate). Considering the computation time and electrode positioning, it is concluded that two arrays of 8 electrodes provide an optimal configuration to classify the isometric flexion and extension of individual fingers.

An important reason for the abandonment of commercial actuated hand prostheses by the users is the lack of sensory feedback. Wearable afferent interfaces capable of providing electro- or vibro-tactile stimulation have high potential to restore the missing tactile and/or proprioceptive information to the user. By definition, these devices can elicit single modality (i.e., either vibrotactile or electrotactile) substitute sensations. In a recent research we have presented a novel approach comprising hybrid vibro-electrotactile (HyVE) combined stimulation, in order to provide multimodal sensory feedback. An important advantage of this approach is in the size of the design: the HyVE interface is much more compact than two separated single-modality interfaces, since electro- and vibro-tactile stimulators are placed one on top of the other. The HyVE approach has been previously tested in healthy subjects and has shown to provide a range of hybrid stimuli that could be properly discriminated. However, this approach has never been assessed as a method to provide multi-channel stimuli, i.e., stimuli from a variety of stimulators, mapping information from a multitude of sensors on a prosthesis. In this study, the ability of ten healthy subjects to discriminate stimuli and patterns of stimuli from four different five-channel interfaces applied on their forearms was evaluated. We showed that multiple HyVE units could be used to provide multi-channel sensory information with equivalent performance (∼95 percent for single stimuli and ∼80 percent for pattern) to single modality interfaces (vibro- or electro-tactile) larger in size and with better performance than vibrotactile interfaces (i.e., 73 percent for single stimuli and 69 percent for pattern) with the same size. These results are promising in relation to the current availability of multi-functional prostheses with multiple sensors.

The focus of this study was to test a novel tool for the analysis of motor coordination with an altered visual input. The altered visual input was created using special glasses that presented the view as recorded by a video camera placed at various positions around the subject. The camera was positioned at a frontal (F), lateral (L), or top (T) position with respect to the subject. We studied the differences between the arm-end (wrist) trajectories while grasping an object between altered vision (F, L, and T conditions) and normal vision (N) in ten subjects. The outcome measures from the analysis were the trajectory errors, the movement parameters, and the time of execution. We found substantial trajectory errors and an increased execution time at the baseline of the study. We also found that trajectory errors decreased in all conditions after three days of practice with the altered vision in the F condition only for 20 minutes per day, suggesting that recalibration of the visual systems occurred relatively quickly. These results indicate that this recalibration occurs via movement training in an altered condition. The results also suggest that recalibration is more difficult to achieve for altered vision in the F and L conditions compared to the T condition. This study has direct implications on the design of new rehabilitation systems.

This study introduces a Functional Electrical Therapy (FET) system based on sensor-driven electrical stimulation for the augmentation of walking. The automatic control relates to the timing of stimulation of four muscles. The sensor system comprises accelerometers and force-sensing resistors. The automatic control implements IF-THEN rules designed by mapping of sensors and muscle activation patterns. The new system was tested in 13 acute stroke patients assigned to a FET group or a control (CON) group. Both groups were treated with a standard rehabilitation program and 45min of walking daily for 5 days over the course of 4 weeks. The FET group received electrical stimulation during walking. The Fugl-Meyer (FM) test for the lower extremities, Barthel Index (BI), mean walking velocity (v(mean)) over a 6-m distance, and Physiological Cost Index (PCI) were assessed at the entry point and at the end of the treatment. Subjects within the FET and CON groups had comparable baseline outcome measures. In the FET group, we determined significant differences in the mean values of all outcomes between the entry and end points of treatment (p<0.05), contrary to the CON group where we found no significant differences (p>0.05). We also found significant differences in the changes of FM, BI, v(mean) and PCI which occurred during the 4 weeks of treatment between the FET and CON groups (p<0.05). The statistical strength of the clinical study was low (<70%), suggesting the need for a larger, randomized clinical trial.

The overall goal of the research is to improve control for electrical stimulation-based assistance of walking in hemiplegic individuals. We present the simulation for generating offline input (sensors)-output (intensity of muscle stimulation) representation of walking that serves in synthesizing a rule-base for control of electrical stimulation for restoration of walking. The simulation uses new algorithm termed moving-window dynamic optimization (MWDO). The optimization criterion was to minimize the sum of the squares of tracking errors from desired trajectories with the penalty function on the total muscle efforts. The MWDO was developed in the MATLAB environment and tested using target trajectories characteristic for slow-to-normal walking recorded in healthy individual and a model with the parameters characterizing the potential hemiplegic user. The outputs of the simulation are piecewise constant intensities of electrical stimulation and trajectories generated when the calculated stimulation is applied to the model. We demonstrated the importance of this simulation by showing the outputs for healthy and hemiplegic individuals, using the same target trajectories. Results of the simulation show that the MWDO is an efficient tool for analyzing achievable trajectories and for determining the stimulation profiles that need to be delivered for good tracking.

We developed the STIMBELT, an electrical stimulation system that comprises a lumbar belt with up to eight pairs of embedded electrodes and an eight-channel electronic stimulator. The STIMBELT is an assistive system for the treatment of low-back pain (LBP). We describe here technical details of the system and summarize the results of its application in individuals with subacute and chronic LBP. The direct goals of the treatment were to relieve pain, reduce muscle spasms, increase strength and range of motion, and educate individuals with LBP in reducing the chances of its reoccurrence. The outcome measures include: a Visual Analogue Scale (VAS), the Oswestry LBP Disability Questionnaire, the Short Form (SF)-12 health survey, and the Manual Muscle Test. The results indicate significant benefits for individuals who use the STIMBELT in addition to the conventional therapy as opposed to only the conventional therapy.

A neural prosthesis (NP) has two applications: permanent assistance of function, and temporary assistance that contributes to long-term recovery of function. Here, we address control issues for a therapeutic NP which uses surface electrodes. We suggest that the effective NP for therapy needs to implement rule-based control. Rule-based control relies on the triggering of preprogrammed sequences of electrical stimulation by the sensory signals. The sensory system in the therapeutic NP needs to be simple for installation, allow self-calibration, it must be robust, and sufficiently redundant in order to guarantee safe operation. The sensory signals need to generate control signals; hence, sensory fusion is needed. MEMS technology today provides sensors that fulfill the technical requirements (accelerometers, gyroscopes, force sensing resistors). Therefore, the task was to design a sensory signal processing method from the mentioned solid state sensors that would recognize phases during the gait cycle. This is necessary for the control of multi channel electrical stimulation. The sensory fusion consists of the following two phases: 1) estimation of vertical and horizontal components of the ground reaction force, center of pressure, and joint angles from the solid-state sensors, and 2) fusion of the estimated signals into a sequence of command signals. The first phase was realized by the use of artificial neural networks and adaptive neuro-fuzzy inference systems, while the second by the use of inductive learning described in our earlier work [1].

We developed user-friendly software that generates stimulation profiles by using user-customized model-based control of walking. The model is a multi-segment structure with pin and ball joints. A pair of an agonist and an antagonistic muscles acts at each joint. Each muscle is modeled by a three-compartment multiplicative model. The control is based on optimization that uses a cost function that minimizes the tracking error of the joint angles and levels of muscles activations. The inputs to the simulation are trajectories and user characteristic model parameters. The outputs of the simulation are levels of muscle activations vs. time. The software allows for interactive testing of various walking trajectories and model parameters since the simulation is integrated into a database of individuals and reference trajectories. The simulation was realized in the MatLab environment with multiple windows graphical user interface. Here we present an example: stimulation patterns for the shank-foot system that is applicable for walking control in hemiplegic individuals.