Search

Objective: Sonomyography, or ultrasound-based sensing of muscle deformation, is an emerging modality for upper limb prosthesis control. Although prior studies have shown that individuals with upper limb loss can achieve successful motion classification with sonomyography, it is important to better understand the time-course over which proficiency develops. In this study, we characterized user performance during their initial and subsequent exposures to sonomyography. Method: Ultrasound images corresponding to a series of hand gestures were collected from individuals with transradial limb loss under three scenarios: during their initial exposure to sonomyography (Experiment 1), during a subsequent exposure to sonomyography where they were provided biofeedback as part of a training protocol (Experiment 2), and during testing sessions held on different days (Experiment 3). User performance was characterized by offline classification accuracy, as well as metrics describing the consistency and separability of the sonomyography signal patterns in feature space. Results: Classification accuracy was high during initial exposure to sonomyography (96.2 ± 5.9%) and did not systematically change with the provision of biofeedback or on different days. Despite this stable classification performance, some of the feature space metrics changed. Conclusions: User performance was strong upon their initial exposure to sonomyography and did not improve with subsequent exposure. Clinical Impact: Prosthetists may be able to quickly assess if a patient will be successful with sonomyography without submitting them to an extensive training protocol, leading to earlier socket fabrication and delivery.

Controlling multi-articulated prosthetic hands with surface electromyography can be challenging for users. Sonomyography, or ultrasound-based sensing of muscle deformation, avoids some of the problems of electromyography and enables classification of multiple motion patterns in individuals with upper limb loss. Because sonomyography has been previously studied only in individuals with transradial limb loss, the purpose of this study was to assess the feasibility of an individual with transhumeral limb loss using this modality for motion classification. A secondary aim was to compare motion classification performance between electromyography and sonomyography. A single individual with transhumeral limb loss created two datasets containing 11 motions each (individual flexion of each finger, thumb abduction, power grasp, key grasp, tripod, point, pinch, wrist pronation). Electromyography or sonomyography signals associated with every motion were acquired and cross-validation accuracy was computed for each dataset. While all motions were usually predicted successfully with both electromyography and sonomyography, the cross-validation accuracies were typically higher for sonomyography. Although this was an exploratory study, the results suggest that controlling an upper limb prosthesis using sonomyography may be feasible for individuals with transhumeral limb loss.

Upper limb prosthesis users currently lack haptic feedback from their terminal devices, which significantly limits their ability to meaningfully interact with their environment. Users therefore rely heavily on visual feedback when using terminal devices. Previously, it has been shown that force-related feedback from an end-effector or virtual environment can help the user minimize errors and improve performance. Currently, myoelectric control systems enable the user to control the velocity of terminal devices. We have developed a novel control method using ultrasound sensing, called sonomyography, that enables position control based on mechanical deformation of muscles. In this paper, we investigated whether the proprioceptive feedback from muscle deformation combined with vibrotactile haptic feedback can minimize the need for visual feedback. Able bodied subjects used sonomyography to control a virtual cursor, and performed a target acquisition task. The effect of visual and haptic feedback on performance of a target acquisition task was systematically tested. We found that subjects made large errors when they tried to reacquire a target without visual feedback, but in the presence of real-time haptic feedback, the precision of the target position improved, and were similar to when visual feedback was used for target acquisition. This result has implications for improving the performance of prosthetic control systems.

Background: Although surface electromyography is commonly used as a sensing strategy for upper limb prostheses, it remains difficult to reliably decode the recorded signals for controlling multi-articulated hands. Sonomyography, or ultrasound-based sensing of muscle deformation, overcomes some of these issues and allows individuals with upper limb loss to reliably perform multiple motion patterns. The purposes of this study were to determine 1) the effect of training on classification performance with sonomyographic control and 2) the effect of training on the underlying muscle deformation patterns.Methods: A series of motion pattern datasets were collected from five individuals with transradial limb loss. Each dataset contained five ultrasound images corresponding each of the following five motions: power grasp, wrist pronation, key grasp, tripod, point. Participants initially performed the motions for the datasets without receiving feedback on their performance (baseline phase), then with visual and verbal feedback (feedback phase), and finally again without feedback (retention phase). Cross-validation accuracy and metrics describing the consistency and separability of the muscle deformation patters were computed for each dataset. Changes in classification performance over the course of the study were assessed using linear mixed models. Associations between classification performance and the consistency and separability metrics were evaluated using Pearson correlations.Results: The average cross-validation accuracy for each phase was 92% or greater and there was no significant change in cross-validation accuracy throughout training. Misclassifications of one motion as another did not persist systematically across datasets. Few of the correlations were significant, although many were moderate or greater in strength and showed a positive association between accuracy and improved consistency and separability metrics.Conclusions: Participants were able to achieve high classification rates upon their initial exposure to sonomyography and training did not affect their performance. Thus, motion classification using sonomyography may be highly intuitive and is unlikely to require a structured training protocol to gain proficiency.

Prosthetics need to incorporate the users sense of proprioception into the control paradigm to provide intuitive control, and reduce training times and prosthetic rejection rates. In the absence of functional tasks with a prosthetic, virtual cursor control tasks have been used to train users to control multiple degrees of freedom. In this study, A proportional position signal was derived from the cross-sectional ultrasound images of the users forearm. We designed a virtual cursor control task with one degree of freedom to measure the users ability to repeatably and accurately acquire different levels of muscle flexion, using only their sense of proprioception. The experiment involved a target acquisition task, where the cursors height corresponded to the extent of muscle flexion. Users were asked to acquire targets on a screen. Visual feedback was disabled at certain times during the experiment, to isolate the effect of proprioception. We found that as visual feedback was taken away from the subjects, position error increased but their stability error did not change significantly. This indicates that users are not perfect at using only their proprioceptive sense to reacquire a level of muscle flexion, in the absence of haptic or visual feedback. However, they are adept at retaining an acquired flexion level without drifting. These results could help to quantify the role of proprioception in target acquisition tasks, in the absence of haptic or visual feedback.

Objective: Sonomyography has been shown to be a promising method for decoding volitional motor intent from analysis of ultrasound images of the forearm musculature. The objectives of this paper are to determine the optimal location for ultrasound transducer placement on the anterior forearm for imaging maximum muscle deformations during different hand motions, and to investigate the effect of using a sparse set of ultrasound scanlines for motion classification for ultrasound-based muscle–computer interfaces (MCIs). Methods: The optimal placement of the ultrasound transducer along the forearm was identified using freehand three-dimensional reconstructions of the muscle thickness during rest and motion completion. Based on the ultrasound images acquired from the optimally placed transducer, classification accuracy with equally spaced scanlines across the cross-sectional field of view was determined. Furthermore, the unique contribution of each scanline to class discrimination using Fisher criterion (FC) and mutual information (MI) with respect to motion discriminability was determined. Results: Experiments with five able-bodied subjects show that the maximum muscle deformation occurred between 40%–50% of the forearm length for multiple degrees-of-freedom. The average classification accuracy was 94% ± 6% with the entire 128-scanline image and 94% ± 5% with four equally spaced scanlines. However, no significant improvement in classification accuracy was observed with optimal scanline selection using FC and MI. Conclusion: For an optimally placed transducer, a small subset of ultrasound scanlines can be used instead of a full imaging array without sacrificing performance in terms of classification accuracy for multiple degrees-of-freedom. Significance: The selection of a small subset of transducer elements can enable the reduction of computation, and simplification of the instrumentation and power consumption of wearable sonomyographic MCIs, particularly for rehabilitation and gesture recognition applications.

Technological advances in multi-articulated prosthetic hands have outpaced the development of methods to intuitively control these devices. In fact, prosthetic users often cite "difficulty of use" as a key contributing factor for abandoning their prostheses. To overcome the limitations of the currently pervasive myoelectric control strategies, namely unintuitive proportional control of multiple degrees-of-freedom, we propose a novel approach: proprioceptive sonomyographiccontrol. Unlike myoelectric control strategies which measure electrical activation of muscles and use the extracted signals to determine the velocity of an end-effector; our sonomyography-based strategy measures mechanical muscle deformation directly with ultrasound and uses the extracted signals to proportionally control the position of an end-effector. Therefore, our sonomyography-based control is congruent with a prosthetic user’s innate proprioception of muscle deformation in the residual limb. In this work, we evaluated proprioceptive sonomyographic control with 5 prosthetic users and 5 able-bodied participants in a virtual target achievement and holding task for 5 different hand motions. We observed that with limited training, the performance of prosthetic users was comparable to that of able-bodied participants and thus conclude that proprioceptive sonomyographic control is a robust and intuitive prosthetic control strategy.

In recent work regarding gesture recognition and muscle computer interfaces, ultrasound-based sensing strategies have been demonstrated as a viable alternative to the pervasive surface electromyography (sEMG) modality. However, in order to facilitate switching between available gestures, both sEMG and ultrasound-based strategies have traditionally relied on unintuitive control mechanisms. The most common among these are: requiring the users to return to rest as an intermediary state between motions; mode switching through co-contraction or other ad-hoc user input; and switching based on muscle activations that are functionally unrelated to the desired motion. The unintuitive nature of such control has historically led to increased user frustration, and is often cited a major reason for device abandonment in the prosthetic control setting. In this work, we propose using an approach inspired by Hidden Markov Models (HMMs) with a novel continuous gesture recognition mechanism, for ultrasound-based sensing. We empirically calculate the average classification accuracy of our novel method during non-transitionary periods to be 99%. We then demonstrate that including predictions made during transition periods reduces this value to 69% Finally, by encoding the temporal dependency of the system within a Hidden Markov Model framework, we show that we can reduce the error caused by the instability of predictions during transitions, measured as the normalized Levenshtein distance from the true ordering, by approximately 98.8%.

Technological advances in multi-articulated prosthetic hands have outpaced the methods available to amputees to intuitively control these devices. Amputees often cite difficulty of use as a key contributing factor for abandoning their prosthesis, creating a pressing need for improved control technology. A major challenge of traditional myoelectric control strategies using surface electromyography electrodes has been the difficulty in achieving intuitive and robust proportional control of multiple degrees of freedom. In this paper, we describe a new control method, proprioceptive sonomyographic control that overcomes several limitations of myoelectric control. In sonomyography, muscle mechanical deformation is sensed using ultrasound, as compared to electrical activation, and therefore the resulting control signals can directly control the position of the end effector. Compared to myoelectric control which controls the velocity of the end-effector device, sonomyographic control is more congruent with residual proprioception in the residual limb. We tested our approach with 5 upper-extremity amputees and able-bodied subjects using a virtual target achievement and holding task. Amputees and able-bodied participants demonstrated the ability to achieve positional control for 5 degrees of freedom with an hour of training. Our results demonstrate the potential of proprioceptive sonomyographic control for intuitive dexterous control of multiarticulated prostheses.

Ultrasound (US) imaging systems have undergone substantial miniaturization recently and have given rise to many potential applications where battery-based operation is desirable. However, current clinical US systems utilizing pulse-echo imaging require high voltage and short duration transmit pulses along with electronics that operate in the MHz frequency range. As an alternative, we have developed an imaging method that employs time-delay spectrometry (TDS), and uses low-voltage (∼5V peak-to-peak) transmit pulses to allow operation at significantly lower acoustical output power levels making it suitable for long-term monitoring applications. Using this technique, we generated M-mode images in real time with 5 cm of depth and a pulse repetition frequency of 50 Hz. TDS produced improved contrast compared to conventional pulseecho US systems at low MI and TI. TDS could lead to a new generation of low-power US imaging systems.

Recently, ultrasound imaging of muscle contractions has been used by several research groups to infer volitional motor intent of the user, and has shown promise as a novel muscle computer interface. Learning spatiotemporal features from ultrasound image sequences is challenging because of deformations introduced by probe repositioning. The image features are sensitive to probe placement and even small displacements during cross-session donning and doffing of the probe could compromise the classification accuracy when using a model trained on a previous session. This requires frequent recalibration. Deep learning based feature extractors have been shown to be invariant to translation, rotation and slight deformations. In this study, we investigated the feasibility of wavelet-based deep scattering features to preserve motion classification accuracy across multiple donning and doffing sessions. It was demonstrated that scattering features, which do not require learning filter weights, can significantly improve the cross-session classification accuracy by 20–30%. It can be concluded that these features are robust to minor probe displacements. They need to be further investigated with a larger data set to investigate their robustness in accurately classifying different muscle movements.

Introduction/Background Recent studies have shown that unintuitive control is a key factor leading to upper-extremity, myoelectric prostheses abandonment. We have developed a non-invasive modality to extract proportional control signals in the residuum using ultrasound imaging. In this study, we investigate the performance of this technology in upper-extremity amputees. Material and method We recruited 4 amputee subjects ( Table 1 ) who currently use myoelectric prostheses. Subjects were instrumented with a portable ultrasound system connected to a low-profile transducer on the volar aspect of their residuum. Ultrasound images were processed in real-time to extract graded muscle activity signal in response to volitional user-intended motions (UIMs). Subjects were trained by iterating each UIM while being provided with visual feedback of ultrasound images and muscle-activity signal. Participants were then given control of an on-screen cursor that responded proportionally to the level of muscle-activity for a particular UIM and asked to reach and hold the cursor at predefined set-points. Control steadiness (standard deviation) and control error (difference between cursor and target) were computed. Results All subjects were able to complete the training phase within 15 minutes (25 iterations or less) for at least 4 degrees-of-freedom (DoF), while achieving motion prediction accuracies greater than 88% ( Table 1 ). 3 subjects participated in the motion control task. The congenital amputee subject performed 2 motions and the rest performed at least 4 motions with control errors between 2.1% and 9.36% ( Table 2 ). Our approach provides direct positional control based on muscle deformation, resulting in improved proprioceptive feedback, unlike myoelectric control. We believe that ultrasound images coupled with the muscle activity signals serve as an intuitive visual feedback mechanism, resulting in reduced training time and improved performance. Conclusion We demonstrated intuitive proportional control using ultrasound-based muscle activity sensing paradigm for multiple DoFs with traumatic and congenital amputees.