Search

Abstract Background In-home rehabilitation systems are a promising, potential alternative to conventional therapy for stroke survivors. Unfortunately, physiological differences between participants and sensor displacement in wearable sensors pose a significant challenge to classifier performance, particularly for people with stroke who may encounter difficulties repeatedly performing trials. This makes it challenging to create reliable in-home rehabilitation systems that can accurately classify gestures. Methods Twenty individuals who suffered a stroke performed seven different gestures (mass flexion, mass extension, wrist volar flexion, wrist dorsiflexion, forearm pronation, forearm supination, and rest) related to activities of daily living. They performed these gestures while wearing EMG sensors on the forearm, as well as FMG sensors and an IMU on the wrist. We developed a model based on prototypical networks for one-shot transfer learning, K-Best feature selection, and increased window size to improve model accuracy. Our model was evaluated against conventional transfer learning with neural networks, as well as subject-dependent and subject-independent classifiers: neural networks, LGBM, LDA, and SVM. Results Our proposed model achieved 82.2% hand—gesture classification accuracy, which was better (P

Abstract Background Telerehabilitation is a promising avenue for improving patient outcomes and expanding accessibility. However, there is currently no spine-related assessment for telerehabilitation that covers multiple exercises. Methods We propose a wearable system with two inertial measurement units (IMUs) to identify IMU locations and estimate spine angles for ten commonly prescribed spinal degeneration rehabilitation exercises (supine chin tuck head lift rotation, dead bug unilateral isometric hold, pilates saw, catcow full spine, wall angel, quadruped neck flexion/extension, adductor open book, side plank hip dip, bird dog hip spinal flexion, and windmill single leg). Twelve healthy subjects performed these spine-related exercises, and wearable IMU data were collected for spine angle estimation and IMU location identification. Results Results demonstrated average mean absolute spinal angle estimation errors of 2.59 $$^\circ$$ ∘ and average classification accuracy of 92.97%. The proposed system effectively identified IMU locations and assessed spine-related rehabilitation exercises while demonstrating robustness to individual differences and exercise variations. Conclusion This inexpensive, convenient, and user-friendly approach to spine degeneration rehabilitation could potentially be implemented at home or provide remote assessment, offering a promising avenue to enhance patient outcomes and improve accessibility for spine-related rehabilitation. Trial registration: No. E2021013P in Shanghai Jiao Tong University.

Gesture recognition is crucial for enhancing human-computer interaction and is particularly pivotal in rehabilitation contexts, aiding individuals recovering from physical impairments and significantly improving their mobility and interactive capabilities. However, current wearable hand gesture recognition approaches are often limited in detection performance, wearability, and generalization. We thus introduce EchoGest, a novel hand gesture recognition system based on soft, stretchable, transparent artificial skin with integrated ultrasonic waveguides. Our presented system is the first to use soft ultrasonic waveguides for hand gesture recognition. EcoflexTM 00–31 and EcoflexTM 00-45 Near ClearTM silicone elastomers were employed to fabricate the artificial skin and ultrasonic waveguides, while 0.1 mm diameter silver-plated copper wires connected the transducers in the waveguides to the electrical system. The wires are enclosed within an additional elastomer layer, achieving a sensing skin with a total thickness of around $500~\mu $ m. Ten participants wore the EchoGest system and performed static hand gestures from two gesture sets: 8 daily life gestures and 10 American Sign Language (ASL) digits 0-9. Leave-One-Subject-Out Cross-Validation analysis demonstrated accuracies of 91.13% for daily life gestures and 88.5% for ASL gestures. The EchoGest system has significant potential in rehabilitation, particularly for tracking and evaluating hand mobility, which could substantially reduce the workload of therapists in both clinical and home-based settings. Integrating this technology could revolutionize hand gesture recognition applications, from real-time sign language translation to innovative rehabilitation techniques.

Wearable lower-limb joint angle estimation using a reduced inertial measurement unit (IMU) sensor set could enable quick, economical sports injury risk assessment and motion capture; however the vast majority of existing research requires a full IMU set attached to every related body segment and is implemented in only a single movement, typically walking. We thus implemented 3-dimensional knee and hip angle estimation with a reduced IMU sensor set during yoga, golf, swimming (simulated lower body swimming in a seated posture), badminton, and dance movements. Additionally, current deep-learning models undergo an accuracy drop when tested with new and unseen activities, which necessitates collecting large amounts of data for the new activity. However, collecting large datasets for every new activity is time-consuming and expensive. Thus, a transfer learning (TL) approach with long short-term memory neural networks was proposed to enhance the model’s generalization ability towards new activities while minimizing the need for a large new-activity dataset. This approach could transfer the generic knowledge acquired from training the model in the source-activity domain to the target-activity domain. The maximum improvement in estimation accuracy (RMSE) achieved by TL is 23.6 degrees for knee flexion/extension and 22.2 degrees for hip flexion/extension compared to without TL. These results extend the application of motion capture with reduced sensor configurations to a broader range of activities relevant to injury prevention and sports training. Moreover, they enhance the capacity of data-driven models in scenarios where acquiring a substantial amount of training data is challenging.

Abstract Anterior cruciate ligament (ACL) injury and ACL reconstruction (ACLR) surgery are common. Laboratory-based biomechanical assessment can evaluate ACL injury risk and rehabilitation progress after ACLR; however, lab-based measurements are expensive and inaccessible to most people. Portable sensors such as wearables and cameras can be deployed during sporting activities, in clinics, and in patient homes. Although many portable sensing approaches have demonstrated promising results during various assessments related to ACL injury, they have not yet been widely adopted as tools for out-of-lab assessment. The purpose of this review is to summarize research on out-of-lab portable sensing applied to ACL and ACLR and offer our perspectives on new opportunities for future research and development. We identified 49 original research articles on out-of-lab ACL-related assessment; the most common sensing modalities were inertial measurement units, depth cameras, and RGB cameras. The studies combined portable sensors with direct feature extraction, physics-based modeling, or machine learning to estimate a range of biomechanical parameters (e.g., knee kinematics and kinetics) during jump-landing tasks, cutting, squats, and gait. Many of the reviewed studies depict proof-of-concept methods for potential future clinical applications including ACL injury risk screening, injury prevention training, and rehabilitation assessment. By synthesizing these results, we describe important opportunities that exist for clinical validation of existing approaches, using sophisticated modeling techniques, standardization of data collection, and creation of large benchmark datasets. If successful, these advances will enable widespread use of portable-sensing approaches to identify ACL injury risk factors, mitigate high-risk movements prior to injury, and optimize rehabilitation paradigms.

Accurate shoulder joint angle estimation is crucial for analyzing joint kinematics and kinetics across a spectrum of movement applications including in athletic performance evaluation, injury prevention, and rehabilitation. However, accurate IMU-based shoulder angle estimation is challenging and the specific influence of key error factors on shoulder angle estimation is unclear. We thus propose an analytical model based on quaternions and rotation vectors that decouples and quantifies the effects of two key error factors, namely sensor-to-segment misalignment and sensor orientation estimation error, on shoulder joint rotation error. To validate this model, we conducted experiments involving twenty-five subjects who performed five activities: yoga, golf, swimming, dance, and badminton. Results showed that improving sensor-to-segment misalignment along the segment’s extension/flexion dimension had the most significant impact in reducing the magnitude of shoulder joint rotation error. Specifically, a 1° improvement in thorax and upper arm calibration resulted in a reduction of 0.40° and 0.57° in error magnitude. In comparison, improving IMU heading estimation was only roughly half as effective (0.23° per 1°). This study clarifies the relationship between shoulder angle estimation error and its contributing factors, and identifies effective strategies for improving these error factors. These findings have significant implications for enhancing the accuracy of IMU-based shoulder angle estimation, thereby facilitating advancements in IMU-based upper limb rehabilitation, human-machine interaction, and athletic performance evaluation.

Biometric-based personal identification models are generally considered to be accurate and secure because biological signals are too complex and person-specific to be fabricated, and EMG signals, in particular, have been used as biological identification tokens due to their high dimension and non-linearity. We investigate the possibility of effectively attacking EMG-based identification models with adversarial biological input via a novel EMG signal individual-style transformer based on a generative adversarial network and tiny leaked data segments. Since two same EMG segments do not exist in nature; the leaked data can’t be used to attack the model directly or it will be easily detected. Therefore, it is necessary to extract the style with the leaked personal signals and generate the attack signals with different contents. With our proposed method and tiny leaked personal EMG fragments, numerous EMG signals with different content can be generated in that person’s style. EMG hand gesture data from eighteen subjects and three well-recognized deep EMG classifiers were used to demonstrate the effectiveness of the proposed attack methods. The proposed methods achieved an average of 99.41% success rate on confusing identification models and an average of 91.51% success rate on manipulating identification models. These results demonstrate that EMG classifiers based on deep neural networks can be vulnerable to synthetic data attacks. The proof-of-concept results reveal that synthetic EMG biological signals must be considered in biological identification system design across a vast array of relevant biometric systems to ensure personal identification security for individuals and institutions.

Most stroke survivors have difficulties completing activities of daily living (ADLs) independently. However, few rehabilitation systems have focused on ADLs-related training for gross and fine motor function together. We propose an ADLs-based serious game rehabilitation system for the training of motor function and coordination of both arm and hand movement where the user performs corresponding ADLs movements to interact with the target in the serious game. A multi-sensor fusion model based on electromyographic (EMG), force myographic (FMG), and inertial sensing was developed to estimate users’ natural upper limb movement. Eight healthy subjects and three stroke patients were recruited in an experiment to validate the system’s effectiveness. The performance of different sensor and classifier configurations on hand gesture classification against the arm position variations were analyzed, and qualitative patient questionnaires were conducted. Results showed that elbow extension/flexion has a more significant negative influence on EMG-based, FMG-based, and EMG+FMG-based hand gesture recognition than shoulder abduction/adduction does. In addition, there was no significant difference in the negative influence of shoulder abduction/adduction and shoulder flexion/extension on hand gesture recognition. However, there was a significant interaction between sensor configurations and algorithm configurations in both offline and real-time recognition accuracy. The EMG+FMG-combined multi-position classifier model had the best performance against arm position change. In addition, all the stroke patients reported their ADLs-related ability could be restored by using the system. These results demonstrate that the multi-sensor fusion model could estimate hand gestures and gross movement accurately, and the proposed training system has the potential to improve patients’ ability to perform ADLs.

Not all wearable fitness devices are suitable for children. Due to the decay of activity recognition accuracy and exercise requirement variety caused by the physical differences between adults and children, devices developed for adults cannot provide appropriate exercise guidance to children. Considering the complexity and time‐consuming nature of developing a new children‐specific model, this article proposes a child adaptation mechanism that can be integrated into current adult wearable devices. Two algorithms using unlabeled children's data for age group recognition and activity recognition inspired by the visual variability in cognitive process and visuo‐tactile coordination phenomenon in cortical plasticity are proposed. During an experiment on 30 adults and children, the age group recognition algorithm achieves 93.33% recognition accuracy, and the activity recognition algorithm achieves 88.57% recognition accuracy. This is an 11.16% improvement compared with the baseline and 2.87–4.1% improvement compared with the state‐of‐the‐art transfer learning or self‐adaptation algorithms. The proposed mechanism ameliorates current wearable devices with minimum cost to make them suitable for children and can serve as an alternative to help children learn healthy exercise habits. It is hoped that the work can draw the attention of academia to vulnerable groups, including children, to build a friendlier artificial intelligence network.

Stroke often leads to hand motor dysfunction, and effective rehabilitation requires keeping patients engaged and motivated. Among the existing automated rehabilitation approaches, data glove-based systems are not easy to wear for patients due to spasticity, and single sensor-based approaches generally provided prohibitively limited information. We thus propose a wearable multimodal serious games approach for hand movement training after stroke. A force myography (FMG), electromyography (EMG), and inertial measurement unit (IMU)-based multi-sensor fusion model was proposed for hand movement classification, which was worn on the user’s affected arm. Two movement recognition-based serious games were developed for hand movement and cognition training. Ten stroke patients with mild to moderate motor impairments (Brunnstrom Stage for Hand II-VI) performed experiments while playing interactive serious games requiring 12 activities-of-daily-living (ADLs) hand movements taken from the Fugl Meyer Assessment. Feasibility was evaluated by movement classification accuracy and qualitative patient questionnaires. The offline classification accuracy using combined FMG-EMG-IMU was 81.0% for the 12 movements, which was significantly higher than any single sensing modality; only EMG, only FMG, and only IMU were 69.6, 63.2, and 47.8%, respectively. Patients reported that they were more enthusiastic about hand movement training while playing the serious games as compared to conventional methods and strongly agreed that they subjectively felt that the proposed training could be beneficial for improving upper limb motor function. These results showed that multimodal-sensor fusion improved hand gesture classification accuracy for stroke patients and demonstrated the potential of this proposed approach to be used as upper limb movement training after stroke.

Goal: In this paper, we introduced a novel ankle band with a vibrational sensor that can achieve low-cost ankle flexion angle estimation, which can be potentially used for automated ankle flexion angle estimation in home-based foot drop rehabilitation scenarios. Methods: Previous ankle flexion angle estimation methods require either professional knowledge or specific equipment and lab environment, which is not feasible for foot drop patients to achieve accurate measurement by themselves in a home-based scenario. To solve the above problems, a prototype was developed based on the assumption that the echo of a vibration signal on the tibialis anterior had different acoustic impedance distribution. By analyzing the frequency spectrum of the echo, the ankle flexion angle can be estimated. Therefore, a surface transducer was utilized to generate frequency-varying active vibration, and a contact microphone was utilized to capture the echo. A portable analog signal processing hub drove the transducer, and was used for echo signal collection from the microphone. Finally, a Random Forest regression model was applied to estimate the ankle flexion angle based on the spectrum amplitude of the echo. Results: Five healthy subjects were recruited in the experiment. The regression estimation error is 4.16 degrees, and the R2 is 0.81. Conclusions: These results demonstrate the feasibility of the proposed ankle band for accurate ankle flexion angle estimation.

Gait retraining to reduce knee loading has been proposed as a conservative treatment option for early-stage knee osteoarthritis. Mounting evidence suggests that a subject-specific approach may be most effective for ensuring consistent knee loading reductions across all individuals within a population. However, it is currently unclear how to determine the required gait modification dosage selection type and amount to both reduce knee loading and satisfy individual preferences. To overcome this challenge, we introduce a novel, mapping-based dosage selection approach to systematically determine multi-parameter gait modifications to reduce knee loading while maintaining individual user preference. In this approach, individuals first explore different dosages of multi-parameter gait modifications, and then a resulting visual map is displayed with a subject-specific dosage selection zone for the target knee loading reduction. Subjects then self-select a preferred gait within their dosage selection zone. To evaluate the feasibility of this approach, fifteen healthy subjects and one knee OA patient performed walking trials on a treadmill involving various dosages of gait modifications to foot progression angle and step width. Subjects quickly selected the subject-specific gait modifications via mapping-based dosage selection during a single 6 min trial, which reduced the knee adduction moment by an average of 14.2%. Resulting subject-specific gait modifications varied, with 6 subjects selecting only toe-in, 5 subjects selecting both toe-in and increased step width, 2 subjects selecting only toe-out, 1 subject selecting both toe-out and increased step width and 1 subject selecting only increased step width. Average perceived exertion was “fairly light” (index was 10.5±2.2). The knee OA patient selected only toe-in and reduced the knee adduction moment by 12.8%. The presented mapping-based dosage selection approach could provide a systematic and practical means to determine subject-specific gait modifications while maintaining individual preferences.

Slip-induced falls, responsible for approximately 40% of falls, can lead to severe injuries and in extreme cases, death. A large foot–floor contact angle (FFCA) during the heel-strike event has been associated with an increased risk of slip-induced falls. The goals of this feasibility study were to design and assess a method for detecting FFCA and providing cues to the user to generate a compensatory FFCA response during a future heel-strike event. The long-term goal of this research is to train gait in order to minimize the likelihood of a slip event due to a large FFCA. An inertial measurement unit (IMU) was used to estimate FFCA, and a speaker provided auditory semi-real-time feedback when the FFCA was outside of a 10–20 degree target range following a heel-strike event. In addition to training with the FFCA feedback during a 10-min treadmill training period, the healthy young participants completed pre- and post-training overground walking trials. Results showed that training with FFCA feedback increased FFCA events within the target range by 16% for “high-risk” walkers (i.e., participants that walked with more than 75% of their FFCAs outside the target range) both during feedback treadmill trials and post-training overground trials without feedback, supporting the feasibility of training FFCA using a semi-real-time FFCA feedback system.

Abstract Background Postural balance and gait training is important for treating persons with functional impairments, however current systems are generally not portable and are unable to train different types of movements. Methods This paper describes a proof-of-concept design of a configurable, wearable sensing and feedback system for real-time postural balance and gait training targeted for home-based treatments and other portable usage. Sensing and vibrotactile feedback are performed via eight distributed, wireless nodes or “Dots” (size: 22.5 × 20.5 × 15.0 mm, weight: 12.0 g) that can each be configured for sensing and/or feedback according to movement training requirements. In the first experiment, four healthy older adults were trained to reduce medial-lateral (M/L) trunk tilt while performing balance exercises. When trunk tilt deviated too far from vertical (estimated via a sensing Dot on the lower spine), vibrotactile feedback (via feedback Dots placed on the left and right sides of the lower torso) cued participants to move away from the vibration and back toward the vertical no feedback zone to correct their posture. A second experiment was conducted with the same wearable system to train six healthy older adults to alter their foot progression angle in real-time by internally or externally rotating their feet while walking. Foot progression angle was estimated via a sensing Dot adhered to the dorsal side of the foot, and vibrotactile feedback was provided via feedback Dots placed on the medial and lateral sides of the mid-shank cued participants to internally or externally rotate their foot away from vibration. Results In the first experiment, the wearable system enabled participants to significantly reduce trunk tilt and increase the amount of time inside the no feedback zone. In the second experiment, all participants were able to adopt new gait patterns of internal and external foot rotation within two minutes of real-time training with the wearable system. Conclusion These results suggest that the configurable, wearable sensing and feedback system is portable and effective for different types of real-time human movement training and thus may be suitable for home-based or clinic-based rehabilitation applications.

The recent explosion of wearable electronics has led to widespread interest in harvesting human movement energy, particularly during walking, for clinical and health applications. However, the amount of energy available to harvest and the required metabolic rate for wearable energy harvesting varies across subjects. In this paper, we utilize custom energy harvesting sliding shoes to develop and evaluate multivariate linear regression models to predict metabolic rate and energy harvesting rate during overground walking outside of the lab. Subjects performed 200 m self-selected normal and fast walking trials on flat ground with custom sliding shoes. Metabolic rate was measured with a portable breathing analysis system and energy harvesting rate was measured directly from the generator on the custom sliding shoes. Model performance was determined by comparing the difference between actual and predicted metabolic and energy harvesting rates. Overall, predictive modeling closely matched the actual values, and there was no statistical difference between actual and predicted average metabolic rate or between actual and predicted average energy harvesting rate. Energy harvesting sliding shoes could potentially be used for a variety of wearable devices to reduce onboard energy storage, and these findings could serve to inform expected energy harvesting rates and associated required metabolic cost for a diverse array of medical and health applications.

Hand gesture classification and finger angle estimation are both critical for intuitive human–computer interaction. However, most approaches study them in isolation. We thus propose a dual-output deep learning model to enable simultaneous hand gesture classification and finger angle estimation. Data augmentation and deep learning were used to detect spatial-temporal features via a wristband with ten modified barometric sensors. Ten subjects performed experimental testing by flexing/extending each finger at the metacarpophalangeal joint while the proposed model was used to classify each hand gesture and estimate continuous finger angles simultaneously. A data glove was worn to record ground-truth finger angles. Overall hand gesture classification accuracy was 97.5% and finger angle estimation R 2 was 0.922, both of which were significantly higher than shallow existing learning approaches used in isolation. The proposed method could be used in applications related to the human–computer interaction and in control environments with both discrete and continuous variables.