Search

The joint moment is a key measurement in locomotion analysis. Transferable prediction across different subjects is advantageous for calibration-free, practical clinical applications. However, even for similar gait motions, intersubject variance presents a significant challenge in maintaining reliable prediction performance. The optimal deep learning models for ankle moment prediction during dynamic gait motions remain underexplored for both intrasubject and intersubject usage. This study evaluates the feasibility of different deep-learning models for estimating ankle moments using sEMG data to find an optimal intrasubject model against the inverse dynamic approach. We verified and compared the performance of 1302 intrasubject models per subject on 597 steps from seven subjects using various architectures and feature sets. The best-performing intrasubject models were recurrent convolutional neural networks trained using signal energy features. They were then transferred to realize intersubject ankle moment estimation.

How our central nervous system efficiently controls our complex musculoskeletal system is still debated. The muscle synergy hypothesis is proposed to simplify this complex system by assuming the existence of functional neural modules that coordinate several muscles. Modularity based on muscle synergies can facilitate motor learning without compromising task performance. However, the effectiveness of modularity in motor control remains debated. This ambiguity can, in part, stem from overlooking that the performance of modularity depends on the mechanical aspects of modules of interest, such as the torque the modules exert. To address this issue, this study introduces two criteria to evaluate the quality of module sets based on commonly used performance metrics in motor learning studies: the accuracy of torque production and learning speed. One evaluates the regularity in the direction of mechanical torque the modules exert, while the other evaluates the evenness of its magnitude. For verification of our criteria, we simulated motor learning of torque production tasks in a realistic musculoskeletal system of the upper arm using feed-forward neural networks while changing the control conditions. We found that the proposed criteria successfully explain the tendency of learning performance in various control conditions. These result suggest that regularity in the direction of and evenness in magnitude of mechanical torque of utilized modules are significant factor for determining learning performance. Although the criteria were originally conceived for an error-based learning scheme, the approach to pursue which set of modules is better for motor control can have significant implications in other studies of modularity in general.

Abstract The use of neurofeedback is an important aspect of effective motor rehabilitation as it offers real-time sensory information to promote neuroplasticity. However, there is still limited knowledge about how the brain’s functional networks reorganize in response to such feedback. To address this gap, this study investigates the reorganization of the brain network during motor imagery tasks when subject to visual stimulation or visual-electrotactile stimulation feedback. This study can provide healthcare professionals with a deeper understanding of the changes in the brain network and help develop successful treatment approaches for brain–computer interface-based motor rehabilitation applications. We examine individual edges, nodes, and the entire network, and use the minimum spanning tree algorithm to construct a brain network representation using a functional connectivity matrix. Furthermore, graph analysis is used to detect significant features in the brain network that might arise in response to the feedback. Additionally, we investigate the power distribution of brain activation patterns using power spectral analysis and evaluate the motor imagery performance based on the classification accuracy. The results showed that the visual and visual-electrotactile stimulation feedback induced subject-specific changes in brain activation patterns and network reorganization in the $$\alpha$$ α band. Thus, the visual-electrotactile stimulation feedback significantly improved the integration of information flow between brain regions associated with motor-related commands and higher-level cognitive functions, while reducing cognitive workload in the sensory areas of the brain and promoting positive emotions. Despite these promising results, neither neurofeedback modality resulted in a significant improvement in classification accuracy, compared with the absence of feedback. These findings indicate that multimodal neurofeedback can modulate imagery-mediated rehabilitation by enhancing motor-cognitive communication and reducing cognitive effort. In future interventions, incorporating this technique to ease cognitive demands for participants could be crucial for maintaining their motivation to engage in rehabilitation.

Abstract A self-organized phenomenon in postural coordination is essential for understanding the auto-switching mechanism of in-phase and anti-phase postural coordination modes during standing and related supra-postural activities. Previously, a model-based approach was proposed to reproduce such self-organized phenomenon. However, if we set this problem including the process of how we establish the internal predictive model in our central nervous system, the learning process is critical to be considered for establishing a neural network for managing adaptive postural control. Particularly when body characteristics may change due to growth or aging or are initially unknown for infants, a learning capability can improve the hyper-adaptivity of human motor control for maintaining postural stability and saving energy in daily living. This study attempted to generate a self-organizing neural network that can adaptively coordinate the postural mode without assuming a prior body model regarding body dynamics and kinematics. Postural coordination modes are reproduced in head-target tracking tasks through a deep reinforcement learning algorithm. The transitions between the postural coordination types, i.e. in-phase and anti-phase coordination modes, could be reproduced by changing the task condition of the head tracking target, by changing the frequencies of the moving target. These modes are considered emergent phenomena existing in human head tracking tasks. Various evaluation indices, such as correlation, and relative phase of hip and ankle joint, are analyzed to verify the self-organizing neural network performance to produce the postural coordination transition between the in-phase and anti-phase modes. In addition, after learning, the neural network can also adapt to continuous task condition changes and even to unlearned body mass conditions keeping consistent in-phase and anti-phase mode alternation.

Humans can generate and sustain a wide range of walking velocities while optimizing their energy efficiency. Understanding the intricate mechanisms governing human walking will contribute to the engineering applications such as energy-efficient biped robots and walking assistive devices. Reflex-based control mechanisms, which generate motor patterns in response to sensory feedback, have shown promise in generating human-like walking in musculoskeletal models. However, the precise regulation of velocity remains a major challenge. This limitation makes it difficult to identify the essential reflex circuits for energy-efficient walking. To explore the reflex control mechanism and gain a better understanding of its energy-efficient maintenance mechanism, we extend the reflex-based control system to enable controlled walking velocities based on target speeds. We developed a novel performance-weighted least squares (PWLS) method to design a parameter modulator that optimizes walking efficiency while maintaining target velocity for the reflex-based bipedal system. We have successfully generated walking gaits from 0.7 to 1.6 m/s in a two-dimensional musculoskeletal model based on an input target velocity in the simulation environment. Our detailed analysis of the parameter modulator in a reflex-based system revealed two key reflex circuits that have a significant impact on energy efficiency. Furthermore, this finding was confirmed to be not influenced by setting parameters, i.e., leg length, sensory time delay, and weight coefficients in the objective cost function. These findings provide a powerful tool for exploring the neural bases of locomotion control while shedding light on the intricate mechanisms underlying human walking and hold significant potential for practical engineering applications.

Abstract The autonomous distillation of physical laws only from data is of great interest in many scientific fields. Data-driven modeling frameworks that adopt sparse regression techniques, such as sparse identification of nonlinear dynamics (SINDy) and its modifications, are developed to resolve difficulties in extracting underlying dynamics from experimental data. However, SINDy faces certain difficulties when the dynamics contain rational functions. The Lagrangian is substantially more concise than the actual equations of motion, especially for complex systems, and it does not usually contain rational functions for mechanical systems. Few proposed methods proposed to date, such as Lagrangian-SINDy we have proposed recently, can extract the true form of the Lagrangian of dynamical systems from data; however, these methods are easily affected by noise as a fact. In this study, we developed an extended version of Lagrangian-SINDy (xL-SINDy) to obtain the Lagrangian of dynamical systems from noisy measurement data. We incorporated the concept of SINDy and used the proximal gradient method to obtain sparse Lagrangian expressions. Further, we demonstrated the effectiveness of xL-SINDy against different noise levels using four mechanical systems. In addition, we compared its performance with SINDy-PI (parallel, implicit) which is a latest robust variant of SINDy that can handle implicit dynamics and rational nonlinearities. The experimental results reveal that xL-SINDy is much more robust than the existing methods for extracting the governing equations of nonlinear mechanical systems from data with noise. We believe this contribution is significant toward noise-tolerant computational method for explicit dynamics law extraction from data.

Policy learning enables agents to learn how to map states to actions, thus enabling adaptive and flexible behavioral generation in complex environments. Policy learning methods are fundamental to reinforcement learning techniques. However, as problem complexity and the requirement for motion flexibility increase, traditional methods that rely on manual design have revealed their limitations. Conversely, data-driven policy learning focuses on extracting strategies from biological behavioral data and aims to replicate these behaviors in real-world environments. This approach enhances the adaptability of agents to dynamic substrates. Furthermore, this approach has been extensively applied in autonomous driving, robot control, and interpretation of biological behavior. In this review, we survey developments in data-driven policy-learning algorithms over the past decade. We categorized them into the following three types according to the purpose of the method: (1) imitation learning (IL), (2) inverse reinforcement learning (IRL), and (3) causal policy learning (CPL). We describe the classification principles, methodologies, progress, and applications of each category in detail. In addition, we discuss the distinct features and practical applications of these methods. Finally, we explore the challenges these methods face and prospective directions for future research.

Imitation learning is a promising approach for robots to learn complex motor skills. Recent techniques allow robots to learn long-term movements comprising multiple sub-behaviors. However, learning the temporal structures of movements from a demonstration is challenging, particularly when sub-behaviors overlap and are not labeled in advance. This study applied time-varying synergies, which are representations of spatial and temporal structures in human behavior in neuroscience, to imitation learning. The proposed method extracts time-varying synergies from human demonstrations, with neural networks that learn their activation patterns. Because time-varying synergies can decompose demonstrations into linear combinations of primitives while allowing overlapping, neural networks can learn demonstrations efficiently. This would make the model compact and improve its generalization ability. The proposed method was evaluated with the task of cursive letter writing requiring overlapping sub-behaviors. Consequently, the proposed method allows a neural network to generate new movements with a higher success rate and fewer parameters than those without the proposed method. Moreover, the neural network worked robustly against control deviations and disturbances in an actual robot.

Abstract The synchronization phenomenon is common to many natural mechanical systems. Joint friction and damping in humans and animals are associated with energy dissipation. A coupled oscillator model is conventionally used to manage multiple joint torque generations to form a limit cycle in an energy dissipation system. The coupling term design and the frequency and phase settings become issues when selecting the oscillator model. The relative coupling relationship between oscillators needs to be predefined for unknown dynamics systems, which is quite challenging problem. We present a simple distributed neural integrators method to induce the limit cycle in unknown energy dissipation systems without using a coupled oscillator. The results demonstrate that synergetic synchronized oscillation could be produced that adapts to different physical environments. Finding the balanced energy injection by neural inputs to form dynamic equilibrium is not a trivial problem, when the dynamics information is not priorly known. The proposed method realized self-organized pattern generation to induce the dynamic equilibrium for different mechanical systems. The oscillation was managed without using the explicit phase or frequency knowledge. However, phase, frequency, and amplitude modulation emerged to form an efficient synchronized limit cycle. This type of distributed neural integrator can be used as a source for regulating multi-joint coordination to induce synergetic oscillations in natural mechanical systems.

The steady increase in the aging population worldwide is expected to cause a shortage of doctors and therapists for older people. This demographic shift requires more efficient and automated systems for rehabilitation and physical ability evaluations. Rehabilitation using mixed reality (MR) technology has attracted much attention in recent years. MR displays virtual objects on a head-mounted see-through display that overlies the user’s field of vision and allows users to manipulate them as if they exist in reality. However, tasks in previous studies applying MR to rehabilitation have been limited to tasks in which the virtual objects are static and do not interact dynamically with the surrounding environment. Therefore, in this study, we developed an application to evaluate cognitive and motor functions with the aim of realizing a rehabilitation system that is dynamic and has interaction with the surrounding environment using MR technology. The developed application enabled effective evaluation of the user’s spatial cognitive ability, task skillfulness, motor function, and decision-making ability. The results indicate the usefulness and feasibility of MR technology to quantify motor function and spatial cognition both for static and dynamic tasks in rehabilitation.

Electroencephalography (EEG) is the most prevalent signal acquisition technique for brain-computer interface (BCI). However, the statistical distribution of EEG data varies across subjects and sessions, resulting in poor generalization of the domain-specific classifier. Although the collection of a large number of recordings may alleviate this issue, it is often impractical and not user-friendly. This study proposes the integration of deep domain adaptation with few-shot learning to address the challenge by leveraging the knowledge from multiple source subjects to enhance the performance of a single target subject. The framework incorporated 3 modules: a feature extractor, domain discriminator, and classifier. The feature extractor utilized the available labeled samples with supervised contrastive loss to map the discriminate features onto a deep representation space, where the features from the same class were more similar than those from different classes. The domain discriminator was used to reduce domain drift, through adversarial training. The classifier predicted the user motor intention, based on EEG features. The framework was extensively evaluated through the BCI Competition IV Datasets 2a and 2b. The results of this study indicate that the framework is capable of enhancing the BCI performance and potentially decreases the calibration effort compared to the traditional approach, but the major limitation of this method is that it requires meticulous selection of source subjects.

Recently, soft robotics has gained considerable attention as it promises numerous applications thanks to unique features originating from the physical compliance of the robots. Biomimetic underwater robots are a promising application in soft robotics and are expected to achieve efficient swimming comparable to the real aquatic life in nature. However, the energy efficiency of soft robots of this type has not gained much attention and has been fully investigated previously. This paper presents a comparative study to verify the effect of soft-body dynamics on energy efficiency in underwater locomotion by comparing the swimming of soft and rigid snake robots. These robots have the same motor capacity, mass, and body dimensions while maintaining the same actuation degrees of freedom. Different gait patterns are explored using a controller based on grid search and the deep reinforcement learning controller to cover the large solution space for the actuation space. The quantitative analysis of the energy consumption of these gaits indicates that the soft snake robot consumed less energy to reach the same velocity as the rigid snake robot. When the robots swim at the same average velocity of 0.024 m/s, the required power for the soft-body robot is reduced by 80.4% compared to the rigid counterpart. The present study is expected to contribute to promoting a new research direction to emphasize the energy efficiency advantage of soft-body dynamics in robot design.

Motion prediction based on kinematic information such as body segment displacement and joint angle has been widely studied. Because motions originate from forces, it is beneficial to estimate dynamic information, such as the ground reaction force (GRF), in addition to kinematic information for advanced motion prediction. In this study, we proposed a method to estimate GRF and ground reaction moment (GRM) from electromyography (EMG) in combination with and without an inertial measurement unit (IMU) sensor using a machine learning technique. A long short-term memory network, which is suitable for processing long time-span data, was constructed with EMG and IMU as input data to estimate GRF during posture control and stepping motion. The results demonstrate that the proposed method can provide the GRF estimation with a root mean square error (RMSE) of 8.22 ± 0.97% (mean ± SE) for the posture control motion and 11.17 ± 2.16% (mean ± SE) for the stepping motion. We could confirm that EMG input is essential especially when we need to predict both GRF and GRM with limited numbers of sensors attached under knees. In addition, we developed a GRF visualization system integrated with ongoing motion in a Unity environment. This system enabled the visualization of the GRF vector in 3-dimensional space and provides predictive motion direction based on the estimated GRF, which can be useful for human motion prediction with portable sensors.

Generating multimodal locomotion in underactuated bipedal robots requires control solutions that can facilitate motion patterns for drastically different dynamical modes, which is an extremely challenging problem in locomotion-learning tasks. Also, in such multimodal locomotion, utilizing body morphology is important because it leads to energy-efficient locomotion. This study provides a framework that reproduces multimodal bipedal locomotion using passive dynamics through deep reinforcement learning (DRL). An underactuated bipedal model was developed based on a passive walker, and a controller was designed using DRL. By carefully planning the weight parameter settings of the DRL reward function during the learning process based on a curriculum learning method, the bipedal model successfully learned to walk, run, and perform gait transitions by adjusting only one command input. These results indicate that DRL can be applied to generate various gaits with the effective use of passive dynamics.

IntroductionEmerging deep learning approaches to decode motor imagery (MI) tasks have significantly boosted the performance of brain-computer interfaces. Although recent studies have produced satisfactory results in decoding MI tasks of different body parts, the classification of such tasks within the same limb remains challenging due to the activation of overlapping brain regions. A single deep learning model may be insufficient to effectively learn discriminative features among tasks.MethodsThe present study proposes a framework to enhance the decoding of multiple hand-MI tasks from the same limb using a multi-branch convolutional neural network. The CNN framework utilizes feature extractors from established deep learning models, as well as contrastive representation learning, to derive meaningful feature representations for classification.ResultsThe experimental results suggest that the proposed method outperforms several state-of-the-art methods by obtaining a classification accuracy of 62.98% with six MI classes and 76.15 % with four MI classes on the Tohoku University MI-BCI and BCI Competition IV datasets IIa, respectively.DiscussionDespite requiring heavy data augmentation and multiple optimization steps, resulting in a relatively long training time, this scheme is still suitable for online use. However, the trade-of between the number of base learners, training time, prediction time, and system performance should be carefully considered.

Underwater snake robots have received attention because of their unique mechanics and locomotion patterns. Given their highly redundant degrees of freedom, designing an energy-efficient gait has been a main challenge for the long-term autonomy of underwater snake robots. We propose a gait design method for an underwater snake robot based on deep reinforcement learning and curriculum learning. For comparison, we consider the gait generated by a conventional parametric gait equation controller as the baseline. Furthermore, inspired by the joints of living organisms, we consider elasticity (stiffness) in the joints of the snake robot to verify whether it contributes to the generation of energy efficiency in the underwater gait. We first demonstrate that the deep reinforcement learning controller can produce a more energy-efficient gait than the gait equation controller in underwater locomotion, by finding the control patterns which maximize the effect of energy efficiency through the exploitation of joint elasticity. In addition, appropriate joint elasticity can increase the maximum velocity achievable by a snake robot. Finally, simulation results in different liquid environments confirm that the deep reinforcement learning controller is superior to the gait equation controller, and it can find adaptive energy-efficient motion even when the liquid environment is changed. The video can be viewed at https://youtu.be/wpwQihhntEY.

The lack of intuitive controllability remains a primary challenge in enabling transhumeral amputees to control a prosthesis for arm reaching with residual limb kinematics. Recent advancements in prosthetic arm control have focused on leveraging the predictive capabilities of artificial neural networks (ANNs) to automate elbow joint motion and wrist pronation–supination during target reaching tasks. However, large quantities of human motion data collected from different subjects for various activities of daily living (ADL) tasks are required to train these ANNs. For example, the reaching motion can be altered when the height of the desk is changed; however, it is cumbersome to conduct human experiments for all conditions. This paper proposes a framework for cloning motion datasets using deep reinforcement learning (DRL) to cater to training data requirements. DRL algorithms have been demonstrated to create human-like synergistic motion in humanoid agents to handle redundancy and optimize movements. In our study, we collected real motion data from six individuals performing multi-directional arm reaching tasks in the horizontal plane. We generated synthetic motion data that mimicked similar arm reaching tasks by utilizing a physics simulation and DRL-based arm manipulation. We then trained a CNN-LSTM network with different configurations of training motion data, including DRL, real, and hybrid datasets, to test the efficacy of the cloned motion data. The results of our evaluation showcase the effectiveness of the cloned motion data in training the ANN to predict natural elbow motion accurately across multiple subjects. Furthermore, motion data augmentation through combining real and cloned motion datasets has demonstrated the enhanced robustness of the ANN by supplementing and diversifying the limited training data. These findings have significant implications for creating synthetic dataset resources for various arm movements and fostering strategies for automatized prosthetic elbow motion.

Transhumeral amputees experience considerable difficulties with controlling a multifunctional prosthesis (powered hand, wrist, and elbow) due to the lack of available muscles to provide electromyographic (EMG) signals. The residual limb motion strategy has become a popular alternative for transhumeral prosthesis control. It provides an intuitive way to estimate the motion of the prosthesis based on the residual shoulder motion, especially for target reaching tasks. Conventionally, a predictive model, typically an artificial neural network (ANN), is directly trained and relied upon to map the shoulder–elbow kinematics using the data from able-bodied subjects without extracting any prior synergistic information. However, it is essential to explicitly identify effective synergies and make them transferable across amputee users for higher accuracy and robustness. To overcome this limitation of the conventional ANN learning approach, this study explicitly combines the kinematic synergies with a recurrent neural network (RNN) to propose a synergy-space neural network for estimating forearm motions (i.e., elbow joint flexion–extension and pronation–supination angles) based on residual shoulder motions. We tested 36 training strategies for each of the 14 subjects, comparing the proposed synergy-space and conventional neural network learning approaches, and we statistically evaluated the results using Pearson’s correlation method and the analysis of variance (ANOVA) test. The offline cross-subject analysis indicates that the synergy-space neural network exhibits superior robustness to inter-individual variability, demonstrating the potential of this approach as a transferable and generalized control strategy for transhumeral prosthesis control.

One of the fundamental limitations in human biomechanics is that we cannot directly obtain joint moments during natural movements without affecting the motion. However, estimating these values is feasible with inverse dynamics computation by employing external force plates, which can cover only a small area of the plate. This work investigated the Long Short-Term Memory (LSTM) network for the kinetics and kinematics prediction of human lower limbs when performing different activities without using force plates after the learning. We measured surface electromyography (sEMG) signals from 14 lower extremities muscles to generate a 112-dimensional input vector from three sets of features: root mean square, mean absolute value, and sixth-order autoregressive model coefficient parameters for each muscle in the LSTM network. With the recorded experimental data from the motion capture system and the force plates, human motions were reconstructed in a biomechanical simulation created using OpenSim v4.1, from which the joint kinematics and kinetics from left and right knees and ankles were retrieved to serve as output for training the LSTM. The estimation results using the LSTM model deviated from labels with average R2 scores (knee angle: 97.25%, knee moment: 94.9%, ankle angle: 91.44%, and ankle moment: 85.44%). These results demonstrate the feasibility of the joint angle and moment estimation based solely on sEMG signals for multiple daily activities without requiring force plates and a motion capture system once the LSTM model is trained.

A self-balancing wheel-legged robot provides higher maneuverability and mobility than legged biped robots. For this reason, wheel-legged systems have attracted enormous interest from academia and commercial sectors in recent years. Most of the past works in this field mainly focused on lower body stabilization. Motivated by the human ability to maintain balance in laborious activities by articulating the arm actively, we explore and analyze the active arm control on top of the wheel-legged system to assist in its balance recovery during external pushes and disturbances. This paper presents a control framework to improve the stability and robustness of an underactuated self-balancing wheel-legged robot using its upper limb arm. Furthermore, we use the centroidal moment pivot (CMP) as a key metric to quantitatively evaluate the effect of the active arm usage on the balance stability improvement of the robot in the ROS-Gazebo environment. The difference from the case of nonusage of arm is verified to clarify the impact of the active arm quantitatively. This concept would lead to the wheel-legged biped robot with an active arm for dual purposes, one is for carrying objects, another is for increasing the balance stability. This point is important for future application in a real-world environment with human-robot interactions.

In Deep Reinforcement Learning (DRL) for robotics application, it is important to find energy-efficient motions. For this purpose, a standard method is to set an action penalty in the reward to find the optimal motion considering the energy expenditure. This method is widely used for the simplicity of implementation. However, since the reward is a linear sum, if the penalty is too large, the system will fall into local minima and no moving solution can be obtained. In contrast, if the penalty is too small, the effect may not be sufficient. Therefore, it is necessary to adjust the amount of the penalty so that the agent always moves dynamically, and the energy-saving effect is sufficient. Nevertheless, since adjusting the hyperparameters is computationally expensive, we need a learning method that is robust to the penalty setting problem. We investigated on the Spiking Neural Network (SNN), which has been attracting attention for its computational efficiency and neuromorphic architecture. We conducted gait experiments using a hexapod agent while varying the energy penalty settings in the simulation environment. By applying SNN to the conventional state-of-the-art DRL algorithms, we examined whether the agent could explore for an optimal gait with a larger penalty variation and obtain an energy-efficient gait verified with Cost of Transport (CoT), a metric of energy efficiency for gait. Soft Actor-Critic (SAC)+SNN resulted in a CoT of 1.64, Twin Delayed Deep Deterministic policy gradient (TD3)+SNN resulted in a CoT of 2.21, and Deep Deterministic policy gradient (DDPG)+SNN resulted in a CoT of 2.08 (1.91 for normal SAC, 2.38 for TD3, and 2.40 for DDPG). DRL combined with SNN succeeded in learning more energy efficient gait with lower CoT.

Humans can rapidly adapt to new situations, even though they have redundant degrees of freedom (d.f.). Previous studies in neuroscience revealed that human movements could be accounted for by low-dimensional control signals, known as motor synergies. Many studies have suggested that humans use the same repertories of motor synergies among similar tasks. However, it has not yet been confirmed whether the combinations of motor synergy repertories can be re-used for new targets in a systematic way. Here we show that the combination of motor synergies can be generalized to new targets that each repertory cannot handle. We use the multi-directional reaching task as an example. We first trained multiple policies with limited ranges of targets by reinforcement learning and extracted sets of motor synergies. Finally, we optimized the activation patterns of sets of motor synergies and demonstrated that combined motor synergy repertories were able to reach new targets that were not achieved with either original policies or single repertories of motor synergies. We believe this is the first study that has succeeded in motor synergy generalization for new targets in new planes, using a full 7-d.f. arm model, which is a realistic mechanical environment for general reaching tasks.

The purpose of this study is to implement a human-like balance recovery controller and analyze its robustness and energy consumption. Three main techniques to maintain balance can be distinguished in humans, namely (i) the ankle strategy, (ii) the hip-ankle strategy, (iii) the stepping strategy. Because we only consider quiet standing balance, then stepping is not included in our balance recovery study. Numerical model predictive control (N-MPC) is proposed to predict the best way to maintain balance against various disturbance forces. To simulate balance recovery, we build a three-link model including a foot with unilateral constraints, the lower body, and the upper body. Subsequently, we derive the dynamical equations of the model and linearize them. Based on human balance capabilities, we set bound constraints on our model, including angles and balance torques of the ankle and hip. Unilateral constraints are set on the foot, which makes our model more similar to the human quiet standing case. Finally, we implemented a simulation of the proposed ankle and hip-ankle strategy in simulation and analyzed the obtained results from kinematic and dynamic indices as well as from an energy consumption perspective. The robustness of the proposed controller was verified through the obtained simulation results. Thus, this study provides a better understanding of human quiet standing balance that could be useful for rehabilitation.

Transcranial direct current stimulation (tDCS) has been shown to evoke hemodynamics response; however, the mechanisms have not been investigated systematically using systems biology approaches. Our study presents a grey-box linear model that was developed from a physiologically detailed multi-compartmental neurovascular unit model consisting of the vascular smooth muscle, perivascular space, synaptic space, and astrocyte glial cell. Then, model linearization was performed on the physiologically detailed nonlinear model to find appropriate complexity (Akaike information criterion) to fit functional near-infrared spectroscopy (fNIRS) based measure of blood volume changes, called cerebrovascular reactivity (CVR), to high-definition (HD) tDCS. The grey-box linear model was applied on the fNIRS-based CVR during the first 150 seconds of anodal HD-tDCS in eleven healthy humans. The grey-box linear models for each of the four nested pathways starting from tDCS scalp current density that perturbed synaptic potassium released from active neurons for Pathway 1, astrocytic transmembrane current for Pathway 2, perivascular potassium concentration for Pathway 3, and voltage-gated ion channel current on the smooth muscle cell for Pathway 4 were fitted to the total hemoglobin concentration (tHb) changes from optodes in the vicinity of 4x1 HD-tDCS electrodes as well as on the contralateral sensorimotor cortex. We found that the tDCS perturbation Pathway 3 presented the least mean square error (MSE, median

The study of human balance recovery strategies is important for human balance rehabilitation and humanoid robot balance control. To date, many efforts have been made to improve balance during quiet standing and walking motions. Arm usage (arm strategy) has been proposed to control the balance during walking motion in the literature. However, limited research exists on the contributions of the arm strategy for balance recovery during quiet standing along with ankle and hip strategy. Therefore, in this study, we built a simplified model with arms and proposed a controller based on nonlinear model predictive control to achieve human-like balance control. Three arm states of the model, namely, active arms, passive arms, and fixed arms, were considered to discuss the contributions of arm usage to human balance recovery during quiet standing. Furthermore, various indexes such as root mean square deviation of joint angles and recovery energy consumption were verified to reveal the mechanism behind arm strategy employment. In this study, we demonstrate to computationally reproduce human-like balance recovery with and without arm rotation during quiet standing while applying different magnitudes of perturbing forces on the upper body. In addition, the conducted human balance experiments are presented as supplementary information in this paper to demonstrate the concept on a typical example of arm strategy.

To obtain biologically inspired robotic control, the architecture of central pattern generators (CPGs) has been extensively adopted to generate periodic patterns for locomotor control. This is attributed to the interesting properties of nonlinear oscillators. Although sensory feedback in CPGs is not necessary for the generation of patterns, it plays a central role in guaranteeing adaptivity to environmental conditions. Nonetheless, its inclusion significantly modifies the dynamics of the CPG architecture, which often leads to bifurcations. For instance, the force feedback can be exploited to derive information regarding the state of the system. In particular, the Tegotae approach can be adopted by coupling proprioceptive information with the state of the oscillation itself in the CPG model. This paper discusses this policy with respect to other types of feedback; it provides higher adaptivity and an optimal energy efficiency for reflex-like actuation. We believe this is the first attempt to analyse the optimal energy efficiency along with the adaptivity of the Tegotae approach.

In this study, we investigated a control algorithm for a semi-active prosthetic knee based on reinforcement learning (RL). Model-free reinforcement Q-learning control with a reward shaping function was proposed as the voltage controller of a magnetorheological damper based on the prosthetic knee. The reward function was designed as a function of the performance index that accounts for the trajectory of the subject-specific knee angle. We compared our proposed reward function to a conventional single reward function under the same random initialization of a Q-matrix. We trained this control algorithm to adapt to several walking speed datasets under one control policy and subsequently compared its performance with that of other control algorithms. The results showed that our proposed reward function performed better than the conventional single reward function in terms of the normalized root mean squared error and also showed a faster convergence trend. Furthermore, our control strategy converged within our desired performance index and could adapt to several walking speeds. Our proposed control structure has also an overall better performance compared to user-adaptive control, while some of its walking speeds performed better than the neural network predictive control from existing studies.

Among end-effector robots for lower limb rehabilitation, systems based on Stewart–Gough platforms enable independent movement of each foot in six degrees of freedom. Nevertheless, control strategies described in recent literature have not been able to fully explore the potential of such a mechatronic system. In this work, we propose two novel approaches for controlling a gait simulator based on Stewart–Gough platforms. The first strategy provides the therapist direct control of each platform using movement data measured by wearable sensors. The following scheme is designed to improve the level of engagement of the patient by enabling a limited degree of control based on trunk inclination. Both strategies are designed to facilitate future studies in tele-rehabilitation settings. Experimental results have illustrated the feasibility of both control interfaces, either in terms of system performance or user subjective evaluation. Technical capacity to deploy in tele-rehabilitation was also verified in this work.

This study is aimed at the detection of single-trial feedback, perceived as erroneous by the user, using a transferable classification system while conducting a motor imagery brain–computer interfacing (BCI) task. The feedback received by the users are relayed from a functional electrical stimulation (FES) device and hence are somato-sensory in nature. The BCI system designed for this study activates an electrical stimulator placed on the left hand, right hand, left foot, and right foot of the user. Trials containing erroneous feedback can be detected from the neural signals in form of the error related potential (ErrP). The inclusion of neuro-feedback during the experiments indicated the possibility that ErrP signals can be evoked when the participant perceives an error from the feedback. Hence, to detect such feedback using ErrP, a transferable (offline) decoder based on optimal transport theory is introduced herein. The offline system detects single-trial erroneous trials from the feedback period of an online neuro-feedback BCI system. The results of the FES-based feedback BCI system were compared to a similar visual-based (VIS) feedback system. Using our framework, the error detector systems for both the FES and VIS feedback paradigms achieved an F1-score of 92.66% and 83.10%, respectively, and are significantly superior to a comparative system where an optimal transport was not used. It is expected that this form of transferable and automated error detection system compounded with a motor imagery system will augment the performance of a BCI and provide a better BCI-based neuro-rehabilitation protocol that has an error control mechanism embedded into it.

An important function missing from current robotic systems is a human-like method for creating behavior from symbolized information. This function could be used to assess the extent to which robotic behavior is human-like because it distinguishes human motion from that of human-made machines created using currently available techniques. The purpose of this research is to clarify the mechanisms that generate automatic motor commands to achieve symbolized behavior. We design a controller with a learning method called tacit learning, which considers system–environment interactions, and a transfer method called mechanical resonance mode, which transfers the control signals into a mechanical resonance mode space (MRM-space). We conduct simulations and experiments that involve standing balance control against disturbances with a two-degree-of-freedom inverted pendulum and bipedal walking control with humanoid robots. In the simulations and experiments on standing balance control, the pendulum can become upright after a disturbance by adjusting a few signals in MRM-space with tacit learning. In the simulations and experiments on bipedal walking control, the robots realize a wide variety of walking by manually adjusting a few signals in MRM-space. The results show that transferring the signals to an appropriate control space is the key process for reducing the complexity of the signals from the environment and achieving diverse behavior.

Poststroke hemiplegic patients often show altered weight distribution with balance disorders, increasing their risk of fall. Conventional balance training, though powerful, suffers from scarcity of trained therapists, frequent visits to clinics to get therapy, one-on-one therapy sessions, and monotony of repetitive exercise tasks. Thus, technology-assisted balance rehabilitation can be an alternative solution. Here, we chose virtual reality as a technology-based platform to develop motivating balance tasks. This platform was augmented with off-the-shelf available sensors such as Nintendo Wii balance board and Kinect to estimate one’s center of mass (CoM). The virtual reality-based CoM-assisted balance tasks (Virtual CoMBaT) was designed to be adaptive to one’s individualized weight-shifting capability quantified through CoM displacement. Participants were asked to interact with Virtual CoMBaT that offered tasks of varying challenge levels while adhering to ankle strategy for weight shifting. To facilitate the patients to use ankle strategy during weight-shifting, we designed a heel lift detection module. A usability study was carried out with 12 hemiplegic patients. Results indicate the potential of our system to contribute to improving one’s overall performance in balance-related tasks belonging to different difficulty levels.

Reliable detection of error from electroencephalography (EEG) signals as feedback while performing a discrete target selection task across sessions and subjects has a huge scope in real-time rehabilitative application of Brain-computer Interfacing (BCI). Error Related Potentials (ErrP) are EEG signals which occur when the participant observes an erroneous feedback from the system. ErrP holds significance in such closed-loop system, as BCI is prone to error and we need an effective method of systematic error detection as feedback for correction. In this paper, we have proposed a novel scheme for online detection of error feedback directly from the EEG signal in a transferable environment (i.e., across sessions and across subjects). For this purpose, we have used a P300-speller dataset available on a BCI competition website. The task involves the subject to select a letter of a word which is followed by a feedback period. The feedback period displays the letter selected and, if the selection is wrong, the subject perceives it by the generation of ErrP signal. Our proposed system is designed to detect ErrP present in the EEG from new independent datasets, not involved in its training. Thus, the decoder is trained using EEG features of 16 subjects for single-trial classification and tested on 10 independent subjects. The decoder designed for this task is an ensemble of linear discriminant analysis, quadratic discriminant analysis, and logistic regression classifier. The performance of the decoder is evaluated using accuracy, F1-score, and Area Under the Curve metric and the results obtained is 73.97, 83.53, and 73.18%, respectively.

The trajectory of the whole body center of mass (CoM) is useful as a reliable metric of postural stability. If the evaluation of a subject-specific CoM were available outside of the laboratory environment, it would improve the assessment of the effects of physical rehabilitation. This paper develops a method that enables tracking CoM position using low-cost sensors that can be moved around by a therapist or easily installed inside a patient’s home. Here, we compare the accuracy of a personalized CoM estimation using the statically equivalent serial chain (SESC) method and measurements obtained with the Kinect to the case of a SESC obtained with high-end equipment (Vicon). We also compare these estimates to literature-based ones for both sensors. The method was validated with seven able-bodied volunteers for whom the SESC was identified using 40 static postures. The literature-based estimation with Vicon measurements had a average error 24.9 ± 3.7 mm; this error was reduced to 12.8 ± 9.1 mm with the SESC identification. When using Kinect measurements, the literature-based estimate had an error of 118.4 ± 50.0 mm, while the SESC error was 26.6 ± 6.0 mm. The subject-specific SESC estimate using low-cost sensors has an equivalent performance as the literature-based one with high-end sensors. The SESC method can improve CoM estimation of elderly and neurologically impaired subjects by considering variations in their mass distribution.

Time-variant muscle responses under electrical stimulation (ES) are often problematic for all the applications of neuroprosthetic muscle control. This situation limits the range of ES usage in relevant areas, mainly due to muscle fatigue and also to changes in stimulation electrode contact conditions, especially in transcutaneous ES. Surface electrodes are still the most widely used in noninvasive applications.Electrical field variations caused by changes in the stimulation contact condition markedly affect the resulting total muscle activation levels. Fatigue phenomena under functional electrical stimulation (FES) are also well known source of time-varying characteristics coming from muscle response under ES. Therefore it is essential to monitor the actual muscle state and assess the expected muscle response by ES so as to improve the current ES system in favour of adaptive muscle-response-aware FES control. To deal with this issue, we have been studying a novel control technique using evoked electromyography (eEMG) signals to compensate for these muscle time-variances under ES for stable neuroprosthetic muscle control. In this perspective article, I overview the background of this topic and highlight important points to be aware of when using ES to induce the desired muscle activation regardless of the time-variance. I also demonstrate how to deal with the common critical problem of ES to move toward robust neuroprosthetic muscle control with the Evoked Electromyographically Controlled Electrical Stimulation paradigm.

Functional Electrical Stimulation (FES) provides a neuroprosthetic interface to non-recovered muscle groups by stimulating the affected region of the human body. FES in combination with Brain-machine interfacing (BMI) has a wide scope in rehabilitation because this system directly links the cerebral motor intention of the users with its corresponding peripheral muscle activations. In this paper, we examine the effect of FES on the electroencephalography (EEG) during motor imagery (left- and right-hand movement) training of the users. Results suggest a significant improvement in the classification accuracy when the subject was induced with FES stimuli as compared to the standard visual one.

As a neuroprosthetic technique, functional electrical stimulation (FES) can restore lost motor performance of impaired patients. Through delivering electrical pulses to target muscles, the joint movement can be eventually elicited. This work presents a real-time FES system which is able to deal with two neuroprosthetic missions: one is estimating FES-induced joint torque with evoked electromyograph (eEMG), and the other is artificially controlling muscle activation with such eEMG feedback. The clinical experiment results on spinal cord injured (SCI) patients and healthy subjects show promising performance of the proposed FES system.