Search

An error-related potential (ErrP) is a form of event-related potential that occurs when an error-related stimulus is encountered during a task. The decoding of ErrP has the potential to apply for Brain Computer System (BCI). Although various methods have been applied to the decoding of ErrPs, existing classification methods have room for improvement. Using a deep convolutional neural network (DCNN) is a viable approach to ErrP classification but its performance can be compromised by insufficient training data being available. Using a form of generative adversarial network (GAN) enables data augmentation, which has offered significant performance improvement in a variety of fields such as signal processing, robotics, and unmanned vehicles. We therefore propose a novel approach to ErrP classification that combines GAN with DCNN to form a data-augmented DCNN. The proposed method has two main components: a GAN that offers data augmentation and a DCNN that performs the classification. We applied our method to the BNCI2020 dataset 22: Monitoring ErrPs, evaluating the results in terms of classification accuracy in various categories, including single-subject single-session, cross-subject single session, cross-session single-subject, and cross-subject cross-session versions for the entire dataset. The evaluations showed that the classification results had been improved comprehensively in comparison with existing published results. The maximum accuracy of the classification performance for the entire dataset reached 87%, which is 3% above the previous best result.

In a meeting, two or more individuals come together to verbally share information. However, discrepancies in perceptions often occur in complicated discussions. Therefore, whether the information transfer succeeded must be reconfirmed. This problem is typically resolved through ex-post measures such as meeting minutes. Our previous results indicated that the degree of information transfer can be predicted using head movement synchrony. In this study, we propose an information transfer estimation method based on head movement synchrony during a conversation evaluated by an omnidirectional camera that can easily obtain body movements. These body movements were quantified using Motion Energy Analysis, which is commonly used in clinical psychology, with additional processes used to reduce noise caused by lighting in the room and resulting from large accidental movements. By these processes, body movement synchrony was quantified. We confirmed that this synchrony occurred between actually interacting pairs and tested whether it was related to the degree of information transfer. In the experimental evaluation, body movement synchrony was found to be high for interacting pairs, and a positive correlation was observed between synchrony and degree of information transfer. Therefore, our proposed method may detect the synchrony and estimate the degree of information transfer using an omnidirectional camera.

Human interaction has two principle functions: building and maintaining relationships with others and exchanging information. The function of building and maintaining relationships with others relates to interpersonal coordination; this behavior pattern is expected to predict the outcome of social relationships, such as between therapists and patients. It is unclear, however, whether the exchange of information is associated with interpersonal coordination. In the present study, we tested a hypothesis of whether body movement synchrony occurs in a natural conversation and whether this synchrony has a positive correlation with the degree of information exchange. Fifty participants were engaged in a conversation task; each had different roles in the conversation. We measured their body movements during this conversation using an optical motion capture system. Similar to methods that can be found in previous research, we calculated body movements and quantified their synchrony applying the methods previously reported that automatically quantified their body movements. Moreover, we determined the participants' degree of information exchange concerning the conversation using a questionnaire. We observed that the body movement synchrony of pairs who talked with each other was significantly higher than that of pairs who did not talk with each other, and that this synchrony was positively associated with the degree of information exchange. These results suggest that body movement synchrony predicted information exchange.

Researchers investigating virtual/augmented reality have shown humans' marked adaptability, especially regarding our sense of body ownership; their cumulative findings have expanded the concept of what it means to have a body. Herein, we report the hand ownership illusion during "two views merged in." In our experiment, participants were presented two first-person perspective views of their arm overlapped, one was the live feed from a camera and the other was a playback video of the same situation, slightly shifted toward one side. The relative visibility of these two views and synchrony of tactile stimulation were manipulated. Participants' level of embodiment was evaluated using a questionnaire and proprioceptive drift. The results show that the likelihood of embodying the virtual hand is affected by the relative visibility of the two views and synchrony of the tactile events. We observed especially strong hand ownership of the virtual hand in the context of high virtual hand visibility with synchronous tactile stimulation.

Gait analysis is used widely in clinical practice to evaluate abnormal gait caused by disease. Conventionally, medical professionals use motion capture systems or make visual observations to evaluate a patient's gait. Recent biomedical engineering studies have proposed easy-to-use gait analysis methods employing wearable sensors with inertial measurement units (IMUs). IMUs placed on the shanks just above the ankles allow for long-term gait monitoring because the participant can walk with or without shoes during the analysis. To the knowledge of the authors, no IMU-based gait analysis method has been reported that estimates stride length, gait speed, stride duration, stance duration, and swing duration simultaneously. In the present study, we tested a proposed gait analysis method that uses IMUs attached on the shanks to estimate foot trajectory and temporal gait parameters. Our proposed method comprises two steps: stepwise dissociation of continuous gait data into multiple steps and three-dimensional trajectory estimation from data obtained from accelerometers and gyroscopes. We evaluated this proposed method by analyzing the gait of 19 able-bodied participants (mean age 23.9 years, 9 men and 10 women). Wearable sensors were attached on the participants' shanks, and we measured three-axis acceleration and three-axis angular velocity with the sensors to estimate foot trajectory during walking. We compared gait parameters estimated from the foot trajectory obtained with the proposed method and those measured with a motion capture system. Mean accuracy (± standard deviation) was 0.054 ± 0.031 m for stride length, 0.034 ± 0.039 m/s for gait speed, 0.002 ± 0.020 s for stride duration, 0.000 ± 0.017 s for stance duration, and 0.002 ± 0.024 s for swing duration. These results suggest that the proposed method is suitable for gait analysis, whereas there is a room for improvement of its accuracy and further development of this IMU-based gait analysis method will enable us to use such systems for clinical gait analysis.

The self is built as an entity independent from the external world using the human ability to experience the senses of agency and ownership. Humans usually experience these senses during movement. Nevertheless, researchers recently reported that another person’s synchronous mirror-symmetrical movements elicited both agency and ownership in research participants. However, it is unclear whether this elicitation was caused by the synchronicity or the mirror symmetry of the movements. To address this question, we investigated the effect of interpersonal synchronization on the self-reported sense of agency and ownership in two conditions, using movements with and without mirror symmetry. Participants performed rhythmic hand movements while viewing the experimenter’s synchronous or random hand movements, and then reported their perceptions of agency and ownership in a questionnaire. We observed that agency and ownership were significantly elicited by the experimenter’s synchronous hand movements in both conditions. The results suggested that the synchronous movements of another person—rather than mirror- or non-mirror-symmetrical movements—appear to elicit the experience of a sense of agency and ownership. The results also suggested that people could experience these senses not only from their own movements but also from another person’s synchronous movements.

This study aimed to investigate whether using a wearable robot applying interactive rhythmic stimulation on the upper limbs of patients with Parkinson's disease (PD) could affect their gait. The wearable robot presented tactile stimuli on the patients' upper limbs, which was mutually synchronized with the swing of their upper limbs. We conducted an evaluation experiment with PD patients (n = 30, Modified Hoehn-Yahr = 1-3, on-state) to investigate the assistance effect by the robot and the immediate after-effect of intervention. The participants were instructed to walk 30 m under four different conditions: (1) not wearing the robot before the intervention (Pre-condition), (2) wearing the robot without the rhythm assistance (RwoA condition), (3) wearing the robot with rhythm assistance (RwA condition), and (4) not wearing the robot immediately after the intervention (Post-condition). These conditions were conducted in this order over a single day. The third condition was performed three times and the others, once. The arm swing amplitude, stride length, and velocity were increased in the RwA condition compared to the RwoA condition. The coefficient of variance (CV) of the stride duration was decreased in the RwA condition compared to the RwoA condition. These results revealed that the assistance by the robot increased the gait performance of PD patients. In addition, the stride length and velocity were increased and the stride duration CV was decreased in the Post-condition compared to the Pre-condition. These results show that the effect of robot assistance on the patient's gait remained immediately after the intervention. These findings suggest that synchronized rhythmic stimulation on the upper limbs could influence the gait of PD patients and that the robot may assist with gait rehabilitation in these patients.

Behavioral errors may cause serious results. For example, while driving a car, if the intention is to stop and the accelerator pedal is mistaken for the brake pedal, a serious traffic accident may result. If electrophysiological phenomena that involve behavioral errors are identified, we may be able to avoid behavioral errors by detecting the error precursor. We have already revealed a neural sign of behavioral errors in our previous study. In the previous study, we applied a spatiotemporal analysis to high-density electroencephalogram (EEG) signals recorded during a visual discrimination task, a d2 test of attention. We demonstrated that, during trials with error outcomes, positive deviation of scalp amplitude with latency of approx. 30 ms was observed in frontal regions, positive deviation with latency of approx. 125 ms was observed in parietal regions, and then a positive deviation with latency of approx. 160 ms was observed in the occipital region. In this study, we propose a single-trial prediction method of behavioral errors by detecting the electrophysiological sign of behavioral errors.

The simultaneous perception of multimodal sensory information has a crucial role for effective reactions to the external environment. Voluntary movements are known to occasionally affect simultaneous perception of auditory and tactile stimuli presented to the moving body part. However, little is known about spatial limits on the effect of voluntary movements on simultaneous perception, especially when tactile stimuli are presented to a non-moving body part. We examined the effect of voluntary movement on the simultaneous perception of auditory and tactile stimuli presented to the non-moving body part. We considered the possible mechanism using a temporal order judgement task under three experimental conditions: voluntary movement, where participants voluntarily moved their right index finger and judged the temporal order of auditory and tactile stimuli presented to their non-moving left index finger; passive movement; and no movement. During voluntary movement, the auditory stimulus needed to be presented before the tactile stimulus so that they were perceived as occurring simultaneously. This subjective simultaneity differed significantly from the passive movement and no movement conditions. This finding indicates that the effect of voluntary movement on simultaneous perception of auditory and tactile stimuli extends to the non-moving body part.

The unusual configuration of body parts can cause illusions. For example, when tactile stimuli are delivered to crossed arms a reversal of subjective temporal ordering occurs. Our group has previously demonstrated that arm crossing without sensory stimuli causes activity changes in the left posterior parietal cortex (PPC) and an assessment of tactile temporal order judgments (TOJs) revealed a positive association between activity in this area, especially the left intraparietal sulcus (IPS) and the degree of the crossed-hand illusion. Thus, the present study investigated how the IPS actively relates to other cortical areas under arms-crossed and -uncrossed conditions by analyzing the functional connectivity of the IPS. Regions showing connectivity with the IPS overlapped with regions within the default mode network (DMN) but the IPS also showed connectivity with other brain areas, including the frontoparietal control network (FPCN). The right middle/inferior frontal gyrus (MFG/IFG), which is included in the FPCN, showed greater connectivity in the arms-crossed condition than in the arms-uncrossed condition. These findings suggest that there is state-dependent connectivity during arm crossing and that the left IPS may play an important role during the spatio-temporal updating of arm positions.

Performance errors may cause serious consequences. It has been reported that ongoing activity of the frontal control regions across trials associates with the occurrence of performance errors. However, neural mechanisms that cause performance errors remain largely unknown. In this study, we hypothesized that some neural functions required for correct outcomes are lacking just before performance errors and to determine this lack of neural function we applied a spatiotemporal analysis to high-density electroencephalogram signals recorded during a visual discrimination task, a d2 test of attention. To our knowledge, this is the first report of a difference in the temporal development of scalp ERP between trials with error and correct outcomes as seen by topography during the d2 test of attention. We observed differences in the signal potential in the frontal region and then the occipital region between reaction times matched with correct and error outcomes. Our observations suggest that lapses of top-down signals from frontal control regions cause performance errors just after the lapses.

The ownership of one's body parts represents a fundamental aspect of self-consciousness. Accumulating empirical evidence supports the existence of this concept in humans and nonhuman primates, but it is unclear whether nonprimate mammals experience similar feelings. Therefore, the present study used rubber tails to investigate body ownership in rodents. When the real tails and rubber tails were synchronously stroked, the mice responded as if their own tails were touched when the rubber tails were grasped. In contrast, when the stimuli were delivered asynchronously, there was a significantly lower mean response rate when the rubber tail was grasped. These findings suggest that mice may experience body ownership of their tails, suggestive of the rubber hand illusion in humans.SIGNIFICANCE STATEMENTTo explore the manner in which the ownership of body parts is experienced, this study specifically used the rubber hand illusion (RHI), in which self-consciousness can be extended out of one's own body. Accumulating empirical evidence supports the existence of this concept in humans and nonhuman primates, but it remains unclear whether nonprimate mammals experience similar feelings. This study demonstrated for the first time that mice may experience body ownership of their tails, which is suggestive of the RHI in humans and provides evidence that may highlight how humans experience the ownership of body parts.