Search

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7–9th, 2021 in a virtual format. The conference continued the BCI Meeting series’ interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Objective. The maintenance of balance is a complicated process in the human brain, which involves multisensory processing such as somatosensory and visual processing, motor planning and execution. It was shown that a specific cortical activity called perturbation-evoked potential (PEP) appears in the electroencephalogram (EEG) during balance perturbation. PEPs are primarily recognized by the N1 component with a negative peak localized in frontal and central regions. There has been a doubt in balance perturbation studies whether the N1 potential of perturbation is elicited due to error processing in the brain. The objective of this study is to test whether the brain perceives postural instability as a cognitive error by imposing two types of perturbations consisting of erroneous and correct perturbations. Approach. We conducted novel research to incorporate the experiment designs of both error and balance studies. To this end, participants encountered errors during balance perturbations at rare moments in the experiment. We induced errors by imposing perturbations to participants in the wrong directions and an erroneous perturbation was considered as a situation when the participant was exposed to an opposite direction of the expected/informed one. In correct perturbations, participants were tilted to the same direction, as they were informed. We analyzed the two conditions in time, time-frequency, and source domains. Main results. We showed that two error-related neural markers were derived from the EEG responses, including error positivity (Pe), and error-related alpha suppression (ERAS) during erroneous perturbations. Consequently, early neural correlates of perturbation cannot be interpreted as error-related responses. We discovered distinct patterns of conscious error processing; both Pe and ERAS are associated with conscious sensations of error. Significance. Our findings indicated that early cortical responses of balance perturbation are not associated with neural error processing of the brain, and errors induce distinct cortical responses that are distinguishable from brain dynamics of N1 potential.

BACKGROUND: People who survive severe brain damage may eventually develop a prolonged consciousness disorder. Others can regain full consciousness but remain unable to speak or move because of the severity of the lesions, as for those with locked-in syndrome (LIS). Brain-computer interface techniques can be useful to disentangle these states by detecting neurophysiological correlates of conscious processing of information to enable communication with these individuals after the diagnosis.OBJECTIVE: The goal of our study was to evaluate with a user-centered design approach the usability of a mental imagery task to detect signs of voluntary information processing and enabling communication in a group of severely disabled individuals.METHODS: Five individuals with LIS participated in the study. Participants were instructed to imagine hand, arm or feet movements during electroencephalography (EEG) to detect patterns of event-related synchronization/desynchronization associated with each task. After the user-centered design, usability was evaluated (i.e., efficiency, effectiveness and satisfaction).RESULTS: Two participants achieved significant levels of accuracy in 2 different tasks. The associated workload and levels of satisfaction perceived by the users were moderate and were mainly related to the time demand of the task.CONCLUSION: Results showed lack of effectiveness of the task to detect voluntary brain activity and thus detect consciousness or communicate with non-responsive individuals. The application must be modified to be sufficiently satisfying for the intended end-users and suggestions are made in this regard.

Neuroimaging studies have provided proof that loss of balance evokes specific neural transient wave complexes in electroencephalography (EEG), called perturbation evoked potentials (PEPs). Online decoding of balance perturbations from ongoing EEG signals can establish the possibility of implementing passive brain-computer interfaces (pBCIs) as a part of aviation/driving assistant systems. In this study, we investigated the feasibility of identifying the existence and expression of perturbations in four different conditions by using EEG signals. Fifteen healthy participants experienced four various postural changes while they sat in a glider cockpit. Sudden perturbations were exposed by a robot connected to a glider and moved to the right and left directions with tilting angles of 5 and 10 degrees. Perturbations occurred in an oddball paradigm in which participants were not aware of the time and expression of the perturbations. We employed a hierarchical approach to separate the perturbation and rest, and then discriminate the expression of perturbations. The performance of the BCI system was evaluated by using classification accuracy and F1 score. Asynchronously, we achieved average accuracies of 89.83 and 73.64% and average F1 scores of 0.93 and 0.60 for binary and multiclass classification, respectively. These results manifest the practicality of pBCI for the detection of balance disturbances in a realistic situation.

Error-related potentials (ErrPs) are the neural signature of error processing. Therefore, the detection of ErrPs is an intuitive approach to improve the performance of brain-computer interfaces (BCIs). The incorporation of ErrPs in discrete BCIs is well established but the study of asynchronous detection of ErrPs is still in its early stages. Here we show the feasibility of asynchronously decoding ErrPs in an online scenario. For that, we measured EEG in 15 participants while they controlled a robotic arm towards a target using their right hand. In 30% of the trials, the control of the robotic arm was halted at an unexpected moment (error onset) in order to trigger error-related potentials. When an ErrP was detected after the error onset, participants regained the control of the robot and could finish the trial. Regarding the asynchronous classification in the online scenario, we obtained an average true positive rate (TPR) of 70% and an average true negative rate (TNR) of 86.8%. These results indicate that the online asynchronous decoding of ErrPs was, on average, reliable, showing the feasibility of the asynchronous decoding of ErrPs in an online scenario.

For Brain-Computer interfaces (BCIs), system calibration is a lengthy but necessary process for successful operation. Co-adaptive BCIs aim to shorten training and imply positive motivation to users by presenting feedback already at early stages: After just 5 min of gathering calibration data, the systems are able to provide feedback and engage users in a mutual learning process. In this work, we investigate whether the retraining stage of co-adaptive BCIs can be adapted to a semi-supervised concept, where only a small amount of labeled data is available and all additional data needs to be labeled by the BCI itself. The aim of the current work was to evaluate whether a semi-supervised co-adaptive BCI could successfully compete with a supervised co-adaptive BCI model. In a supporting two-class (190 trials per condition) BCI study based on motor imagery tasks, we evaluated both approaches in two separate groups of 10 participants online, while we simulated the other approach in each group offline. Our results indicate that despite the lack of true labeled data, the semi-supervised driven BCI did not perform significantly worse (p > 0.05) than the supervised counterpart. We believe that these findings contribute to developing BCIs for long-term use, where continuous adaptation becomes imperative for maintaining meaningful BCI performance. Graphical abstractIn this work, we investigate whether the retraining stage of a co-adaptive BCI can be adapted to a semi-supervised concept, where only a small amount of labeled data is available and all additional data needs to be labeled by the BCI itself. In two groups of 10 persons, we evaluate a supervised as well as a semi-supervised approach. Our results indicate that despite the lack of true labeled data, the semi-supervised driven BCI did not perform significantly worse (p > 0.05) than the supervised counterpart. Electronic supplementary material The online version of this article (10.1007/s11517-019-02047-1) contains supplementary material, which is available to authorized users.

Movement decoders exploit the tuning of neural activity to various movement parameters with the ultimate goal of controlling end-effector action. Invasive approaches, typically relying on spiking activity, have demonstrated feasibility. Results of recent functional neuroimaging studies suggest that information about movement parameters is even accessible non-invasively in the form of low-frequency brain signals. However, their spatiotemporal tuning characteristics to single movement parameters are still unclear. Here, we extend the current understanding of low-frequency electroencephalography (EEG) tuning to position and velocity signals. We recorded EEG from 15 healthy participants while they performed visuomotor and oculomotor pursuit tracking tasks. Linear decoders, fitted to EEG signals in the frequency range of the tracking movements, predicted positions and velocities with moderate correlations (0.2–0.4; above chance level) in both tasks. Predictive activity in terms of decoder patterns was significant in superior parietal and parieto-occipital areas in both tasks. By contrasting the two tracking tasks, we found that predictive activity in contralateral primary sensorimotor and premotor areas exhibited significantly larger tuning to end-effector velocity when the visuomotor tracking task was performed.

Decoding intended movements from individuals with spinal cord injury (SCI) has been a central topic in braincomputer interface research for decades. Recent works, relying on neural spiking activity, demonstrated that the kinematics of intended movements can be detected in neural spiking activity and used by individuals with SCI to control end-effectors. Whether, and to which degree this approach translates to EEG remains an open question. In this work, we summarize our attempts towards realizing an EEG-based movement decoder. We summarize our efforts to address this topic from various perspectives, and we present results of a single case study with a non-disabled participant, where we decoded the intended movement trajectories, while the participant's arm was fixed.

Movement intention detection using electroencephalography (EEG) is a challenging but essential component of brain–computer interfaces (BCIs) for people with motor disabilities. Objective. The goal of this study is to develop a new experimental paradigm to perform asynchronous online detection of movement based on low-frequency time-domain EEG features, concretely on movement-related cortical potentials. The paradigm must be easily transferable to people without any residual upper-limb movement function and the BCI must be independent of upper-limb movement onset measurements and external cues. Approach. In a study with non-disabled participants, we evaluated a novel BCI paradigm to detect self-initiated reach-and-grasp movements. Two experimental conditions were involved. In one condition, participants performed reach-and-grasp movements to a target and simultaneously shifted their gaze towards it. In a control condition, participants solely shifted their gaze towards the target (oculomotor task). The participants freely decided when to initiate the tasks. After eye artefact correction, the EEG signals were time-locked to the saccade onset and the resulting amplitude features were exploited on a hierarchical classification approach to detect movement asynchronously. Main results. With regards to BCI performance, 54.1% (14.4% SD) of the movements were correctly identified, and all participants achieved a performance above chance-level (around 12%). An average of 21.5% (14.1% SD) of the oculomotor tasks were falsely detected as upper-limb movement. In an additional rest condition, 1.7 (1.6 SD) false positives per minute were measured. Through source imaging, movement information was mapped to sensorimotor, posterior parietal and occipital areas. Significance. We present a novel approach for movement detection using EEG signals which does not rely on upper-limb movement onset measurements or on the presentation of external cues. The participants’ behaviour closely matches the natural behaviour during goal-directed reach-and-grasp movements, which also constitutes an advantage with respect to current BCI protocols.

With neuroscientific investigations of analyzing neuronal activity of motor areas directly in the cortex in the 1980s, the foundation of invasive brain–computer interfaces (BCIs) or brain–machine interfaces (BMIs) was laid. Since then, researchers were studying motor cortex activity with multielectrode arrays, first in primates and later in humans, and applying decoders that allow multidimensional control of robotic limbs. Less invasive, but still with high signal-to-noise ratio, electrocorticography (ECoG) signals have been used for the investigation of motor cortex activity since the 2000s. After many basic investigations and decoding examples, this type of brain signal also seems to be promising for motor restoration.

Although motor imagery was used for the first BCI-controlled neuroprosthetic applications, it has turned out that it is a limited experimental strategy when it comes to control of more than two degrees of freedom. More natural ways for controlling the hand and ultimately the whole arm function have been identified in using attempted movement and even the attempt to move the whole arm. First evidence on the feasibility of these new BCI principles for neuroprosthesis control is discussed in this chapter.

There is an ongoing discussion in the NeuroIS (Neuro-Information-Systems) discipline on whether consumer-grade EEG instruments are as suitable for scientific research as research-grade instruments. Considering the increasing adoption of consumer-grade instruments along with the fact that many NeuroIS EEG papers used such tools, this debate is fundamental. We report on a study in which we contrasted a 61-channel EEG recording with a 14-channel recording that should simulate the electrode layout of the EPOC headset, the presumably worldwide most widely used consumer-grade tool. The contrast was carried out based on topographic mapping, because this kind of EEG data analysis does not only play a significant role in cognitive neuroscience, but also in NeuroIS research. Our findings show noticeable differences in the topoplots between both conditions. The current research results are limited by the fact that our task context is a non-IS context (i.e., upper limb movements). Hence, future research should validate our results based on IS tasks and situations in order to confirm, revise, or falsify the present results.

This chapter gives an overview on the foundation of the electroencephalogram (EEG), describes its phenomena, and explains how EEG recordings can be performed. This includes types of electrodes, amplifiers, and known artifacts. Further on, the term brain–computer interface (BCI) is introduced and its components and useful experimental paradigms get explained. With an outlook of application scenarios we build the basis for further chapters in this book.

Restoration of grasping has the highest priority for people with cervical spinal cord injury (SCI). This chapter describes the non-invasive brain–computer interface (BCI)-controlled grasp neuroprosthesis developed within the European Horizon 2020 project MoreGrasp. Based on former projects of the collaborators, several innovative technologies were developed within the MoreGrasp project with the aim to achieve an intuitive thought-controlled restoration of hand function in end users with tetraplegia for supporting activities of daily living. The end users in the focus of this project have been people with sufficiently preserved elbow and shoulder movements, but missing hand and finger functions.

The classification of different fine hand movements from electroencephalogram (EEG) signals represents a relevant research challenge, e.g., in BCI applications for motor rehabilitation. Here, we analyzed two different datasets where fine hand movements (touch, grasp, palmar, and lateral grasp) were performed in a self-paced modality. We trained and tested a newly proposed CNN, and we compared its classification performance with two well-established machine learning models, namely, shrinkage-linear discriminant analysis (LDA) and Random Forest (RF). Compared to previous literature, we included neuroscientific evidence, and we trained our Convolutional Neural Network (CNN) model on the so-called movement-related cortical potentials (MRCPs). They are EEG amplitude modulations at low frequencies, i.e., (0.3,3) Hz that have been proved to encode several properties of the movements, e.g., type of grasp, force level, and speed. We showed that CNN achieved good performance in both datasets (accuracy of 0.70±0.11 and 0.64±0.10, for the two datasets, respectively), and they were similar or superior to the baseline models (accuracy of 0.68±0.10 and 0.62±0.07 with sLDA, accuracy of 0.70±0.15 and 0.61±0.07 with RF, with comparable performance in precision and recall). In addition, compared to the baseline, our CNN requires a faster pre-processing procedure, paving the way for its possible use in online BCI applications.

Objective. One of the main goals in brain-computer interface (BCI) research is the replacement or restoration of lost function in individuals with paralysis. One line of research investigates the inference of movement kinematics from brain activity during different volitional states. A growing number of electroencephalography (EEG) and magnetoencephalography (MEG) studies suggest that information about directional (e.g., velocity) and nondirectional (e.g., speed) movement kinematics is accessible noninvasively. We sought to assess if the neural information associated with both types of kinematics can be combined to improve the decoding accuracy. Approach. In an offline analysis, we reanalyzed the data of two previous experiments containing the recordings of 34 healthy participants (15 EEG, 19 MEG). We decoded 2D movement trajectories from low-frequency M/EEG signals in executed and observed tracking movements, and compared the accuracy of an unscented Kalman filter (UKF) that explicitly modeled the nonlinear relation between directional and nondirectional kinematics to the accuracies of linear Kalman (KF) and Wiener filters (WF) which did not combine both types of kinematics. Main results. At the group level, posterior-parietal and parieto-occipital (executed and observed movements) and sensorimotor areas (executed movements) encoded kinematic information. Correlations between the recorded position and velocity trajectories and the UKF decoded ones were on average 0.49 during executed and 0.36 during observed movements. Compared to the other filters, the UKF could achieve the best trade-off between maximizing the signal to noise ratio and minimizing the amplitude mismatch between the recorded and decoded trajectories. Significance. We present direct evidence that directional and nondirectional kinematic information is simultaneously detectable in low-frequency M/EEG signals. Moreover, combining directional and nondirectional kinematic information significantly improves the decoding accuracy upon a linear KF.

OBJECTIVE An important part of restoring motor control via a brain-computer interface is to close the sensorimotor feedback loop. As part of our investigations into vibrotactile kinaesthetic feedback of arm movements, we studied electroencephalographic signals in the δ, µ and β bands obtained during a center-out movement task with four conditions: movement with real-time kinaesthetic feedback, movement with static vibrations, movement without vibrotactile input, and no movement with sham feedback. APPROACH Participants performed center-out movements with their palm on a flat table surface. One of three movement directions was cued visually before the movement. The palm position was tracked in order to provide real-time vibrotactile feedback. All analyses were performed offline. MAIN RESULTS Movement-related cortical potentials exhibit minor discrepancies between movement conditions as well as between movement directions, in peak amplitude and shape. Classification of each movement condition and each direction against rest yields peak accuracies of 60%-65% using low-frequency amplitude features, and 90% using µ and β power features. Within-class accuracies of four-way classification between conditions based on low-frequency amplitude features are around chance level for the movement conditions with vibrotactile stimulation, slightly above chance level for the movement condition without stimulation, and considerably higher for the non-movement condition. Four-way classification between conditions based on µ and β power features yields within-class accuracies slightly above chance level for all movement conditions, and considerably higher for the non-movement condition. Within-class accuracies of three-way classification between directions are slightly above chance level for low-frequency amplitude features, and at chance level for power features. SIGNIFICANCE We found that the vibrotactile stimulation does not interfere with movement-related features in the δ, µ and β frequency ranges. Our feedback system may therefore feasibly be deployed in conjunction with a BCI based on movement-related cortical potentials or sensorimotor rhythms, without adversely affecting control.

Decoding upper-limb movements in invasive recordings has become a reality, but neural tuning in non-invasive low-frequency recordings is still under discussion. Recent studies managed to decode movement positions and velocities using linear decoders, even developing an online system. The decoded signals, however, exhibited smaller amplitudes than actual movements, affecting feedback and user experience. Recently, we showed that a non-linear offline decoder can combine directional (e.g., velocity) and non-directional (e.g., speed) information. In this study, it is assessed if the non-linear decoder can be used online to provide real-time feedback. Five healthy subjects were asked to track a moving target by controlling a robotic arm. Initially, the robot was controlled by their right hand; then, the control was gradually switched until it was entirely controlled by the electroencephalogram (EEG). Correlations between actual and decoded movements were generally above chance level. Results suggest that information about speed was also encoded in the EEG, demonstrating that the proposed non-linear decoder is suitable for decoding real-time arm movements.

Brain-computer interfaces (BCIs) provide more independence to people with severe motor disabilities but current BCIs' performance is still not optimal and often the user's intentions are misinterpreted. Error-related potentials (ErrPs) are the neurophysiological signature of error processing and their detection can help improving a BCI's performance.A major inconvenience of BCIs is that they commonly require a long calibration period, before the user can receive feedback of their own brain signals. Here, we use the data of 15 participants and compare the performance of a personalized ErrP classifier with a generic ErrP classifier. We concluded that there was no significant difference in classification performance between the generic and the personalized classifiers (Wilcoxon signed rank tests, two-sided and one-sided left and right). This results indicate that the use of a generic ErrP classifier is a good strategy to remove the calibration period of a ErrP classifier, allowing participants to receive immediate feedback of the ErrP detections.

Objective. Loss of balance control can have serious consequences on interaction between humans and machines as well as the general well-being of humans. Perceived balance perturbations are always accompanied by a specific cortical activation, the so-called perturbation-evoked potential (PEP). In this study, we investigate the possibility to classify PEPs from ongoing EEG. Approach. Fifteen healthy subjects were exposed to seated whole-body perturbations. Each participant performed 120 trials; they were rapidly tilted to the right and left, 60 times respectively. Main results. We achieved classification accuracies of more than 85% between PEPs and rest EEG using a window-based classification approach. Different window lengths and electrode layouts were compared. We were able to achieve excellent classification performance (87.6 ± 8.0% accuracy) by using a short window length of 200 ms and a minimal electrode layout consisting of only the Cz electrode. The peak classification accuracy coincides in time with the strongest component of PEPs, called N1. Significance. We showed that PEPs can be discriminated against ongoing EEG with high accuracy. These findings can contribute to the development of a system that can detect balance perturbations online.

Virtual conference; International audience; When persons interact with the environment and experience or witness an error (e.g. an unexpected event), a specific brain pattern, known as error-related potential (ErrP) can be observed in the electroencephalographic signals (EEG). Virtual Reality (VR) technology enables users to interact with computer-generated simulated environments and to provide multi-modal sensory feedback. Using VR systems can, however, be error-prone. In this paper, we investigate the presence of ErrPs when Virtual Reality users face 3 types of visualization errors: (Te) tracking errors when manipulating virtual objects, (Fe) feedback errors, and (Be) background anomalies. We conducted an experiment in which 15 participants were exposed to the 3 types of errors while performing a center-out pick and place task in virtual reality. The results showed that tracking errors generate error-related potentials, the other types of errors did not generate such discernible patterns. In addition, we show that it is possible to detect the ErrPs generated by tracking losses in single trial, with an accuracy of 85%. This constitutes a first step towards the automatic detection of error-related potentials in VR applications, paving the way to the design of adaptive and self-corrective VR/AR applications by exploiting information directly from the user’s brain.

The aim of the MoreGrasp project is to develop a non-invasive, multimodal user interface including a brain-computer interface (BCI) for control of a grasp neuroprostheses in individuals with high spinal cord injury (SCI). The first results of the ongoing MoreGrasp clinical feasibility study involving end users with SCI are presented. This includes BCI screening sessions, in which we investigate the electroencephalography (EEG) patterns associated with single, natural movements of the upper limb. These patterns will later be used to control the neuroprosthesis. Additionally, the MoreGrasp grasp neuroprosthesis consisting of electrode arrays embedded in an individualized textile forearm sleeve is presented. The general feasibility of this electrode array in terms of corrections of misalignments during donning is shown together with the functional results in end users of the electrode forearm sleeve.

A high fraction of artifact-free signals is highly desirable in functional neuroimaging and brain-computer interfacing (BCI). We present the high-variance electrode artifact removal (HEAR) algorithm to remove transient electrode pop and drift (PD) artifacts from electroencephalographic (EEG) signals. Transient PD artifacts reflect impedance variations at the electrode scalp interface that are caused by ion concentration changes. HEAR and its online version (oHEAR) are open-source and publicly available. Both outperformed state of the art offline and online transient, high-variance artifact correction algorithms for simulated EEG signals. (o)HEAR attenuated PD artifacts by approx. 25 dB, and at the same time maintained a high SNR during PD artifact-free periods. For real-world EEG data, (o)HEAR reduced the fraction of outlier trials by half and maintained the waveform of a movement related cortical potential during a center-out reaching task. In the case of BCI training, using oHEAR can improve the reliability of the feedback a user receives through reducing a potential negative impact of PD artifacts.

An affective brain-computer interface (aBCI) is a direct communication pathway between human brain and computer, via which the computer tries to recognize the affective states of its user and respond accordingly. As aBCI introduces personal affective factors into human-computer interaction, it could potentially enrich the user’s experience during the interaction. Successful emotion recognition plays a key role in such a system. The state-of-the-art aBCIs leverage machine learning techniques which consist in acquiring affective electroencephalogram (EEG) signals from the user and calibrating the classifier to the affective patterns of the user. Many studies have reported satisfactory recognition accuracy using this paradigm. However, affective neural patterns are volatile over time even for the same subject. The recognition accuracy cannot be maintained if the usage of aBCI prolongs without recalibration. Existing studies have overlooked the performance evaluation of aBCI during long-term use. In this paper, we propose SAFE—an EEG dataset for stable affective feature selection. The dataset includes multiple recording sessions spanning across several days for each subject. Multiple sessions across different days were recorded so that the long-term recognition performance of aBCI can be evaluated. Based on this dataset, we demonstrate that the recognition accuracy of aBCIs deteriorates when re-calibration is ruled out during long-term usage. Then, we propose a stable feature selection method to choose the most stable affective features, for mitigating the accuracy deterioration to a lesser extent and maximizing the aBCI performance in the long run. We invite other researchers to test the performance of their aBCI algorithms on this dataset, and especially to evaluate the long-term performance of their methods.

Passive brain-computer interfaces (pBCIs) can be used to inform humans about their current mental state or in human-machine interaction (HMI) scenario. We introduced the perturbation-evoked potential (PEP) in the context of pBCI and are further investigating how neural correlates in a HMI could interact. In the current study we investigate the neural correlates after perturbation followed by motor reaction. We found that the PEP as well as a movement-related cortical potential appear, and that the latter has an influence on the shape of the PEP.

Objective Continuous decoding of voluntary movement is desirable for closed-loop, natural control of neuroprostheses. Recent studies showed the possibility to reconstruct the hand trajectories from low-frequency (LF) electroencephalographic (EEG) signals. So far this has only been performed offline. Here, we attempt for the first time continuous online control of a robotic arm with LF-EEG-based decoded movements. Approach The study involved ten healthy participants, asked to track a moving target by controlling a robotic arm. At the beginning of the experiment, the robot was fully controlled by the participant's hand trajectories. After calibrating the decoding model, that control was gradually replaced by LF-EEG-based decoded trajectories, first with 33%, 66% and finally 100% EEG control. Likewise with other offline studies, we regressed the movement parameters (two-dimensional positions, velocities, and accelerations) from the EEG with partial least squares (PLS) regression. To integrate the information from the different movement parameters, we introduced a combined PLS and Kalman filtering approach (named PLSKF). Main results We obtained moderate yet overall significant (α = 0.05) online correlations between hand kinematics and PLSKF-decoded trajectories of 0.32 on average. With respect to PLS regression alone, the PLSKF had a stable correlation increase of Δr = 0.049 on average, demonstrating the successful integration of different models. Parieto-occipital activations were highlighted for the velocity and acceleration decoder patterns. The level of robot control was above chance in all conditions. Participants finally reported to feel enough control to be able to improve with training, even in the 100% EEG condition. Significance Continuous LF-EEG-based movement decoding for the online control of a robotic arm was achieved for the first time. The potential bottlenecks arising when switching from offline to online decoding, and possible solutions, were described. The effect of the PLSKF and its extensibility to different experimental designs were discussed.

The activation of different brain areas during kinaesthetic and visual motor imagery has been extensively studied, whereas little is known about affective motor imagery, i.e. the imagery of pleasant/unpleasant movements. In the present neuroimaging study we investigated cortical activation of kinaesthetic motor imagery (KMI) based on emotional stimulus content by means of functional near infrared spectroscopy (fNIRS). Twenty healthy adult participants were instructed to imagine affective, and neutral motor tasks while multichannel fNIRS was recorded simultaneously. In the affective MI condition they had to imagine e.g. squeezing a cactus with their right hand several times, eliciting an unpleasant emotion. In the neutral condition their task was to imagine squeezing a ball. Significant differences in oxy-hemoglobin [oxy-Hb] concentration changes during KMI including affective objects in different brain regions were found. Specifically activation in left parietal and frontal regions was increased during the imagery of squeezing a cactus which induced a painful feeling. Both areas are also involved in the perception of pain and commonly labelled as parts of the "pain matrix". Our study provides novel insights in cortical activation patterns during affective motor imagery and its psychological and cognitive mechanisms underlying pain experience.

Background Recently, fNIRS has been proposed as a promising approach for awareness detection, and a possible method to establish basic communication in patients with disorders of consciousness (DOC). Aim Using fNIRS, the present study evaluated the applicability of auditory presented mental-arithmetic tasks in this respect. Methods We investigated the applicability of active attention to serial subtractions for awareness detection in ten healthy controls (HC, 21–32 y/o), by comparing the measured patterns to patterns induced by self-performance of the same task. Furthermore, we examined the suitability of ignoring the given task as additional control signal to implement a two-class brain-computer interface (BCI) paradigm. Finally, we compared our findings in HC with recordings in one DOC patient (78 y/o). Results and conclusion Results of the HC revealed no differences between the self-performance and the attention condition, making the attention task suitable for awareness detection. However, there was no general difference between the ignore and attend condition, making the tasks less suitable for BCI control. Despite inconsistent correlations between the patient data and the HC group, single runs of the patient recordings revealed task-synchronous patterns – however, we cannot conclude whether the measured activation derives from instruction based task performance and thus awareness.

Grasping movements are known to activate the fronto-parietal brain networks both in human and non-human primates. However, it is unclear if these activations represent properties of the objects or hand postures or both at different stages of the movement. We manipulated the intrinsic properties of the objects and the grasping types in order to create twelve unique combinations of grasping movements and we investigated, in healthy adult humans, the low-frequency time-domain EEG representation of grasping over different stages of the movement. Next, we implemented two multiclass decoders for the grasp type and objects' properties and evaluated them over time. Furthermore, we investigated the similarity between these grasping EEG representations and categorical models that encode properties of the movement and intrinsic properties of the objects. We found that properties of the grasping movement (grasp types, number of fingers) and intrinsic object properties (shape and size) as represented in EEG are encoded in different brain areas throughout the movement stages. Both object properties and grasp types can be decoded significantly above chance level using low-frequency EEG activity during the planning and execution of the movement. Moreover, we found that this preferential time-wise encoding allows the decoding of object properties already from the observation stage, while the grasp type can also be accurately decoded at the object release stage. These findings contribute to the understanding of the grasping representation based on noninvasive EEG brain signals, and its evolution over the course of movement in relation to categorical models that describe the grasped object's properties or that encode properties of the grasping movement. Moreover, our multiclass grasping decoders are informative for the design and implementation of noninvasive motor control strategies.

Patients who suffer from a high spinal cord injury have severe motor disabilities in the lower as well as in the upper extremities. Thus they rely on the help of other people in everyday life. Restoring the function of the upper limbs, especially the grasp function can help them to gain some independence. Using EEG-based neuroprosthetics is a way to help tetraplegic people restore different grasp types as well as moving the arm and the elbow. In this work an overview of non-invasive EEG-based methods for restoring the hand and arm function with the use of neuroprosthetics in individuals with high spinal cord injury is given. Since the Graz BCI group is leading in this area of non-invasive research mainly, the work of this group is represented.

Brain-computer interfaces (BCIs) can be used to improve human-machine interactions (HMIs) by providing implicit information about the mental state. We introduce a brain activity, perturbation-evoked potentials (PEPs), that was not yet investigated in the context of BCIs although it has the required properties. An experimental setup for studying PEPs is proposed and validated and two possible use cases for this brain activity are introduced.

While most tasks of daily life can be handled through a small number of different grasps, many tasks require the action of both hands. In these bimanual tasks, the second hand has either a supporting role (e.g. for fixating a jar) or a more active role (e.g. grasping a pot on both handles). In this study we attempt to discriminate the neural correlates of unimanual (performed with left and right hand) from bimanual reach-and-grasp actions using the low-frequency time-domain electroencephalogram (EEG). In a self-initiated movement task, 15 healthy participants were asked to perform unimanual (palmar and lateral grasps with left and right hand) and bimanual (double lateral, mixed palmar/lateral) reach-and-grasps on objects of daily life. Using EEG time-domain features in the frequency range of 0.3-3 Hz, we achieved multiclass-classification accuracies of 38.6 $\pm$ 6.6% (7 classes, 17.1% chance level) for a combination of 6 movements and 1 rest condition. The grand average confusion matrix shows highest true positive rates (TPR) for the rest (63%) condition while TPR for the movement classes varied between 33 to 41%. The underlying movement-related cortical potentials (MRCPs) show significant differences between unimanual (e.g left hand vs. right hand grasps) as well unimanual vs. bimanual conditions which both can be attributed to lateralization effects. We believe that these findings can be exploited and further used for attempts in providing persons with spinal cord injury a form of natural control for bimanual neuroprostheses.

Brain-computer interfaces (BCIs) might provide an intuitive way for severely motor impaired persons to operate assistive devices to perform daily life activities. Recent studies have shown that complex hand movements, such as reach-and-grasp tasks, can be decoded from the low frequency of the electroencephalogram (EEG). In this work we investigated whether additional features extracted from the frequency-domain of alpha and beta bands could improve classification performance of rest vs. palmar vs. lateral grasp. We analysed two multi-class classification approaches, the first using features from the low frequency time-domain, and the second in which we combined the time-domain with frequency-domain features from alpha and beta bands. We measured EEG of ten participants without motor disability which performed self-paced reach-and-grasp actions on objects of daily life. For the time-domain classification approach, participants reached an average peak accuracy of 65%. For the combined approach, an average peak accuracy of 75% was reached. In both approaches and for all subjects, performance was significantly higher than chance level (38.1%, 3-class scenario). By computing the confusion matrices as well as feature rankings through the Fisher score, we show that movement vs. rest classification performance increased considerably in the combined approach and was the main responsible for the multi-class higher performance. These findings could help the development of BCIs in real-life scenarios, where decreasing false movement detections could drastically increase the end-user acceptance and usability of BCIs.

We show that persons with spinal cord injury (SCI) retain decodable neural correlates of attempted arm and hand movements. We investigated hand open, palmar grasp, lateral grasp, pronation, and supination in 10 persons with cervical SCI. Discriminative movement information was provided by the time-domain of low-frequency electroencephalography (EEG) signals. Based on these signals, we obtained a maximum average classification accuracy of 45% (chance level was 20%) with respect to the five investigated classes. Pattern analysis indicates central motor areas as the origin of the discriminative signals. Furthermore, we introduce a proof-of-concept to classify movement attempts online in a closed loop, and tested it on a person with cervical SCI. We achieved here a modest classification performance of 68.4% with respect to palmar grasp vs hand open (chance level 50%).

Affective brain-computer interface (aBCI) introduces personal affective factors to human-computer interaction. The state-of-the-art aBCI tailors its classifier to each individual user to achieve accurate emotion classification. A subject-independent classifier that is trained on pooled data from multiple subjects generally leads to inferior accuracy, due to the fact that electroencephalography patterns vary from subject to subject. Transfer learning or domain adaptation techniques have been leveraged to tackle this problem. Existing studies have reported successful applications of domain adaptation techniques on SEED dataset. However, little is known about the effectiveness of the domain adaptation techniques on other affective datasets or in a cross-dataset application. In this paper, we focus on a comparative study on several state-of-the-art domain adaptation techniques on two datasets: 1) DEAP and 2) SEED. We demonstrate that domain adaptation techniques can improve the classification accuracy on both datasets, but not so effective on DEAP as on SEED. Then, we explore the efficacy of domain adaptation in a cross-dataset setting when the data are collected under different environments using different devices and experimental protocols. Here, we propose to apply domain adaptation to reduce the intersubject variance as well as technical discrepancies between datasets, and then train a subject-independent classifier on one dataset and test on the other. Experiment results show that using domain adaptation technique in a transductive adaptation setting can improve the accuracy significantly by 7.25%-13.40% compared to the baseline accuracy where no domain adaptation technique is used. National Research Foundation (NRF) Accepted version This work was supported by the National Research Foundation, Prime Minister’s Office, Singapore, through Its International Research Centers in Singapore Funding Initiative.

Objective The simultaneous application of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) opens up new ways to investigate the human brain. The EEG recordings of simultaneous EEG-fMRI, however, are overlaid to a great degree by fMRI related artifacts and an artifact reduction is mandatory before any EEG analysis. The most severe artifacts-the gradient artifact and the pulse artifact-are repetitive. Average artifact subtraction (AAS) technique exploits the repetitiveness and is presumably the most often used artifact reduction technique. In this method artifact templates are calculated by averaging over adjacent artifact epochs and subsequently the templates are subtracted to reduce the artifacts. Although the AAS technique is one of the best performing methods, artifact residuals are usually present in the resulting EEG after applying the AAS technique. This work aims at identifying sources of the artifact residuals. Approach Application of the AAS technique to artificial EEG that is contaminated with artificial fMRI related artifacts. Main results A new source of artifact residuals was identified. It was found that the AAS technique itself adds artifacts to the EEG during gradient artifact reduction, because the gradient artifact template is corrupted by pulse artifact remainders. Significance This work shows that using a standard number of 25 epochs to calculate the gradient artifact template-as suggested by the inventors of AAS-results in substantial artifact residuals and consequently to a low EEG quality. Furthermore, the work discusses how potential solutions to this problem have serious side effects such as loss of adaptivity of the AAS technique. Hence, this problem must be considered carefully already in the design of simultaneous EEG-fMRI experiments.

Movement covariates, such as electromyographic or kinematic activity, have been proposed as candidates for the neural representation of hand control. However, it remains unclear how these movement covariates are reflected in electroencephalographic (EEG) activity during different stages of grasping movements. In this exploratory study, we simultaneously acquired EEG, kinematic and electromyographic recordings of human subjects performing 33 types of grasps, yielding the largest such dataset to date. We observed that EEG activity reflected different movement covariates in different stages of grasping. During the pre-shaping stage, centro-parietal EEG in the lower beta frequency band reflected the object’s shape and size, whereas during the finalization and holding stages, contralateral parietal EEG in the mu frequency band reflected muscle activity. These findings contribute to the understanding of the temporal organization of neural grasping patterns, and could inform the design of noninvasive neuroprosthetics and brain-computer interfaces with more natural control.

Personen mit hoher Querschnittlahmung mussen neben dem Verlust der Steh- und Gehfunktion und einer Funktionsminderung des Blasen-Darm-Trakts auch massive Einschrankungen in den oberen Extremitaten hinnehmen. Mit dem Verlust der Greiffunktion und einhergehender Reduktion der Armfunktion droht die lebenslange Abhangigkeit von Dritten. Brain-Computer Interfaces (BCI) konnten bei der Steuerung von Neuroprothesen hilfreich sein.

Objective: This work aimed to find and evaluate a new method for detecting errors in continuous brain–computer interface (BCI) applications. Instead of classifying errors on a single-trial basis, the new method was based on multiple events (MEs) analysis to increase the accuracy of error detection. Methods: In a BCI-driven car game, based on motor imagery (MI), discrete events were triggered whenever subjects collided with coins and/or barriers. Coins counted as correct events, whereas barriers were errors. This new method, termed ME method, combined and averaged the classification results of single events (SEs) and determined the correctness of MI trials, which consisted of event sequences instead of SEs. The benefit of this method was evaluated in an offline simulation. In an online experiment, the new method was used to detect erroneous MI trials. Such MI trials were discarded and could be repeated by the users. Results: We found that, even with low SE error potential (ErrP) detection rates, feasible accuracies can be achieved when combining MEs to distinguish erroneous from correct MI trials. Online, all subjects reached higher scores with error detection than without, at the cost of longer times needed for completing the game. Conclusion: Findings suggest that ErrP detection may become a reliable tool for monitoring continuous states in BCI applications when combining MEs. Significance: This paper demonstrates a novel technique for detecting errors in online continuous BCI applications, which yields promising results even with low single-trial detection rates.

Everyday locomotion and obstacle avoidance requires effective gait adaptation in response to sensory cues. Many studies have shown that efficient motor actions are associated with μ rhythm (8–13 Hz) and β band (13–35 Hz) local field desynchronizations in sensorimotor and parietal cortex, whereas a number of cognitive task studies have reported higher behavioral accuracy to be associated with increases in β band power in prefrontal and sensory cortex. How these two distinct patterns of β band oscillations interplay during gait adaptation, however, has not been established. Here we recorded 108 channel EEG activity from 18 participants (10 males, 22–35 years old) attempting to walk on a treadmill in synchrony with a series of pacing cue tones, and quickly adapting their step rate and length to sudden shifts in pacing cue tempo. Independent component analysis parsed each participant's EEG data into maximally independent component (IC) source processes, which were then grouped across participants into distinct spatial/spectral clusters. Following cue tempo shifts, mean β band power was suppressed for IC sources in central midline and parietal regions, whereas mean β band power increased in IC sources in or near medial prefrontal and dorsolateral prefrontal cortex. In the right dorsolateral prefrontal cortex IC cluster, the β band power increase was stronger during (more effortful) step shortening than during step lengthening. These results thus show that two distinct patterns of β band activity modulation accompany gait adaptations: one likely serving movement initiation and execution; and the other, motor control and inhibition.SIGNIFICANCE STATEMENTUnderstanding brain dynamics supporting gait adaptation is crucial for understanding motor deficits in walking, such as those associated with aging, stroke, and Parkinson's. Only a few electromagnetic brain imaging studies have examined neural correlates of human upright walking. Here, application of independent component analysis to EEG data recorded during treadmill walking allowed us to uncover two distinct β band oscillatory cortical networks that are active during gait adaptation to shifts in the tempo of an auditory pacing cue: (8–13 Hz) μ rhythm and (13–35 Hz) β band power decreases in central and parietal cortex and (14–20 Hz) β band power increases in frontal brain areas. These results provide a fuller framework for electrophysiological studies of cortical gait control and its disorders.

Background While visualization of brain activity has well established practical applications such as real-time functional mapping or neurofeedback, visual representation of brain connectivity is not widely used. In addition, technically challenging single-trial connectivity estimation may have hindered practical usage of connectivity in online applications. New method In this work, we developed algorithms that are capable of estimating and visualizing (effective) connectivity between independent cortical sources during online EEG recordings. Results The core routines of our procedure, such as CSPVARICA source extraction and regularized connectivity estimation, are available in our open source Python-based toolbox SCoT. We demonstrate for the first time that online connectivity visualization is feasible. We show this in a feasibility study with twelve participants performing two different tasks, namely motor execution and resting with eyes open or closed. Connectivity patterns were significantly different between two motor tasks in four participants, whereas significant differences between resting task patterns were found in seven participants. Comparison with existing methods Existing connectivity studies have focused on offline methods. In contrast, there are only a small number of examples in the literature that explored online connectivity estimation. For example, a system based on wearable EEG has been demonstrated to work for one subject, and the Glass Brain project has received considerable attention in popular sciences last year. However, none of these attempts validate their methods on multiple subjects. Conclusions Our results show that causal connectivity patterns can be observed online during EEG measurements, which is a first step towards real-time connectivity analysis.

Movement preparation and initiation have been shown to involve large scale brain networks. Recent findings suggest that movement preparation and initiation are represented in functionally distinct cortical networks. In electroencephalographic (EEG) recordings, movement initiation is reflected as a strong negative potential at medial central channels that is phase-locked to the movement onset - the movement-related cortical potential (MRCP). Movement preparation describes the process of transforming high level movement goals to low level commands. An integral part of this transformation process is directional processing (i.e., where to move). The processing of movement direction during visuomotor and oculomotor tasks is associated with medial parieto-occipital cortex (PO) activity, phase-locked to the presentation of potential movement goals. We surmised that the network generating the MRCP (movement initiation) would encode less information about movement direction than the parieto-occipital network processing movement direction. Here, we studied delta band EEG activity during center-out reaching movements (2D; 4 directions) in visuomotor and oculomotor tasks. In 15 healthy participants, we found a consistent representation of movement direction in PO 300–400 ​ms after the direction cue irrespective of the task. Despite generating the MRCP, sensorimotor areas (SM) encoded less information about the movement direction than PO. Moreover, the encoded directional information in SM was less consistent across participants and specific to the visuomotor task. In a classification approach, we could infer the four movement directions from the delta band EEG activity with moderate accuracies up to 55.9%. The accuracies for cue-aligned data were significantly higher than for movement onset-aligned data in either task, which also suggests a stronger representation of movement direction during movement preparation. Therefore, we present direct evidence that EEG delta band amplitude modulations carry information about both arm movement initiation and movement direction, and that they are represented in two distinct cortical networks., Highlights • Delta band EEG carries information about arm movement initiation and direction. • Movement initiation and direction are represented in two distinct cortical networks. • Information about movement direction is primarily encoded in parieto-occipital areas. • The activity in parieto-occipital areas is phase-locked to the direction cue.

Eye movements and blinks contaminate electroencephalographic (EEG) and magnetoencephalographic (MEG) activity. As the eye moves, the corneo-retinal dipole (CRD) and eyelid introduce potential/field changes in the M/EEG activity. These eye artifacts can affect a brain-computer interface and thereby impinge on neurofeedback quality. Here, we introduce the sparse generalized eye artifact subspace subtraction (SGEYESUB) algorithm that can correct these eye artifacts offline and in real time. We provide an open source reference implementation of the algorithm and the paradigm to obtain calibration data. Once the algorithm is fitted to calibration data (approx. 5 ​min), the eye artifact correction reduces to a matrix multiplication. We compared SGEYESUB with 4 state-of-the-art algorithms using M/EEG activity of 69 participants. SGEYESUB achieved the best trade-off between correcting the eye artifacts and preserving brain activity. Residual correlations between the corrected M/EEG channels and the eye artifacts were below 0.1. Error-related and movement-related cortical potentials were attenuated by less than 0.5 ​μV. Our results furthermore demonstrate that CRD and eyelid-related artifacts can be assumed to be stationary for at least 1–1.5 ​h, validating the feasibility of our approach in offline and online eye artifact correction.

Objective. Since the discovery of the population vector that directly relates neural spiking activity with arm movement direction, it has become feasible to control robotic arms and neuroprostheses using invasively recorded brain signals. For non-invasive approaches, a direct relation between human brain signals and arm movement direction is yet to be established. Approach. Here, we investigated electroencephalographic (EEG) signals in temporal and spectral domains in a continuous, circular arm movement task. Using machine learning methods that respect the linear mixture of brain activity within EEG signals, we show that directional information is represented in the temporal domain in amplitude modulations of the same frequency as the arm movement, and in the spectral domain in power modulations of the 20–24 Hz frequency band. Main results. In the temporal domain, the directional information was mainly expressed in primary sensorimotor cortex (SM1) and posterior parietal cortex (PPC) contralateral to the moving arm, while in the spectral domain SM1 and PPC of both hemispheres predicted arm movement direction. The different cortical representations suggest distinct neural representations in both domains. Significance. This direct relation between neural activity and arm movement direction in both domains demonstrates the potential of machine learning to reveal neuroscientific insights about the dynamics of human arm movements.