Search

EEG is an established method for quantifying large-scale neuronal dynamics which enables diverse real-world biomedical applications including brain-computer interfaces, epilepsy monitoring and sleep staging. Advances in sensor technology have freed EEG from traditional laboratory settings, making low-cost ambulatory or at-home assessments of brain function possible. While ecologically valid brain assessments are becoming more practical, the impact of their reduced spatial resolution and susceptibility to noise remain to be investigated. This study set out to explore the potential of at-home EEG assessments for biomarker discovery using the brain age framework and four-channel consumer EEG data. We analyzed recordings from more than 5200 human subjects (18-81 years) during meditation and sleep, focusing on the age prediction task. With cross-validatedR2scores between 0.3 - 0.5, prediction performance was within the range of results obtained by recent benchmarks focused on laboratory-grade EEG. While age prediction was successful from both meditation and sleep recordings, the latter led to higher performance. Analysis by sleep stage uncovered that N2-N3 stages contained most of the signal. When combined, EEG features extracted from all sleep stages gave the best performance, suggesting that the entire night of sleep contains valuable age-related information. Furthermore, model comparisons suggested that information was spread out across electrodes and frequencies, supporting the use of multivariate modeling approaches. Thanks to our unique dataset of longitudinal repeat sessions spanning 153 to 529 days from eight subjects, we finally evaluated the variability of EEG-based age predictions, showing that they reflect both trait- and state-like information. Overall, our results demonstrate that state-of-the-art machine learning approaches based on age prediction can be readily applied to real-world EEG recordings obtained during at-home sleep and meditation practice.

Objective. Supervised learning paradigms are often limited by the amount of labeled data that is available. This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG), where labeling can be costly in terms of specialized expertise and human processing time. Consequently, deep learning architectures designed to learn on EEG data have yielded relatively shallow models and performances at best similar to those of traditional feature-based approaches. However, in most situations, unlabeled data is available in abundance. By extracting information from this unlabeled data, it might be possible to reach competitive performance with deep neural networks despite limited access to labels. Approach. We investigated self-supervised learning (SSL), a promising technique for discovering structure in unlabeled data, to learn representations of EEG signals. Specifically, we explored two tasks based on temporal context prediction as well as contrastive predictive coding on two clinically-relevant problems: EEG-based sleep staging and pathology detection. We conducted experiments on two large public datasets with thousands of recordings and performed baseline comparisons with purely supervised and hand-engineered approaches. Main results. Linear classifiers trained on SSL-learned features consistently outperformed purely supervised deep neural networks in low-labeled data regimes while reaching competitive performance when all labels were available. Additionally, the embeddings learned with each method revealed clear latent structures related to physiological and clinical phenomena, such as age effects. Significance. We demonstrate the benefit of self-supervised learning approaches on EEG data. Our results suggest that SSL may pave the way to a wider use of deep learning models on EEG data., 32 pages, 9 figures

Supervised learning paradigms are often limited by the amount of labeled data that is available. This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG), where labeling can be costly in terms of specialized expertise and human processing time. Consequently, deep learning architectures designed to learn on EEG data have yielded relatively shallow models and performances at best similar to those of traditional feature-based approaches. However, in most situations, unlabeled data is available in abundance. By extracting information from this unlabeled data, it might be possible to reach competitive performance with deep neural networks despite limited access to labels. Here we report results using self-supervised learning (SSL), a promising technique for discovering structure in unlabeled data, to learn representations of EEG signals. Specifically, we consider a contrastive approach and provide results on two clinically-relevant problems: EEG-based sleep staging and pathology detection. Results report that linear classifiers trained on SSL-learned features consistently outperformed purely supervised deep neural networks in low-labeled data regimes while reaching competitive performance when all labels were available. Our results suggest that self-supervision may pave the way to a wider use of deep learning models on EEG data.

Building machine learning models using EEG recorded outside of the laboratory setting requires methods robust to noisy data and randomly missing channels. This need is particularly great when working with sparse EEG montages (1-6 channels), often encountered in consumer-grade or mobile EEG devices. Neither classical machine learning models nor deep neural networks trained end-to-end on EEG are typically designed or tested for robustness to corruption, and especially to randomly missing channels. While some studies have proposed strategies for using data with missing channels, these approaches are not practical when sparse montages are used and computing power is limited (e.g., wearables, cell phones). To tackle this problem, we propose dynamic spatial filtering (DSF), a multi-head attention module that can be plugged in before the first layer of a neural network to handle missing EEG channels by learning to focus on good channels and to ignore bad ones. We tested DSF on public EEG data encompassing ~4,000 recordings with simulated channel corruption and on a private dataset of ~100 at-home recordings of mobile EEG with natural corruption. Our proposed approach achieves the same performance as baseline models when no noise is applied, but outperforms baselines by as much as 29.4% accuracy when significant channel corruption is present. Moreover, DSF outputs are interpretable, making it possible to monitor channel importance in real-time. This approach has the potential to enable the analysis of EEG in challenging settings where channel corruption hampers the reading of brain signals., Comment: 42 pages, 9 figures

With the growing complexity of various text-to-speech systems, it is becoming more important to understand the underlying perceptual and judgement processes that drive user Quality-of-Experience (QoE) perception. Typical QoE assessment techniques, such as listening tests with self-report ratings, are useful but provide limited insight into these underlying processes. Recent advances in neuroimaging and physiological monitoring technologies, however, have opened new doors and allowed us to better understand and measure QoE perception. In this paper, we explore the use of two neuroimaging techniques, namely electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), to better understand neuronal and cerebral haemodynamic changes resultant from synthesized speech of varying quality. Neural correlates of several QoE dimensions were derived and validated on the publicly available PhySyQX database. Fusion of EEG, fNIRS, and fNIRS-derived physiological parameters, combined with conventional features extracted from the synthesized speech signal showed to accurately represent several QoE dimensions, including those related to listener affective states. It is hoped that these findings will help researchers build better instrumental QoE models that incorporate technological, contextual, and human influence factors.

International audience; The supervised learning paradigm is limited by the cost - and sometimes the impracticality - of data collection and labeling in multiple domains. Self-supervised learning, a paradigm which exploits the structure of unlabeled data to create learning problems that can be solved with standard supervised approaches, has shown great promise as a pretraining or feature learning approach in fields like computer vision and time series processing. In this work, we present self-supervision strategies that can be used to learn informative representations from multivariate time series. One successful approach relies on predicting whether time windows are sampled from the same temporal context or not. As demonstrated on a clinically relevant task (sleep scoring) and with two electroencephalography datasets, our approach outperforms a purely supervised approach in low data regimes, while capturing important physiological information without any access to labels.; Le paradigme de l’apprentissage supervisé est limité par le coût - et parfois l’impraticabilité - de la collecte de données et de l’étiquetage dans de multiples domaines. L'apprentissage auto-supervisé, un paradigme qui exploite la structure de données non étiquetées pour créer des problèmes d'apprentissage qui peuvent être résolus avec des approches supervisées standard, s'est révélé très prometteur en tant qu'approche de pré-entraînement ou d'apprentissage de traits caractéristiques dans des domaines tels que la vision par ordinateur et le traitement de séries temporelles. Dans ce travail, nous présentons des stratégies d'auto-supervision pouvant être utilisées pour apprendre des représentations informatives à partir de séries temporelles multivariées. Une approche fructueuse consiste à prédire si des fenêtres temporelles sont échantillonnées dans le même contexte temporel ou non. Comme le démontre une tâche cliniquement pertinente (classification des stades du sommeil) et avec deux jeux de données d'électroencéphalographie, notre approche surpasse une approche purement supervisée dans des régimes de données faibles, tout en capturant des informations physiologiques importantes sans accès aux étiquettes.

Context. Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question. Objective. In this work, we review 154 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, brain–computer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations. Methods. Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to (1) the data, (2) the preprocessing methodology, (3) the DL design choices, (4) the results, and (5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends. Results. Our analysis reveals that the amount of EEG data used across studies varies from less than ten minutes to thousands of hours, while the number of samples seen during training by a network varies from a few dozens to several millions, depending on how epochs are extracted. Interestingly, we saw that more than half the studies used publicly available data and that there has also been a clear shift from intra-subject to inter-subject approaches over the last few years. About of the studies used convolutional neural networks (CNNs), while used recurrent neural networks (RNNs), most often with a total of 3–10 layers. Moreover, almost one-half of the studies trained their models on raw or preprocessed EEG time series. Finally, the median gain in accuracy of DL approaches over traditional baselines was across all relevant studies. More importantly, however, we noticed studies often suffer from poor reproducibility: a majority of papers would be hard or impossible to reproduce given the unavailability of their data and code. Significance. To help the community progress and share work more effectively, we provide a list of recommendations for future studies and emphasize the need for more reproducible research. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly. A planned follow-up to this work will be an online public benchmarking portal listing reproducible results.

Context. In recent years, hybrid brain-computer interfaces (hBCIs) have proven to be a promising path towards practical brain-computer interfacing. These hybrid interfaces capitalize on the concurrent recording of various physiological signals, or of the elicitation of more than one mental process, to increase the number of possible input commands and achieve more flexible and robust systems. Although hBCIs have previously been reviewed in some articles, a more recent, and complete survey of the literature is missing to lay the foundations for further research. Objective. This work aims at systematically reviewing recent articles on the topic of non-invasive hBCIs, to comprehensively identify the current trends, limitations and challenges that these studies report. Methods. Three major databases covering the fields of science and engineering were queried. From these and others sources, 55 journal articles from 2008 to November 2014 were selected and analyzed. Results. Twenty-two items were investigated to o...

A key goal of neuroergonomics is to develop tools to measure mental workload during task execution, potentially enabling safer and more efficient human–machine interaction in diverse working environments. A promising approach to developing these high-performance and robust cognitive monitoring systems is to combine neurophysiological modalities, such as electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), that can offer complementary information on brain activity. Here, innovative temporal dynamics-related EEG and fNIRS features are described that will enable development of highly effective multimodal mental-workload measures.

Based on recent electroencephalography (EEG) and near-infrared spectroscopy (NIRS) studies that showed that tasks such as motor imagery and mental arithmetic induce specific neural response patterns, we propose a hybrid brain-computer interface (hBCI) paradigm in which EEG and NIRS data are fused to improve binary classification performance. We recorded simultaneous NIRS-EEG data from nine participants performing seven mental tasks (word generation, mental rotation, subtraction, singing and navigation, and motor and face imagery). Classifiers were trained for each possible pair of tasks using (1) EEG features alone, (2) NIRS features alone, and (3) EEG and NIRS features combined, to identify the best task pairs and assess the usefulness of a multimodal approach. The NIRS-EEG approach led to an average increase in peak kappa of 0.03 when using features extracted from one-second windows (equivalent to an increase of 1.5% in classification accuracy for balanced classes). The increase was much stronger (0.20, corresponding to an 10% accuracy increase) when focusing on time windows of high NIRS performance. The EEG and NIRS analyses further unveiled relevant brain regions and important feature types. This work provides a basis for future NIRS-EEG hBCI studies aiming to improve classification performance toward more efficient and flexible BCIs.

A product's success in the market can be predicted based on the Quality-of-Experience (QoE) it offers to its users. With the burgeoning market for text-to-speech (TTS) systems, it has become extremely important to characterise new TTS systems in terms of their QoE. To this end, many objective models for quality estimation have been developed. These state-of-the art models are developed considering the system and contextual factors which influence the users' experience. Such models generally lack inputs from human factors, as these are not directly observable and are manifested inside users' brains. Therefore, in this study a multi-modal database was developed for neuro-physiological identification of the human factors which influence user perceived QoE and also to probe into the users' internal quality formation processes. It is hoped that the database will help improve the pre-existing models for quality estimation. The database utilizes neuro-physiological tools, such as electroencephalography and functional near infrared spectroscopy, to record users' brain activity while experiencing synthesised speech produced from various commercially available TTS systems. Moreover, an extensive analysis of participants' ratings has been reported in the paper. Also, the database has been made publicly available online to encourage other researchers to utilize the neuro-physiological insights while developing new quality estimation algorithms.

The past few years have seen the availability of consumer electroencephalography (EEG) devices increase significantly. These devices, usually with a compact form factor and affordable price, now allow researchers and enthusiasts to use EEG in various new contexts and environments. However, the many consumer headsets often require extensive programming experience to interface with their respective drivers; moreover, standardization of the access and recording of EEG data across the devices still remains to be done. Consequently, a tool is needed to facilitate the recording and streaming of EEG data from consumer headsets that can easily be interfaced with different programming languages and software, and that allows interchangeability between devices. This paper describes the open source MuSAE Lab EEG Server (MuLES), an EEG acquisition and streaming server that aims at creating a standard interface for portable EEG headsets, in order to accelerate the development of brain-computer interfaces (BCIs) and of general EEG use in novel contexts. In addition to the EEG server interface which currently supports five different consumer devices and session playback, clients are developed for use on different platforms and in various programming languages, making prototyping and recording a quick and simple task. To validate the functionality and usability of the EEG server, a use case highlighting its main features is presented. The developed tool simplifies the acquisition and recording of EEG data from portable consumer devices by providing a single efficient interface, with applications in areas such as basic and behavioural research, prototyping, neurogaming, live performance and arts.

As new speech technologies emerge, telecommunication service providers have to provide superior user experience in order to remain competitive. To this end, quality-of-experience (QoE) perception modelling and measurement has become a key priority. QoE models rely on three influence factors: technological, contextual and human. Existing solutions have typically relied on the former two and human influence factors (HIFs) have been mostly neglected due to difficulty in measuring them. In this paper, we show that measuring human affective states is important for QoE measurement and propose the use of affective brain-computer interfaces (aBCIs) for objective measurement of perceived QoE for two emerging speech technologies, namely far-field hands-free communications and text-to-speech systems. When incorporating subjectively-derived HIFs into the QoE model, gains of up to 26.3 % could be found relative to utilizing only technological factors. When utilizing HIFs derived from an electroencephalography (EEG) based aBCI, in turn, gains of up to 14.5 % were observed. These findings show the importance of using aBCIs in QoE measurement and also highlight that further improvement may be warranted once improved affective state correlates are found from EEGs and/or other neurophysiological modalities.