Search

Cross-subject EEG emotion recognition suffers a major setback due to high inter-subject variability in emotional responses. Many prior studies have endeavored to alleviate the inter-subject discrepancies of EEG feature distributions, ignoring the variable EEG connectivity and prediction deviation caused by individual differences, which may cause poor generalization to the unseen subject. This article proposes a graph domain disentanglement network (GDDN) to generalize EEG emotion recognition across subjects in terms of EEG connectivity, representation, and prediction. More specifically, a graph domain disentanglement module is proposed to extract common-specific characteristics on both EEG graph connectivity and graph representation, enabling a more comprehensive network transferability to the unseen individual. Meanwhile, to strengthen stable emotion prediction capability, a domain-adaptive classifier aggregation module is developed to facilitate adaptive emotional prediction for the unseen individual conditioned on the domain weights of the input individuals. Finally, an auxiliary supervision module is imposed to alleviate the domain discrepancy and reduce information loss during the disentanglement learning. Extensive experiments on three public EEG emotion datasets, i.e., SEED, SEED-IV, and MPED, validate the superior generalizability of GDDN compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

EEG emotion recognition has been hampered by the clear individual differences in the electroencephalogram (EEG). Nowadays, domain adaptation is a good way to deal with this issue because it aligns the distribution of data across subjects. However, the performance for EEG emotion recognition is limited by the existing research, which mainly focuses on the global alignment between the source domain and the target domain and ignores much fine-grained information. In this study, we propose a method called Graph-based Unsupervised Subdomain Adaptation (Gusa), which simultaneously aligns the distribution between the source and target domains in a fine-grained way from both the channel and emotion subdomains. Gusa employs three modules, such as the Node-wise Domain Constraints Module to align each EEG channel and obtain a domain-variant representation, the Class-level Distribution Constraints Module, and the Emotion-wise Domain Constraints Module, to collect more fine-grained information, create more discriminative representations for each emotion, and lessen the impact of noisy emotion labels. The studies on the SEED, SEED-IV, and MPED datasets demonstrate that Gusa significantly improves the ability of EEG to recognize emotions and can extract more granular and discriminative representations for EEG. [ABSTRACT FROM AUTHOR]

Emotion recognition systems are typically trained to classify a given psychophysiological state into emotion categories. Current platforms for emotion ground-truth collection show limitations for real-world scenarios of long-duration content (e.g. $> $ > 10 minutes), namely: 1) Real-time annotation tools are distracting and become exhausting; 2) Perform retrospective annotation of the whole content in bulk (providing highly coarse annotations); or 3) Are used by external experts (depending on the number of annotators and their subjective experience). We explore a novel approach, the EmotiphAI Annotator, that allows undisturbed content visualisation and simplifies the annotation process by using segmentation algorithms that select brief clips for emotional annotation retrospectively. We compare three methods for content segmentation based on physiological data (Electrodermal Activity (EDA), emotion-based), scene (time-based), and random (control) selection. The EmotiphAI Annotator attained a B+ System Usability Scale score and low-average mental workload as per the NASA Task Load Index (40%). The reliability of the self-report was analysed by the inter-rater agreement (STD $< $ < 0.75), coherence across time segmentation methods (STD $< $ < 0.17), comparison against the state-of-the-art ground truth (STD $< $ < 0.7), and correlation to EDA ($> $ > 0.3 to 0.8), where the EDA-based method obtained the overall best performance. [ABSTRACT FROM AUTHOR]

Multi-modal classifiers for emotion recognition have become prominent, as the emotional states of subjects can be more comprehensively inferred from Electroencephalogram (EEG) signals and eye movements. However, existing classifiers experience a decrease in performance due to the distribution shift when applied to new users. Unsupervised domain adaptation (UDA) emerges as a solution to address the distribution shift between subjects by learning a shared latent feature space. Nevertheless, most UDA approaches focus on a single modality, while existing multi-modal approaches do not consider that fine-grained structures should also be explicitly aligned and the learned feature space must be discriminative. In this paper, we propose Coarse and Fine-grained Distribution Alignment with Correlated and Separable Features (CFDA-CSF), which performs a coarse alignment over the global feature space, and a fine-grained alignment between modalities from each domain distribution. At the same time, the model learns intra-domain correlated features, while a separable feature space is encouraged on new subjects. We conduct an extensive experimental study across the available sessions on three public datasets for multi-modal emotion recognition: SEED, SEED-IV, and SEED-V. Our proposal effectively improves the recognition performance in every session, achieving an average accuracy of 93.05%, 85.87% and 91.20% for SEED; 85.72%, 89.60%, and 86.88% for SEED-IV; and 88.49%, 91.37% and 91.57% for SEED-V. [ABSTRACT FROM AUTHOR]

Most existing multi-modal emotion recognition studies are targeted at a classification task that aims to assign a specific emotion category to a combination of several heterogeneous input data, including multimedia signals and physiological signals. A growing number of recent psychological evidence suggests that different discrete emotions may co-exist at the same time, which promotes the development of mixed-emotion recognition to identify a mixture of basic emotions. In this work, we focus on a challenging situation where both positive and negative emotions are presented simultaneously, and propose a multi-modal mixed emotion recognition framework, namely EmotionDict. The key characteristics of our EmotionDict include the following. (1) Inspired by the psychological evidence that such a mixed state can be represented by combinations of basic emotions, we address mixed emotion recognition as a label distribution learning task. An emotion dictionary has been designed to disentangle the mixed emotion representations into a weighted sum of a set of basic emotion elements in a shared latent space and their corresponding weights. (2) We incorporate physiological and overt behavioral multi-modal signals, including electroencephalogram (EEG), peripheral physiological signals, and facial videos, which directly display the subjective emotions. These modalities have diverse characteristics given that they are related to the central or peripheral nervous system, and the motor cortex. (3) We further design auxiliary tasks to learn modality attentions for modality integration. Experiments on two datasets show that our method outperforms existing state-of-the-art approaches on mixed-emotion recognition. [ABSTRACT FROM AUTHOR]

Multi-modal emotion recognition has attracted increasing attention in human-computer interaction, as it extracts complementary information from physiological and behavioral features. Compared to single modal approaches, multi-modal fusion methods are more susceptible to uncertainty in emotion recognition, such as heterogeneity and inconsistent predictions across different modalities. Previous multi-modal approaches ignore systematic modeling of uncertainty in fusion and revelation of dynamic variations in emotion process. In this article, we propose a dynamic confidence-aware fusion network for robust recognition of heterogeneous emotion features, including electroencephalogram (EEG) and facial expression. First, we develop a self-attention based multi-channel LSTM network to preliminarily align the heterogeneous emotion features. Second, we propose a confidence regression network to estimate true class probability (TCP) on each modality, which helps explore the uncertainty at modality level. Then, different modalities are weighted fused according to above two types of uncertainty. Finally, we adopt self-paced learning (SPL) mechanism to further improve the model robustness by alleviating negative effect from the hard learning samples. The experimental results on several multi-modal emotion datasets demonstrate the proposed method outperforms the state-of-the-art methods in emotion recognition performance and explicitly reveals the dynamic variation of emotion with uncertainty estimation. [ABSTRACT FROM AUTHOR]

Emotions play an essential role in affecting the performance of cognitive abilities in continuous cognitive tasks. Most previous studies share a common issue in that the evoked emotions are simply presumed to be real emotions, without taking into account the observation that emotions may be changed when carrying out cognitive activities. This may lead to the inaccurate detection of true emotions, which further adversely affects the investigation of interactions between emotion and cognition. To address this challenging problem, the present work develops an innovative study using EEG measurement to investigate the interaction between dynamic affection and cognitive ability. In particular, a real-time emotion detection model by the use of physiological signals (i.e., EEG) is constructed, to dynamically monitor the current emotional state. Given the observed emotion, the analysis of the interaction between cognitive abilities and dynamic emotions is undertaken from the perspectives of both behavioral performance and brain mechanisms. Research outcomes indicate that emotions are not stable, and are indeed dynamically changed by cognitive performance. Meanwhile, cognitive activities also influence the brain activation pattern revealed under different emotions, which validates the necessity of introducing the dynamic emotion monitoring model. In addition, the best performance has been found when the emotional state is neutral in terms of accuracy and response time. The results of this study provide a potential basis for assessing the cognitive abilities of individuals with different emotions in a variety of applications of cognitive scenarios. [ABSTRACT FROM AUTHOR]

Emotions are important factors in decision-making. With the advent of brain-computer interface (BCI) techniques, researchers developed a strong interest in predicting decisions based on emotions, which is a challenging task. To predict decision-making performance using emotion, we have proposed the Maximizing Mutual Information between Emotion and Decision relevant features (MMI-ED) method, with three modules: (1) Temporal-spatial encoding module captures spatial correlation and temporal dependence from electroencephalogram (EEG) signals; (2) Relevant feature decomposition module extracts emotion-relevant features and decision-relevant features; (3) Relevant feature fusion module maximizes the mutual information to incorporate useful emotion-related feature information during the decision-making prediction process. To construct a dataset that uses emotions to predict decision-making performance, we designed an experiment involving emotion elicitation and decision-making tasks and collected EEG, behavioral, and subjective data. We performed a comparison of our model with several emotion recognition and motion imagery models using our dataset. The results demonstrate that our model achieved state-of-the-art performance, achieving a classification accuracy of 92.96 $\%$ % . This accuracy is 6.83 $\%$ % higher than the best-performing model. Furthermore, we conducted an ablation study to demonstrate the validity of each module and provided explanations for the brain regions associated with the relevant features. [ABSTRACT FROM AUTHOR]

Stress, especially chronic stress, is a high risk factor of many physical and mental health problems. This work acquired 702 days of full-day ambulatory electrocardiogram (ECG) and Tri-axial acceleration (T-ACC) data from 104 healthy college students and realized chronic stress recognition through signal processing, statistical test and machine learning. We divided the 24 hours of a day into 153 time slots, and calculated 30 features from ECG and T-ACC data in each time slot. Statistical test of the above 30 features of the subjects with chronic stress and no chronic stress labels showed that chronic stress altered the autonomic nervous control of the heart not only in the daily activity time but also in the rest time at night, leading to smaller heart rate variability, faster heart rate and less complexity of the heartbeat rhythm. More specifically, the parasympathetic nervous activity at night was weakened by chronic stress. We expressed the ECG and T-ACC data of a day as a 30 × 153 data matrix, applied a four-layer fully connected neural network to classify the data of 702 days with chronic stress and no chronic stress labels, and obtained 88.17% chronic stress detection accuracy in the leave-one-subject-out cross test. [ABSTRACT FROM AUTHOR]

Olfactory-enhanced virtual reality (OVR) creates a complex and rich emotional experience, thus promoting a new generation of human-computer interaction experiences in real-world scenarios. However, with the rise of virtual reality (VR) as a mood induction procedure (MIP), few studies have incorporated olfactory stimuli into emotion induction in three-dimensional (3D) environments. Considering the differences in electroencephalography (EEG) dynamics between sensory stimuli, all previous two-dimensional (2D) and 3D emotional studies have been less effective in reality because they only use visual and audio senses. To overcome these limitations, we developed a novel EEG signal dataset based on OVR. We systematically analyzed the influence of olfactory stimuli on emotion induction in a VR environment from a neurophysiological perspective. Specifically, synchronous EEG signals were collected from 65 participants as they watched positive and negative videos in traditional VR and OVR. Their power spectral densities (PSDs) were then calculated to compare the differences in brain activation between their VR and OVR modes during the induction of positive and negative emotions, while their brain states were classified after feature selection. The results showed that olfactory stimuli enhanced EEG responses for positive emotions, but the opposite was true for negative emotions. Additionally, the recognition rate of brain emotional states was more than 90% under both positive and negative emotions, while the high-frequency β and γ bands could effectively distinguish VR and OVR modes. This study introduced the olfaction into the field of human-computer interaction, which could promote research on emotion induction and recognition in real-world environments. [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG) signals have been widely studied in human emotion recognition. The majority of existing EEG emotion recognition algorithms utilize dozens or hundreds of electrodes covering the whole scalp region (denoted as full-channel EEG devices in this paper). Nowadays, more and more portable and miniature EEG devices with only a few electrodes (denoted as few-channel EEG devices in this paper) are emerging. However, emotion recognition from few-channel EEG data is challenging because the device can only capture EEG signals from a portion of the brain area. Moreover, existing full-channel algorithms cannot be directly adapted to few-channel EEG signals due to the significant inter-variation between full-channel and few-channel EEG devices. To address these challenges, we propose a novel few-channel EEG emotion recognition framework from the perspective of knowledge transfer. We leverage full-channel EEG signals to provide supplementary information, available online, for few-channel signals via a transfer learning-based model CD-EmotionNet, which consists of a base emotion model for efficient emotional feature extraction and a cross-device transfer learning strategy. This strategy helps to enhance emotion recognition performance on few-channel EEG data by utilizing knowledge learned from full-channel EEG data. To evaluate our cross-device EEG emotion transfer learning framework, we construct an emotion dataset containing paired 18-channel and 5-channel EEG signals from 25 subjects, as well as 5-channel EEG signals from 13 other subjects. Extensive experiments show that our framework outperforms state-of-the-art EEG emotion recognition methods by a large margin. [ABSTRACT FROM AUTHOR]

We present a dataset for the analysis of human affective states using functional near-infrared spectroscopy (fNIRS). Data were recorded from thirty-one participants who engaged in two tasks. In the emotional perception task the participants passively viewed images sampled from the standard international affective picture system database, which provided ground-truth valence and arousal annotation for the stimuli. In the affective imagery task the participants actively imagined emotional scenarios followed by rating these for subjective valence and arousal. Correlates between the fNIRS signal and the valence-arousal ratings were investigated to estimate the validity of the dataset. Source-code and summaries are provided for a processing pipeline, brain activity group analysis, and estimating baseline classification performance. For classification, prediction experiments are conducted for single-trial 4-class classification of arousal and valence as well as cross-participant classifications, and comparisons between high and low arousal variants of the valence prediction tasks. Finally, classification results are presented for subject-specific and cross-participant models. The dataset is made publicly available to encourage research on affective decoding and downstream applications using fNIRS data. [ABSTRACT FROM AUTHOR]

Recognizing human emotions from complex, multivariate, and non-stationary electroencephalography (EEG) time series is essential in affective brain-computer interface. However, because continuous labeling of ever-changing emotional states is not feasible in practice, existing methods can only assign a fixed label to all EEG timepoints in a continuous emotion-evoking trial, which overlooks the highly dynamic emotional states and highly non-stationary EEG signals. To solve the problems of high reliance on fixed labels and ignorance of time-changing information, in this paper we propose a time-aware sampling network (TAS-Net) using deep reinforcement learning (DRL) for unsupervised emotion recognition, which is able to detect key emotion fragments and disregard irrelevant and misleading parts. Specifically, we formulate the process of mining key emotion fragments from EEG time series as a Markov decision process and train a time-aware agent through DRL without label information. First, the time-aware agent takes deep features from a feature extractor as input and generates sample-wise importance scores reflecting the emotion-related information each sample contains. Then, based on the obtained sample-wise importance scores, our method preserves top-X continuous EEG fragments with relevant emotion and discards the rest. Finally, we treat these continuous fragments as key emotion fragments and feed them into a hypergraph decoding model for unsupervised clustering. Extensive experiments are conducted on three public datasets (SEED, DEAP, and MAHNOB-HCI) for emotion recognition using leave-one-subject-out cross-validation, and the results demonstrate the superiority of the proposed method against previous unsupervised emotion recognition methods. The proposed TAS-Net has great potential in achieving a more practical and accurate affective brain-computer interface in a dynamic and label-free circumstance. [ABSTRACT FROM AUTHOR]

Affective computing is an important subfield of artificial intelligence, and with the rapid development of brain-computer interface technology, emotion recognition based on EEG signals has received broad attention. It is still a great challenge to effectively explore the multi-dimensional information in the EEG data in spite of a large number of deep learning methods. In this article, we propose a deep learning model called Attention-based Multiple Dimensions EEG Transformer (AMDET), which can leverage the complementarity among the spectral-spatial-temporal features of EEG data by employing the multi-dimensional global attention mechanism. We first transform the original EEG data into 3D temporal-spectral-spatial representations and then the AMDET would use spectral-spatial transformer blocks to extract effective features in the EEG signal and focus on the critical time frame with the temporal attention block. We conduct extensive experiments on the DEAP, SEED, and SEED-IV datasets to evaluate the performance of AMDET and the results outperform the state-of-the-art baseline on three datasets. Accuracy rates of 97.48%, 96.85%, 97.17%, 87.32% were achieved in the DEAP-Arousal, DEAP-Valence, SEED, and SEED-IV datasets, respectively. Based on AMDET, we can achieve over 90% accuracy with only eight channels, significantly improving the possibility of practical applications. [ABSTRACT FROM AUTHOR]

An affective brain-computer interface (aBCI) can detect affective states based on brain signals and might assist people in improving their emotion regulation abilities. However, individual differences in emotional brain patterns make cross-subject emotion identification extremely challenging. Traditional supervised single-subject classification schemes require considerable calibration samples from new individuals to train subject-dependent models. Individuals are easily fatigued with long-term EEG collection processes, which may affect performance in subsequent online experiments. In this study, we propose a real-time aBCI system using domain-fusion-based multisource style transfer mapping (DF-MS-STM) to detect positive, neutral, and negative emotional states without the need for additional training sessions. Sixteen subjects participated in our online experiments to test the performance of our aBCI system and an average online prediction accuracy of 72.17±12.25% was obtained for three-class emotion recognition tasks in the last three experimental sessions. Our proposed algorithm significantly outperformed numerous baseline methods in terms of cross-subject emotion classification. In addition, we identified distinct brain patterns in response to different emotional stimuli based on the results of event-related spectral perturbation (ERSP) analyses. These neural patterns might provide new insights for emotional brain mechanistic studies and related aBCIs. [ABSTRACT FROM AUTHOR]

Exploring effective approaches to enhance face-to-face interactions and interpersonal relationships is an important topic in the applications of affective computing. According to the co-actualization model, we propose a face-to-face co-participation serious game for relationship enhancement, with a focus on battling COVID-19. Moreover, a prototype system is developed using an immersive virtual environment and a low-cost brain-computer interface. Through this system, a dynamic flow experience enhancement tool is utilized to involve partners in the cooperative task. To evaluate the system performance, two studies are conducted with schoolmates as participants. Study 1 compares the cooperative and competitive modes, and demonstrates that the former elicited higher level of decision-making challenge and affections, which are beneficial for forming relationships. Study 2 further examines the effect of the dynamic flow enhancement tool in the cooperative task and the results show its effectiveness in promoting flow experience, perceived closeness, and intimacy in relationships. Given this short-term participation, participants felt a greater sense of closeness and intimacy than they had before the test. In conclusion, our proposed system is effective in enhancing schoolmate relationships. [ABSTRACT FROM AUTHOR]

Workers with demanding jobs are at risk of experiencing mental stress, leading to decreased performance, mental illness, and disrupted sleep. To detect elevated stress levels in the workplace, studies have explored stress measurement from physiological, psychological, and behavioral perspectives. This paper reviews the assessment methods and strategies for mitigating mental stress in the workplace and provides recommendations for early detection and mitigation of mental stress. Among the modalities, Electroencephalography (EEG), Electrocardiography (ECG) and Galvanic Skin Response (GSR) were found to be the most used in assessing mental stress in the workplace. Nevertheless, these modalities are sensitive to motion artifacts and are difficult to be integrated into real work environments. To further improve stress level assessment in the workplace, multimodality integration with a reduced number of sensors such as EEG, GSR and Functional near infrared spectroscopy (fNIRS) can be utilized. This would lead to developing strategies for stress management in real-time. Furthermore, combining EEG with fNIRS would improve source localization of mental stress. To mitigate stress, we recommend developing a closed loop system that incorporates brain data acquisition systems and machine learning with physical stimulations such as audio Binaural Beats Stimulation and/or Transcranial Electric Stimulation. [ABSTRACT FROM AUTHOR]

Multimodal emotion recognition has attracted increasing interest from academia and industry in recent years, since it enables emotion detection using various modalities, such as facial expression images, speech and physiological signals. Although research in this field has grown rapidly, it is still challenging to create a multimodal database containing facial electrical information due to the difficulty in capturing natural and subtle facial expression signals, such as optomyography (OMG) signals. To this end, we present a newly developed Multimodal Genuine Emotion and Expression Detection (MGEED) database in this paper, which is the first publicly available database containing the facial OMG signals. MGEED consists of 17 subjects with over 150 K facial images, 140 K depth maps and different modalities of physiological signals including OMG, electroencephalography (EEG) and electrocardiography (ECG) signals. The emotions of the participants are evoked by video stimuli and the data are collected by a multimodal sensing system. With the collected data, an emotion recognition method is developed based on multimodal signal synchronisation, feature extraction, fusion and emotion prediction. The results show that superior performance can be achieved by fusing the visual, EEG and OMG features. [ABSTRACT FROM AUTHOR]

EEG emotion recognition plays a significant role in various mental health services. Deep learning-based methods perform excellently, but still suffer from interpretability. Although methods such as Gradient-weighted Class Activation Mapping(Grad-CAM) can cope with the above problem, their coarse granularity cannot accurately reveal the mechanism to promote emotional intelligence. In this paper, fine-grained interpretability is proposed, called Concat-aided Grad-CAM. Specifically, the multi-level feature mapping before the fully connected layer is concatenated to obtain the gradients of the target concept so that the discriminant information can be directly located in the high-precision area. Unlike coarse-grained interpretability methods applied in EEG emotion recognition, it can accurately highlight the EEG channels related to emotion rather than an obscure area. In addition, a systematic brain functional network is proposed to reveal the relationship between those channels and to further improve emotion recognition performance. The channels with greater contributions are connected, and those connections are learned by dynamic graph convolutional networks, while the others are independent to eliminate interference. Experiments on four EEG emotion recognition datasets manifest that Concat-aided Grad-CAM can be interpreted by the fine-grained. In addition, it has been shown that the learned brain functional network can improve the performance of the baselines. Significantly, the experiment results achieve state-of-the-art performance in multiple experiments. [ABSTRACT FROM AUTHOR]

Affective brain-computer interface based on electroencephalography (EEG) is an important branch in the field of affective computing. However, the individual differences in EEG emotional data and the noisy labeling problem in the subjective feedback seriously limit the effectiveness and generalizability of existing models. To tackle these two critical issues, we propose a novel transfer learning framework with Prototypical Representation based Pairwise Learning (PR-PL). The discriminative and generalized EEG features are learned for emotion revealing across individuals and the emotion recognition task is formulated as pairwise learning for improving the model tolerance to the noisy labels. More specifically, a prototypical learning is developed to encode the inherent emotion-related semantic structure of EEG data and align the individuals’ EEG features to a shared common feature space under consideration of the feature separability of both source and target domains. Based on the aligned feature representations, pairwise learning with an adaptive pseudo labeling method is introduced to encode the proximity relationships among samples and alleviate the label noises effect on modeling. Extensive results on two benchmark databases (SEED and SEED-IV) under four different cross-validation evaluation protocols validate the model reliability and stability across subjects and sessions. Compared to the literature, the average enhancement of emotion recognition across four different evaluation protocols is 2.04% (SEED) and 2.58% (SEED-IV). [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG) data annotation demands considerable expertise and is a time-intensive process. Moreover, inter-subject variability intensifies the challenge of domain shift, adversely impacting the generalization performance of deep learning models on unseen subjects. Current methods in EEG data analysis often struggle to handle the complex nature of brain activity without relying on EEG feature engineering. In this paper, we present a hybrid semi-supervised framework for seizure type classification, which relies on minimal domain knowledge provided by exploiting spectral and spatial patch-level representations of raw unlabeled EEG data, while leveraging a small amount of labeled data. Our method, IMPRESS, enhances EEG representation learning by combining multi-patch mutual information maximization with adversarial distribution alignment. We assessed the framework’s performance for cross-patient seizure classification using publicly accessible Temple University Seizure Corpus. IMPRESS surpasses the best-performing semi-supervised learning method by 1.92% and 0.72% using balanced accuracy and macro-F1 metrics, respectively, with 40 labeled samples per class. Remarkably, IMPRESS surpasses the fully-supervised method while requiring only 25 labeled samples per class. Additionally, we visualize the learned feature embeddings, highlighting the underlying dynamics across different seizure types, aiding in understanding the model’s behavior. This demonstrates the potential of leveraging multi-patch information from unlabeled data through a contrastive data-driven approach, alleviating the burden of annotating large amounts of EEG data.

Accurate diagnosis of upper extremity motor function in stroke patients is important for effective rehabilitation. However, the approach to correctly perform clinical assessments is still a matter of discussion and requires both trained personnel and specialized materials, thus, limiting the availability of stroke upper extremity diagnosis. Computer-aided methods have been scarcely reported for stroke upper extremity motor function estimation and could support personnel training and clinical decision-making. For these reasons, in the present study, linear regression and regression tree ensembles were applied to estimate upper extremity assessments’ scores using neurophysiological measurements, including electroencephalography (EEG) and transcranial magnetic stimulation (TMS). A database was used to evaluate these approaches and was comprised by measurements of upper extremity sensorimotor and functional performance of stroke patients assessed with the Fugl-Meyer Assessment for the Upper Extremity (FMA-UE) and the Action Research Arm Test (ARAT). Regression tree ensembles outperformed linear models, estimating 66.7% of the FMA-UE scores and 70% of the ARAT scores with errors below the minimal clinically important difference. The median absolute errors were 3.5 points for the FMA-UE and 1.8 points for the ARAT, within clinically acceptable ranges. Variables that were associated with a higher upper extremity function measured with FMA-UE and ARAT were a higher corticospinal integrity in patients’ affected hemisphere, lower interhemispheric functional connectivity in the central region of the cortex during hand motor intention and, higher alpha activation in the central and lower activation in the parietal regions of the cortex during hand motor intention. Limitations of the study considered, the performance of the proposed approach implied that computer-aided estimation of upper extremity motor function is feasible using physiological information and nonlinear models. These models could be used to create expert systems that support clinical personnel training and decision making regarding upper extremity assessment in stroke.

Workload and distraction are problems that affect road driving and can cause very serious accidents. Drive assistance systems are very important to help drivers relieve fatigue of driving by increasing road safety and reducing the number of accidents. The Adaptive Cruise Control (ACC) is a useful driver assistance system to set and maintain a safe distance from the front vehicle, modulating speed and limiting sudden braking. However, the effects of the ACC on overall driver behaviour have not yet been thoroughly investigated. In this study, the authors assessed the influence of ACC on mental workload and attention of drivers in a real driving environment. The results of several evaluations were considered: subjective assessment of workload (NASA-TLX), physiological measurements of workload (brain activity through electroencephalographic technique and the analysis of visual behaviour through an eye monitoring device) and performance-based measurements (via Vbox Pro Video mounted on the vehicle). 52 drivers were involved in the study, 26 ACC experts and 26 non-experts, who drove along the ring road of Bologna (Tangenziale di Bologna) (Italy), both using ACC and manually, without the help of any device. During the test a secondary task was introduced: the sudden arrival of a braking car. Results showed that the use of ACC increased distraction when driving.

Consumer-grade Electroencephalography (EEG) devices equipped with few electrodes often suffer from low spatial resolution, hindering the accurate capture of intricate brain activity patterns. To address this issue, we propose MASER, a novel super-resolution approach for EEG recording. In MASER, we design the eMamba block for extracting EEG features based on the principles of state space models (SSMs). We further stack eMamba blocks to form a low-resolution feature extractor and a high-resolution signal predictor, which enhances the feature representation. During the training of MASER, we fully consider the characteristics of multidimensional biological series signals, incorporating a smoothness constraint loss to achieve more consistent high-resolution reconstructions. MASER pioneers EEG-oriented state space modeling, effectively capturing the temporal dynamics and latent states, thereby revealing complex neural interactions over time. Extensive experiments show that the proposed MASER outperforms the state-of-the-art methods in super-resolution quality on two public EEG datasets, with normalized mean square error reduced by 16.25% and Pearson correlation improved by 1.13%. Moreover, a case study of motor imagery recognition highlights the advantages conferred by high-resolution EEG signals. With a 4x increase in spatial resolution by MASER, the recognition accuracy improves by 5.74%, implying a significant performance elevation in brain-computer interface (BCI) command mapping. By enhancing the spatial resolution of EEG signals, MASER makes EEG-based applications more accessible, reducing cost and setup time while maintaining high performance across various domains such as gaming, education, and healthcare.

Epilepsy is a chronic brain disorder characterized by recurrent unprovoked seizures. The treatment for epilepsy is influenced by the types of seizures. Therefore, developing a reliable, explainable, and automated system to identify seizure types is necessary. This study aims to automate the process of classification of five seizure types: focal non-specific, generalized, complex partial, absence, and tonic-clonic using electroencephalogram (EEG) signals and machine learning algorithms. The EEG signals of 2933 seizures from 327 patients were obtained from the publicly available Temple University Hospital dataset. Initially, the signals were preprocessed using a standard pipeline, and 110 features from the time, frequency, and time-frequency domain were computed from each seizure. Further, the features were ranked using the statistical test and extreme Gradient Boosting (XGBoost) algorithm to identify the significant features. We built binary and multiclass seizure-type classification systems using the identified features and machine learning algorithms. Our study revealed that the EEG band power between 11–13 Hz, 27–29 Hz, intrinsic mode function (IMF) band power 19–21 Hz, and delta band (1-4 Hz) played a crucial role in discriminating the seizures. We achieved an average accuracy of 88.21% and 69.43% for the binary and multiclass seizure-type classification, respectively, using the XGBoost classifier. We also found that the combination of features performed well compared to any single domain. This automated system has the potential to aid neurologists in making diagnosis of epileptic seizure types. The proposed methodology can be applied alongside the established clinical approach of visual evaluation for the classification of seizure-types.

Motor imagery-based brain-computer interface (MI-BCI) systems convert user intentions into computer commands, aiding the communication and rehabilitation of individuals with motor disabilities. Traditional MI classification relies on supervised learning; however, it faces challenges in acquiring large volumes of reliably labeled data and ensuring generalized high performance owing to specific experimental paradigms. To address these issues, this study proposes a contrastive self-supervised learning (SSL) method that does not require negative samples. A MultiResolutionCNN backbone, designed to capture temporal and spatial electroencephalogram (EEG) features, is introduced. By using Barlow Twins loss-based contrastive SSL, features are effectively extracted from EEG signals without labels. MI classification was performed with data from two datasets that were not used for pretraining: an in-domain dataset with two classes and out-domain dataset with four classes. The proposed framework outperformed conventional supervised learning, achieving accuracies of 82.47% and 43.19% on the in-domain and out-domain datasets, respectively. Performance variations were observed in response to variations in the values of key hyperparameters during training. The maximum differences in lambda, learning rate, and batch size were 0.93%, 3.28%, and 3.38%, respectively, demonstrating robust generalization. This study facilitates the development and application of more practical and robust MI-BCI systems. This code is available at https://github.com/dlcjfgmlnasa/Barlow_Twins_EEG.git.

Through phase-amplitude analysis, this study investigated how low-frequency postural fluctuations interact with high-frequency scalp electroencephalography (EEG) amplitudes, shedding light on age-related mechanic differences in balance control during uneven surface navigation. Twenty young ( $24.1~\pm ~1.9$ years) and twenty older adults ( $66.2~\pm ~2.7$ years) stood on a training stabilometer with visual guidance, while their scalp EEG and stabilometer plate movements were monitored. In addition to analyzing the dynamics of the postural fluctuation phase, phase-amplitude coupling (PAC) for postural fluctuations below 2 Hz and within EEG sub-bands (theta: 4-7 Hz, alpha: 8-12 Hz, beta: 13-35 Hz) was calculated. The results indicated that older adults exhibited significantly larger postural fluctuation amplitudes(p ${p} =0.005$ ) than young adults. The PAC between postural fluctuation and theta EEG (FCz and bilateral temporal-parietal-occipital area), as well as that between postural fluctuation and alpha EEG oscillation, was lower in older adults than in young adults (p ${p}=0.006$ ), was higher in older adults than in young adults. In summary, the postural fluctuation phase and phase-amplitude coupling between postural fluctuation and EEG are sensitive indicators of the age-related decline in postural adjustments, reflecting less flexible motor state transitions and adaptive changes in error monitoring and visuospatial attention.

The level of human consciousness is a critical indicator in medicine, essential for the accurate diagnosis of specific conditions such as brain death. This study aims to provide an objective and visual consciousness indicator for medical applications by proposing a new algorithm to quantify consciousness. Traditionally, consciousness levels have been qualitatively assessed using scales like the Glasgow Coma Scale (GCS). In this study, we introduce a novel algorithm, termed D-MaxApEn, designed to quantify consciousness. This algorithm was applied to EEG data from four different states: wakefulness, anesthesia, coma, and brain death. Through calculation and analysis, we objectively quantified these four levels of consciousness and validated the effectiveness of this algorithm in a real-time brain death diagnosis system. The results demonstrate the significant potential of this algorithm for the future diagnosis of the level of consciousness. Its use could revolutionize early intervention, improve patient outcomes, and reduce the burden on individuals and healthcare systems.

The internet of medical things (IoMT) has become a pivotal aspect of IoT applications, playing a crucial role in cutting down healthcare expenses, enhancing access to clinical services, and refining operational efficiency within the healthcare domain. An early detection of neurological brain disorders continues to present a formidable challenge. In response, our research endeavors are directed towards the IoMT-based system used for the real-time diagnosis of neurological disorders. In this paper, IoMT-based real-time diagnosis model is developed for neurological brain disorders using systematic deep learning and electroencephalogram (EEG) signal (RT-NeuroDDSM). We first introduce the time domain, channel and spatial attention network (TCSNet) for feature extraction which extracts the high-level time series, channel and spatial features, respectively. TCSNet aims to learn more valuable features from the input data to achieve good classification results. Furthermore, in order to maximize features, we create the modified normative fish swarm (MNFS) feature selection algorithm. Next, the detection of various neurological brain problems, including neuro-typical, epilepsy, and autism spectrum disorder (ASD), is accomplished by applying the hinging hyperplane neural network (HHNN). To verify the performance, we used the publicly accessible EEG datasets from University of Bonn-Germany, CHB-MIT EEG repository, and King Abdulaziz University. The RT-NeuroDDSM has an overall classification accuracy of 99.956%, making it 5.471% more in efficiency compared to the existing state-of-the-art model.

The deficit in social interaction skills among individuals with autism spectrum disorder (ASD) is strongly influenced by personal experiences and social environments. Neuroimaging studies have previously highlighted the link between social impairment and brain activity in ASD. This study aims to develop a method for assessing and identifying ASD using a social cognitive game-based paradigm combined with electroencephalo-graphy (EEG) signaling features. Typically developing (TD) participants and autistic preadolescents and teenagers were recruited to participate in a social game while 12-channel EEG signals were recorded. The EEG signals underwent preprocessing to analyze local brain activities, including event-related potentials (ERPs) and time-frequency features. Additionally, the global brain network’s functional connectivity between brain regions was evaluated using phase-lag indices (PLIs). Subsequently, machine learning models were employed to assess the neurophysiological features. Results indicated pronounced ERP components, particularly the late positive potential (LPP), in parietal regions during social training. Autistic preadolescents and teenagers exhibited lower LPP amplitudes and larger P200 amplitudes compared to TD participants. Reduced theta synchronization was also observed in the ASD group. Aberrant functional connectivity within certain time intervals was noted in the ASD group. Machine learning analysis revealed that support-vector machines achieved a sensitivity of 100%, specificity of 91.7%, and accuracy of 95.8% as part of the performance evaluation when utilizing ERP and brain oscillation features for ASD characterization. These findings suggest that social interaction difficulties in autism are linked to specific brain activation patterns. Traditional behavioral assessments face challenges of subjectivity and accuracy, indicating the potential use of social training interfaces and EEG features for cognitive assessment in ASD.

Individuals with mild cognitive impairment (MCI), the preclinical stage of Alzheimer disease (AD), suffer decline in their visual working memory (WM) functions. Using large-scale network analysis of electroencephalography (EEG), the current study intended to investigate if there are differences in functional connectivity properties extracted during visual WM coding stages between MCI patients and normal controls (NC). A total of 21 MCI patients and 20 NC performed visual memory tasks of load four, while 32-channel EEG recordings were acquired. The functional connectivity properties were extracted from the acquired EEGs by the directed transform function (DTF) via spectral Granger causal analysis. Brain network analyses revealed distinctive brain network patterns between the two groups during the WM coding stage. Compared with the NC, MCI patients exhibited a reduced visual network connectivity of the frontal-temporal in $\theta $ (4-7Hz) band. A likely compensation mechanism was observed in MCI patients, with a strong brain functional connectivity of the frontal-occipital and parietal-occipital in both $\theta $ and $\alpha $ (8-13Hz) band. Further analyses of the network core node properties based on the differential brain network showed that, in $\theta $ band, there was a significant difference in the out-degree of the frontal lobe and parietal lobe between the two groups, while in $\alpha $ band, such difference was located only in the parietal lobe. The current study found that, in MCI patients, dysconnectivity is found from the prefrontal lobe to bilateral temporal lobes, leading to increased recruitment of functional connectivity in the frontal-occipital and parietal-occipital direction. The dysconnectivity pattern of MCI is more complex and primarily driven by core nodes Pz and Fz. These results significantly expanded previous knowledge of MCI patients’ EEG dynamics during WM tasks and provide new insights into the underpinning neural mechanism MCI. It further provided a potential therapeutic target for clinical interventions of the condition.

We propose a Transfer learning-enabled electroencephalography-based intuitive brain-computer interface system by utilizing phase-based brain functional connectivity methods such as phase lag index (PLI) and Intersite phase clustering (ISPC) along with power features to explore both phase and power-based information from electroencephalography (EEG) signals. Time-frequency decomposition using a complex morlet wavelet is applied to analyze the signal components in both the time and frequency domains and extract phase connectivity and power features. Functional connectivity methods aim to recognize functional interactions and statistical mutuality among signals acquired across various brain areas. The phase-based connectivity features are extracted simultaneously for multiple channels to investigate the phase synchronization among EEG signals across the entire brain. Next, Graph theory is adopted to trace connectivity between brain regions by calculating the connectivity degree of extracted PLI and ISPC features with other electrodes. In Parallel, Discrete wavelet convolution is performed to calculate the time variable frequency band’s specific power from the imagined speech EEG data. Finally, Time-frequency images of the above-mentioned PLI, ISPC, and EEG power features are fed as input to DenseNet-121 architecture for classification. Dense Net architecture overcomes the problem of ‘vanishing gradient’ by connecting each layer directly with other layers, making the network densely connected. The maximum classification accuracy achieved is 100%, 99.14%, and 98.72% for binary, three-class, and four-class classifications, respectively. The experimental results indicate that the proposed phase-based connectivity features, EEG power, and the DenseNet-121 model have achieved excellent accuracy for two public datasets, outperforming the state-of-the-art methods. The outstanding results strengthen the possibility of real-time EEG-based intuitive brain-computer interface communication.

Walking with an exoskeleton represents a sophisticated interplay between human physiology and mechanical augmentation, yet understanding of cortical responses in this context remains limited. To address this gap, this study aimed to explore cortical responses during walking with an ankle exoskeleton, examining how these responses evolve with familiarity to the augmentation. Healthy participants without prior exoskeleton experience underwent EEG, EMG, and motion capture analysis while walking with exoskeleton assistance at 1.2m/s. Initially, exoskeleton-assisted walking induced significant biomechanical changes accompanied by corresponding cortical alterations, leading to increased cortical involvement. In addition, after a brief familiarization period, significant increases in alpha band cortical power were observed, indicating decreased cortical engagement. These findings hold significance for elucidating the cortical mechanisms involved in exoskeleton-assisted walking and may contribute to the development of more seamlessly integrated augmentation devices.

Approximately one third of the population is prone to motion sickness (MS), which is associated with the dysfunction in the integration of sensory inputs. Transcranial alternating current stimulation (tACS) has been widely used to modulate neurological functions by affecting neural oscillation. However, it has not been applied in the treatment of motion sickness. This study aims to investigate changes in brain oscillations during exposure to MS stimuli and to further explore the potential impact of tACS with the corresponding frequency and site on MS symptoms. A total of 19 subjects were recruited to be exposed to Coriolis stimuli to complete an inducing session. After that, they were randomly assigned to tACS stimulation group or sham stimulation group to complete a stimulation session. Electroencephalography (EEG), electrocardiogram, and galvanic skin response were recorded during the experiment. All the subjects suffering from obvious MS symptoms after inducing session were observed that alpha power of four channels of parieto-occipital lobe significantly decreased (P7: t =3.589, p

Visual imagery, or the mental simulation of visual information from memory, could serve as an effective control paradigm for a brain-computer interface (BCI) due to its ability to directly convey the user’s intention with many natural ways of envisioning an intended action. However, multiple initial investigations into using visual imagery as a BCI control strategies have been unable to fully evaluate the capabilities of true spontaneous visual mental imagery. One major limitation in these prior works is that the target image is typically displayed immediately preceding the imagery period. This paradigm does not capture spontaneous mental imagery as would be necessary in an actual BCI application but something more akin to short-term retention in visual working memory. Results from the present study show that short-term visual imagery following the presentation of a specific target image provides a stronger, more easily classifiable neural signature in EEG than spontaneous visual imagery from long-term memory following an auditory cue for the image. We also show that short-term visual imagery and visual perception share commonalities in the most predictive electrodes and spectral features. However, visual imagery received greater influence from frontal electrodes whereas perception was mostly confined to occipital electrodes. This suggests that visual perception is primarily driven by sensory information whereas visual imagery has greater contributions from areas associated with memory and attention. This work provides the first direct comparison of short-term and long-term visual imagery tasks and provides greater insight into the feasibility of using visual imagery as a BCI control strategy.

Automated sleep staging is essential to assess sleep quality and treat sleep disorders, so the issue of electroencephalography (EEG)-based sleep staging has gained extensive research interests. However, the following difficulties exist in this issue: 1) how to effectively learn the intrinsic features of salient waves from single-channel EEG signals; 2) how to learn and capture the useful information of sleep stage transition rules; 3) how to address the class imbalance problem of sleep stages. To handle these problems in sleep staging, we propose a novel method named SleepFC. This method comprises convolutional feature pyramid network (CFPN), cross-scale temporal context learning (CSTCL), and class adaptive fine-tuning loss function (CAFTLF) based classification network. CFPN learns the multi-scale features from salient waves of EEG signals. CSTCL extracts the informative multi-scale transition rules between sleep stages. CAFTLF-based classification network handles the class imbalance problem. Extensive experiments on three public benchmark datasets demonstrate the superiority of SleepFC over the state-of-the-art approaches. Particularly, SleepFC has a significant performance advantage in recognizing the N1 sleep stage, which is challenging to distinguish.

Epilepsy impacts around 6.38 per 1,000 people globally, presenting diagnostic challenges due to the complexity of seizures. Accurate classification of seizure types via Electroencephalogram (EEG) is critical for effective treatment and enhancing patient quality of life. However, the intricate characteristics of EEG data necessitate expert interpretation, a process that is both time-intensive and susceptible to human error. Recent advancements in Deep Learning (DL) have shown promise in EEG analysis, offering new avenues for seizure type classification. This study introduces two innovative deep learning architectures designed for seizure type classification within a seven-class framework: Network 1D Raw, which applies 1D Convolutions to Raw EEG signals, and Network 2D Conv, utilizing 2D convolutions on pre-computed spectrograms. Both architectures employ Separable Convolutions to enhance feature extraction efficiency, with the Network 1D Raw also incorporating a dilation rate technique for expanded analysis. Tested on the Temple University Hospital Seizure (TUSZ) dataset, these methods are evaluated using inter-patient 3-fold cross-validation. The Network 1D Raw achieved a weighted f1-score of $0.611 \pm 0.037$ , while the Network 2D Conv reached $0.599 \pm 0.052$ , both surpassing existing benchmarks. The Network 1D Raw demonstrated superior classification for Absence Seizure (ABSZ), Focal Non-specific Seizures (FNSZ), and Generalized Non-specific Seizures types (GNSZ) with AUPRs of $0.9400 \pm 0.0500$ , $0.8100\pm 0.0300$ , and $0.5500\pm 0.1800$ , with Network 2D Conv excelling in FNSZ Seizures and GNSZ classes with AUPRs of $0.8100 \pm 0.1100$ and $0.4000 \pm 0.0200$ . Our DL models advance seizure type classification, blending efficiency with accuracy. Network 1D Raw’s compact design suits low-resource environments, aiding quick, precise diagnoses. Future work will focus on expanding dataset diversity, particularly for underrepresented seizure types, to further refine classification performance.

The diagnosis and treatment of brain diseases represent the forefront of brain science research, with EEG-related research occupying a uniquely significant position. In recent years, deep learning technology has been widely applied to the study of EEG signals, yet the integration of information from multiple EEG channels remains a challenging task. Based on the dynamic routing algorithm, this study established a deep neural network. Subsequently, leveraging this network, an epileptic seizure recognition method, VarChanNet, was proposed. Seizure recognition experiments were conducted using the Bonn and CHB-MIT databases. The experimental results demonstrate that the proposed VarChanNet method maintains high recognition accuracy even when the number of channels involved in the recognition process changes. It reliably functions on both the Bonn and CHB-MIT databases, indicating its potential for generalization. Furthermore, the method provides recommendations for channel selection during the recognition process. For instance, in the case of CHB-MIT, Channel 21 can be selected for single-channel recognition, Channels 2 and 3 for dual-channel, and Channels 1, 2, and 3 for triple-channel epileptic seizure recognition. In a word, the proposed VarChanNet method enables the fusion of information from different EEG channels, supporting recognition tasks even when the number of channels varies. It offers a new perspective for EEG analysis and holds the potential for generalization.

The olfactory system enables humans to smell different odors, which are closely related to emotions. The high temporal resolution and non-invasiveness of Electroencephalogram (EEG) make it suitable to objectively study human preferences for odors. Effectively learning the temporal dynamics and spatial information from EEG is crucial for detecting odor-induced emotional valence. In this paper, we propose a deep learning architecture called Temporal Attention with Spatial Autoencoder Network (TASA) for predicting odor-induced emotions using EEG. TASA consists of a filter-bank layer, a spatial encoder, a time segmentation layer, a Long Short-Term Memory (LSTM) module, a multi-head self-attention (MSA) layer, and a fully connected layer. We improve upon the previous work by utilizing a two-phase learning framework, using the autoencoder module to learn the spatial information among electrodes by reconstructing the given input with a latent representation in the spatial dimension, which aims to minimize information loss compared to spatial filtering with CNN. The second improvement is inspired by the continuous nature of the olfactory process; we propose to use LSTM-MSA in TASA to capture its temporal dynamics by learning the intercorrelation among the time segments of the EEG. TASA is evaluated on an existing olfactory EEG dataset and compared with several existing deep learning architectures to demonstrate its effectiveness in predicting olfactory-triggered emotional responses. Interpretability analyses with DeepLIFT also suggest that TASA learns spatial-spectral features that are relevant to olfactory-induced emotion recognition.

Sleep is vital to our daily activity. Lack of proper sleep can impair functionality and overall health. While stress is known for its detrimental impact on sleep quality, the precise effect of pre-sleep stress on subsequent sleep structure remains unknown. This study introduced a novel approach to study the pre-sleep stress effect on sleep structure, specifically slow-wave sleep (SWS) deficiency. To achieve this, we selected forehead resting EEG immediately before and upon sleep onset to extract stress-related neurological markers through power spectra and entropy analysis. These markers include beta/delta correlation, alpha asymmetry, fuzzy entropy (FuzzEn) and spectral entropy (SpEn). Fifteen subjects were included in this study. Our results showed that subjects lacking SWS often exhibited signs of stress in EEG, such as an increased beta/delta correlation, higher alpha asymmetry, and increased FuzzEn in frontal EEG. Conversely, individuals with ample SWS displayed a weak beta/delta correlation and reduced FuzzEn. Finally, we employed several supervised learning models and found that the selected neurological markers can predict subsequent SWS deficiency. Our investigation demonstrated that the classifiers could effectively predict varying levels of slow-wave sleep (SWS) from pre-sleep EEG segments, achieving a mean balanced accuracy surpassing 0.75. The SMOTE-Tomek resampling method could improve the performance to 0.77. This study suggests that stress-related neurological markers derived from pre-sleep EEG can effectively predict SWS deficiency. Such information can be integrated with existing sleep-improving techniques to provide a personalized sleep forecasting and improvement solution.

Robot-assisted motor training is applied for neurorehabilitation in stroke patients, using motor imagery (MI) as a representative paradigm of brain-computer interfaces to offer real-life assistance to individuals facing movement challenges. However, the effectiveness of training with MI may vary depending on the location of the stroke lesion, which should be considered. This paper introduces a multi-task electroencephalogram-based heterogeneous ensemble learning (MEEG-HEL) specifically designed for cross-subject training. In the proposed framework, common spatial patterns were used for feature extraction, and the features according to stroke lesions are shared and selected through sequential forward floating selection. The heterogeneous ensembles were used as classifiers. Nine patients with chronic ischemic stroke participated, engaging in MI and motor execution (ME) paradigms involving finger tapping. The classification criteria for the multi-task were established in two ways, taking into account the characteristics of stroke patients. In the cross-subject session, the first involved a direction recognition task for two-handed classification, achieving a performance of 0.7419 (±0.0811) in MI and 0.7061 (±0.1270) in ME. The second task focused on motor assessment for lesion location, resulting in a performance of 0.7457 (±0.1317) in MI and 0.6791 (±0.1253) in ME. Comparing the specific-subject session, except for ME on the motor assessment task, performance on both tasks was significantly higher than the cross-subject session. Furthermore, classification performance was similar to or statistically higher in cross-subject sessions compared to baseline models. The proposed MEEG-HEL holds promise in improving the practicality of neurorehabilitation in clinical settings and facilitating the detection of lesions.

Canonical correlation analysis (CCA), Multivariate synchronization index (MSI), and their extended methods have been widely used for target recognition in Brain-computer interfaces (BCIs) based on Steady State Visual Evoked Potentials (SSVEP), and covariance calculation is an important process for these algorithms. Some studies have proved that embedding time-local information into the covariance can optimize the recognition effect of the above algorithms. However, the optimization effect can only be observed from the recognition results and the improvement principle of time-local information cannot be explained. Therefore, we propose a time-local weighted transformation (TT) recognition framework that directly embeds the time-local information into the electroencephalography signal through weighted transformation. The influence mechanism of time-local information on the SSVEP signal can then be observed in the frequency domain. Low-frequency noise is suppressed on the premise of sacrificing part of the SSVEP fundamental frequency energy, the harmonic energy of SSVEP is enhanced at the cost of introducing a small amount of high-frequency noise. The experimental results show that the TT recognition framework can significantly improve the recognition ability of the algorithms and the separability of extracted features. Its enhancement effect is significantly better than the traditional time-local covariance extraction method, which has enormous application potential.

The objective of this paper is to develop a novel emotion recognition system from electroencephalogram (EEG) signals using effective connectivity and deep learning methods. Emotion recognition is an important task for various applications such as human-computer interaction and, mental health diagnosis. The paper aims to improve the accuracy and robustness of emotion recognition by combining different effective connectivity (EC) methods and pre-trained convolutional neural networks (CNNs), as well as long short-term memory (LSTM). EC methods measure information flow in the brain during emotional states using EEG signals. We used three EC methods: transfer entropy (TE), partial directed coherence (PDC), and direct directed transfer function (dDTF). We estimated a fused image from these methods for each five-second window of 32-channel EEG signals. Then, we applied six pre-trained CNNs to classify the images into four emotion classes based on the two-dimensional valence-arousal model. We used the leave-one-subject-out cross-validation strategy to evaluate the classification results. We also used an ensemble model to select the best results from the best pre-trained CNNs using the majority voting approach. Moreover, we combined the CNNs with LSTM to improve recognition performance. We achieved the average accuracy and F-score of 98.76%, 98.86%, 98.66 and 98.88% for classifying emotions using DEAP and MAHNOB-HCI datasets, respectively. Our results show that fused images can increase the accuracy and that an ensemble and combination of pre-trained CNNs and LSTM can achieve high accuracy for automated emotion recognition. Our model outperformed other state-of-the-art systems using the same datasets for four-class emotion classification.

Perceived stress is the predominant mental health concern in this age of development and progress. Timely and precise recognition of perceived stress is vital for appropriate and effective treatment. Previously, many studies intended to classify perceived stress with considerable accuracy using electroencephalography (EEG). This research study aims to accurately classify perceived stress with a lesser number of electrodes by using significant features identified by the information-gain technique. The dataset employed in this research comprises EEG signals from twenty-eight participants in a closed-eye state, utilizing commercially available Muse EEG headbands. We have preprocessed EEG data and performed analysis on EEG data spanning 210 seconds. Two segmentation techniques have been employed: non-overlap and overlap. After segmentation, twenty-time domain features have been extracted, and feature selection has been performed using an information-gain-based method. It has been applied to enhance feature relevance and to reduce the dimensionality of feature vectors. To label the EEG data into stressed and non-stressed groups, the Perceived Stress Scale (PSS) questionnaire has been utilized. Employing a Random Forest classifier alongside the overlap segmentation technique, our proposed method attained a maximum classification accuracy of 93.8%. This accuracy surpasses existing stress classification schemes found in the literature with a similar number of electrodes.

Predicting the potential for recovery of motor function in stroke patients who undergo specific rehabilitation treatments is an important and major challenge. Recently, electroencephalography (EEG) has shown potential in helping to determine the relationship between cortical neural activity and motor recovery. EEG recorded in different states could more accurately predict motor recovery than single-state recordings. Here, we design a multi-state (combining eyes closed, EC, and eyes open, EO) fusion neural network for predicting the motor recovery of patients with stroke after EEG-brain-computer-interface (BCI) rehabilitation training and use an explainable deep learning method to identify the most important features of EEG power spectral density and functional connectivity contributing to prediction. The prediction accuracy of the multi-states fusion network was 82%, significantly improved compared with a single-state model. The neural network explanation result demonstrated the important region and frequency oscillation bands. Specifically, in those two states, power spectral density and functional connectivity were shown as the regions and bands related to motor recovery in frontal, central, and occipital. Moreover, the motor recovery relation in bands, the power spectrum density shows the bands at delta and alpha bands. The functional connectivity shows the delta, theta, and alpha bands in the EC state; delta, theta, and beta mid at the EO state are related to motor recovery. Multi-state fusion neural networks, which combine multiple states of EEG signals into a single network, can increase the accuracy of predicting motor recovery after BCI training, and reveal the underlying mechanisms of motor recovery in brain activity.

Attention deficit hyperactivity disorder (ADHD) is a chronic neurological and psychiatric disorder that affects children during their development. To find neural patterns for ADHD and provide subjective features as decision references to assist specialists and physicians. Many studies have been devoted to investigating the neural dynamics of the brain through resting-state or continuous performance tests (CPT) with EEG or functional magnetic resonance imaging (fMRI). The present study used coherence, which is one of the functional connectivity (FC) methods, to analyze the neural patterns of children and adolescents (8-16 years old) under CPT and continuous auditory test of attention (CATA) task. In the meantime, electroencephalography (EEG) oscillations were recorded by a wireless brain-computer interface (BCI). 72 children were enrolled, of which 53 participants were diagnosed with ADHD and 19 presented to be typical developing (TD). The experimental results exhibited a higher difference in alpha and theta bands between the TD group and the ADHD group. While the differences between the TD group and the ADHD group in all four frequency domains were greater than under CPT conditions. Statistically significant differences ( $\text{p}< 0.05$ ) were observed between the ADHD and TD groups in the alpha rhythm during the CATA task in the short-range of coherence. For the temporal lobe FC during the CATA task, the TD group exhibited statistically significantly FC ( $\text{p}< 0.05$ ) in the alpha rhythm compared to the ADHD group. These findings offering new possibilities for more techniques and diagnostic methods in finding more ADHD features. The differences in alpha and beta frequencies were more pronounced in the ADHD group during the CPT task compared to the CATA task. Additionally, the disparities in brain activity were more evident across delta, theta, alpha and beta frequency domains when the task given was a CATA as opposed to a CPT. The findings presented the underlying mechanisms of the FC differences between children and adolescents with ADHD. Moreover, these findings should extend to use machine learning approaches to assist the ADHD classification and diagnosis.

Transcranial electrical stimulation has demonstrated the potential to enhance cognitive functions such as working memory, learning capacity, and attentional allocation. Recently, it was shown that periodic stimulation within a specific duration could augment the human brain’s neuroplasticity. This study investigates the effects of repetitive transcranial alternating current stimulation (tACS; 1 mA, 5 Hz, 2 min duration) on cognitive function, functional connectivity, and topographic changes using both electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). Fifteen healthy subjects were recruited to measure brain activity in the pre-, during-, and post-stimulation sessions under tACS and sham stimulation conditions. Fourteen trials of working memory tasks and eight repetitions of tACS/sham stimulation with a 1-minute intersession interval were applied to the frontal cortex of the participants. The working memory score, EEG band-wise powers, EEG topography, concentration changes of oxygenated hemoglobin, and functional connectivity (FC) were individually analyzed to quantify the behavioral and neurophysiological effects of tACS. Our results indicate that tACS increases: i) behavioral scores (i.e., 15.08, ${p} < 0.001$ ) and EEG band-wise powers (i.e., theta and beta bands) compared to the sham stimulation condition, ii) FC of both EEG-fNIRS signals, especially in the large-scale brain network communication and interhemispheric connections, and iii) the hemodynamic response in comparison to the pre-stimulation session and the sham condition. Conclusively, the repetitive theta-band tACS stimulation improves the working memory capacity regarding behavioral and neuroplasticity perspectives. Additionally, the proposed fNIRS biomarkers (mean, slope), EEG band-wise powers, and FC can be used as neuro-feedback indices for closed-loop brain stimulation.

Emotion recognition based on electroencephalography (EEG) signals has garnered substantial attention in recent years and finds extensive applications in the domains of medicine and psychology. However, individual differences in EEG signals pose a challenge to accurate emotion recognition and limit the widespread adoption of such techniques. To address this issue, this study proposes a model that combines random forest weights (RFWs) and four-dimensional convolutional recurrent neural network (4DCRNN) to minimize individual differences and captures emotion-relevant information. By integrating, the proposed model aims to improve the accuracy and generalization capability of emotion recognition. To evaluate the performance of the proposed model, experiments were conducted using the DEAP and SEED datasets. The results demonstrate the effectiveness of the RFW-4DCRNN in emotion recognition. Specifically, the proposed model achieves mean accuracy of 94.98% and 94.21% for Subject-dependent recognition using the DEAP and SEED datasets, respectively. For Subject-independent emotion recognition, the model achieved mean accuracy of 81.70% and 91.12% using two datasets, respectively. The result highlights the capability of the RFW-4DCRNN to effectively recognize emotions and improves generalization performance. Overall, this study presents an approach to addressing individual differences in EEG-based emotion recognition. The RFW-4DCRNN demonstrates promising results in terms of accuracy and generalization capability, offering potential for the advancement and application of emotion recognition techniques.

The rapidly evolving landscape of artificial intelligence (AI) and machine learning has placed data at the forefront of healthcare innovation. Electroencephalography (EEG) has gained significant attention for its potential to revolutionize healthcare applications. However, the effective utilization of EEG data in advancing medical diagnoses and treatment hinges on the availability and quality of relevant datasets. In this context, we conducted a scoping review to explore the wealth of EEG datasets designed for healthcare applications. This review serves as a critical exploration of the current landscape, aiming to identify datasets related to healthcare conditions while assessing their reusability. Our findings highlight both the opportunities and limitations in the wealth of open access EEG datasets. Available. As AI increasingly relies on high-quality, well labelled data, barriers impeding the sharing and utilization of EEG data for healthcare (such as lack of comprehensive documentation or adherence to FAIR principles) must be addressed so as to leverage the potential of advanced deep learning models to unlock new possibilities for diagnosis and analysis of a wide array of medical conditions.

Micro-expression recognition based on ima- ges has made some progress, yet limitations persist. For instance, image-based recognition of micro-expressions is affected by factors such as ambient light, changes in head posture, and facial occlusion. The high temporal resolution of electroencephalogram (EEG) technology can record brain activity associated with micro-expressions and identify them objectively from a neurophysiological standpoint. Accordingly, this study introduces a novel method for recognizing micro-expressions using node efficiency features of brain networks derived from EEG signals. We designed a real-time Supervision and Emotional Expression Suppression (SEES) experimental paradigm to collect video and EEG data reflecting micro- and macro-expression states from 70 participants experiencing positive emotions. By constructing functional brain networks based on graph theory, we analyzed the network efficiencies at both macro- and micro-levels. The participants exhibited lower connection density, global efficiency, and nodal efficiency in the alpha, beta, and gamma networks during micro-expressions compared to macro-expressions. We then selected the optimal subset of nodal efficiency features using a random forest algorithm and applied them to various classifiers, including Support Vector Machine (SVM), Gradient-Boosted Decision Tree (GBDT), Logistic Regression (LR), Random Forest (RF), and eXtreme Gradient Boosting (XGBoost). These classifiers achieved promising accuracy in micro-expression recognition, with SVM exhibiting the highest accuracy of 92.6% when 15 channels were selected. This study provides a new neuroscientific indicator for recognizing micro-expressions based on EEG signals, thereby broadening the potential applications for micro-expression recognition.