Search

Decoding natural language from noninvasive brain signals has been an exciting topic with the potential to expand the applications of brain-computer interface (BCI) systems. However, current methods face limitations in decoding sentences from electroencephalography (EEG) signals. Improving decoding performance requires the development of a more effective encoder for the EEG modality. Nonetheless, learning generalizable EEG representations remains a challenge due to the relatively small scale of existing EEG datasets. In this paper, we propose enhancing the EEG encoder to improve subsequent decoding performance. Specifically, we introduce the discrete Conformer encoder (D-Conformer) to transform EEG signals into discrete representations and bootstrap the learning process by imposing EEG-language alignment from the early training stage. The D-Conformer captures both local and global patterns from EEG signals and discretizes the EEG representation, making the representation more resilient to variations, while early-stage EEG-language alignment mitigates the limitations of small EEG datasets and facilitates the learning of the semantic representations from EEG signals. These enhancements result in improved EEG representations and decoding performance. We conducted extensive experiments and ablation studies to thoroughly evaluate the proposed method. Utilizing the D-Conformer encoder and bootstrapping training strategy, our approach demonstrates superior decoding performance across various tasks, including word-level, sentence-level, and sentiment-level decoding from EEG signals. Specifically, in word-level classification, we show that our encoding method produces more distinctive representations and higher classification performance compared to the EEG encoders from existing methods. At the sentence level, our model outperformed the baseline by 5.45%, achieving a BLEU-1 score of 42.31%. Furthermore, in sentiment classification, our model exceeded the baseline by 14%, achieving a sentiment classification accuracy of 69.3%.

Previous studies have reported a role of alterations in the brain’s inhibitory control mechanism in addiction. Mounting evidence from neuroimaging studies indicates that its key components can be evaluated with brain oscillations and connectivity during inhibitory control. In this study, we developed an internet-related stop-signal task with electroencephalography (EEG) signal recorded to investigate inhibitory control. Healthy controls and participants with Internet addiction were recruited to participate in the internet-related stop-signal task with 19-channel EEG signal recording, and the corresponding event-related potentials and spectral perturbations were analyzed. Brain effective connections were also evaluated using direct directed transfer function. The results showed that, relative to the healthy controls, participants with Internet addiction had increased Stop-P3 during inhibitory control, suggesting that they have an altered neural mechanism in impulsive control. Furthermore, participants with Internet addiction showed increased low-frequency synchronization and decreased alpha and beta desynchronization in the middle and right frontal regions compared to healthy controls. Aberrant brain effective connectivity was also observed, with increased occipital-parietal and intra-occipital connections, as well as decreased frontal-paracentral connection in participants with Internet addiction. These results suggest that physiological signals are essential in future implementations of cognitive assessment of Internet addiction to further investigate the underlying mechanisms and effective biomarkers.

Social interaction enables the smooth progression of our daily lives. Mounting evidence from recent hyperscanning neuroimaging studies indicates that key components of social behavior can be evaluated using inter-brain oscillations and connectivity. However, mapping out inter-brain networks and developing neurocognitive theories that explain how humans co-create and share information during social interaction remains challenging. In this study, we developed a jigsaw puzzle-solving game with hyperscanning electroencephalography (EEG) signals recorded to investigate inter-brain activities during social interactions involving cooperation and competition. Participants were recruited and paired into dyads to participate in the multiplayer jigsaw puzzle game with 32-channel EEG signals recorded. The corresponding event-related potentials (ERPs), brain oscillations, and inter-brain functional connectivity were analyzed. The results showed different ERP morphologies of P3 patterns in competitive and cooperative contexts, and brain oscillations in the low-frequency band may be an indicator of social cognitive activities. Furthermore, increased inter-brain functional connectivity in the delta, theta, alpha, and beta frequency bands was observed in the competition mode compared to the cooperation mode. By presenting comparable and valid hyperscanning EEG results alongside those of previous studies using traditional paradigms, this study demonstrates the potential of utilizing hyperscanning techniques in real-life game-playing scenarios to quantitatively assess social cognitive interactions involving cooperation and competition. Our approach offers a promising platform with potential applications in the flexible assessment of psychiatric disorders related to social functioning.

Drowsy driving is one of the primary causes of driving fatalities. Electroencephalography (EEG), a method for detecting drowsiness directly from brain activity, has been widely used for detecting driver drowsiness in real-time. Recent studies have revealed the great potential of using brain connectivity graphs constructed based on EEG data for drowsy state predictions. However, traditional brain connectivity networks are irrelevant to the downstream prediction tasks. This article proposes a connectivity-aware graph neural network (CAGNN) using a self-attention mechanism that can generate task-relevant connectivity networks via end-to-end training. Our method achieved an accuracy of 72.6% and outperformed other convolutional neural networks (CNNs) and graph generation methods based on a drowsy driving dataset. In addition, we introduced a squeeze-and-excitation (SE) block to capture important features and demonstrated that the SE attention score can reveal the most important feature band. We compared our generated connectivity graphs in the drowsy and alert states and found drowsiness connectivity patterns, including significantly reduced occipital connectivity and interregional connectivity. Additionally, we performed a post hoc interpretability analysis and found that our method could identify drowsiness features such as alpha spindles. Our code is available online at https://github.com/ALEX95GOGO/CAGNN.

Orthogonal frequency-division multiplexing (OFDM) is a key technology for cellular and Wi-Fi systems, but its performance may be degraded by hardware impairments. Existing works focus mostly on single hardware impairment in OFDM systems, without considering the joint effect of hardware impairments on the entire system. In this paper, hardware impairments including nonlinear power amplification, clipping, in-phase/quadrature-phase (IQ) imbalance, phase noise, carrier frequency offset, and sampling clock offset in OFDM systems are simultaneously considered. We propose end-to-end deep learning-based designs, which jointly optimize transmitter and receiver, to effectively mitigate the performance loss due to hardware impairments. For single-antenna systems and $2\times 2$ multiple-input and multiple-output (MIMO) systems, the proposed design featuring the dense layer neural network (DLNN) significantly outperforms traditional impairment-mitigating methods under both the additive white Gaussian noise (AWGN) channel and the Rayleigh fading channel. Meanwhile, the complexity of the proposed scheme is six times smaller. For $2\times 4$ MIMO systems, the proposed design featuring the residual dense convolution dense neural network (ResNet-DCDNN) outperforms the traditional methods by a large margin. Additionally, transfer learning is applied to effectively address the issue of time-varying impairment levels.

Identifying meaningful brain activities is critical in brain-computer interface (BCI) applications. Recently, an increasing number of neural network approaches have been proposed to recognize EEG signals. However, these approaches depend heavily on using complex network structures to improve the performance of EEG recognition and suffer from the deficit of training data. Inspired by the waveform characteristics and processing methods shared between EEG and speech signals, we propose Speech2EEG, a novel EEG recognition method that leverages pretrained speech features to improve the accuracy of EEG recognition. Specifically, a pretrained speech processing model is adapted to the EEG domain to extract multichannel temporal embeddings. Then, several aggregation methods, including the weighted average, channelwise aggregation, and channel-and-depthwise aggregation, are implemented to exploit and integrate the multichannel temporal embeddings. Finally, a classification network is used to predict EEG categories based on the integrated features. Our work is the first to explore the use of pretrained speech models for EEG signal analysis as well as the effective ways to integrate the multichannel temporal embeddings from the EEG signal. Extensive experimental results suggest that the proposed Speech2EEG method achieves state-of-the-art performance on two challenging motor imagery (MI) datasets, the BCI IV-2a and BCI IV-2b datasets, with accuracies of ${89}.{5}\%$ and ${84}.{07}\%$ , respectively. Visualization analysis of the multichannel temporal embeddings show that the Speech2EEG architecture can capture useful patterns related to MI categories, which can provide a novel solution for subsequent research under the constraints of a limited dataset scale.

Motor imagery-based brain-computer interface (MI-BCI) currently represents a new trend in rehabilitation. However, individual differences in the responsive frequency bands and a poor understanding of the communication between the ipsilesional motor areas and other regions limit the use of MI-BCI therapy. Objective: Bimanual training has recently attracted attention as it achieves better outcomes as compared to repetitive one-handed training. This study compared the effects of three MI tasks with different visual feedback. Methods: Fourteen healthy subjects performed single hand motor imagery tasks while watching single static hand (traditional MI), single hand with rotation movement (rmMI), and bimanual coordination with a hand pedal exerciser (bcMI). Functional connectivity is estimated by Transfer Entropy (TE) analysis for brain information flow. Results: Brain connectivity of conducting three MI tasks showed that the bcMI demonstrated increased communications from the parietal to the bilateral prefrontal areas and increased contralateral connections between motor-related zones and spatial processing regions. Discussion/Conclusion: The results revealed bimanual coordination operation events increased spatial information and motor planning under the motor imagery task. And the proposed bimanual coordination MI-BCI (bcMI-BCI) can also achieve the effect of traditional motor imagery tasks and promotes more effective connections with different brain regions to better integrate motor-cortex functions for aiding the development of more effective MI-BCI therapy. Clinical and Translational Impact Statement The proposed bcMI-BCI provides more effective connections with different brain areas and integrates motor-cortex functions to promote motor imagery rehabilitation for patients’ impairment.

Unmanned aerial vehicle (UAV) is lately one of many popular research topics. High variety of its usage makes it attentive to be studied on its construction or the control. However, without knowing the energy that will be consumed in each mission, the available flight duration will be unknown and the usages of this vehicle will be limited. A mission-based black box modeling of UAV's energy consumption prediction was proposed in this paper. The setup consists of ArduPilot with Mission Planner Firmware installed to a custom built hexarotor. The method consists of three consecutive steps: data collection, data preprocessing, and regression. To collect the required data, flight patterns that contained several types of movements were defined, where the flight data log that contained missions, GPS, and battery, was collected. The preprocessing included the movement separation and also included the acceleration and the deceleration of horizontal movement. Finally, the regression was done using the Elastic Net Regression from Sklearn. The model was then tested on two flight patterns to simulate a surveillance application of a UAV and could predict with 98.773% mean of energy accuracy of the missions that started from the takeoff and ended with the return to launch command.

Polysomnography (PSG) is considered the gold standard in the diagnosis of obstructive sleep apnea (OSA). The diagnosis of OSA requires an overnight sleep experiment in a laboratory. However, due to limitations in relation to the number of labs and beds available, patients often need to wait a long time before being diagnosed and eventually treated. In addition, the unfamiliar environment and restricted mobility when a patient is being tested with a polysomnogram may disturb their sleep, resulting in an incomplete or corrupted test. Therefore, it is posed that a PSG conducted in the patient's home would be more reliable and convenient. The Internet of Things (IoT) plays a vital role in the e-Health system. In this paper, we implement an IoT-based wireless polysomnography system for sleep monitoring, which utilizes a batterypowered, miniature, wireless, portable, and multipurpose recorder. A Java-based PSG recording program in the personal computer is designed to save several bio-signals and transfer them into the European data format. These PSG records can be used to determine a patient's sleep stages and diagnose OSA. This system is portable, lightweight, and has low power-consumption. To demonstrate the feasibility of the proposed PSG system, a comparison was made between the standard PSG-Alice 5 Diagnostic Sleep System and the proposed system. Several healthy volunteer patients participated in the PSG experiment and were monitored by both the standard PSG-Alice 5 Diagnostic Sleep System and the proposed system simultaneously, under the supervision of specialists at the Sleep Laboratory in Taipei Veteran General Hospital. A comparison of the results of the time-domain waveform and sleep stage of the two systems shows that the proposed system is reliable and can be applied in practice. The proposed system can facilitate the long-term tracing and research of personal sleep monitoring at home.