Search

Pandemic-related sleep disorders affect human physical and mental health. The artificial intelligence (AI) based sleep staging with multimodal electrophysiological signals help people diagnose and treat sleep disorders. However, the existing AI-based methods could not capture more discriminative modalities and adaptively correlate these multimodal features. This paper introduces a multimodal attention network (MMASleepNet) to efficiently extract, perceive and fuse multimodal features of electrophysiological signals. The MMASleepNet has a multi-branch feature extraction (MBFE) module followed by an attention-based feature fusing (AFF) module. In the MBFE module, branches are designed to extract multimodal signals' temporal and spectral features. Each branch has two-stream convolutional networks with a unique kernel to perceive features of different time scales. The AFF module contains a modal-wise squeeze and excitation (SE) block to adjust the weights of modalities with more discriminative features and a Transformer encoder (TE) to generate attention matrices and extract the inter-dependencies among multimodal features. Our MMASleepNet outperforms state-of-the-art models in terms of different evaluation matrices on the datasets of Sleep-EDF and ISRUC-Sleep. The implementation code is available at: https://github.com/buptantEEG/MMASleepNet/.

Contrastive learning, as an unsupervised technique, has emerged as a prominent method in time series representation learning tasks, serving as a viable solution to the scarcity of annotated data. However, the application of data augmentation methods during training can distort the distribution of raw data. This discrepancy between the representations learned from augmented data in contrastive learning and those obtained from supervised learning results in an incomplete understanding of the information contained in the real data from the trained encoder. We refer to this as the data augmentation bias (DAB), representing the disparity between the two sets of learned representations. To mitigate the influence of DAB, we propose a DAB-aware contrastive learning framework for time series representation (DABaCLT). This framework leverages a raw features stream (RFS) to extract features from raw data, which are then combined with augmented data to create positive and negative pairs for DAB-aware contrastive learning. Additionally, we introduce a DAB-minimizing loss function (DABMinLoss) within the contrasting module to minimize the DAB of the extracted temporal and contextual features. Our proposed method is evaluated on three time series classification tasks, including sleep staging classification (SSC) and epilepsy seizure prediction (ESP) based on EEG and human activity recognition (HAR) based on sensors signals. The experimental results demonstrate that our DABaCLT achieves strong performance in self-supervised time series representation, 0.19% to 22.95% accuracy improvement for SSC, 2.96% to 5.05% for HAR, 1.00% to 2.46% for ESP, and achieves comparable performance to the supervised approach. The source code for our framework is open-source. [ABSTRACT FROM AUTHOR]

Drowsy driving is one of the major causes in traffic accidents worldwide. Various electroencephalography (EEG)-based feature extraction methods are proposed to detect driving drowsiness, to name a few, spectral power features and fuzzy entropy features. However, most existing studies only concentrate on features in each channel separately to identify drowsiness, making them vulnerable to variability across different sessions and subjects without sufficient data. In this paper, we propose a method called Tensor Network Features (TNF) to exploit underlying structure of drowsiness patterns and extract features based on tensor network. This TNF method first introduces Tucker decomposition to tensorized EEG channel data of training set, then features of training and testing tensor samples are extracted from the corresponding subspace matrices through tensor network summation. The performance of the proposed TNF method was evaluated through a recently published EEG dataset during a sustained-attention driving task. Compared with spectral power features and fuzzy entropy features, the accuracy of TNF method is improved by 6.7% and 10.3% on average with maximum value 17.3% and 29.7% respectively, which is promising in developing practical and robust cross-session driving drowsiness detection system.

Accurate and reliable detecting of driving fatigue using Electroencephalography (EEG) signals is a method to reduce traffic accidents. So far, it is natural to cut the part of operating the steering wheel data away for achieving the relatively high accuracy in detecting driving fatigue using EEG data. However, the data segment during operating the steering wheel also contains valuable information. Moreover, operating the steering wheel is a common practice during actual driving. In this study, we utilize the part of data operating the steering wheel to detecting fatigue. The feature used is the spectral band power calculates from the data. For each experiment and each experimental participant, the data and features are divided into sessions and subjects. Using the divided features, this work performs cross-session and cross-subject verification and comparison on the two classification methods of logistic regression and multi-layer perceptron. To compare the effect, the experiment is conducted on the data both operating the steering wheel and not operating the steering wheel. The result shows that the bias between the average accuracy of two types of data is only 2.27%, and the effect of using multi-layer perceptron is 10.37% better than using logistic regression. This proves that the data segment during operating the steering wheel also contains valid information and can be used for driving fatigue detection.