Search

The application of intracranial electroencephalogram (iEEG) to predict seizures remains challenging. Although channel selection has been utilized in seizure prediction and detection studies, most of them focus on the combination with conventional machine learning methods. Thus, channel selection combined with deep learning methods can be further analyzed in the field of seizure prediction. Given this, in this work, a novel iEEG-based deep learning method of One-Dimensional Convolutional Neural Networks (1D-CNN) combined with channel increment strategy was proposed for the effective seizure prediction. First, we used 4-sec sliding windows without overlap to segment iEEG signals. Then, 4-sec iEEG segments with an increasing number of channels (channel increment strategy, from one channel to all channels) were sequentially fed into the constructed 1D-CNN model. Next, the patient-specific model was trained for classification. Finally, according to the classification results in different channel cases, the channel case with the best classification rate was selected for each patient. Our method was tested on the Freiburg iEEG database, and the system performances were evaluated at two levels (segment- and event-based levels). Two model training strategies (Strategy-1 and Strategy-2) based on the K-fold cross validation (K-CV) were discussed in our work. (1) For the Strategy-1, a basic K-CV, a sensitivity of 90.18%, specificity of 94.81%, and accuracy of 94.42% were achieved at the segment-based level. At the event-based level, an event-based sensitivity of 100%, and false prediction rate (FPR) of 0.12/h were attained. (2) For the Strategy-2, the difference from the Strategy-1 is that a trained model selection step is added during model training. We obtained a sensitivity, specificity, and accuracy of 86.23%, 96.00% and 95.13% respectively at the segment-based level. At the event-based level, we achieved an event-based sensitivity of 98.65% with 0.08/h FPR. Our method also showed a better performance in seizure prediction compared to many previous studies and the random predictor using the same database. This may have reference value for the future clinical application of seizure prediction. peerReviewed

For real-world automatic sleep-stage classification tasks, various existing deep learning-based models are biased toward the majority with a high proportion. Because of the unique sleep structure, most of the current polysomnography (PSG) datasets suffer an inherent class imbalance problem (CIP), in which the number of each sleep stage is severely unequal. In this study, we first define the class imbalance factor (CIF) to describe the level of CIP quantitatively. Afterward, we propose two balancing methods to alleviate this problem from the dataset quantity and the relationship between the class distribution and the applied model, respectively. The first one is to employ the data augmentation (DA) with the generative adversarial network (GAN) model and different intensities of Gaussian white noise (GWN) to balance samples, thereinto, GWN addition is specifically tailored to deep learning-based models, which can work on raw electroencephalogram (EEG) data while preserving their properties. In addition, we try to balance the relationship between the imbalanced class and biased network model to achieve a balanced state with the help of class distribution and neuroscience principles. We further propose an effective deep convolutional neural network (CNN) model utilizing bidirectional long short-term memory (Bi-LSTM) with single-channel EEG as the baseline. It is used for evaluating the efficiency of two balancing approaches on three imbalanced PSG datasets (CCSHS, Sleep-EDF, and Sleep-EDF-V1). The qualitative and quantitative evaluation of experimental results demonstrates that the proposed methods could not only show the superiority of class balancing through the confusion matrix and classwise metrics, but also get better N1 stage and whole stages classification accuracies compared to other state-of-the-art approaches. peerReviewed

Pain is a complex phenomenon, the experience of which varies widely across individuals. At worst, chronic pain can lead to anxiety and depression. Cost-effective strategies are urgently needed to improve the treatment of pain, and thus we propose a novel home-based pain measurement system for the longitudinal monitoring of pain experience and variation in different patients with chronic low back pain. The autonomous nervous system and audio-visual features are analyzed from heart rate signals, voice characteristics and facial expressions using a unique measurement protocol. Self-reporting is utilized for the follow-up of changes in pain intensity, induced by well-designed physical maneuvers, and for studying the consecutive trends in pain. We describe the study protocol, including hospital measurements and questionnaires and the implementation of the home measurement devices. We also present different methods for analyzing the multimodal data: electroencephalography, audio, video and heart rate. Our intention is to provide new insights using technical methodologies that will be beneficial in the future not only for patients with low back pain but also patients suffering from any chronic pain.

Funding Information: We wish to thank the reviewers and editors for the useful comments to improve the paper a lot. We thank Dr. Hiroshi Morioka for the useful discussion at the beginning of the project. L.P. was funded in part by the European Research Council (No. 678578 ). A.H. was supported by a Fellowship from CIFAR, and the Academy of Finland. The authors acknowledge the computational resources provided by the Aalto Science-IT project, and also wish to thank the Finnish Grid and Cloud Infrastructure (FGCI) for supporting this project with computational and data storage resources. | openaire: EC/H2020/678578/EU//HRMEG Resting-state magnetoencephalography (MEG) data show complex but structured spatiotemporal patterns. However, the neurophysiological basis of these signal patterns is not fully known and the underlying signal sources are mixed in MEG measurements. Here, we developed a method based on the nonlinear independent component analysis (ICA), a generative model trainable with unsupervised learning, to learn representations from resting-state MEG data. After being trained with a large dataset from the Cam-CAN repository, the model has learned to represent and generate patterns of spontaneous cortical activity using latent nonlinear components, which reflects principal cortical patterns with specific spectral modes. When applied to the downstream classification task of audio-visual MEG, the nonlinear ICA model achieves competitive performance with deep neural networks despite limited access to labels. We further validate the generalizability of the model across different datasets by applying it to an independent neurofeedback dataset for decoding the subject's attentional states, providing a real-time feature extraction and decoding mindfulness and thought-inducing tasks with an accuracy of around 70% at the individual level, which is much higher than obtained by linear ICA or other baseline methods. Our results demonstrate that nonlinear ICA is a valuable addition to existing tools, particularly suited for unsupervised representation learning of spontaneous MEG activity which can then be applied to specific goals or tasks when labelled data are scarce.

In diagnosing sleep disorders, sleep stage classification is a very essential yet time-consuming process. Various existing state-of-the-art approaches rely on hand-crafted features and multi-modality polysomnography (PSG) data, where prior knowledge is compulsory and high computation cost can be expected. Besides, it is a big challenge to handle the task with raw single-channel electroencephalogram (EEG). To overcome these shortcomings, this paper proposes an end-to-end framework with a deep neural network, namely SingleChannelNet, for automatic sleep stage classification based on raw single-channel EEG. The proposed model utilizes a 90s epoch as the textual input and employs two multi-convolution (MC) blocks and several max-average pooling (M-Apooling) layers to learn different scales of feature representations. To demonstrate the efficiency of the proposed model, we evaluate our model using different raw single-channel EEGs (C4/A1 and Fpz-Cz) on two public PSG datasets (Cleveland children’s sleep and health study: CCSHS and Sleep-EDF database expanded: Sleep-EDF). Experimental results show that the proposed architecture can achieve better overall accuracy and Cohen’s kappa (CCSHS: 90.2%–86.5%, Sleep-EDF: 86.1%–80.5%) compared with state-of-the-art approaches. Additionally, the proposed model can learn features automatically for sleep stage classification using different single-channel EEGs with distinct sampling rates and without using any hand-engineered features. peerReviewed

Seizure prediction using intracranial electroencephalogram (iEEG) is still challenging because of complicated signals in spatial and time domains. Feature selection in the spatial domain (i.e., channel selection) has been largely ignored in this field. Hence, in this paper, a novel approach of iEEG channel selection strategy combined with one-dimensional convolutional neural networks (1D-CNN) was presented for seizure prediction. First, 15-sec and 30-sec iEEG segments with an increasing number of channels (from one channel to all channels) were sequentially fed into 1D-CNN models for training and testing. Then, the channel case with the best classification rate was selected for each participant. We tested our method on the Freiburg iEEG dataset. A sensitivity of 89.03-90.84%, specificity of 98.99-99.73%, and accuracy of 98.07-98.99% were achieved at the segment-based level. At the event-based level, we attained a sensitivity of 98.48-98.85% and a false prediction rate (FPR) of 0-0.02/h. peerReviewed

Seizure prediction using intracranial electroencephalogram (iEEG) has attracted an increasing attention during recent years. iEEG signals are commonly recorded in the form of multiple channels. Many previous studies generally used the iEEG signals of all channels to predict seizures, ignoring the consideration of channel selection. In this study, a method of one-dimensional convolutional neural networks (1D-CNN) combined with channel selection strategy was proposed for seizure prediction. First, we used 30-s sliding windows to segment the raw iEEG signals. Then, the 30-s iEEG segments, which were in three channel forms (single channel, channels only from seizure onset or free zone and all channels from seizure onset and free zones), were used as the inputs of 1D-CNN for classification, and the patient-specific model was trained. Finally, the channel form with the best classification was selected for each patient. The proposed method was evaluated on the Freiburg Hospital iEEG dataset. In the situation of seizure occurrence period (SOP) of 30min and seizure prediction horizon (SPH) of 5min, 98.60% accuracy, 98.85% sensitivity and 0.01/h false prediction rate (FPR) were achieved. In the situation of SOP of 60min and SPH of 5min, 98.32% accuracy, 98.48% sensitivity and 0.01/h FPR were attained. Compared with the many existing methods using the same iEEG dataset, our method showed a better performance. peerReviewed

Ongoing electroencephalography (EEG) signals are recorded as a mixture of stimulus-elicited EEG, spontaneous EEG and noises, which poses a huge challenge to current data analyzing techniques, especially when different groups of participants are expected to have common or highly correlated brain activities and some individual dynamics. In this study, we proposed a data-driven shared and unshared feature extraction framework based on nonnegative and coupled tensor factorization, which aims to conduct group-level analysis for the EEG signals from major depression disorder (MDD) patients and healthy controls (HC) when freely listening to music. Constrained tensor factorization not only preserves the multilinear structure of the data, but also considers the common and individual components between the data. The proposed framework, combined with music information retrieval, correlation analysis, and hierarchical clustering, facilitated the simultaneous extraction of shared and unshared spatio-temporal-spectral feature patterns between/in MDD and HC groups. Finally, we obtained two shared feature patterns between MDD and HC groups, and obtained totally three individual feature patterns from HC and MDD groups. The results showed that the MDD and HC groups triggered similar brain dynamics when listening to music, but at the same time, MDD patients also brought some changes in brain oscillatory network characteristics along with music perception. These changes may provide some basis for the clinical diagnosis and the treatment of MDD patients.

Spatial source phase, the phase information of spatial maps extracted from functional magnetic resonance imaging (fMRI) data by data-driven methods such as independent component analysis (ICA), has rarely been studied. While the observed phase has been shown to convey unique brain information, the role of spatial source phase in representing the intrinsic activity of the brain is yet not clear. This study explores the spatial source phase for identifying spatial differences between patients with schizophrenia (SZs) and healthy controls (HCs) using complex-valued resting-state fMRI data from 82 individuals. ICA is first applied to preprocess fMRI data, and post-ICA phase de-ambiguity and denoising are then performed. The ability of spatial source phase to characterize spatial differences is examined by the homogeneity of variance test (voxel-wise F-test) with false discovery rate correction. Resampling techniques are performed to ensure that the observations are significant and reliable. We focus on two components of interest widely used in analyzing SZs, including the default mode network (DMN) and auditory cortex. Results show that the spatial source phase exhibits more significant variance changes and higher sensitivity to the spatial differences between SZs and HCs in the anterior areas of DMN and the left auditory cortex, compared to the magnitude of spatial activations. Our findings show that the spatial source phase can potentially serve as a new brain imaging biomarker and provide a novel perspective on differences in SZs compared to HCs, consistent with but extending previous work showing increased variability in patient data.

The study of brain network interactions during naturalistic stimuli facilitates a deeper understanding of human brain function. To estimate large-scale brain networks evoked with naturalistic stimuli, a tensor component analysis (TCA) based framework was used to characterize shared spatio-temporal patterns across subjects in a purely data-driven manner. In this framework, a third-order tensor is constructed from the timeseries extracted from all brain regions from a given parcellation, for all participants, with modes of the tensor corresponding to spatial distribution, time series and participants. TCA then reveals spatially and temporally shared components, i.e., evoked networks with the naturalistic stimuli, their time courses of activity and subject loadings of each component. To enhance the reproducibility of the estimation with the adaptive TCA algorithm, a novel spectral clustering method, tensor spectral clustering, was proposed and applied to evaluate the stability of the TCA algorithm. We demonstrated the effectiveness of the proposed framework via simulations and real fMRI data collected during a motor task with a traditional fMRI study design. We also applied the proposed framework to fMRI data collected during passive movie watching to illustrate how reproducible brain networks are evoked by naturalistic movie viewing. peerReviewed

Sleep scoring is a fundamental but time-consuming process in any sleep laboratory. Automatic sleep scoring is crucial and urgent to help address the increasing unmet need for sleep research. Therefore, this paper aims to develop an end-to-end deep learning architecture using raw polysomnographic recordings to automate sleep scoring. The proposed model adopts two-dimensional convolutional neural networks (2D-CNN) to automatically learn features from multi-modality signals, together with a "squeeze and excitation" block for recalibrating channel-wise feature responses. The learnt representations are finally fed to a softmax classifier to generate predictions for each sleep stage. The model performance is evaluated on two public sleep datasets (SHHS and Sleep-EDF) with different available channels. The results have shown that our model achieves an overall accuracy of 85.2% on the SHHS dataset and an accuracy of 85% on the Sleep-EDF dataset. We have also demonstrated that the proposed architecture not only is able to handle various numbers of input channels and several signal modalities from different datasets but also exhibits short runtimes and low computational cost. peerReviewed

Funding Information: The work of KN was supported by the CRoNoS COST Action IC1408 and the Austrian Science Fund P31881-N32. The work of ST was supported by the CRoNoS COST Action IC1408. The work of JV was supported by Academy of Finland (grant 321883). We would like to thank the anonymous reviewers for their comments which improved the paper and package considerably. Publisher Copyright: © 2021, American Statistical Association. All rights reserved. Multivariate time series observations are increasingly common in multiple fields of science but the complex dependencies of such data often translate into intractable models with large number of parameters. An alternative is given by first reducing the dimension of the series and then modelling the resulting uncorrelated signals univariately, avoiding the need for any covariance parameters. A popular and effective framework for this is blind source separation. In this paper we review the dimension reduction tools for time series available in the R package tsBSS. These include methods for estimating the signal dimension of second-order stationary time series, dimension reduction techniques for stochastic volatility models and supervised dimension reduction tools for time series regression. Several examples are provided to illustrate the functionality of the package.

Epileptic seizure detection using scalp electroencephalogram (sEEG) and intracranial electroencephalogram (iEEG) has attracted widespread attention in recent two decades. The accurate and rapid detection of seizures not only reflects the efficiency of the algorithm, but also greatly reduces the burden of manual detection during long-term electroencephalogram (EEG) recording. In this work, a stacked one-dimensional convolutional neural network (1D-CNN) model combined with a random selection and data augmentation (RS-DA) strategy is proposed for seizure onset detection. Firstly, we segmented the long-term EEG signals using 2-sec sliding windows. Then, the 2-sec interictal and ictal segments were classified by the stacked 1D-CNN model. During model training, a RS-DA strategy was applied to solve the problem of sample imbalance, and the patient-specific model was trained with event-based K-fold (K is the number of seizures per patient) cross validation for detecting all seizures of each patient. Finally, we evaluated the performances of the proposed approach in the two levels: the segment-based level and the event-based level. The proposed method was tested on two long-term EEG datasets: the CHB-MIT sEEG dataset and the SWEC-ETHZ iEEG dataset. For the CHB-MIT sEEG dataset, we achieved 88.14 sensitivity, 99.62 specificity and 99.54 accuracy in the segment-based level. From the perspective of the event-based level, 99.31 sensitivity, 0.2/h false detection rate (FDR) and mean 8.1-sec latency were achieved. For the SWEC-ETHZ iEEG dataset, in the segment-based level, 90.09 sensitivity, 99.81 specificity and 99.73 accuracy were obtained. In the event-based level, 97.52 sensitivity, 0.07/h FDR and mean 13.2-sec latency were attained. From these results, we can see that our method can effectively use both sEEG and iEEG data to detect epileptic seizures, and this may provide a reference for the clinical application of seizure onset detection. peerReviewed

Proprioceptive paired-stimulus paradigm was used for 30 children (10–17 years) and 21 adult (25–45 years) volunteers in magnetoencephalography (MEG). Their right index finger was moved twice with 500-ms interval every 4 ± 25 s (repeated 100 times) using a pneumatic-movement actuator. Spatial-independent component analysis (ICA) was applied to identify stimulus-related components from MEG cortical responses. Clustering was used to identify spatiotemporally consistent components across subjects. We found a consistent primary response in the primary somatosensory (SI) cortex with similar gating ratios of 0.72 and 0.69 for the children and adults, respectively. Secondary responses with similar transient gating behavior were centered bilaterally in proximity of the lateral sulcus. Delayed and prolonged responses with strong gating were found in the frontal and parietal cortices possibly corresponding to larger processing network of somatosensory afference. No significant correlation between age and gating ratio was found. We confirmed that cortical gating to proprioceptive stimuli is comparable to other somatosensory and auditory domains, and between children and adults. Gating occurred broadly beyond SI cortex. Spatial ICA revealed several consistent response patterns in various cortical regions which would have been challenging to detect with more commonly applied equivalent current dipole or distributed source estimates.

Mental fatigue is a common phenomenon with implicit and multidimensional properties. It brings dynamic changes of functional brain networks. However, the challenging problem of false positives appears when the connectivity is estimated by Electroencephalography (EEG). In this paper, we propose a novel framework based on spatial clustering to explore the sources of mental fatigue and functional activity changes caused by them. To suppress the false positive observations, spatial clustering is implemented in brain networks. The nodes extracted by spatial clustering are registered back to functional magnetic resonance imaging (fMRI) source space to determined the sources of mental fatigue. The wavelet entropy of EEG in a sliding window is calculated to find the temporal features of mental fatigue. Our experimental results show that the extracted nodes correspond to the fMRI sources across different subjects and different tasks. The entropy values on the extracted nodes demonstrate clearer staged decreasing changes (deactivation). Additionally, the synchronization among the extracted nodes is stronger than that among all the nodes in the deactivation stage. The initial time of the strong synchronized deactivation is consistent with the subjective fatigue time reported by the subjects themselves. It means the synchronization and deactivation corresponds to the subjective feelings of fatigue. Therefore, this functional activity pattern may be caused by the sources of mental fatigue. The proposed framework is useful for a wide range of prolonged functional imaging and fatigue detection studies. peerReviewed

Previous researches demonstrate that major depression disorder (MDD) is associated with widespread network dysconnectivity, and the dynamics of functional connectivity networks are important to delineate the neural mechanisms of MDD. Cortical electroencephalography (EEG) oscillations act as coordinators to connect different brain regions, and various assemblies of oscillations can form different networks to support different cognitive tasks. Studies have demonstrated that the dysconnectivity of EEG oscillatory networks is related with MDD. In this study, we investigated the oscillatory hyperconnectivity and hypoconnectivity networks in MDD under a naturalistic and continuous stimuli condition of music listening. With the assumption that the healthy group and the MDD group share similar brain topology from the same stimuli and also retain individual brain topology for group differences, we applied the coupled nonnegative tensor decomposition algorithm on two adjacency tensors with the dimension of time × frequency × connectivity × subject, and imposed double-coupled constraints on spatial and spectral modes. The music-induced oscillatory networks were identified by a correlation analysis approach based on the permutation test between extracted temporal factors and musical features. We obtained three hyperconnectivity networks from the individual features of MDD and three hypoconnectivity networks from common features. The results demonstrated that the dysfunction of oscillation-modulated networks could affect the involvement in music perception for MDD patients. Those oscillatory dysconnectivity networks may provide promising references to reveal the pathoconnectomics of MDD and potential biomarkers for the diagnosis of MDD.

Background Independent component analysis (ICA) has been widely used for blind source separation in the field of medical imaging. However, despite of previous substantial efforts, the stability of ICA components remains a critical issue which has not been adequately addressed, despite numerous previous efforts. Most critical is the inconsistency of some of the extracted components when ICA is run with different model orders (MOs). New Method In this study, a novel method of determining the consistency of component analysis (CoCA) is proposed to evaluate the consistency of extracted components with different model orders. In the method, “consistent components” (CCs) are defined as those which can be extracted repeatably over a range of model orders. Result The efficacy of the method was evaluated with simulation data and fMRI datasets. With our method, the simulation result showed a clear difference of consistency between ground truths and noise. Comparison with existing methods The information criteria were implemented to provide suggestions for the optimal model order, where some of the ICs were revealed inconsistent in our proposed method. Conclusions This method provided an objective protocol for choosing CCs of an ICA decomposition of a data matrix, independent of model order. This is especially useful with high model orders, where noise or other disturbances could possibly lead to an instability of the components. peerReviewed

Evoked event-related oscillations (EROs) have been widely used to explore the mechanisms of brain activities for both normal people and neuropsychiatric disease patients. In most previous studies, the calculation of the regions of evoked EROs of interest is commonly based on a predefined time window and a frequency range given by the experimenter, which tends to be subjective. Additionally, evoked EROs sometimes cannot be fully extracted using the conventional time-frequency analysis (TFA) because they may be overlapped with each other or with artifacts in time, frequency, and space domains. To further investigate the related neuronal processes, a novel approach was proposed including three steps: (1) extract the temporal and spatial components of interest simultaneously by temporal principal component analysis (PCA) and Promax rotation and project them to the electrode fields for correcting their variance and polarity indeterminacies, (2) calculate the time-frequency representations (TFRs) of the back-projected components, and (3) compute the regions of evoked EROs of interest on TFRs objectively using the edge detection algorithm. We performed this novel approach, conventional TFA, and TFA-PCA to analyse both the synthetic datasets with different levels of SNR and an actual ERP dataset in a two-factor paradigm of waiting time (short/long) and feedback (loss/gain) separately. Synthetic datasets results indicated that N2-theta and P3-delta oscillations can be stably detected from different SNR-simulated datasets using the proposed approach, but, by comparison, only one oscillation was obtained via the last two approaches. Furthermore, regarding the actual dataset, the statistical results for the proposed approach revealed that P3-delta was sensitive to the waiting time but not for that of the other approaches. This study manifested that the proposed approach could objectively extract evoked EROs of interest, which allows a better understanding of the modulations of the oscillatory responses. peerReviewed

Clustering is a promising tool for grouping the sequence of similar time-points aimed to identify the attention blocks in spatiotemporal event-related potentials (ERPs) analysis. It is most likely to elicit the appropriate time window for ERP of interest if a suitable clustering method is applied to spatiotemporal ERP. However, how to reliably estimate a proper time window from entire individual subjects’ data is still challenging. In this study, we developed a novel multiset consensus clustering method in which several clustering results of multiple subjects were combined to retrieve the best fitted clustering for all the subjects within a group. Then, the obtained clustering was processed by a newly proposed time-window detection method to determine the most suitable time window for identifying the ERP of interest in each condition/group. Applying the proposed method to the simulated ERP data and real data indicated that the brain responses from the individual subjects can be collected to determine a reliable time window for different conditions/groups. Our results revealed more precise time windows to identify N2 and P3 components in the simulated data compared to the state-of-the-art methods. Additionally, our proposed method achieved more robust performance and outperformed statistical analysis results in the real data for N300 and prospective positivity components. To conclude, the proposed method successfully estimates the time window for ERP of interest by processing the individual data, offering new venues for spatiotemporal ERP processing. peerReviewed

In independent component analysis (ICA), the selection of model order (i.e., number of components to be extracted) has crucial effects on functional magnetic resonance imaging (fMRI) brain network analysis. Model order selection (MOS) algorithms have been used to determine the number of estimated components. However, simulations show that even when the model order equals the number of simulated signal sources, traditional ICA algorithms may misestimate the spatial maps of the signal sources. In principle, increasing model order will consider more potential information in the estimation, and should therefore produce more accurate results. However, this strategy may not work for fMRI because large-scale networks are widely spatially distributed and thus have increased mutual information with noise. As such, conventional ICA algorithms with high model orders may not extract these components at all. This conflict makes the selection of model order a problem. We present a new strategy for model order free ICA, called Snowball ICA, that obviates these issues. The algorithm collects all information for each network from fMRI data without the limitations of network scale. Using simulations and in vivo resting-state fMRI data, our results show that component estimation using Snowball ICA is more accurate than traditional ICA. The Snowball ICA software is available at https://github.com/GHu-DUT/Snowball-ICA. peerReviewed

Canonical polyadic decomposition (CPD) of multi-subject complex-valued fMRI data can be used to provide spatially and temporally shared components among groups with both magnitude and phase information. However, the CPD model is not well formulated due to the large subject variability in the spatial and temporal modalities, as well as the high noise level in complex-valued fMRI data. Considering that the shift-invariant CPD can model temporal variability across subjects, we propose to further impose a phase sparsity constraint on the shared spatial maps to denoise the complex-valued components and to model the inter-subject spatial variability as well. More precisely, subject-specific time delays are first estimated for the complex-valued shared time courses in the framework of real-valued shift-invariant CPD. Source phase sparsity is then imposed on the complex-valued shared spatial maps. A smoothed $\ell _{\mathbf {{0}}}$ norm is specifically used to reduce voxels with large phase values after phase de-ambiguity based on the small phase characteristic of BOLD-related voxels. The results from both the simulated and experimental fMRI data demonstrate improvements of the proposed method over three complex-valued algorithms, namely, tensor-based spatial ICA, shift-invariant CPD and CPD without spatiotemporal constraints. When comparing with a real-valued algorithm combining shift-invariant CPD and ICA, the proposed method detects 178.7% more contiguous task-related activations. peerReviewed

Background Ongoing EEG data are recorded as mixtures of stimulus-elicited EEG, spontaneous EEG and noises, which require advanced signal processing techniques for separation and analysis. Existing methods cannot simultaneously consider common and individual characteristics among/within subjects when extracting stimulus-elicited brain activities from ongoing EEG elicited by 512-s long modern tango music. New method Aiming to discover the commonly music-elicited brain activities among subjects, we provide a comprehensive framework based on fast double-coupled nonnegative tensor decomposition (FDC-NTD) algorithm. The proposed algorithm with a generalized model is capable of simultaneously decomposing EEG tensors into common and individual components. Results With the proposed framework, the brain activities can be effectively extracted and sorted into the clusters of interest. The proposed algorithm based on the generalized model achieved higher fittings and stronger robustness. In addition to the distribution of centro-parietal and occipito-parietal regions with theta and alpha oscillations, the music-elicited brain activities were also located in the frontal region and distributed in the 4∼11 Hz band. Comparison with existing method(s) The present study, by providing a solution of how to separate common stimulus-elicited brain activities using coupled tensor decomposition, has shed new light on the processing and analysis of ongoing EEG data in multi-subject level. It can also reveal more links between brain responses and the continuous musical stimulus. Conclusions The proposed framework based on coupled tensor decomposition can be successfully applied to group analysis of ongoing EEG data, as it can be reliably inferred that those brain activities we obtained are associated with musical stimulus.

Sustained attention encompasses a cascade of fundamental functions. The human ability to implement a sustained attention task is supported by brain networks that dynamically formed and dissolved through oscillatory synchronization. The decrement of vigilance induced by prolonged task engagement affects sustained attention. However, little is known about which stage or combinations are affected by vigilance decrement. Here, we applied an analysis framework composed of weighted phase lag index (wPLI) and tensor component analysis (TCA) to an EEG dataset collected during 80 min sustained attention task to examine the electrophysiological basis of such effect. We aimed to characterize the phase-coupling networks to untangle different phases involved in sustained attention and study how they are modulated by vigilance decrement. We computed the time–frequency domain wPLI from each block and subject and constructed a fourth-order tensor, containing the time, frequency, functional connectivity (FC), and blocks × subjects. This tensor was subjected to the TCA to identify the interacted and low-dimensional components representing the frequency-specific dynamic FC (fdFC). We extracted four types of neuromakers during a sustained attention task, namely the pre-stimulus alpha right-lateralized parieto-occipital FC, the post-stimulus theta fronto-parieto-occipital FC, delta fronto-parieto-occipital FC, and beta right/left sensorimotor FCs. All these fdFCs were impaired by vigilance decrement. These fdFCs, except for the beta left sensorimotor network, were restored by rewards, although the restoration by reward in the beta right sensorimotor network was transient. These findings provide implications for dissociable effects of vigilance decrement on sustained attention by utilizing the tensor-based framework. Electronic supplementary material The online version of this article (10.1007/s10548-020-00795-0) contains supplementary material, which is available to authorized users.

Deep learning for the automated detection of epileptic seizures has received much attention during recent years. In this work, one dimensional convolutional neural network (1D-CNN) and two dimensional convolutional neural network (2D-CNN) are simultaneously used on electroencephalogram (EEG) data for seizure detection. Firstly, using sliding windows without overlap on raw EEG to obtain the definite one-dimension time EEG segments (1D-T), and continuous wavelet transform (CWT) for 1D-T signals to obtain the two-dimension time-frequency representations (2D-TF). Then, 1D-CNN and 2D-CNN model architectures are used on 1D-T and 2D-TF signals for automatic classification, respectively. Finally, the classification results from 1D-CNN and 2D-CNN are showed. In the two-classification and three-classification problems of seizure detection, the highest accuracy can reach 99.92% and 99.55%, respectively. It shows that the proposed method for a benchmark clinical dataset can achieve good performance in terms of seizure detection. peerReviewed

Background and objective. It is challenging to conduct real-time identification of myocardial infarction (MI) due to artifact corruption and high dimensionality of multi-lead electrocardiogram (ECG). In the present study, we proposed an automated single-beat MI detection and localization system using dual-Q tunable Q-factor wavelet transformation (Dual-Q TQWT) denoising algorithm. Methods. After denoising and segmentation of ECG, a fourth-order wavelet tensor (leads × subbands × samples × beats) was constructed based on thediscretewavelet packet transform (DWPT), to represent the features considering the information of inter-beat, intra-beat, inter-frequency, and inter-lead. To reduce the tensor dimension and preserve the intrinsic information, the multilinear principal component analysis (MPCA) was employed. Afterward, 84 discriminate features were fed into a classifier of bootstrap-aggregated decision trees (Treebagger). A total of 78 healthy and 328 MI (6types) records including 57557 beats were chosen from PTB diagnostic ECG database for evaluation. Results.The validation results demonstratedthat our proposed MI detection and localization system embedded with Dual-Q TQWT and wavelet packet tensor decomposition outperformedcommonly used discrete wavelet transform (DWT), empirical mode decomposition (EMD) denoising methods and vector-based PCA method. With the Treebagger classifier, we obtained an accuracy of 99.98% in beat level and an accuracy of 97.46% in record level training/testing for MI detection. We also achieved an accuracy of 99.87% in beat level and an accuracy of 90.39% in record level for MI localization. Conclusion. Altogether, the automated system brings potential improvement in automated detectionand localization of MI in clinical practice. peerReviewed

Spatial source phase, the phase information of spatial maps extracted from functional magnetic resonance imaging (fMRI) data by data‐driven methods such as independent component analysis (ICA), has rarely been studied. While the observed phase has been shown to convey unique brain information, the role of spatial source phase in representing the intrinsic activity of the brain is yet not clear. This study explores the spatial source phase for identifying spatial differences between patients with schizophrenia (SZs) and healthy controls (HCs) using complex‐valued resting‐state fMRI data from 82 individuals. ICA is first applied to preprocess fMRI data, and post‐ICA phase de‐ambiguity and denoising are then performed. The ability of spatial source phase to characterize spatial differences is examined by the homogeneity of variance test (voxel‐wise F‐test) with false discovery rate correction. Resampling techniques are performed to ensure that the observations are significant and reliable. We focus on two components of interest widely used in analyzing SZs, including the default mode network (DMN) and auditory cortex. Results show that the spatial source phase exhibits more significant variance changes and higher sensitivity to the spatial differences between SZs and HCs in the anterior areas of DMN and the left auditory cortex, compared to the magnitude of spatial activations. Our findings show that the spatial source phase can potentially serve as a new brain imaging biomarker and provide a novel perspective on differences in SZs compared to HCs, consistent with but extending previous work showing increased variability in patient data. peerReviewed

Sleep scoring is a fundamental but time-consuming process in any sleep laboratory. To speed up the process of sleep scoring without compromising accuracy, this paper develops an automatic sleep scoring toolbox with the capability of multi-signal processing. It allows the user to choose signal types and the number of target classes. Then, an automatic process containing signal pre-processing, feature extraction, classifier training (or prediction) and result correction will be performed. Finally, the application interface displays predicted sleep structure, related sleep parameters and the sleep quality index for reference. To improve the identification accuracy of minority stages, a layer-wise classification strategy is proposed according to the signal characteristics of sleep stages. The context of the current stage is taken into consideration in the correction phase by employing a Hidden Markov Model to study the transition rules of sleep stages in the training dataset. These transition rules will be used for logic classification results. The performance of proposed toolbox has been tested on 100 subjects with an average accuracy of 85.76%. The proposed automatic scoring toolbox would alleviate the burden of the physicians, speed up sleep scoring, and expedite sleep research. peerReviewed

New approach to identify different lifetime stages of rolling element bearings, to improve early bearing fault detection, is presented. We extract characteristic features from vibration signals generated by rolling element bearings. This data is first pre-labelled with an unsupervised clustering method. Then, supervised methods are used to improve the labelling. Moreover, we assess feature importance with each classifier. From the practical point of view, the classifiers are compared on how early emergence of a bearing fault is being suggested. The results show that all of the classifiers are usable for bearing fault detection and the importance of the features was consistent. peerReviewed

Background. Stability of spatial components is frequently used as a post-hoc selection criteria for choosing the dimensionality of an independent component analysis (ICA) of functional magnetic resonance imaging (fMRI) data. Although the stability of the ICA temporal courses differs from that of spatial components, temporal stability has not been considered during dimensionality decisions. New method. The current study aims to (1) develop an algorithm to incorporate temporal course stability into dimensionality selection and (2) test the impact of temporal course on the stability of the ICA decomposition of fMRI data via tensor clustering. Resting state fMRI data were analyzed with two popular ICA algorithms, InfomaxICA and FastICA, using our new method and results were compared with model order selection based on spatial or temporal criteria alone. Results. Hierarchical clustering indicated that the stability of the ICA decomposition incorporating spatiotemporal tensor information performed similarly when compared to current best practice. However, we found that component spatiotemporal stability and convergence of the model varied significantly with model order. Considering both may lead to methodological improvements for determining ICA model order. Selected components were also significantly associated with relevant behavioral variables. Comparison with Existing Method: The Kullback–Leibler information criterion algorithm suggests the optimal model order for group ICA is 40, compared to the proposed method with an optimal model order of 20. Conclusion. The current study sheds new light on the importance of temporal course variability in ICA of fMRI data. peerReviewed

The cognitive processing mechanism of humor refers to how the system of neural circuitry and pathways in the brain deals with the incongruity in a humorous manner. The past research has revealed different stages and corresponding functional brain activities involved in humor-processing in terms of time and space dimensions, highlighting the effects of the time windows of about 400 ms, 600 ms, and 900 ms. However, much less is known about humor processing in light of the frequency dimension. A total of 36 Chinese participants were recruited in this experiment, with Chinese jokes, nonjokes, and nonsensical sentences used as the stimuli. The experimental results showed that there were significant differences among conditions in the P200 effect, which signified that the incongruity detection had already been integrated and perceived at about 200 ms, prior to the semantic integration at about 400 ms. This pre-processing is specific to Chinese verbal jokes due to the simultaneous involvement of both orthographic and phonologic parts in processing Chinese characters. The analysis on the frequency dimension indicated that beta’s power particularly reflected the characteristics of different stages in Chinese verbal humor processing. Jokes’ and nonsensical sentences’ relative power changes on the beta band ranked significantly higher than that of nonjokes at about 200 ms, which suggested the existence of more difficulties in meaning construction in pre-processing the incongruities. This indicated a continuity between the analysis of event related potential (ERP) components and neural oscillations and revealed the key role of the beta frequency band in Chinese verbal joke processing. peerReviewed

Background: Sleep scoring is an essential but time-consuming process, and therefore automatic sleep scoring is crucial and urgent to help address the growing unmet needs for sleep research. This paper aims to develop a versatile deep-learning architecture to automate sleep scoring using raw polysomnography recordings. Method: The model adopts a linear function to address different numbers of inputs, thereby extending model applications. Two-dimensional convolution neural networks are used to learn features from multi-modality polysomnographic signals, a “squeeze and excitation” block to recalibrate channel-wise features, together with a long short-term memory module to exploit long-range contextual relation. The learnt features are finally fed to the decision layer to generate predictions for sleep stages. Result: Model performance is evaluated on three public datasets. For all tasks with different available channels, our model achieves outstanding performance not only on healthy subjects but even on patients with sleep disorders (SHHS: Acc-0.87, K-0.81; ISRUC: Acc-0.86, K-0.82; Sleep-EDF: Acc-0.86, K-0.81). The highest classification accuracy is achieved by a fusion of multiple polysomnographic signals. Comparison: Compared to state-of-the-art methods that use the same dataset, the proposed model achieves a comparable or better performance, and exhibits low computational cost. Conclusions: The model demonstrates its transferability among different datasets, without changing model architecture or hyper-parameters across tasks. Good model transferability promotes the application of transfer learning on small group studies with mismatched channels. Due to demonstrated availability and versatility, the proposed method can be integrated with diverse polysomnography systems, thereby facilitating sleep monitoring in clinical or routine care. peerReviewed

To examine the electrophysiological underpinnings of the functional networks involved in music listening, previous approaches based on spatial independent component analysis (ICA) have recently been used to ongoing electroencephalography (EEG) and magnetoencephalography (MEG). However, those studies focused on healthy subjects, and failed to examine the group-level comparisons during music listening. Here, we combined group-level spatial Fourier ICA with acoustic feature extraction, to enable group comparisons in frequency-specific brain networks of musical feature processing. It was then applied to healthy subjects and subjects with major depressive disorder (MDD). The music-induced oscillatory brain patterns were determined by permutation correlation analysis between individual time courses of Fourier-ICA components and musical features. We found that (1) three components, including a beta sensorimotor network, a beta auditory network and an alpha medial visual network, were involved in music processing among most healthy subjects; and that (2) one alpha lateral component located in the left angular gyrus was engaged in music perception in most individuals with MDD. The proposed method allowed the statistical group comparison, and we found that: (1) the alpha lateral component was activated more strongly in healthy subjects than in the MDD individuals, and that (2) the derived frequency-dependent networks of musical feature processing seemed to be altered in MDD participants compared to healthy subjects. The proposed pipeline appears to be valuable for studying disrupted brain oscillations in psychiatric disorders during naturalistic paradigms.

Background Component splitting at higher model orders is a widely accepted finding for independent component analysis (ICA) of functional magnetic resonance imaging (fMRI) data. However, our recent study found that intact components occurred with subcomponents at higher model orders. New method This study investigated model order effects on ICA of resting-state complex-valued fMRI data from 82 subjects, which included 40 healthy controls (HCs) and 42 schizophrenia patients. In addition, we explored underlying causes for distinct component splitting between complex-valued data and magnitude-only data by examining model order effects on ICA of phase fMRI data. A best run selection method was proposed to combine subject averaging and a one-sample t-test. We selected the default mode network (DMN)-, visual-, and sensorimotor-related components from the best run of ICA at varying model orders from 10 to 140. Results Results show that component integration occurred in complex-valued and phase analyses, whereas component splitting emerged in magnitude-only analysis with increasing model order. Incorporation of phase data appears to play a complementary role in preserving integrity of brain networks. Comparison with existing method(s) When compared with magnitude-only analysis, the intact DMN component obtained in complex-valued analysis at higher model orders exhibited highly significant subject-level differences between HCs and patients with schizophrenia. We detected significantly higher activity and variation in anterior areas for HCs and in posterior areas for patients with schizophrenia. Conclusions These results demonstrate the potential of complex-valued fMRI data to contribute generally and specifically to brain network analysis in identification of schizophrenia-related changes.

Independent component analysis (ICA) on group-level voxel-based morphometry (VBM) produces the coefficient matrix and the component matrix. The former contains variability among multiple subjects for further statistical analysis, and the latter reveals spatial maps common for all subjects. ICA algorithms converge to local optimization points in practice and the mostly applied stability investigation approach examines the stability of the extracted components. We found that the practically stable components do not guarantee to produce the practically stable coefficients of ICA decomposition for the further statistical analysis. Consequently, we proposed a novel approach including two steps: 1), the stability index for the coefficient matrix and the stability index for the component matrix were examined, respectively; 2) the two indices were multiplied to analyze the stability of ICA decomposition. The proposed approach was used to study the sMRI data of Type II diabetes mellitus group (DM) and the healthy control group (HC). Group differences in VBM were found in the superior temporal gyrus. Besides, it was revealed that the VBMs of the region of the HC group were significantly correlated with Montreal Cognitive Assessment (MoCA) describing the level of cognitive disorder. In contrast to the widely applied approach to investigating the stability of the extracted components for ICA decomposition, we proposed to examine the stability of ICA decomposition by fusion the stability of both coefficient matrix and the component matrix. Therefore, the proposed approach can examine the stability of ICA decomposition sufficiently. peerReviewed

Interest towards higher ecological validity in functional magnetic resonance imaging (fMRI) experiments has been steadily growing since the turn of millennium. The trend is reflected in increasing amount of naturalistic experiments, where participants are exposed to the real-world complex stimulus and/or cognitive tasks such as watching movie, playing video games, or listening to music. Multifaceted stimuli forming parallel streams of input information, combined with reduced control over experimental variables introduces number of methodological challenges associated with isolating brain responses to individual events. This exploratory work demonstrated some of those methodological challenges by applying widely used data-driven methods to real fMRI data elicited from continuous music listening experiment. Under the general goal of finding functional networks of brain regions involved in music processing, this work contributed to improvement of the methodology from two perspectives. One is to produce a set of representative features for stimulus audio that can capture different aspects of music, such as timbre and tonality. Another is to improve reliability and quality of separation of the observed brain activations into independent spatial patterns. Improved separation in turn enables better differentiation of stimulus-related activations from the ones originating from unrelated physiological, cognitive, or technical processes. More specifically, part of the research explored an application of a nonlinear method for generating perceptually relevant stimulus features representing high-level concepts in music. Another part addressed dimensionality reduction and model order estimation problem before subjecting fMRI data to source separation and offered few methodological developments in this regard.

Tietokoneen käyttäjän varmentaminen silmänliikkeiden perusteella Tutkimus käsittää tietokoneen käyttäjän biometrisen verifioinnin kehittämistä perustuen käyttäjän nopeisiin silmänliikkeisiin, sakkadeihin. Tällöin käyttäjästä tehdyillä silmänliikemittauksilla todennetaan, onko kyseinen henkilö oikea (autentikoitu) käyttäjä vai ei. Biometrista verifiointia eli käyttäjän vahvistamista on aiemmin tutkittu varsinkin sormenjälkikuvista, joista on kaupallisia sovelluksia, mutta myös mm. kasvo-, silmänpohja- ja kämmenkuvista. Nämä kaikki ovat biometrisiä datalähteitä. Tässä tutkimuksessa biometrisenä datana olivat henkilön silmänliikkeet. Tähän mennessä näitä ovat tutkineet biometristä verifiointia varten vain muutama muu tutkija maailmassa. Silmänliiketutkimuksia on luonnollisesti lääketieteessä ja psykologiassa tehty jo vuosikymmeniä ja tietokoneiden käyttöliittymienkin yhteydessä parikymmentä vuotta. Verrattuna muihin biometrisiin datoihin silmänliikkeiden hyvät puolet verifioinnin mielessä voivat olla seuraavat: vaikeampi imitoida ja datan yksinkertaisuuden vuoksi helpompi prosessointi, mikä tuottaa nopean laskennan. Väitöstutkimuksen tekijä on tutkinut ja kuvaa tutkimuksessaan laskennallisia menetelmiä, joilla saadaan laskettua silmänliikkeistä henkilöiden välillä vaihtelevia ominaisuuksia verifiointia varten. Tekijä mittasi silmänliikevideokamerasysteemillä koehenkilöiden silmänliikkeet näiden katsellessa tarkoitukseen sopivia stimulaatioita (ärsykkeitä), kuten kuvaruudulla liikkuvia pieniä kohteita. Vertailudatana oli jo aiemmissa tutkimuksissa elektro-okulograafisesti mitattuja eli silmän etu- takaosan väliseen jännite-eroon perustuvia signaaleja, jolloin oli käytetty samanlaisia stimulaatioita kuin silmänliikekameroilla. Tekijä käytti signaalianalyysia silmänliikkeiden tunnistamiseksi signaalidatasta. Väitöstutkimuksen tekijä kehitti testausproseduurit, tietokoneohjelmat, kuinka tehdä mittaukset ja erottaa kulloinenkin oikea käyttäjä muista koehenkilöistä, jotka esittivät testeissä laittomia tunkeilijoita. Hän perusti tämän verifioinnin useisiin tiedonlouhinnan ja koneoppimisen luokitusmenetelmiin, kuten suoraviivainen lähimmän naapurin etsintäalgoritmi ja edellistä kehittyneempi, mutta monimutkaisempi algoritmi nimeltä tukivektorikone. Tekijä testasi kehittämillään laskentamenetelmillä, kuinka usein oli mahdollista verifioida oikein asianomainen, oikea käyttäjä (oikeiden positiivisten määrät) ja kuinka usein pystyttiin erottamaan väärät yrittäjät (oikeiden negatiivisten määrät). Parhaimmillaan testit ylsivät erinomaiseen tulokseen, noin 95 prosenttia. Lisäksi hän toisti testejä samoille koehenkilöille päivien ja viikkojenkin väliajoilla voidakseen tutkia, miten koehenkilöiden silmänliikkeiden ominaisuuksien mahdollinen ajasta riippuva vaihtelevuus vaikuttaisi biometrisen verifiointiin. Vaihtelevuus vaikutti vain suhteellisen vähän laskien luokitustarkkuutta noin 90 prosenttiin. Tätä saattoi kuitenkin vähän vielä parantaa käyttämällä toista silmänliikevideokamerasysteemiä, jonka näytteenottotaajuus on 250 Hz. Aluksi tekijä käytti systeemiä, jossa se oli ainoastaan 30 Hz. Tätä matalaa taajuutta oli kuitenkin mielekästä kokeilla, sillä se vastaa halpojen, tietokoneissa olevien kamerasysteemien (eivät vielä silmänliikekameroita) ominaisuuksia. Kun voidaan olettaa tulevaisuudessa tietokoneiden ja älypuhelimien käyttöliittymään liitetyn silmänliikekameran käyttäjän katseen eli silmänliikkeiden tunnistamiseksi, tähän voitaisiin kytkeä myös biometrinen verifiointi. On jo olemassa älypuhelimia, joissa käyttöliittymä hyödyntää käyttäjän katseensuuntaa tunnistaen käyttäjän kuvaruudulla kulloinkin katsoman paikan. Väitöstutkimus osoitti, että tietokoneen käyttäjä on mahdollista verifioida luotettavasti henkilöstä mitattujen silmänliikkeiden perusteella. This thesis focuses on biometric verification of subjects based on saccadic eye movements. Verification corresponds to two-class classification to recognize an authenticated user and to classify other subjects as impostors. Compared with other biometric signals or data, the possible advantages of eye movements can be as follows: harder to imitate, easier processing and faster computation. The thesis describes a procedure to use variables of saccade eye movements recorded. It analyses the variabilities between electro-oculography (EOG) and video-oculography (VOG) signals: i.e. eye movements were recorded with skin electrodes or with two special video cameras. When a signal was recorded with a low-frequency video camera device simulating a web camera, the sampling frequency of signals was enhanced using interpolation. The techniques of signal processing and statistics were also applied to analysis. In order to evaluate biometric accuracy, the test procedures for true positive rate (TPR) and true negative rate (TNR) were designed separately. Many classification methods were explored for verification performance, including both modified simple methods such as k-nearest neighbour searching and advanced methods such as neural networks and support vector machines. Approaches and other details in the verification procedure were improved through multiple tests and comparisons of the verification accuracies. Optimal parameters and settings of the classification methods used were found. With more and more saccades and subjects collected into training sets, a high TNR accuracy was gained, which was close to 95% at its best. It showed that, using saccade eye movements, it was possible to distinguish between an authenticated user and impostors. On the other hand, after multiple recordings of subjects, the high accuracy of TPR – close to 90% – also confirmed that an authenticated user can be recognized notwithstanding the variability of variable values of saccades between different sessions. Finally, better results given by signals with a relatively high sampling frequency of 250 Hz were obtained, and this could allow user verification based on eye movements to be applied in practice, along with the development of eye movement video cameras in future.