Search

Deformable medical image registration plays an essential role in clinical diagnosis and treatment. However, due to the large difference in image deformation, unsupervised convolutional neural network (CNN)-based methods cannot extract global features and local features simultaneously and cannot capture long-distance dependencies to solve the problem of excessive deformation. In this paper, an unsupervised end-to-end registration network is proposed for 3D MRI medical image registration, named AEAU-Net, which includes two-stage operations, i.e., an affine transformation and a deformable registration. These two operations are implemented by an affine transformation subnetwork and a deformable registration subnetwork, respectively. In the deformable registration subnetwork, termed as EAU-Net, we designed an efficient attention mechanism (EAM) module and a recursive residual path (RSP) module. The EAM module is embedded in the bottom layer of the EAU-Net to capture long-distance dependencies. The RSP model is used to obtain effective features by fusing deep and shallow features. Extensive experiments on two datasets, LPBA40 and Mindboggle101, were conducted to verify the effectiveness of the proposed method. Compared with baseline methods, this proposed method could obtain better registration performance. The ablation study further demonstrated the reasonability and validity of the designed architecture of the proposed method., (© 2023. International Federation for Medical and Biological Engineering.)

Steady-state visual evoked potential (SSVEP) is one of the most commonly used control signals in the brain-computer interface (BCI) systems. However, the conventional spatial filtering methods for SSVEP classification highly depend on the subject-specific calibration data. The need for the methods that can alleviate the demand for the calibration data becomes urgent. In recent years, developing the methods that can work in inter-subject scenario has become a promising new direction. As a popular deep learning model nowadays, Transformer has been used in EEG signal classification tasks owing to its excellent performance. Therefore, in this study, we proposed a deep learning model for SSVEP classification based on Transformer architecture in inter-subject scenario, termed as SSVEPformer, which was the first application of Transformer on the SSVEP classification. Inspired by previous studies, we adopted the complex spectrum features of SSVEP data as the model input, which could enable the model to simultaneously explore the spectral and spatial information for classification. Furthermore, to fully utilize the harmonic information, an extended SSVEPformer based on the filter bank technology (FB-SSVEPformer) was proposed to improve the classification performance. Experiments were conducted using two open datasets (Dataset 1: 10 subjects, 12 targets; Dataset 2: 35 subjects, 40 targets). The experimental results show that the proposed models could achieve better results in terms of classification accuracy and information transfer rate than other baseline methods. The proposed models validate the feasibility of deep learning models based on Transformer architecture for SSVEP data classification, and could serve as potential models to alleviate the calibration procedure in the practical application of SSVEP-based BCI systems., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 Elsevier Ltd. All rights reserved.)

Auditory verbal hallucinations (AVH) are a core positive symptom of schizophrenia and are regarded as a consequence of the functional breakdown in the related sensory process. Yet, the potential mechanism of AVH is still lacking. In the present study, we explored the difference between AVHs (n = 23) and non-AVHs (n = 19) in schizophrenia and healthy controls (n = 29) by using multidimensional electroencephalograms data during an auditory oddball task. Compared to healthy controls, both AVH and non-AVH groups showed reduced P300 amplitudes. Additionally, the results from brain networks analysis revealed that AVH patients showed reduced left frontal to posterior parietal/temporal connectivity compared to non-AVH patients. Moreover, using the fused network properties of both delta and theta bands as features for in-depth learning made it possible to identify the AVH from non-AVH patients at an accuracy of 80.95%. The left frontal-parietal/temporal networks seen in the auditory oddball paradigm might be underlying biomarkers of AVH in schizophrenia. This study demonstrated for the first time the functional breakdown of the auditory processing pathway in the AVH patients, leading to a better understanding of the atypical brain network of the AVH patients., Competing Interests: Declaration of Competing Interest The authors declared that they have no conflicts of interest to this work. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted, (Copyright © 2023 Elsevier B.V. All rights reserved.)

In recent years, emotion recognition based on electroencephalography (EEG) has received growing interests in the brain-computer interaction (BCI) field. The neuroscience researches indicate that the left and right brain hemispheres demonstrate activity differences under different emotional activities, which could be an important principle for designing deep learning (DL) model for emotion recognition. Besides, owing to the nonstationarity of EEG signals, using convolution kernels of a single size may not sufficiently extract the abundant features for EEG classification tasks. Based on these two angles, we proposed a model termed Multi-Scales Bi-hemispheric Asymmetric Model (MSBAM) based on convolutional neural network (CNN) structure. Evaluated on the public DEAP and DREAMER datasets, MSBAM achieved over 99% accuracy for the two-class classification of low-level and high-level states in each of four emotional dimensions, i.e., arousal, valence, dominance and liking, respectively. This study further demonstrated the promising potential to design the DL model from the multi-scale characteristics of the EEG data and the neural mechanisms of the emotion cognition., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

As one of the important psychological stress reactions, Micro-expressions (MEs) are spontaneous and subtle facial movements, which usually occur in a high-stake situation and can reveal genuine human feelings and cognition. ME, Recognition (MER) has essential applications in many fields such as lie detection, criminal investigation, and psychological healing. However, due to the challenges of learning discriminative ME features via fleeting facial subtle reactions as well as the shortage of available MEs data, this research topic is still far from well-studied. To this end, in this paper, we propose a deep prototypical learning framework, namely ME-PLAN, with a local attention mechanism for the MER problem. Specifically, ME-PLAN consists of two components, i.e., a 3D residual prototypical network and a local-wise attention module, where the former aims to learn the precise ME feature prototypes through expression-related knowledge transfer and episodic training, and the latter could facilitate the attention to the local facial movements. Furthermore, to alleviate the dilemma that most MER methods need to depend on manually annotated apex frames, we propose an apex frame spotting method with Unimodal Pattern Constrained (UPC) and further extract ME key-frames sequences based on the detected apex frames to train our proposed ME-PLAN in an end-to-end manner. Finally, through extensive experiments and interpretable analysis regarding the apex frame spotting and MER on composite-database, we demonstrate the superiority and effectiveness of the proposed methods., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

Hemiplegia is a common dysfunction caused by the brain stroke and leads to movement disability. Although the lateralization of movement-related potential, the event-related desynchronization, and more complicated inter-regional information coupling have been investigated, seldom studies have focused on investigating the dynamic information exchanging among multiple brain regions during motor execution for post-stroke hemiplegic patients. With high temporal-resolution electroencephalogram (EEG), the time-varying network is able to reflect the dynamical complex network modalities corresponding to the movements at a millisecond level. In our present study, the wrist extension experiment was designed, along with related EEG datasets being collected. Thereafter, the corresponding time-varying networks underlying the wrist extension were accordingly constructed by adopting the adaptive directed transfer function and then statistically explored, to further uncover the dynamic network deficits (i.e., motor dysfunction) in post-stroke hemiplegic patients. Results of this study found the effective connectivity between the stroked motor area and other areas decreased in patients when compared to healthy controls; on the contrary, the enhanced connectivity between non-stroked motor areas and other areas, especially the frontal and parietal-occipital lobes, were further identified for patients during their accomplishing the designed wrist extension, which might dynamically compensate for the deficited patients' motor behaviors. These findings not only helped deepen our knowledge of the mechanism underlying the patients' motor behaviors, but also facilitated the real-time strategies for clinical therapy of brain stroke, as well as providing a reliable biomarker to predict the future rehabilitation., Supplementary Information: The online version contains supplementary material available at 10.1007/s11571-021-09738-2., Competing Interests: Conflicts of interestThe authors declare that they have no competing interests., (© The Author(s), under exclusive licence to Springer Nature B.V. 2021.)

The clinical therapy of schizophrenia (SCZ) replies on the corresponding accurate and reliable recognition. Although efforts have been paid, the diagnosis of SCZ is still roughly subjective, it is thus urgent to search for related objective physiological parameters. Motivated by the great potential of resting-state networks in underling the brain deficits among different SCZ groups, in this study, we then developed a multi-class feature extraction approach that could effectively extract the spatial network topology and facilitate the recognition of the SCZ, by combining a network structure based supervised learning with an ensemble co-decision strategy. The results demonstrated that the multi-class spatial pattern of the network (MSPN) features outperformed the other conventional electrophysiological features, such as relative power spectrums and network properties, and achieved the highest classification accuracy of 71.58% in the alpha band. These findings did validate that the resting-state MSPN is a promising tool for the clinical assessment of the SCZ., (© 2022. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.)

Steady-state visual evoked potential (SSVEP) is one of the commonly used control signals in brain-computer interface (BCI) systems. The SSVEP-based BCI has the advantages of high information transmission rate and short training time, which has become an important branch of BCI research field. In this review paper, the main progress on frequency recognition algorithm for SSVEP in past five years are summarized from three aspects, i.e., unsupervised learning algorithms, supervised learning algorithms and deep learning algorithms. Finally, some frontier topics and potential directions are explored.

The detection of attentive mental state plays an essential role in the neurofeedback process and the treatment of Attention Deficit and Hyperactivity Disorder (ADHD). However, the performance of the detection methods is still not satisfactory. One of the challenges is to find a proper representation for the electroencephalogram (EEG) data, which could preserve the temporal information and maintain the spatial topological characteristics. Inspired by the deep learning (DL) methods in the research of brain-computer interface (BCI) field, a 3D representation of EEG signal was introduced into attention detection task, and a 3D convolutional neural network model with cascade and parallel convolution operations was proposed. The model utilized three cascade blocks, each consisting of two parallel 3D convolution branches, to simultaneously extract the multi-scale features. Evaluated on a public dataset containing twenty-six subjects, the proposed model achieved better performance compared with the baseline methods under the intra-subject, inter-subject and subject-adaptive classification scenarios. This study demonstrated the promising potential of the 3D CNN model for detecting attentive mental state., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2021 Elsevier Ltd. All rights reserved.)

Periodic visual stimulation can induce stable steady-state visual evoked potentials (SSVEPs) distributed in multiple brain regions and has potential applications in both neural engineering and cognitive neuroscience. However, the underlying dynamic mechanisms of SSVEPs at the whole-brain level are still not completely understood. Here, we addressed this issue by simulating the rich dynamics of SSVEPs with a large-scale brain model designed with constraints of neuroimaging data acquired from the human brain. By eliciting activity of the occipital areas using an external periodic stimulus, our model was capable of replicating both the spatial distributions and response features of SSVEPs that were observed in experiments. In particular, we confirmed that alpha-band (8-12 Hz) stimulation could evoke stronger SSVEP responses; this frequency sensitivity was due to nonlinear entrainment and resonance, and could be modulated by endogenous factors in the brain. Interestingly, the stimulus-evoked brain networks also exhibited significant superiority in topological properties near this frequency-sensitivity range, and stronger SSVEP responses were demonstrated to be supported by more efficient functional connectivity at the neural activity level. These findings not only provide insights into the mechanistic understanding of SSVEPs at the whole-brain level but also indicate a bright future for large-scale brain modeling in characterizing the complicated dynamics and functions of the brain., Competing Interests: Declaration of Competing Interest The authors declare no competing financial interests., (Copyright © 2021 The Authors. Published by Elsevier Inc. All rights reserved.)