Search

Neuroimaging is revolutionizing our ability to investigate the brain's structural and functional properties, enabling us to visualize brain activity during diverse mental processes and actions. One of the most widely used neuroimaging techniques is electroencephalography (EEG), which records electrical activity from the brain using electrodes positioned on the scalp. EEG signals capture both spatial (brain region) and temporal (time-based) data. While a high temporal resolution is achievable with EEG, spatial resolution is comparatively limited. Consequently, capturing both spatial and temporal information from EEG data to recognize mental activities remains challenging. In this paper, we represent spatial and temporal information obtained from EEG signals by transforming EEG data into sequential topographic brain maps. We then apply hybrid deep learning models to capture the spatiotemporal features of the EEG topographic images and classify imagined English words. The hybrid framework utilizes a sequential combination of three-dimensional convolutional neural networks (3DCNNs) and recurrent neural networks (RNNs). The experimental results reveal the effectiveness of the proposed approach, achieving an average accuracy of 77.8% in identifying imagined English speech. [ABSTRACT FROM AUTHOR]

Speech involves the synchronization of the brain and the oral articulators. Inner speech, also known as imagined speech or covert speech, refers to thinking in the form of sound without intentional movement of the lips, tongue, or hands. Decoding human thoughts is a powerful technique that can help individuals who have lost the ability to speak. This paper introduces a high-performance brain wave decoder based on inner speech, using a novel feature extraction method. The approach combined Support Vector Machine (SVM) and multi-wavelet feature extraction techniques to decode two EEG-based inner speech datasets (Data 1 and Data 2) into internally spoken words. The proposed approach achieved an overall classification accuracy of 68.20%, precision of 68.22%, recall of 68.20%, and F1-score of 68.21% for Data 1, and accuracy of 97.5%, precision of 97.73%, recall of 97.50%, and F1-score of 97.61% for Data 2. Additionally, the Area Under the Curve of the Receiver Operating Characteristic (AUC-ROC) demonstrated the validity of the proposed approach for classifying inner speech commands by achieving a macro-average of 78.76% and 99.32% for Data 1 and Data 2, respectively. The EEG-based inner speech classification method proposed in this research has the potential to improve communication for patients with speech disorders, mutism, cognitive development issues, executive function problems, and mental disorder.

Neuroimaging is revolutionizing our ability to investigate the brain’s structural and functional properties, enabling us to visualize brain activity during diverse mental processes and actions. One of the most widely used neuroimaging techniques is electroencephalography (EEG), which records electrical activity from the brain using electrodes positioned on the scalp. EEG signals capture both spatial (brain region) and temporal (time-based) data. While a high temporal resolution is achievable with EEG, spatial resolution is comparatively limited. Consequently, capturing both spatial and temporal information from EEG data to recognize mental activities remains challenging. In this paper, we represent spatial and temporal information obtained from EEG signals by transforming EEG data into sequential topographic brain maps. We then apply hybrid deep learning models to capture the spatiotemporal features of the EEG topographic images and classify imagined English words. The hybrid framework utilizes a sequential combination of three-dimensional convolutional neural networks (3DCNNs) and recurrent neural networks (RNNs). The experimental results reveal the effectiveness of the proposed approach, achieving an average accuracy of 77.8% in identifying imagined English speech.

IntroductionParalyzed and physically impaired patients face communication difficulties, even when they are mentally coherent and aware. Electroencephalographic (EEG) brain–computer interfaces (BCIs) offer a potential communication method for these people without invasive surgery or physical device controls.MethodsAlthough virtual keyboard protocols are well documented in EEG BCI paradigms, these implementations are visually taxing and fatiguing. All English words combine 44 unique phonemes, each corresponding to a unique EEG pattern. In this study, a complete phoneme-based imagined speech EEG BCI was developed and tested on 16 subjects.ResultsUsing open-source hardware and software, machine learning models, such as k-nearest neighbor (KNN), reliably achieved a mean accuracy of 97 ± 0.001%, a mean F1 of 0.55 ± 0.01, and a mean AUC-ROC of 0.68 ± 0.002 in a modified one-versus-rest configuration, resulting in an information transfer rate of 304.15 bits per minute. In line with prior literature, the distinguishing feature between phonemes was the gamma power on channels F3 and F7.DiscussionHowever, adjustments to feature selection, trial window length, and classifier algorithms may improve performance. In summary, these are iterative changes to a viable method directly deployable in current, commercially available systems and software. The development of an intuitive phoneme-based EEG BCI with open-source hardware and software demonstrates the potential ease with which the technology could be deployed in real-world applications.

In this paper, we propose an imagined speech-based brain wave pattern recognition using deep learning. Multiple features were extracted concurrently from eight-channel electroencephalography (EEG) signals. To obtain classifiable EEG data with fewer sensors, we placed the EEG sensors on carefully selected spots on the scalp. To decrease the dimensions and complexity of the EEG dataset and to avoid overfitting during the deep learning algorithm, we utilized the wavelet scattering transformation. A low-cost 8-channel EEG headset was used with MATLAB 2023a to acquire the EEG data. The long-short term memory recurrent neural network (LSTM-RNN) was used to decode the identified EEG signals into four audio commands: up, down, left, and right. Wavelet scattering transformation was applied to extract the most stable features by passing the EEG dataset through a series of filtration processes. Filtration was implemented for each individual command in the EEG datasets. The proposed imagined speech-based brain wave pattern recognition approach achieved a 92.50% overall classification accuracy. This accuracy is promising for designing a trustworthy imagined speech-based brain–computer interface (BCI) future real-time systems. For better evaluation of the classification performance, other metrics were considered, and we obtained 92.74%, 92.50%, and 92.62% for precision, recall, and F1-score, respectively. [ABSTRACT FROM AUTHOR]

Recent studies have demonstrated that it is possible to decode and synthesize various aspects of acoustic speech directly from intracranial measurements of electrophysiological brain activity. In order to continue progressing toward the development of a practical speech neuroprosthesis for the individuals with speech impairments, better understanding and modeling of imagined speech processes are required. The present study uses intracranial brain recordings from participants that performed a speaking task with trials consisting of overt, mouthed, and imagined speech modes, representing various degrees of decreasing behavioral output. Speech activity detection models are constructed using spatial, spectral, and temporal brain activity features, and the features and model performances are characterized and compared across the three degrees of behavioral output. The results indicate the existence of a hierarchy in which the relevant channels for the lower behavioral output modes form nested subsets of the relevant channels from the higher behavioral output modes. This provides important insights for the elusive goal of developing more effective imagined speech decoding models with respect to the better-established overt speech decoding counterparts.

In this paper, we propose an imagined speech-based brain wave pattern recognition using deep learning. Multiple features were extracted concurrently from eight-channel electroencephalography (EEG) signals. To obtain classifiable EEG data with fewer sensors, we placed the EEG sensors on carefully selected spots on the scalp. To decrease the dimensions and complexity of the EEG dataset and to avoid overfitting during the deep learning algorithm, we utilized the wavelet scattering transformation. A low-cost 8-channel EEG headset was used with MATLAB 2023a to acquire the EEG data. The long-short term memory recurrent neural network (LSTM-RNN) was used to decode the identified EEG signals into four audio commands: up, down, left, and right. Wavelet scattering transformation was applied to extract the most stable features by passing the EEG dataset through a series of filtration processes. Filtration was implemented for each individual command in the EEG datasets. The proposed imagined speech-based brain wave pattern recognition approach achieved a 92.50% overall classification accuracy. This accuracy is promising for designing a trustworthy imagined speech-based brain–computer interface (BCI) future real-time systems. For better evaluation of the classification performance, other metrics were considered, and we obtained 92.74%, 92.50%, and 92.62% for precision, recall, and F1-score, respectively.

Objective: Brain-computer interfaces (BCI) studies are increasingly leveraging different attributes of multiple signal modalities simultaneously. Bimodal data acquisition protocols combining the temporal resolution of electroencephalography (EEG) with the spatial resolution of functional near-infrared spectroscopy (fNIRS) require novel approaches to decoding. Methods: We present an EEG-fNIRS Hybrid BCI that employs a new bimodal deep neural network architecture consisting of two convolutional sub-networks (subnets) to decode overt and imagined speech. Features from each subnet are fused before further feature extraction and classification. Nineteen participants performed overt and imagined speech in a novel cue-based paradigm enabling investigation of stimulus and linguistic effects on decoding. Results: Using the hybrid approach, classification accuracies (46.31% and 34.29% for overt and imagined speech, respectively (chance: 25%)) indicated a significant improvement on EEG used independently for imagined speech (p = 0.020) while tending towards significance for overt speech (p = 0.098). In comparison with fNIRS, significant improvements for both speech-types were achieved with bimodal decoding (p<0.001). There was a mean difference of ∼12.02% between overt and imagined speech with accuracies as high as 87.18% and 53%. Deeper subnets enhanced performance while stimulus effected overt and imagined speech in significantly different ways. Conclusion: The bimodal approach was a significant improvement on unimodal results for several tasks. Results indicate the potential of multi-modal deep learning for enhancing neural signal decoding. Significance: This novel architecture can be used to enhance speech decoding from bimodal neural signals. [ABSTRACT FROM AUTHOR]

Currently, the most used method to measure brain activity under a non-invasive procedure is the electroencephalogram (EEG). This is because of its high temporal resolution, ease of use, and safety. These signals can be used under a Brain Computer Interface (BCI) framework, which can be implemented to provide a new communication channel to people that are unable to speak due to motor disabilities or other neurological diseases. Nevertheless, EEG-based BCI systems have presented challenges to be implemented in real life situations for imagined speech recognition due to the difficulty to interpret EEG signals because of their low signal-to-noise ratio (SNR). As consequence, in order to help the researcher make a wise decision when approaching this problem, we offer a review article that sums the main findings of the most relevant studies on this subject since 2009. This review focuses mainly on the pre-processing, feature extraction, and classification techniques used by several authors, as well as the target vocabulary. Furthermore, we propose ideas that may be useful for future work in order to achieve a practical application of EEG-based BCI systems toward imagined speech decoding. [ABSTRACT FROM AUTHOR]

We present a transfer learning-based approach for decoding imagined speech from electroencephalogram (EEG). Features are extracted simultaneously from multiple EEG channels, rather than separately from individual channels. This helps in capturing the interrelationships between the cortical regions. To alleviate the problem of lack of enough data for training deep networks, sliding window-based data augmentation is performed. Mean phase coherence and magnitude-squared coherence, two popular measures used in EEG connectivity analysis, are used as features. These features are compactly arranged, exploiting their symmetry, to obtain a three dimensional “image-like” representation. The three dimensions of this matrix correspond to the alpha, beta and gamma EEG frequency bands. A deep network with ResNet50 as the base model is used for classifying the imagined prompts. The proposed method is tested on the publicly available ASU dataset of imagined speech EEG, comprising four different types of prompts. The accuracy of decoding the imagined prompt varies from a minimum of 79.7% for vowels to a maximum of 95.5% for short-long words across the various subjects. The accuracies obtained are better than the state-of-the-art methods, and the technique is good in decoding prompts of different complexities.

Currently, the most used method to measure brain activity under a non-invasive procedure is the electroencephalogram (EEG). This is because of its high temporal resolution, ease of use, and safety. These signals can be used under a Brain Computer Interface (BCI) framework, which can be implemented to provide a new communication channel to people that are unable to speak due to motor disabilities or other neurological diseases. Nevertheless, EEG-based BCI systems have presented challenges to be implemented in real life situations for imagined speech recognition due to the difficulty to interpret EEG signals because of their low signal-to-noise ratio (SNR). As consequence, in order to help the researcher make a wise decision when approaching this problem, we offer a review article that sums the main findings of the most relevant studies on this subject since 2009. This review focuses mainly on the pre-processing, feature extraction, and classification techniques used by several authors, as well as the target vocabulary. Furthermore, we propose ideas that may be useful for future work in order to achieve a practical application of EEG-based BCI systems toward imagined speech decoding.

Interpretation of neural signals to a form that is as intelligible as speech facilitates the development of communication mediums for the otherwise speech/motor-impaired individuals. Speech perception, production, and imagination often constitute phases of human communication. The primary goal of this article is to analyze the similarity between these three phases by studying electroencephalogram(EEG) patterns across these modalities, in order to establish their usefulness for brain computer interfaces. Neural decoding of speech using such non-invasive techniques necessitates the optimal choice of signal analysis and translation protocols. By employing selection-by-exclusion based temporal modeling algorithms, we discover fundamental syllable-like units that reveal similar set of signal signatures across all the three phases. Significantly higher than chance accuracies are recorded for single trial multi-unit EEG classification using machine learning approaches over three datasets across 30 subjects. Repeatability and subject independence tests performed at every step of the analysis further strengthens the findings and holds promise for translating brain signals to speech non-invasively.

Imagined speech can be used to send commands without any muscle movement or emitting audio. The current status of research is in the early stage, and there is a shortage of open-access datasets for imagined speech analysis. We have proposed an openly accessible electroencephalograph (EEG) dataset for six imagined words in this work. We have selected six phonetically distributed, monosyllabic, and emotionally neutral words from W-22 CID word lists. The phonetic distribution of words consisted of the different places of consonants’ articulation and different positions of tongue advancement for vowel pronunciation. The selected words were “could,” “yard,” “give,” “him,” “there,” and “toe.” The experiment was performed over 15 subjects who performed the overt and imagined speech task for the displayed word. Each word was presented 50 times in random order. EEG signals were recorded during the experiment using a 64-channel EEG acquisition system with a sampling rate of 2,048 Hz. A preliminary analysis of the recorded data is presented by performing the classification of EEGs corresponding to the imagined words. The achieved accuracy is above the chance level for all subjects, which suggests that the recorded EEGs contain distinctive information about the imagined words.

The use of brain-computer interfaces to produce imagined speech from brain waves has the potential to assist individuals with difficulty producing speech or communicating silently. The decoding of covert speech has been observed to have limited efficacy due to the diverse nature of the associated measured brain waves and the limited number of covert speech databases. As a result, traditional machine learning algorithms for learning and inference are challenging, and one of the real alternatives could be to leverage transfer of learning. The main goals of this research were to create a new deep learning (DL) framework for decoding imagined speech electroencephalography (EEG) signals tasks using transfer learning and to transfer the model learning of the source task of an imagined speech EEG dataset to the model training on the target task of another imagined speech EEG dataset, essentially the cross-task learning transfer of discriminative characteristics of the source task to the target task of imagined speech. The experiment was carried out using two distinct open-access EEG datasets, FEIS and KaraOne, that recorded the imagined speech classes of neural signals from multiple individuals. The target FEIS model and the target KaraOne model for multiclass classification exhibit overall accuracy of 89.01% and 82.35%, respectively, according to the proposed transfer learning. The experiment results indicate that the cross-task deep transfer learning design reliably classifies the imagined speech EEG signals by applying the source task learning to the target task learning. The findings suggest the feasibility of a consistent strategy for classifying multiclass imagined speech with transfer learning, which could thereby open up the possibility of future investigation into cross-task imagined speech classification knowledge usability for generalization of new imagined speech prompts.

Over the past decade, many researchers have come up with different implementations of systems for decoding covert or imagined speech from EEG (electroencephalogram). They differ from each other in several aspects, from data acquisition to machine learning algorithms, due to which, a comparison between different implementations is often difficult. This review article puts together all the relevant works published in the last decade on decoding imagined speech from EEG into a single framework. Every important aspect of designing such a system, such as selection of words to be imagined, number of electrodes to be recorded, temporal and spatial filtering, feature extraction and classifier are reviewed. This helps a researcher to compare the relative merits and demerits of the different approaches and choose the one that is most optimal. Speech being the most natural form of communication which human beings acquire even without formal education, imagined speech is an ideal choice of prompt for evoking brain activity patterns for a BCI (brain-computer interface) system, although the research on developing real-time (online) speech imagery based BCI systems is still in its infancy. Covert speech based BCI can help people with disabilities to improve their quality of life. It can also be used for covert communication in environments that do not support vocal communication. This paper also discusses some future directions, which will aid the deployment of speech imagery based BCI for practical applications, rather than only for laboratory experiments.

Over the past decade, many researchers have come up with different implementations of systems for decoding covert or imagined speech from EEG (electroencephalogram). They differ from each other in several aspects, from data acquisition to machine learning algorithms, due to which, a comparison between different implementations is often difficult. This review article puts together all the relevant works published in the last decade on decoding imagined speech from EEG into a single framework. Every important aspect of designing such a system, such as selection of words to be imagined, number of electrodes to be recorded, temporal and spatial filtering, feature extraction and classifier are reviewed. This helps a researcher to compare the relative merits and demerits of the different approaches and choose the one that is most optimal. Speech being the most natural form of communication which human beings acquire even without formal education, imagined speech is an ideal choice of prompt for evoking brain activity patterns for a BCI (brain-computer interface) system, although the research on developing real-time (online) speech imagery based BCI systems is still in its infancy. Covert speech based BCI can help people with disabilities to improve their quality of life. It can also be used for covert communication in environments that do not support vocal communication. This paper also discusses some future directions, which will aid the deployment of speech imagery based BCI for practical applications, rather than only for laboratory experiments. [ABSTRACT FROM AUTHOR]

Imagined speech is the inner pronunciation of words (unspoken speech, silent speech, or covert speech) without emitting sounds or making movements of face. Speech is the simple, normal and effective way people can communicate with one another. But some brain injuries caused by brain stroke, traumatic brain injuries, brain paralyses, stroke and ALS (amyotrophic lateral sclerosis) restrict vocal speech, even when the patient is completely conscious. People that cannot communicate due to neural conditions will benefit from a device that can specifically infer brain impulses from internal expression. For this speech impaired population, Brain Computer Interface(BCI) may be a possible communication alternative. A Brain Computer Interface that uses brain impulses to manipulate electronic equipment by a detection of complex variations of brain electrical activity needs to be applied in order to provide quiet communication skills between the two individuals to gather data relating to imaginary talk. This study document focuses on BCI research on the identification of imagined expression and explanations of end-to-end EEG signal processing methodology. [ABSTRACT FROM AUTHOR]

Brain-computer interface (BCI) is oriented toward intuitive systems that users can easily operate. Imagined speech and visual imagery are emerging paradigms that can directly convey a user’s intention. We investigated the underlying characteristics that affect the decoding performance of these two paradigms. Twenty-two subjects performed imagined speech and visual imagery of twelve words/phrases frequently used for patients’ communication. Spectral features were analyzed with thirteen-class classification (including rest class) using EEG filtered in six frequency ranges. In addition, cortical regions relevant to the two paradigms were analyzed by classification using single-channel and pre-defined cortical groups. Furthermore, we analyzed the word properties that affect the decoding performance based on the number of syllables, concrete and abstract concepts, and the correlation between the two paradigms. Finally, we investigated multiclass scalability in both paradigms. The high-frequency band displayed a significantly superior performance to that in the case of any other spectral features in the thirteen-class classification (imagined speech: 39.73 ± 5.64%; visual imagery: 40.14 ± 4.17%). Furthermore, the performance of Broca’s and Wernicke’s areas and auditory cortex was found to have improved among the cortical regions in both paradigms. As the number of classes increased, the decoding performance decreased moderately. Moreover, every subject exceeded the confidence level performance, implying the strength of the two paradigms in BCI inefficiency. These two intuitive paradigms were found to be highly effective for multiclass communication systems, having considerable similarities between each other. The results could provide crucial information for improving the decoding performance for practical BCI applications. [ABSTRACT FROM AUTHOR]

The use of imagined speech with electroencephalographic (EEG) signals is a promising field of brain-computer interfaces (BCI) that seeks communication between areas of the cerebral cortex related to language and devices or machines. However, the complexity of this brain process makes the analysis and classification of this type of signals a relevant topic of research. The goals of this study were: to develop a new algorithm based on Deep Learning (DL), referred to as CNNeeg1-1, to recognize EEG signals in imagined vowel tasks; to create an imagined speech database with 50 subjects specialized in imagined vowels from the Spanish language (/a/,/e/,/i/,/o/,/u/); and to contrast the performance of the CNNeeg1-1 algorithm with the DL Shallow CNN and EEGNet benchmark algorithms using an open access database (BD1) and the newly developed database (BD2). In this study, a mixed variance analysis of variance was conducted to assess the intra-subject and inter-subject training of the proposed algorithms. The results show that for intra-subject training analysis, the best performance among the Shallow CNN, EEGNet, and CNNeeg1-1 methods in classifying imagined vowels (/a/,/e/,/i/,/o/,/u/) was exhibited by CNNeeg1-1, with an accuracy of 65.62% for BD1 database and 85.66% for BD2 database.

Highlights • Bag of Features is able the discrimination of imagined speech in electroencephalograms. • Bag of Features aids the transfer learning of imagined speech to recognize new imagined words. • Using time domain features of the electroencephalogram for a Bag of Features model achieves a competitive classification performance. Abstract The Brain–Computer Interfaces (BCI) based on electroencephalograms (EEG) are systems which aim is to provide a communication channel to any person with a computer, initially it was proposed to aid people with disabilities, but actually wider applications have been proposed. These devices allow to send messages or to control devices using the brain signals. There are different neuro-paradigms which evoke brain signals of interest for such purposes. Imagined speech is one of the most recent paradigms, and it is explored in this work, it consists of the internal pronunciation of a word, i.e. a subject imagines the utterance of a word without emitting sounds or articulating facial movements. Under this neuro-paradigm, to increase the initial vocabulary reducing drastically the training time using few or none new data is an open challenge. The proposed method extracts characteristic units (i.e. codewords) of the EEGs associated with the words of an initial vocabulary. Subsequently, a new imagined word is represented with these codewords, and then a classification algorithm is applied. The method was tested both, with and without calibration examples, in a 27 subjects dataset. An initial vocabulary of 4 words, with 33 epochs for each word was considered. The results were obtained by averaging the accuracies of every subject, without calibration data a 65.65% accuracy was achieved. In comparison to the baseline method, which obtained an average accuracy of 68.9%, the proposed method showed no statistical difference. [ABSTRACT FROM AUTHOR]

Highlights • An EEG-based imagined speech system was assessed for subjects recognition. • More than one word can be used as a password without decreasing the method performance. • Impact of instances/channels reduction and feature extraction techniques were studied. Abstract Due to the problems presented in current traditional/biometric security systems, the interest to use new security systems, have been increasing. This paper explores the use of brain signals EEG-based during imagined speech in order to use it as a new biometric measure for Subjects identification and thus create a new biometric security system. The main contribution of this paper are two methods for feature extraction, first to improve the signal-to-noise ratio the Common Average Reference was applied. The first method was based on Discrete Wavelet Transform, and the second method was based on statistical features directly from the raw signal. The proposed methods were tested in a dataset of 27 Subjects who performed 33 repetitions of 5 imagined words in Spanish. The results show the feasibility of the task with accurate identification of the Subject, regardless of the imagined word used and using a commercial EEG system (EMOTIV EPOC). In addition, the scope of the method is displayed by decreasing the training data, as well as the number of active sensors for the identification task. Using the proposed method with future improvements and implementing it in a low-cost device can be a new and valuable biometric security system. [ABSTRACT FROM AUTHOR]

Imagined speech is a relatively new electroencephalography (EEG) neuro-paradigm, which has seen little use in Brain-Computer Interface (BCI) applications. Imagined speech can be used to allow physically impaired patients to communicate and to use smart devices by imagining desired commands and then detecting and executing those commands in a smart device. The goal of this research is to verify previous classification attempts made and then design a new, more efficient neural network that is noticeably less complex (fewer number of layers) that still achieves a comparable classification accuracy. The classifiers are designed to distinguish between EEG signal patterns corresponding to imagined speech of different vowels and words. This research uses a dataset that consists of 15 subjects imagining saying the five main vowels (a, e, i, o, u) and six different words. Two previous studies on imagined speech classifications are verified as those studies used the same dataset used here. The replicated results are compared. The main goal of this study is to take the proposed convolutional neural network (CNN) model from one of the replicated studies and make it much more simpler and less complex, while attempting to retain a similar accuracy. The pre-processing of data is described and a new CNN classifier with three different transfer learning methods is described and used to classify EEG signals. Classification accuracy is used as the performance metric. The new proposed CNN, which uses half as many layers and less complex pre-processing methods, achieved a considerably lower accuracy, but still managed to outperform the initial model proposed by the authors of the dataset by a considerable margin. It is recommended that further studies investigating classifying imagined speech should use more data and more powerful machine learning techniques. Transfer learning proved beneficial and should be used to improve the effectiveness of neural networks.

Introduction: Paralyzed and physically impaired patients face communication difficulties, even when they are mentally coherent and aware. Electroencephalographic (EEG) brain-computer interfaces (BCIs) offer a potential communication method for these people without invasive surgery or physical device controls., Methods: Although virtual keyboard protocols are well documented in EEG BCI paradigms, these implementations are visually taxing and fatiguing. All English words combine 44 unique phonemes, each corresponding to a unique EEG pattern. In this study, a complete phoneme-based imagined speech EEG BCI was developed and tested on 16 subjects., Results: Using open-source hardware and software, machine learning models, such as k-nearest neighbor (KNN), reliably achieved a mean accuracy of 97 ± 0.001%, a mean F1 of 0.55 ± 0.01, and a mean AUC-ROC of 0.68 ± 0.002 in a modified one-versus-rest configuration, resulting in an information transfer rate of 304.15 bits per minute. In line with prior literature, the distinguishing feature between phonemes was the gamma power on channels F3 and F7., Discussion: However, adjustments to feature selection, trial window length, and classifier algorithms may improve performance. In summary, these are iterative changes to a viable method directly deployable in current, commercially available systems and software. The development of an intuitive phoneme-based EEG BCI with open-source hardware and software demonstrates the potential ease with which the technology could be deployed in real-world applications., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2023 LaRocco, Tahmina, Lecian, Moore, Helbig and Gupta.)

• This paper proposes a supervised CatPCA approach for performing the imagined speech classification. • The objective function of an optimization problem for designing the unitary matrix is formulated based on both the PCA algorithm and the k means algorithm. • To find the analytical solution of the optimization problem, a property on computing the singular value decomposition (SVD) of a symmetric matrix is employed. • Then, the design of the cluster centers in the k means algorithm is formulated as a quadratic programming. • Finally, the above two procedures are iterated until the algorithm converges. • The required computational power for finding the solution of the optimization problem is low. The dimension of the feature vectors for performing the imagined speech is large. Hence, it is required to reduce their dimension as well as perform the classification simultaneously. This is a categorized principal component analysis (CatPCA) problem. However, the conventional CatPCA methods are unsupervised learning methods and the design of the unitary matrix in the conventional principal component analysis (PCA) algorithms is only based on the PCA approach. Therefore, the classification performance is limited. This paper proposes a supervised CatPCA approach for performing the imagined speech classification. Also, the objective function of an optimization problem for designing the unitary matrix is formulated based on both the PCA algorithm and the k means algorithm. To find the analytical solution of the optimization problem, a property on computing the singular value decomposition (SVD) of a symmetric matrix is employed. Then, the design of the cluster centers in the k means algorithm is formulated as a quadratic programming. Finally, the above two procedures are iterated until the algorithm converges. Since the analytical solutions of these two optimization problems can be derived analytically, the required computational power for finding the solution of the optimization problem is low. To demonstrate the effectiveness of our proposed CatPCA algorithm, the Track 3 of the International brain computer interface (BCI) Competition Database is employed as the dataset for performing the performance evaluation. The computer numerical simulation results show that our proposed CatPCA algorithm yields the higher classification accuracy compared to PCA variants and the dimension reduction related algorithms. Also, the required execution time of our proposed method is lower than those of the states of the arts methods. This demonstrates the effectiveness and the efficiency of our proposed CatPCA algorithm. [ABSTRACT FROM AUTHOR]

Brain-computer interface (BCI) technology has many applications in various scientific fields, such as used in communication (speech recognition). The data of imagery speech has been collected in electroencephalogram (EEG) signals. In this paper, we propose an approach for EEG feature extraction of imagined speech with high accuracy and efficiency. In this way, we improve the common spatial pattern (CSP) binary algorithm to multiclass level in two parts 'One-vs-One' and 'One-vs-All'. The "Kara One" dataset is used in this research that includes EEG signals of thirteen subjects with twelve trials and sixty-four channels for any four English words signals and seven English phonemes signals. We compared our proposed CSP to other imagined speech feature methods. The classification accuracy of the second part of the proposed method is 97.34% in the subject-wise overall model which is 19.97% better than the best previous result. We have obtained the highest classification accuracy for sixty-four channels, which is the highest accuracy ever achieved using this database. Our proposed model is ready to be tested with more EEG data. This proposed work, which includes an ensemble method for classifying speech imagery words, can greatly contribute to intuitive BCI development using silent speech. [ABSTRACT FROM AUTHOR]

EEG-based Brain–Computer Interfaces (BCI) are an alternative technique that aims to integrate people with severe motor disabilities to their environment. However, they still are not used in everyday life because BCIs are controlled by means of few intuitive electrophysiological sources. To address this problem, work has been carried out with the objective of classifying EEG signals recorded while imagining speech. In this paper, the emerging techniques of sonification and textification are applied to EEG signals, which allows us to characterize EEG signals as either an audio signal or a text document. The aim is to analyze whether these techniques can help to do a better discrimination or to highlight patterns, and assess which of the two methods performs better in order to improve classification results of unspoken words. For proving this we processed signals, from 27 subjects, with and without the application of the proposed methods of sonification and textification. The average accuracy rate using the original EEG signals, the EEG sonified signals and the EEG textified signals are 58.41 ± 12.41%, 63.82 ± 13.44% and 83.34 ± 6.12%, respectively. It can be clearly noticed that the methods of sonification and textification of EEG improve the classification rates obtained using the EEG signals in their original form. [ABSTRACT FROM AUTHOR]

• Stereotactic electroencephalography was recorded during overt, mouthed, and imagined speech. • Relevant channels form a nested hierarchy according to degree of behavioral output. • Findings suggest a common neural substrate of related speech processes. • Broad and deep brain regions are relevant for speech activity detection models. • Overt speech can be a viable surrogate for imagined speech detection models. Recent studies have demonstrated that it is possible to decode and synthesize various aspects of acoustic speech directly from intracranial measurements of electrophysiological brain activity. In order to continue progressing toward the development of a practical speech neuroprosthesis for the individuals with speech impairments, better understanding and modeling of imagined speech processes are required. The present study uses intracranial brain recordings from participants that performed a speaking task with trials consisting of overt, mouthed, and imagined speech modes, representing various degrees of decreasing behavioral output. Speech activity detection models are constructed using spatial, spectral, and temporal brain activity features, and the features and model performances are characterized and compared across the three degrees of behavioral output. The results indicate the existence of a hierarchy in which the relevant channels for the lower behavioral output modes form nested subsets of the relevant channels from the higher behavioral output modes. This provides important insights for the elusive goal of developing more effective imagined speech decoding models with respect to the better-established overt speech decoding counterparts. [ABSTRACT FROM AUTHOR]

Brain–computer interfaces (BCIs) aim to decode brain signals and transform them into commands for device operation. The present study aimed to decode the brain activity during imagined speech. The BCI must identify imagined words within a given vocabulary and thus perform the requested action. A possible scenario when using this approach is the gradual addition of new words to the vocabulary using incremental learning methods. An issue with incremental learning methods is degradation of the decoding capacity of the original model when new classes are added. In this study, a class-incremental neural network method is proposed to increase the vocabulary of imagined speech. The results indicate a stable model that did not degenerate when a new word was integrated. The proposed method allows for the inclusion of newly imagined words without a significant loss of total accuracy for the two datasets. • Model based on neural networks for imagined speech discrimination. • Intra-subject incremental learning approach of imagined speech BCIs. • Incremental twin neural networks for incremental learning. [ABSTRACT FROM AUTHOR]

One of the main purposes of brain-computer interfaces (BCI) is to provide persons of an alternative communication channel. This objective was firstly focused on handicapped subjects but nowadays its scope has increased to healthy persons. Usually, BCIs record brain activity using electroencephalograms (EEG), according to four main neuro-paradigms (slow cortical potentials, motor imagery, P300 component and visual evoked potentials). These analytical paradigms are not intuitive and are difficult to implement. Accordingly, this work researches an alternative neuro-paradigm called imagined speech, which refers to the internal pronunciation of words without emitting sounds or doing facial movements. Specifically, the present research is focused on the recognition of five Spanish words corresponding to the English words “up,” “down,” “left,” “right” and “select”, with which a computer cursor could be controlled. We perform an offline computer automatic classification procedure of a dataset of EEG signals from 27 subjects. The method implements a channel selection composed of two stages; the first one obtains a Pareto front and is approached as a multi-objective optimization problem dealing with the error rate and the number of channels; the second stage selects a single solution (channel combination) from the front, applying a fuzzy inference system (FIS). We assess the method’s performance through a channel combination and a test set not used to generate the front. Several FIS configurations were explored to evaluate if a FIS is able to select channel combinations that improve or, at least, keep the obtained accuracies using all channels for each subject’s data. We found that a FIS configuration, FIS3×3 (three membership functions for both input variables: error rate and the number of channels), obtained the best trade-off between the number of fuzzy rules and its accuracy (68.18% using around 7 channels). Also, the FIS3×3 obtained a similar statistically accuracy compared to the use of all channels (70.33%). Results of our method demonstrate the feasibility of using a FIS to automatically select a solution from the Pareto front to select channels applied to a problem of imagined speech classification. The presented method outperforms previous works in accuracy and showed a dependence relationship between EEG data and imagined words. [ABSTRACT FROM AUTHOR]

Speech production involves the careful orchestration of sophisticated systems, yet overt speech errors rarely occur under naturalistic conditions. The present functional magnetic resonance imaging study sought neural evidence for internal error detection and correction by leveraging a tongue twister paradigm that induces the potential for speech errors while excluding any overt errors from analysis. Previous work using the same paradigm in the context of silently articulated and imagined speech production tasks has demonstrated forward predictive signals in auditory cortex during speech and presented suggestive evidence of internal error correction in left posterior middle temporal gyrus (pMTG) on the basis that this area tended toward showing a stronger response when potential speech errors are biased toward nonwords compared to words (Okada et al., 2018). The present study built on this prior work by attempting to replicate the forward prediction and lexicality effects in nearly twice as many participants but introduced novel stimuli designed to further tax internal error correction and detection mechanisms by biasing speech errors toward taboo words. The forward prediction effect was replicated. While no evidence was found for a significant difference in brain response as a function of lexical status of the potential speech error, biasing potential errors toward taboo words elicited significantly greater response in left pMTG than biasing errors toward (neutral) words. Other brain areas showed preferential response for taboo words as well but responded below baseline and were less likely to reflect language processing as indicated by a decoding analysis, implicating left pMTG in internal error correction., Competing Interests: Competing Interests: The authors have declared that no competing interests exist., (© 2023 Massachusetts Institute of Technology.)