Search

Objective. When listening to continuous speech, populations of neurons in the brain track different features of the signal. Neural tracking can be measured by relating the electroencephalography (EEG) and the speech signal. Recent studies have shown a significant contribution of linguistic features over acoustic neural tracking using linear models. However, linear models cannot model the nonlinear dynamics of the brain. To overcome this, we use a convolutional neural network (CNN) that relates EEG to linguistic features using phoneme or word onsets as a control and has the capacity to model non-linear relations. Approach. We integrate phoneme- and word-based linguistic features (phoneme surprisal, cohort entropy (CE), word surprisal (WS) and word frequency (WF)) in our nonlinear CNN model and investigate if they carry additional information on top of lexical features (phoneme and word onsets). We then compare the performance of our nonlinear CNN with that of a linear encoder and a linearized CNN. Main results. For the non-linear CNN, we found a significant contribution of CE over phoneme onsets and of WS and WF over word onsets. Moreover, the non-linear CNN outperformed the linear baselines. Significance. Measuring coding of linguistic features in the brain is important for auditory neuroscience research and applications that involve objectively measuring speech understanding. With linear models, this is measurable, but the effects are very small. The proposed non-linear CNN model yields larger differences between linguistic and lexical models and, therefore, could show effects that would otherwise be unmeasurable and may, in the future, lead to improved within-subject measures and shorter recordings., (© 2023 IOP Publishing Ltd.)

Objective. When a person listens to continuous speech, a corresponding response is elicited in the brain and can be recorded using electroencephalography (EEG). Linear models are presently used to relate the EEG recording to the corresponding speech signal. The ability of linear models to find a mapping between these two signals is used as a measure of neural tracking of speech. Such models are limited as they assume linearity in the EEG-speech relationship, which omits the nonlinear dynamics of the brain. As an alternative, deep learning models have recently been used to relate EEG to continuous speech. Approach. This paper reviews and comments on deep-learning-based studies that relate EEG to continuous speech in single- or multiple-speakers paradigms. We point out recurrent methodological pitfalls and the need for a standard benchmark of model analysis. Main results. We gathered 29 studies. The main methodological issues we found are biased cross-validations, data leakage leading to over-fitted models, or disproportionate data size compared to the model's complexity. In addition, we address requirements for a standard benchmark model analysis, such as public datasets, common evaluation metrics, and good practices for the match-mismatch task. Significance. We present a review paper summarizing the main deep-learning-based studies that relate EEG to speech while addressing methodological pitfalls and important considerations for this newly expanding field. Our study is particularly relevant given the growing application of deep learning in EEG-speech decoding., (© 2023 IOP Publishing Ltd.)

Speech comprehension is a complex neural process on which relies on activation and integration of multiple brain regions. In the current study, we evaluated whether speech comprehension can be investigated by neural tracking. Neural tracking is the phenomenon in which the brain responses time-lock to the rhythm of specific features in continuous speech. These features can be acoustic, i.e., acoustic tracking, or derived from the content of the speech using language properties, i.e., language tracking. We evaluated whether neural tracking of speech differs between a comprehensible story, an incomprehensible story, and a word list. We evaluated the neural responses to speech of 19 participants (six men). No significant difference regarding acoustic tracking was found. However, significant language tracking was only found for the comprehensible story. The most prominent effect was visible to word surprisal, a language feature at the word level. The neural response to word surprisal showed a prominent negativity between 300 and 400 ms, similar to the N400 in evoked response paradigms. This N400 was significantly more negative when the story was comprehended, i.e., when words could be integrated in the context of previous words. These results show that language tracking can capture the effect of speech comprehension., (Copyright © 2023 Gillis et al.)

Objectives: Audiological rehabilitation includes sensory management, auditory training (AT), and counseling and can alleviate the negative consequences associated with (untreated) hearing impairment. AT aims at improving auditory skills through structured analytical (bottom-up) or synthetic (top-down) listening exercises. The evidence for AT to improve auditory outcomes of postlingually deafened adults with a cochlear implant (CI) remains a point of debate due to the relatively limited number of studies and methodological shortcomings. There is a general agreement that more rigorous scientific study designs are needed to determine the effectiveness, generalization, and consolidation of AT for CI users. The present study aimed to investigate the effectiveness of a personalized AT program compared to a nonpersonalized Active Control program with adult CI users in a stratified randomized controlled clinical trial., Design: Off-task outcomes were sentence understanding in noise, executive functioning, and health-related quality of life. Participants were tested before and after 16 weeks of training and after a further 8 months without training. Participant expectations of the training program were assessed before the start of training., Results: The personalized and nonpersonalized AT programs yielded similar results. Significant on-task improvements were observed. Moreover, AT generalized to improved speech understanding in noise for both programs. Half of the CI users reached a clinically relevant improvement in speech understanding in noise of at least 2 dB SNR post-training. These improvements were maintained 8 months after completion of the training. In addition, a significant improvement in quality of life was observed for participants in both treatment groups. Adherence to the training programs was high, and both programs were considered user-friendly., Conclusions: Training in both treatments yielded similar results. For half of the CI users, AT transferred to better performance with generalization of learning for speech understanding in noise and quality of life. Our study supports the previous findings that AT can be beneficial for some CI users., Competing Interests: The authors have no conflicts of interest to disclose., (Copyright © 2022 Wolters Kluwer Health, Inc. All rights reserved.)

Objective. The human brain tracks the temporal envelope of speech, which contains essential cues for speech understanding. Linear models are the most common tool to study neural envelope tracking. However, information on how speech is processed can be lost since nonlinear relations are precluded. Analysis based on mutual information (MI), on the other hand, can detect both linear and nonlinear relations and is gradually becoming more popular in the field of neural envelope tracking. Yet, several different approaches to calculating MI are applied with no consensus on which approach to use. Furthermore, the added value of nonlinear techniques remains a subject of debate in the field. The present paper aims to resolve these open questions. Approach. We analyzed electroencephalography (EEG) data of participants listening to continuous speech and applied MI analyses and linear models. Main results. Comparing the different MI approaches, we conclude that results are most reliable and robust using the Gaussian copula approach, which first transforms the data to standard Gaussians. With this approach, the MI analysis is a valid technique for studying neural envelope tracking. Like linear models, it allows spatial and temporal interpretations of speech processing, peak latency analyses, and applications to multiple EEG channels combined. In a final analysis, we tested whether nonlinear components were present in the neural response to the envelope by first removing all linear components in the data. We robustly detected nonlinear components on the single-subject level using the MI analysis. Significance. We demonstrate that the human brain processes speech in a nonlinear way. Unlike linear models, the MI analysis detects such nonlinear relations, proving its added value to neural envelope tracking. In addition, the MI analysis retains spatial and temporal characteristics of speech processing, an advantage lost when using more complex (nonlinear) deep neural networks., (© 2023 IOP Publishing Ltd.)

Hearing aids (HA) are a fundamental component in restoring auditory function; however, they cannot completely alleviate all problems encountered by adults with hearing impairment. The aim of this study is twofold. Firstly, we assess the health-related quality of life and coping strategies of experienced HA users. Secondly, we assess whether HA users can benefit from auditory training. To this end, 40 participants who had worn HAs for more than 6 months participated in this study. Half of the participants received auditory training, while the other half served as a passive control. The training consisted of a personalized training scheme, with outcome measures including speech in noise perception in free-field and via direct streaming to the HA, phoneme identification, cognitive control, and health-related quality of life. Results showed that experienced HA users reported a relatively good quality of life. Health-related quality of life was correlated with aided speech perception in noise, but not with aided pure tone audiometry. Coping strategies were adaptive, leading to improved communication. Participants showed improvements in trained tasks, consonant identification, and speech in noise perception. While both groups yielded improved speech in noise perception at the end, post hoc analysis following a three-way interaction showed a significantly larger pre-post difference for the trained group in the streaming condition. Although training showed some improvements, the study suggests that the training paradigm was not sufficiently challenging for HA users. To optimize daily life listening, we recommend that future training should incorporate more exercises in noise and focus on cognitive control.

The goal of auditory attention decoding (AAD) is to determine to which speaker out of multiple competing speakers a listener is attending based on the brain signals recorded via, e.g., electroencephalography (EEG). AAD algorithms are a fundamental building block of so-called neuro-steered hearing devices that would allow identifying the speaker that should be amplified based on the brain activity. A common approach is to train a subject-specific stimulus decoder that reconstructs the amplitude envelope of the attended speech signal. However, training this decoder requires a dedicated 'ground-truth' EEG recording of the subject under test, during which the attended speaker is known. Furthermore, this decoder remains fixed during operation and can thus not adapt to changing conditions and situations. Therefore, we propose an online time-adaptive unsupervised stimulus reconstruction method that continuously and automatically adapts over time when new EEG and audio data are streaming in. The adaptive decoder does not require ground-truth attention labels obtained from a training session with the end-user and instead can be initialized with a generic subject-independent decoder or even completely random values. We propose two different implementations: a sliding window and recursive implementation, which we extensively validate on three independent datasets based on multiple performance metrics. We show that the proposed time-adaptive unsupervised decoder outperforms a time-invariant supervised decoder, representing an important step toward practically applicable AAD algorithms for neuro-steered hearing devices.

Speech-perception testing is essential for monitoring outcomes with a hearing aid or cochlear implant (CI). However, clinical care is time-consuming and often challenging with an increasing number of clients. A potential approach to alleviating some clinical care and possibly making room for other outcome measures is to employ technologies that assess performance in the home environment. In this study, we investigate 3 different speech perception indices in the same 40 CI users: phoneme identification (vowels and consonants), digits in noise (DiN) and sentence recognition in noise (SiN). The first two tasks were implemented on a tablet and performed multiple times by each client in their home environment, while the sentence task was administered at the clinic. Speech perception outcomes in the same forty CI users showed that DiN assessed at home can serve as an alternative to SiN assessed at the clinic. DiN scores are in line with the SiN ones by 3-4 dB improvement and are useful to monitor performance at regular intervals and to detect changes in auditory performance. Phoneme identification in quiet also explains a significant part of speech perception in noise, and provides additional information on the detectability and discriminability of speech cues. The added benefit of the phoneme identification task, which also proved to be easy to administer at home, is the information transmission analysis in addition to the summary score. Performance changes for the different indices can be interpreted by comparing against measurement error and help to target personalized rehabilitation. Altogether, home-based speech testing is reliable and proves powerful to complement care in the clinic for CI users., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2021 van Wieringen, Magits, Francart and Wouters.)

Objective. Currently, only behavioral speech understanding tests are available, which require active participation of the person being tested. As this is infeasible for certain populations, an objective measure of speech intelligibility is required. Recently, brain imaging data has been used to establish a relationship between stimulus and brain response. Linear models have been successfully linked to speech intelligibility but require per-subject training. We present a deep-learning-based model incorporating dilated convolutions that operates in a match/mismatch paradigm. The accuracy of the model's match/mismatch predictions can be used as a proxy for speech intelligibility without subject-specific (re)training. Approach. We evaluated the performance of the model as a function of input segment length, electroencephalography (EEG) frequency band and receptive field size while comparing it to multiple baseline models. Next, we evaluated performance on held-out data and finetuning. Finally, we established a link between the accuracy of our model and the state-of-the-art behavioral MATRIX test. Main results. The dilated convolutional model significantly outperformed the baseline models for every input segment length, for all EEG frequency bands except the delta and theta band, and receptive field sizes between 250 and 500 ms. Additionally, finetuning significantly increased the accuracy on a held-out dataset. Finally, a significant correlation ( r = 0.59, p = 0.0154) was found between the speech reception threshold (SRT) estimated using the behavioral MATRIX test and our objective method. Significance. Our method is the first to predict the SRT from EEG for unseen subjects, contributing to objective measures of speech intelligibility., (© 2021 IOP Publishing Ltd.)

When multiple speakers talk simultaneously, a hearing device cannot identify which of these speakers the listener intends to attend to. Auditory attention decoding (AAD) algorithms can provide this information by, for example, reconstructing the attended speech envelope from electroencephalography (EEG) signals. However, these stimulus reconstruction decoders are traditionally trained in a supervised manner, requiring a dedicated training stage during which the attended speaker is known. Pre-trained subject-independent decoders alleviate the need of having such a per-user training stage but perform substantially worse than supervised subject-specific decoders that are tailored to the user. This motivates the development of a new unsupervised self-adapting training/updating procedure for a subject-specific decoder, which iteratively improves itself on unlabeled EEG data using its own predicted labels. This iterative updating procedure enables a self-leveraging effect, of which we provide a mathematical analysis that reveals the underlying mechanics. The proposed unsupervised algorithm, starting from a random decoder, results in a decoder that outperforms a supervised subject-independent decoder. Starting from a subject-independent decoder, the unsupervised algorithm even closely approximates the performance of a supervised subject-specific decoder. The developed unsupervised AAD algorithm thus combines the two advantages of a supervised subject-specific and subject-independent decoder: it approximates the performance of the former while retaining the 'plug-and-play' character of the latter. As the proposed algorithm can be used to automatically adapt to new users, as well as over time when new EEG data is being recorded, it contributes to more practical neuro-steered hearing devices.

Objective: Noise reduction algorithms in current hearing devices lack informationabout the sound source a user attends to when multiple sources are present. To resolve this issue, they can be complemented with auditory attention decoding (AAD) algorithms, which decode the attention using electroencephalography (EEG) sensors. State-of-the-art AAD algorithms employ a stimulus reconstruction approach, in which the envelope of the attended source is reconstructed from the EEG and correlated with the envelopes of the individual sources. This approach, however, performs poorly on short signal segments, whilelonger segments yield impractically long detection delays when the user switches attention., Methods: We propose decoding the directional focus of attention using filterbank common spatial pattern filters (FB-CSP) as an alternative AAD paradigm, whichdoes not require access to the clean source envelopes., Results: The proposed FB-CSP approach outperforms both the stimulus reconstruction approach on short signal segments, as well as a convolutional neural network approach on the same task. We achieve a high accuracy (80% for [Formula: see text] windows and 70% for quasi-instantaneous decisions), which is sufficient to reach minimal expected switch durations below [Formula: see text]. We also demonstrate that the decoder can adapt to unlabeled data from anunseen subject and works with only a subset of EEG channels located around the ear to emulate a wearable EEG setup., Conclusion: The proposed FB-CSP method provides fast and accurate decoding of the directional focus of auditory attention., Significance: The high accuracy on very short data segments is a major step forward towards practical neuro-steered hearing devices.

Objectives: Recently, an objective measure of speech intelligibility (SI), based on brain responses derived from the electroencephalogram (EEG), has been developed using isolated Matrix sentences as a stimulus. We investigated whether this objective measure of SI can also be used with natural speech as a stimulus, as this would be beneficial for clinical applications., Design: We recorded the EEG in 19 normal-hearing participants while they listened to two types of stimuli: Matrix sentences and a natural story. Each stimulus was presented at different levels of SI by adding speech weighted noise. SI was assessed in two ways for both stimuli: (1) behaviorally and (2) objectively by reconstructing the speech envelope from the EEG using a linear decoder and correlating it with the acoustic envelope. We also calculated temporal response functions (TRFs) to investigate the temporal characteristics of the brain responses in the EEG channels covering different brain areas., Results: For both stimulus types, the correlation between the speech envelope and the reconstructed envelope increased with increasing SI. In addition, correlations were higher for the natural story than for the Matrix sentences. Similar to the linear decoder analysis, TRF amplitudes increased with increasing SI for both stimuli. Remarkable is that although SI remained unchanged under the no-noise and +2.5 dB SNR conditions, neural speech processing was affected by the addition of this small amount of noise: TRF amplitudes across the entire scalp decreased between 0 and 150 ms, while amplitudes between 150 and 200 ms increased in the presence of noise. TRF latency changes in function of SI appeared to be stimulus specific: the latency of the prominent negative peak in the early responses (50 to 300 ms) increased with increasing SI for the Matrix sentences, but remained unchanged for the natural story., Conclusions: These results show (1) the feasibility of natural speech as a stimulus for the objective measure of SI; (2) that neural tracking of speech is enhanced using a natural story compared to Matrix sentences; and (3) that noise and the stimulus type can change the temporal characteristics of the brain responses. These results might reflect the integration of incoming acoustic features and top-down information, suggesting that the choice of the stimulus has to be considered based on the intended purpose of the measurement.

Objectives: To investigate the mechanisms behind binaural and spatial effects in speech understanding for bimodal cochlear implant listeners. In particular, to test our hypothesis that their speech understanding can be characterized by means of monaural signal to noise ratios, rather than complex binaural cue processing such as binaural unmasking., Design: We applied a semantic framework to characterize binaural and spatial effects in speech understanding on an extensive selection of the literature on bimodal listeners. In addition, we performed two experiments in which we measured speech understanding in different masker types (1) using head-related transfer functions, and (2) while adapting the broadband signal to noise ratios in both ears independently. We simulated bimodal hearing with a vocoder in one ear (the cochlear implant side) and a low-pass filter in the other ear (the hearing aid side). By design, the cochlear implant side was the main contributor to speech understanding in our simulation., Results: We found that spatial release from masking can be explained as a simple trade-off between a monaural change in signal to noise at the cochlear implant side (quantified as the head shadow effect) and an opposite change in signal to noise at the hearing aid side (quantified as a change in bimodal benefit). In simulated bimodal listeners, we found that for every 1 dB increase in signal to noise ratio at the hearing aid side, the bimodal benefit improved by approximately 0.4 dB in signal to noise ratio., Conclusions: Although complex binaural cue processing is often implicated when discussing speech intelligibility in adverse listening conditions, performance can simply be explained based on monaural signal to noise ratios for bimodal listeners.

Objectives: To establish a framework to unambiguously define and relate the different spatial effects in speech understanding: head shadow, redundancy, squelch, spatial release from masking (SRM), and so on. Next, to investigate the contribution of interaural time and level differences to these spatial effects in speech understanding and how this is influenced by the type of masking noise., Design: In our framework, SRM is uniquely characterized as a linear combination of head shadow, binaural redundancy, and binaural squelch. The latter two terms are combined into one binaural term, which we define as binaural contrast: a benefit of interaural differences. In this way, SRM is a simple sum of a monaural and a binaural term. We used the framework to quantify these spatial effects in 10 listeners with normal hearing. The participants performed speech intelligibility tasks in different spatial setups. We used head-related transfer functions to manipulate the presence of interaural time and level differences. We used three spectrally matched masker types: stationary speech-weighted noise, a competing talker, and speech-weighted noise that was modulated with the broadband temporal envelope of the competing talker., Results: We found that (1) binaural contrast was increased by interaural time differences, but reduced by interaural level differences, irrespective of masker type, and (2) large redundancy (the benefit of having identical information in two ears) could reduce binaural contrast and thus also reduce SRM., Conclusions: Our framework yielded new insights in binaural processing in speech intelligibility. First, interaural level differences disturb speech intelligibility in realistic listening conditions. Therefore, to optimize speech intelligibility in hearing aids, it is more beneficial to improve monaural signal-to-noise ratios rather than to preserve interaural level differences. Second, although redundancy is mostly ignored when considering spatial hearing, it might explain reduced SRM in some cases.

Objective: When listening to speech, the brain tracks the speech envelope. It is possible to reconstruct this envelope from EEG recordings. However, in people who hear using a cochlear implant (CI), the artifacts caused by electrical stimulation of the auditory nerve contaminate the EEG. The objective of this study is to develop and validate a method for assessing the neural tracking of speech envelope in CI users., Approach: To obtain EEG recordings free of stimulus artifacts, the electrical stimulation is periodically interrupted. During these stimulation gaps, artifact-free EEG can be sampled and used to train a linear envelope decoder. EEG recordings obtained during audible and inaudible (i.e. sub-threshold) stimulation were used to characterize the artifacts and their influence on the envelope reconstruction., Main Results: The present study demonstrates for the first time that neural tracking of the speech envelope can be measured in response to ongoing electrical stimulation. The responses were validated to be truly neural and not affected by stimulus artifact., Significance: Besides applications in audiology and neuroscience, the characterization and elimination of stimulus artifacts will enable future EEG studies involving continuous speech in CI users. Measures of neural tracking of the speech envelope reflect interesting properties of the listener's perception of speech, such as speech intelligibility or attentional state. Successful decoding of neural envelope tracking will open new possibilities to investigate the neural mechanisms of speech perception with a CI.

Currently there is a growing population of cochlear-implant (CI) users with (near) normal hearing in the non-implanted ear. This configuration is often called SSD (single-sided deafness) CI. The goal of the CI is often to improve spatial perception, so the question raises to what extent SSD CI listeners are sensitive to interaural time differences (ITDs). In a controlled lab setup, sensitivity to ITDs was investigated in 11 SSD CI listeners. The stimuli were 100-pps pulse trains on the CI side and band-limited click trains on the acoustic side. After determining level balance and the delay needed to achieve synchronous stimulation of the two ears, the just noticeable difference in ITD was measured using an adaptive procedure. Seven out of 11 listeners were sensitive to ITDs, with a median just noticeable difference of 438 μs. Out of the four listeners who were not sensitive to ITD, one listener reported binaural fusion, and three listeners reported no binaural fusion. To enable ITD sensitivity, a frequency-dependent delay of the electrical stimulus was required to synchronize the electric and acoustic signals at the level of the auditory nerve. Using subjective fusion measures and refined by ITD sensitivity, it was possible to match a CI electrode to an acoustic frequency range. This shows the feasibility of these measures for the allocation of acoustic frequency ranges to electrodes when fitting a CI to a subject with (near) normal hearing in the contralateral ear.

Speech intelligibility is currently measured by scoring how well a person can identify a speech signal. The results of such behavioral measures reflect neural processing of the speech signal, but are also influenced by language processing, motivation, and memory. Very often, electrophysiological measures of hearing give insight in the neural processing of sound. However, in most methods, non-speech stimuli are used, making it hard to relate the results to behavioral measures of speech intelligibility. The use of natural running speech as a stimulus in electrophysiological measures of hearing is a paradigm shift which allows to bridge the gap between behavioral and electrophysiological measures. Here, by decoding the speech envelope from the electroencephalogram, and correlating it with the stimulus envelope, we demonstrate an electrophysiological measure of neural processing of running speech. We show that behaviorally measured speech intelligibility is strongly correlated with our electrophysiological measure. Our results pave the way towards an objective and automatic way of assessing neural processing of speech presented through auditory prostheses, reducing confounds such as attention and cognitive capabilities. We anticipate that our electrophysiological measure will allow better differential diagnosis of the auditory system, and will allow the development of closed-loop auditory prostheses that automatically adapt to individual users.

Objectives: Auditory steady state responses (ASSRs) are used in clinical practice for objective hearing assessments. The response is called steady state because it is assumed to be stable over time, and because it is evoked by a stimulus with a certain periodicity, which will lead to discrete frequency components that are stable in amplitude and phase over time. However, the stimuli commonly used to evoke ASSRs are also known to be able to induce loudness adaptation behaviorally. Researchers and clinicians using ASSRs assume that the response remains stable over time. This study investigates (1) the stability of ASSR amplitudes over time, within one recording, and (2) whether loudness adaptation can be reflected in ASSRs., Design: ASSRs were measured from 14 normal-hearing participants. The ASSRs were evoked by the stimuli that caused the most loudness adaptation in a previous behavioral study, that is, mixed-modulated sinusoids with carrier frequencies of either 500 or 2000 Hz, a modulation frequency of 40 Hz, and a low sensation level of 30 dB SL. For each carrier frequency and participant, 40 repetitions of 92 sec recordings were made. Two types of analyses were used to investigate the ASSR amplitudes over time: with the more traditionally used Fast Fourier Transform and with a novel Kalman filtering approach. Robust correlations between the ASSR amplitudes and behavioral loudness adaptation ratings were also calculated., Results: Overall, ASSR amplitudes were stable. Over all individual recordings, the median change of the amplitudes over time was -0.0001 μV/s. Based on group analysis, a significant but very weak decrease in amplitude over time was found, with the decrease in amplitude over time around -0.0002 μV/s. Correlation coefficients between ASSR amplitudes and behavioral loudness adaptation ratings were significant but low to moderate, with r = 0.27 and r = 0.39 for the 500 and 2000 Hz carrier frequency, respectively., Conclusions: The decrease in amplitude of ASSRs over time (92 sec) is small. Consequently, it is safe to use ASSRs in clinical practice, and additional correction factors for objective hearing assessments are not needed. Because only small decreases in amplitudes were found, loudness adaptation is probably not reflected by the ASSRs.

State-of-the-art hearing prostheses are equipped with acoustic noise reduction algorithms to improve speech intelligibility. Currently, one of the major challenges is to perform acoustic noise reduction in so-called cocktail party scenarios with multiple speakers, in particular because it is difficult-if not impossible-for the algorithm to determine which are the target speaker(s) that should be enhanced, and which speaker(s) should be treated as interfering sources. Recently, it has been shown that electroencephalography (EEG) can be used to perform auditory attention detection, i.e., to detect to which speaker a subject is attending based on recordings of neural activity. In this paper, we combine such an EEG-based auditory attention detection (AAD) paradigm with an acoustic noise reduction algorithm based on the multi-channel Wiener filter (MWF), leading to a neuro-steered MWF. In particular, we analyze how the AAD accuracy affects the noise suppression performance of an adaptive MWF in a sliding-window implementation, where the user switches his attention between two speakers.

Recent research has shown that it is possible to detect which of two simultaneous speakers a person is attending to, using brain recordings and the temporal envelope of the separate speech signals. However, a wide range of possible methods for extracting this speech envelope exists. This paper assesses the effect of different envelope extraction methods with varying degrees of auditory modelling on the performance of auditory attention detection (AAD), and more specifically on the detection accuracy. It is found that sub-band envelope extraction with proper power-law compression yields best performance, and that the use of several more detailed auditory models does not yield a further improvement in performance.

Interaural timing cues are important for sound source localization and for binaural unmasking of speech that is spatially separated from interfering sounds. Users of a cochlear implant (CI) with residual hearing in the non-implanted ear (bimodal listeners) can only make very limited use of interaural timing cues with their clinical devices. Previous studies showed that bimodal listeners can be sensitive to interaural time differences (ITDs) for simple single- and three-channel stimuli. The modulation enhancement strategy (MEnS) was developed to improve the ITD perception of bimodal listeners. It enhances temporal modulations on all stimulated electrodes, synchronously with modulations in the acoustic signal presented to the non-implanted ear, based on measurement of the amplitude peaks occurring at the rate of the fundamental frequency in voiced phonemes. In the first experiment, ITD detection thresholds were measured using the method of constant stimuli for five bimodal listeners for an artificial vowel, processed with either the advanced combination encoder (ACE) strategy or with MEnS. With MEnS, detection thresholds were significantly lower, and for four subjects well within the physically relevant range. In the second experiment, the extent of lateralization was measured in three subjects with both strategies, and ITD sensitivity was determined using an adaptive procedure. All subjects could lateralize sounds based on ITD and sensitivity was significantly better with MEnS than with ACE. The current results indicate that ITD cues can be provided to bimodal listeners with modified sound processing.

Users of a cochlear implant together with a hearing aid in the non-implanted ear currently use devices that were developed separately and are often fitted separately. This results in very different growth of loudness with level in the two ears, potentially leading to decreased wearing comfort and suboptimal perception of interaural level differences. A loudness equalisation strategy, named 'SCORE bimodal', is proposed. It equalises loudness growth for the two modalities using existing models of loudness for acoustic and electric stimulation, and is suitable for implementation in wearable devices. Loudness balancing experiments were performed with six bimodal listeners to validate the strategy. In a first set of experiments, the function of each loudness model used was validated by balancing the loudness of four harmonic complexes of different bandwidths, ranging from 200 Hz to 1000 Hz, separately for each ear. Both the electric and acoustic loudness models predicted the data well. In a second set of experiments, binaural balancing was done for the same stimuli. It was found that SCORE significantly improved binaural balance., (Copyright © 2012 Elsevier B.V. All rights reserved.)

Previous work suggests that bilateral cochlear implant users are sensitive to interaural cues if experimental speech processors are used to preserve accurate interaural information in the electrical stimulation pattern. Binaural unmasking occurs in adults and children when an interaural delay is applied to the envelope of a high-rate pulse train. Nevertheless, for speech perception, binaural unmasking benefits have not been demonstrated consistently, even with coordinated stimulation at both ears. The present study aimed at bridging the gap between basic psychophysical performance on binaural signal detection tasks on the one hand and binaural perception of speech in noise on the other hand. Therefore, binaural signal detection was expanded to multi-channel stimulation and biologically relevant interaural delays. A harmonic complex, consisting of three sinusoids (125, 250, and 375 Hz), was added to three 125-Hz-wide noise bands centered on the sinusoids. When an interaural delay of 700 μs was introduced, an average BMLD of 3 dB was established. Outcomes are promising in view of real-life benefits. Future research should investigate the generalization of the observed benefits for signal detection to speech perception in everyday listening situations and determine the importance of coordination of bilateral speech processors and accentuation of envelope cues.

The interaural time difference (ITD) is an important cue to localize sound sources. Sensitivity to ITD was measured in eight users of a cochlear implant (CI) in the one ear and a hearing aid (HA) in the other severely impaired ear. The stimulus consisted of an electric pulse train of 100 pps and an acoustic filtered click train. Just-noticeable differences (JNDs) in ITD were measured using a lateralization paradigm. Four subjects exhibited median JNDs in ITD of 156, 341, 254, and 91 mus; the other subjects could not lateralize the stimuli consistently. Only the subjects who could lateralize had average acoustic hearing thresholds at 1,000 and 2,000 Hz better than 100-dB SPL. The electric signal had to be delayed by 1.5 ms to achieve synchronous stimulation at the auditory nerves.