Search

Reading directions vary across writing systems. Through long-term experience readers adjust their visual systems to the dominant reading direction in their writing systems. However, little is known about the neural correlates underlying these adjustments because different writing systems do not just differ in reading direction, but also regarding visual and linguistic properties. Here, we took advantage that Chinese is read to different degrees in left-right or top-down directions in different regions. We investigated visual word processing in participants from Taiwan (both top-down and left-right directions) and from mainland China (only left-right direction). Combined EEG/eye tracking was used together with a saccade-contingent parafoveal preview manipulation to investigate neural correlates, while participants read 5-word lists. Fixation-related potentials (FRPs) showed a reduced late N1 effect (preview positivity), but this effect was modulated by the prior experience with a specific reading direction. Results replicate previous findings that valid previews facilitate visual word processing, as indicated by reduced FRP activation. Critically, the results indicate that this facilitation effect depends on experience with a given reading direction, suggesting a specific mechanism how cultural experience shapes the way people process visual information.

The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using the RSVP-with-flanker paradigms have shown that a semantically implausible word in a sentence elicits a more negative N400 component than a plausible one already before the word enters foveal vision. While this finding suggests that word meaning can is accessed in parafoveal vision and used to rapidly update the sentence representation, evidence of similar effects in natural reading situations is still scarce. In the present study, we combined the co-registration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during natural Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pre-target fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 effect was absent whenever the preview had been implausible, indicating that when a word’s (im)plausibility is processed in parafoveal vision, this information is not revised anymore upon direct fixation. Our results provide convergent neural and behavioral evidence for the parafoveal processing of semantic information in natural reading.

The emotional expression and gaze direction of a face are important cues for human social interactions. However, the interplay of these factors and their neural correlates are only partially understood. In the current study, we investigated ERP correlates of gaze and emotion processing following the initial presentation of faces with different emotional expressions (happy, neutral, angry) and an averted or direct gaze direction as well as following a subsequent change in gaze direction that occurred in half of the trials. We focused on the time course and scalp topography of the N170 and EPN components. The N170 amplitude was larger to averted than direct gaze for the initial face presentation and larger to gaze changes from direct to averted than from averted to direct in response to the gaze change. For the EPN component in response to the initial face presentation, we replicate classic effects of emotion, which did not interact with gaze direction. As a major new finding, changes from direct to averted gaze elicited an EPN-like effect when the face showed a happy expression. No such effect was seen for angry expressions. We conclude that happy faces reflexively attract attention when they look at the observer rather than away from the observer. These results for happy expressions are in line with the shared signal hypothesis that posits a better processing of expressions if their approach or avoidance tendency is consistent with gaze direction. However, the shared signal hypothesis is not supported by the present results for angry faces.

In many situations of everyday life, it is important to quickly understand a spoken message despite distraction by an already ongoing activity. Previous dual-task studies recording the N400 component of the event-related potential (ERP) have shown that auditory language comprehension can be strongly delayed by temporally overlapping additional tasks. In the current study, we investigated whether this interference is aggravated by the need for saccadic eye movements and visuospatial attention shifts in the overlapping task. In two dual-task experiments, a synonymy judgment task for spoken words was combined with a visual discrimination task at different stimulus onset asynchronies. In half of the trials, the primary visual task required a 10° exogenous saccade towards a peripheral stimulus. The timing of semantic processing in the secondary task was assessed by recording the N400. We replicated that with increasing temporal overlap between the tasks, the N400 component was strongly delayed. However, additional saccade-related processes in the primary task had no detrimental effects on concurrent spoken language processing on their own. Unexpectedly, we also observed that a preceding saccade consistently facilitated subsequent motoric processing in the visual task (as indexed by manual reaction times and the lateralized readiness potential), suggesting that saccade execution as such can significantly enhance subsequent processing.

In vision science, a particularly controversial topic is whether and how quickly the semantic information about objects is available outside foveal vision. Here, we aimed at contributing to this debate by coregistering eye movements and EEG while participants viewed photographs of indoor scenes that contained a semantically consistent or inconsistent target object. Linear deconvolution modeling was used to analyze the ERPs evoked by scene onset as well as the fixation-related potentials (FRPs) elicited by the fixation on the target object (t) and by the preceding fixation (t − 1). Object–scene consistency did not influence the probability of immediate target fixation or the ERP evoked by scene onset, which suggests that object–scene semantics was not accessed immediately. However, during the subsequent scene exploration, inconsistent objects were prioritized over consistent objects in extrafoveal vision (i.e., looked at earlier) and were more effortful to process in foveal vision (i.e., looked at longer). In FRPs, we demonstrate a fixation-related N300/N400 effect, whereby inconsistent objects elicit a larger frontocentral negativity than consistent objects. In line with the behavioral findings, this effect was already seen in FRPs aligned to the pretarget fixation t − 1 and persisted throughout fixation t, indicating that the extraction of object semantics can already begin in extrafoveal vision. Taken together, the results emphasize the usefulness of combined EEG/eye movement recordings for understanding the mechanisms of object–scene integration during natural viewing.

An influential theory in the field of visual object recognition proposes that fast magnocellular (M) information facilitates neural processing of spatially more fine-grained but slower parvocellular (P) information. While written words can be considered as a special type of visual objects, it is unknown whether magnocellular facilitation also plays a role in reading. We used a masked priming paradigm that has been shown to result in neural facilitation in visual word processing and tested whether these facilitating effects are mediated by the magnocellular system. In two experiments, we manipulated the influence of magnocellular and parvocellular systems on visual processing of a contextually predictable target character by contrasting high versus low spatial frequency and luminance versus color contrast, respectively. In addition, unchanged (normal) primes were included in both experiments as a manipulation check. As expected, unchanged primes elicited typical repetition effects in the N1, N250 and P3 components of the ERP in both experiments. In the experiment manipulating spatial contrast, we obtained repetition effects only for the N1 component for both M- and P-biased primes. In the luminance versus color contrast experiment, repetition effects were found in N1 and N250 for both M- and P- biased primes. Furthermore, no interactions were found between M- vs. P-biased prime types and repetition. Together these results indicate that M- and P- information contributes jointly to early neural processes underlying visual word recognition.

During natural reading, readers can take up some visual information from not-yet-fixated words to the right of the current fixation and it is well-established that this parafoveal preview facilitates the subsequent foveal processing of the word. However, the extraction and integration of word meaning from parafoveal words and their possible influence on the semantic content of the sentence are controversial. In the current study, we recorded event-related potentials (ERPs) in the RSVP-with-flankers paradigm to test whether and how updates of sentential meaning, based only on parafoveal information, may influence the subsequent foveal processing. In Chinese sentences, the congruency of parafoveal and foveal target words with the sentence was orthogonally manipulated. In contrast to previous research, we also controlled for potentially confounding effects of parafoveal-to-foveal repetition priming (identity preview effects) on the N400. Crucially, we found that the classic effect of foveal congruency on the N400 component only appeared when the word in preview had been congruent with sentence meaning; in contrast, there was no N400 as a function of foveal incongruency when the preview word had also been incongruent. These results indicate that sentence meaning rapidly adapts to parafoveal preview, altering the semantic context for the subsequently fixated word. We also show that correct parafoveal preview generally attenuates the N400 once a word is fixated, regardless of congruency. Taken together, our findings underline the highly generative and adaptive framework of language comprehension.

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Covert shifts of attention that follow the presentation of a cue are associated with lateralized components in the event-related potential (ERP): the "early directing attention negativity" (EDAN) and the "anterior directing attention negativity" (ADAN). Traditionally, these shifts are thought to take place while gaze is fixated and, thus, in the absence of saccades. However, microsaccades of small amplitude (1°) occur frequently and involuntarily also during fixation and are closely correlated with spatial attention. To investigate potential links between microsaccades and lateralized ERP components, we simultaneously recorded eye movements and ERPs in a spatial cueing task. As a first major result, we show that both the posterior EDAN and the orientation of microsaccades align more strongly with the location of the task-relevant part of the cue stimulus than with the direction of the attention shift indicated by that cue. A coupling between microsaccades and EDAN was also present on the single-trial level: The EDAN was largest when microsaccades were oriented toward the relevant cue, but absent when microsaccades were oriented away from it, suggesting that EDAN and microsaccades are generated by the same neural network, which selects relevant stimuli and orients behavior toward them. As a second major result, we show that small corneoretinal artifacts from microsaccades, which fall below conventional EOG rejection thresholds, contaminate the measurement of the ADAN. After correcting the EEG for microsaccade-related artifacts with an optimized variant of independent component analysis, ADAN was abolished at frontal sites, but a genuine ADAN was still present at central sites. Thus, the combined measurement of microsaccades and lateralized ERPs sheds new light onto cue-elicited shifts of covert attention.

Visuospatial attention is an important mechanism in reading that governs the uptake of information from foveal and parafoveal regions of the visual field. However, the spatiotemporal dynamics of how attention is allocated during eye fixations are not completely understood. The current study explored the use of EEG alpha-band oscillations to investigate the spatial distribution of attention during reading. We reanalyzed two data sets, focusing on the lateralization of alpha activity at posterior scalp sites. In each experiment, participants read short lists of German nouns in two paradigms: either by freely moving their eyes (saccadic reading) or by fixating the screen center while the text moved passively from right to left at the same average speed (RSVP paradigm). In both paradigms, upcoming words were either visible or masked, and foveal processing load was manipulated by varying the words' lexical frequencies. Posterior alpha lateralization revealed a sustained rightward bias of attention during saccadic reading, but not in the RSVP paradigm. Interestingly, alpha lateralization was not influenced by word frequency (foveal load) or preview during the preceding fixation. Hence, alpha did not reflect transient attention shifts within a given fixation. However, in both experiments, we found that in the saccadic reading condition a stronger alpha lateralization shortly before a saccade predicted shorter fixations on the subsequently fixated word. These results indicate that alpha lateralization can serve as a measure of attention deployment and its link to oculomotor behavior in reading.

Eye contact is a salient social cue, which is assumed to influence already early neural correlates of face perception. Specifically, the N170 component of the event-related potential (ERP) has often been found to be larger for faces with an averted gaze as compared to faces that directly look at the observer. In most existing ERP studies, effects of eye contact were investigated under comparatively artificial conditions where participants were instructed to maintain a steady fixation while they passively observed gaze changes in the stimulus face. It is therefore unclear to what extent neural correlates of eye contact generalize to more naturalistic situations that involve a continuous interplay between directed and averted gaze between the communication partners. To start bridging this gap, the present study compared the passive viewing of gaze changes to an active condition in which the participant's own gaze (measured online with an eye tracker) interacted with the gaze position of a continuously presented stimulus face. We also investigated whether eye contact effects were modulated by the face's emotional expression. In both the passive and the active viewing condition, N170 amplitudes were larger when the gaze of the stimulus faces was averted rather than directed towards the participant. Furthermore, eye contact decreased P300 amplitudes in both conditions. The emotional expression of the face also modulated the N170, but this effect did not interact with that of gaze direction. We conclude that the neural correlates of gaze perception during active gaze interactions are comparable to those found during passive viewing, encouraging the further study of eye contact effects in more naturalistic settings.

Fixation-related potentials (FRPs), neural responses aligned to saccade offsets, are a promising tool to study the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/EEG experiments: (i) the synchronization of data streams, (ii) the removal of ocular artifacts, (iii) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, (iv) and the fact that numerous low-level stimulus and saccade properties also influence the post-saccadic neural responses. While effective solutions exist for the first two problems, the latter ones are only beginning to be addressed. In the current paper, we present and review a unified framework for FRP analysis that allows us to deconvolve overlapping potentials and control for linear and nonlinear confounds on the FRPs. An open software implementation is provided for all procedures. We then demonstrate the advantages of this approach for data from three commonly studied paradigms: face perception, scene viewing, and natural sentence reading. First, for a traditional ERP face recognition experiment, we show how deconvolution can separate stimulus-ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in scene viewing, we isolate multiple non-linear influences of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable fixation-related brain potentials during natural vision.

Humans actively sample their environment with saccadic eye movements to bring relevant information into high-acuity foveal vision. Despite being lower in resolution, peripheral information is also available prior to each saccade. How pre-saccadic extrafoveal preview of a visual object influences its post-saccadic processing is still an unanswered question. Here, we investigated this question by simultaneously recording behavior and fixation-related brain potentials while human subjects made saccades to face stimuli. We manipulated the relationship between pre-saccadic “previews” and post-saccadic images to explicitly isolate the influences of the former. Subjects performed a gender discrimination task on a newly foveated face under three preview conditions: phase-scrambled face, incongruent face (different identity from the foveated face), and congruent face (same identity). As expected, reaction times were faster after a congruent-face preview compared to the phase-scrambled and incongruent conditions. Importantly, a face preview (either incongruent or congruent) resulted in a strong reduction of post-saccadic neural responses. Specifically, we analyzed the classic face-selective N170 component at occipito-temporal EEG electrodes, which was still present in our experiments with active looking. We found that this component was strongly attenuated for face preview conditions compared to scrambled conditions. This large and long-lasting decrease in evoked activity is consistent with an active prediction mechanism influencing category-specific neural processing at the start of a new fixation. These findings constrain theories of visual stability and show that the extrafoveal preview methodology can be a useful tool to investigate its underlying mechanisms.Significance StatementNeural correlates of object recognition have traditionally been studied by flashing stimuli to the central visual field. This procedure differs in fundamental ways from natural vision, where viewers actively sample the environment with eye movements and also obtain a low-resolution preview of soon-to-be-fixated objects. Here we show that the N170, a classic electrophysiological marker of the structural processing of faces, also occurs during a more natural viewing condition but is massively reduced due to extrafoveal preprocessing (preview benefit). Our results therefore highlight the importance of peripheral vision during trans-saccadic processing in building a coherent and stable representation of the world around us.

During reading, the parafoveal processing of an upcoming word n+1 can influence word recognition in two ways: It can affect fixation behavior during the preceding fixation on word n (parafovea-on-fovea effect, POF), and it can facilitate subsequent foveal processing once word n+1 is fixated (preview benefit). While preview benefits are established, evidence for POF effects is mixed. Recently, it has been suggested that POF effects exist, but have a delayed impact on saccade planning and thus coincide with preview benefits measured on word n+1. We combined eye movement and EEG recordings to investigate and separate neural correlates of POF and preview benefit effects. Participants read lists of nouns either in a boundary paradigm or the RSVP-with-flankers paradigm, while we recorded fixation- or event-related potentials (FRPs/ERPs), respectively. The validity and lexical frequency of the word shown as preview for the upcoming word n+1 were orthogonally manipulated. Analyses focused on the first fixation on word n+1. Preview validity (correct vs. incorrect preview) strongly modulated fixation times and electrophysiological N1 amplitudes, replicating previous findings. Importantly, gaze durations and FRPs measured on word n+1 were also affected by the frequency of the word shown as preview, with low-frequency previews eliciting a sustained, N400-like centroparietal negativity. Results support the idea that POF effects exist but affect word recognition with a delay. Lastly, once word n+1 was fixated, its frequency also modulated N1 amplitudes in ERPs and FRPs. Taken together, we separated immediate and delayed effects of parafoveal processing on brain correlates of word recognition.

The world appears stable despite saccadic eye-movements. One possible explanation for this phenomenon is that the visual system predicts upcoming input across saccadic eye-movements based on peripheral preview of the saccadic target. We tested this idea using concurrent electroencephalography (EEG) and eye-tracking. Participants made cued saccades to peripheral upright or inverted face stimuli that changed orientation (invalid preview) or maintained orientation (valid preview) while the saccade was completed. Experiment 1 demonstrated better discrimination performance and a reduced fixation-locked N170 component (fN170) with valid than with invalid preview, demonstrating integration of pre- and post-saccadic information. Moreover, the early fixation-related potentials (FRP) showed a preview face inversion effect suggesting that some pre-saccadic input was represented in the brain until around 170 ms post fixation-onset. Experiment 2 replicated Experiment 1 and manipulated the proportion of valid and invalid trials to test whether the preview effect reflects context-based prediction across trials. A whole-scalp Bayes factor analysis showed that this manipulation did not alter the fN170 preview effect but did influence the face inversion effect before the saccade. The pre-saccadic inversion effect declined earlier in the mostly invalid block than in the mostly valid block, which is consistent with the notion of pre-saccadic expectations. In addition, in both studies, we found strong evidence for an interaction between the pre-saccadic preview stimulus and the post-saccadic target as early as 50 ms (Experiment 2) or 90 ms (Experiment 1) into the new fixation. These findings suggest that visual stability may involve three temporal stages: prediction about the saccadic target, integration of pre-saccadic and post-saccadic information at around 50-90 ms post fixation onset, and post-saccadic facilitation of rapid categorization.

Combining EEG with eye-tracking is a promising approach to study neural correlates of natural vision, but the resulting recordings are also heavily contaminated by activity of the eye balls, eye lids, and extraocular muscles. While Independent Component Analysis (ICA) is commonly used to suppress these ocular artifacts, its performance under free viewing conditions has not been systematically evaluated and many published findings display residual artifacts. Here I evaluated and optimized ICA-based correction for two tasks with unconstrained eye movements: visual search in images and sentence reading. In a first step, four parameters of the ICA pipeline were systematically varied: the (1) high-pass and (2) low-pass filter applied to the training data, (3) the proportion of training data containing myogenic saccadic spike potentials (SP), and (4) the threshold for eye tracker-based component rejection. In a second step, the eye-tracker was used to objectively quantify correction quality of each ICA solution, both in terms of undercorrection (residual artifacts) and overcorrection (removal of neurogenic activity). As a benchmark, results were compared to those obtained with an alternative spatial filter, Multiple Source Eye Correction (MSEC). With commonly used settings, Infomax ICA not only left artifacts in the data of both tasks, but also distorted neurogenic activity during eye movement-free intervals. However, correction could be drastically improved by training the ICA on optimally filtered data in which SPs were massively overweighted. With optimized procedures, ICA removed virtually all artifacts, including the SP and its associated spectral broadband artifact, with little distortion of neural activity. It also outperformed MSEC in terms of SP correction. Matlab code is provided.Author NoteI would like to acknowledge Maarten De Schuymer, who conducted early explorations of the effects of high-pass filtering that helped to initiate the current work. I am also grateful to Lisa Spiering for assisting with the MSEC correction and Werner Sommer for providing an excellent working environment. Collection of one of the datasets was supported by a grant from DFG (FG-868-A2). Comments or corrections are highly appreciated.

Attentional cueing tasks using gaze direction as spatial cues have sometimes yielded an early directing attention negativity (EDAN) component in the ERP, presumably reflecting the initial orienting toward the cued location. However, other studies have failed to identify an EDAN component for gaze cues, yielding an inconsistent picture. In the present study, we re-examined the EDAN to gaze cueing, using a continuous task where the specific direction of the gaze changes was task irrelevant. Face stimuli changed gaze direction several times during each trial between direct, left-, and right-averted positions. Participants counted the number of gaze shifts during the trial. Results showed an unusually large EDAN-like ERP asymmetry at posterior scalp sites that was of similar amplitude for large and small gaze shifts into the periphery. Shifts from an averted position toward a direct gaze elicited a qualitatively similar but smaller effect than shifts into the periphery. Together, these findings shed new light on gaze-elicited spatial attention as they indicate a reflexive attention orienting, following the direction of gaze motion, even when the gaze direction itself is irrelevant for the task.

Electrophysiological research with event-related brain potentials (ERPs) is increasingly moving from simple, strictly orthogonal stimulation paradigms towards more complex, quasi-experimental designs and naturalistic situations that involve fast, multisensory stimulation and complex motor behavior. As a result, electrophysiological responses from subsequent events often overlap with each other. In addition, the recorded neural activity is typically modulated by numerous covariates, which influence the measured responses in a linear or nonlinear fashion. Examples of paradigms where systematic temporal overlap variations and low-level confounds between conditions cannot be avoided include combined EEG/eye-tracking experiments during natural vision, fast multisensory stimulation experiments, and mobile brain/body imaging studies. However, even “traditional”, highly controlled ERP datasets often contain a hidden mix of overlapping activity (e.g. from stimulus onsets, involuntary microsaccades, or button presses) and it is helpful or even necessary to disentangle these components for a correct interpretation of the results. In this paper, we introduce unfold, a powerful, yet easy-to-use MATLAB toolbox for regression-based EEG analyses that combines existing concepts of massive univariate modeling (“regression ERPs”), linear deconvolution modeling, and non-linear modeling with the generalized additive model (GAM) into one coherent and flexible analysis framework. The toolbox is modular, compatible with EEGLAB and can handle even large datasets efficiently. It also includes advanced options for regularization and the use of temporal basis functions (e.g. Fourier sets). We illustrate the advantages of this approach for simulated data as well as data from a standard face recognition experiment. In addition to traditional and non-conventional EEG/ERP designs, unfold can also be applied to other overlapping physiological signals, such as pupillary or electrodermal responses. It is available as open-source software at http://www.unfoldtoolbox.org.

Natural reading involves the preprocessing of upcoming words, resulting in shorter fixations on words visible in the parafovea during preceding fixations. While this preview benefit is established in behavior, its brain-electric correlates have only recently been investigated. Using fixation-related potentials, an attenuation of the occipitotemporal N1 component for words that were parafoveally visible during preceding fixations has been demonstrated. In contrast, another study, using an RSVP paradigm with parafoveal flanker words, observed no such general preview benefit in ERPs, but instead reported N400 effects triggered by semantically incongruous parafoveal words. To follow up on these discrepant findings and to extend them to a nonalphabetic writing system, we conducted two ERP experiments with Chinese readers using the RSVP-with-flankers paradigm and rigorous fixation control via eye tracking. We replicate robust parafoveal N400 semantic congruency effects in Chinese participants. Additionally, we found that, once a word was directly looked at, words after a valid preview elicited a smaller N1 and a weaker N400 than those after an invalid preview. Results underline the importance of considering parafoveal vision in ERP studies on reading.

Cognitive conflict control in flanker tasks has often been described using the zoom-lens metaphor of selective attention. However, whether and how selective attention – in terms of suppression and enhancement – operates in this context has remained unclear. To examine the dynamic interplay of selective attention and cognitive control we used electrophysiological measures and presented task-irrelevant visual probe stimuli at foveal, parafoveal, and peripheral display positions. Target–flanker congruency varied either randomly from trial to trial (mixed-block) or block-wise (fixed-block) in order to induce reactive versus proactive control modes, respectively. Three EEG measures were used to capture ad-hoc adjustments within trials as well as effects of context-based predictions: the N1 component of the visual evoked potential (VEP) to probes, the VEP to targets, and the conflict-related midfrontal N2 component. Results from probe-VEPs indicate that enhanced processing of the foveal target rather than suppression of the peripheral flankers supports interference control. In incongruent mixed-block trials VEPs were larger to probes near the targets. In the fixed-blocks probe-VEPs were not modulated, but contrary to the mixed-block the preceding target-related VEP was affected by congruency. Results of the control-related N2 reveal largest amplitudes in the unpredictable context, which did not differentiate for stimulus and response incongruency. In contrast, in the predictable context, N2 amplitudes were reduced overall and differentiated between stimulus and response incongruency. Taken together these results imply that predictability alters interference control by a reconfiguration of stimulus processing. During unpredictable sequences participants adjust their attentional focus dynamically on a trial-by-trial basis as reflected in congruency-dependent probe-VEP-modulation. This reactive control mode also elicits larger N2 amplitudes. In contrast, when task demands are predictable, participants focus selective attention earlier as reflected in the target-related VEPs. This proactive control mode leads to smaller N2 amplitudes and absent probe effects.

Visuospatial attention is an important mechanism in reading that governs the uptake of information from foveal and parafoveal regions of the visual field. However, the spatiotemporal dynamics of how attention is allocated during eye fixations are not completely understood. The current study explored the use of EEG alpha-band oscillations to investigate the spatial distribution of attention during reading. We reanalyzed two data sets, focusing on the lateralization of alpha activity at posterior scalp sites. In each experiment, participants read short lists of German nouns in two paradigms: either by freely moving their eyes (saccadic reading) or by fixating the screen center while the text moved passively from right to left at the same average speed (RSVP paradigm). In both paradigms, upcoming words were either visible or masked, and foveal processing load was manipulated by varying the words' lexical frequencies. Posterior alpha lateralization revealed a sustained rightward bias of attention during saccadic reading, but not in the RSVP paradigm. Interestingly, alpha lateralization was not influenced by word frequency (foveal load) or preview during the preceding fixation. Hence, alpha did not reflect transient attention shifts within a given fixation. However, in both experiments, we found that in the saccadic reading condition a stronger alpha lateralization shortly before a saccade predicted shorter fixations on the subsequently fixated word. These results indicate that alpha lateralization can serve as a measure of attention deployment and its link to oculomotor behavior in reading.

Neural correlates of word recognition are commonly studied with (rapid) serial visual presentation (RSVP), a condition that eliminates three fundamental properties of natural reading: parafoveal preprocessing, saccade execution, and the fast changes in attentional processing load occurring from fixation to fixation. We combined eye-tracking and EEG to systematically investigate the impact of all three factors on brain-electric activity during reading. Participants read lists of words either actively with eye movements (eliciting fixation-related potentials) or maintained fixation while the text moved passively through foveal vision at a matched pace (RSVP-with-flankers paradigm, eliciting ERPs). The preview of the upcoming word was manipulated by changing the number of parafoveally visible letters. Processing load was varied by presenting words of varying lexical frequency. We found that all three factors have strong interactive effects on the brain's responses to words: Once a word was fixated, occipitotemporal N1 amplitude decreased monotonically with the amount of parafoveal information available during the preceding fixation; hence, the N1 component was markedly attenuated under reading conditions with preview. Importantly, this preview effect was substantially larger during active reading (with saccades) than during passive RSVP with flankers, suggesting that the execution of eye movements facilitates word recognition by increasing parafoveal preprocessing. Lastly, we found that the N1 component elicited by a word also reflects the lexical processing load imposed by the previously inspected word. Together, these results demonstrate that, under more natural conditions, words are recognized in a spatiotemporally distributed and interdependent manner across multiple eye fixations, a process that is mediated by active motor behavior.

The recent years have seen the emergence of graph theoretical analysis of complex, functional brain networks estimated from neurophysiological measurements. The research has mainly focused on the graph characterization of the resting-state/default network, and its potential for clinical application. Functional resting-state networks usually display the characteristics of small-world networks and their statistical properties have been observed to change due to pathological conditions or aging. In the present paper we move forward in the application of graph theoretical tools in functional connectivity by investigating high-level cognitive processing in healthy adults, in a manner similar to that used in psychological research in the framework of event-related potentials (ERPs). More specifically we aim at investigating how graph theoretical approaches can help to discover systematic and task-dependent differences in high-level cognitive processes such as language perception. We will show that such an approach is feasible and that the results coincide well with the findings from neuroimaging studies. © 2011 Elsevier B.V. All rights reserved.

It has been suggested that cognitive conflicts require effortful processing and, therefore, are aversive (Botvinick, 2007). In the present study, we compared conflicts emerging from the inhibition of a predominant response tendency in a go/no-go task with those between incompatible response activations in a Simon task in a within-subjects design, using the same type of stimuli. Whereas no-go trials elicited reduced skin conductance and pupillometric responses, but prolonged corrugator muscle activity, as compared with go trials, incompatible and compatible Simon trials were indistinguishable with respect to these parameters. Furthermore, the conflict-sensitive N2 components of the event-related brain potential were similar in amplitude, but showed significantly different scalp distributions, indicating dissociable neural generator systems. The present findings suggest the involvement of different emotional and cognitive processes in both types of cognitive conflicts-none being aversive, however. In addition, the N2 findings call into question claims of common monitoring systems for all kinds of cognitive conflicts.

In the recent past, recurrence quantification analysis (RQA) has gained an increasing interest in various research areas. The complexity measures the RQA provides have been useful in describing and analysing a broad range of data. It is known to be rather robust to noise and nonstationarities. Yet, one key question in empirical research concerns the confidence bounds of measured data. In the present Letter we suggest a method for estimating the confidence bounds of recurrence-based complexity measures. We study the applicability of the suggested method with model and real- life data.

It has recently been demonstrated that the presentation of visual oddballs induces a prolonged inhibition of microsaccades. The amplitude of the P300 component in event-related potentials (ERPs) has been shown to be sensitive to the category (target vs. nontarget) of the eliciting stimulus, its overall probability, and the preceding stimulus sequence. In the present study we further specify the functional underpinnings of the prolonged microsaccadic inhibition in the visual oddball task, showing that the stimulus category, the frequency of a stimulus, and the preceding stimulus sequence influence microsaccade rate. Furthermore, by co-recording ERPs and eye movements, we were able to demonstrate that, despite being largely sensitive to the same experimental manipulation, the amplitude of P300 and the microsaccadic inhibition predict each other only weakly. Copyright © 2009 Society for Psychophysiological Research.

Over the last years recurrence plots (RPs)and recurrence quantification analysis (RQA)have become quite popular in various branches of science. One key problem in applying RPs and RQA is the selection of suitable parameters for the data under investigation. Whereas various well-established methods for the selection of embedding parameters exists, the question of choosing an appropriate threshold has not yet been answered satisfactorily. The recommendations found in the literature are rather rules of thumb than actual guidelines. In this paper we address the issue of threshold selection in RP/RQA. The core criterion for choosing a threshold is the power in signal detection that threshold yields. We will validate our approach by applying it to model as well as real-life data.

There is mounting evidence that under some conditions the processing of facial identity and facial emotional expressions may not be independent; however, the nature of this interaction remains to be established. By using event-related brain potentials (ERP) we attempted to localize these interactions within the information processing system. During an expression discrimination task (Experiment 1) categorization was faster for portraits of personally familiar vs. unfamiliar persons displaying happiness. The peak latency of the P300 (trend) and the onset of the stimulus-locked LRP were shorter for familiar than unfamiliar faces. This implies a late perceptual but pre-motoric locus of the facilitating effect of familiarity on expression categorization. In Experiment 2 participants performed familiarity decisions about portraits expressing different emotions. Results revealed an advantage of happiness over disgust specifically for familiar faces. The facilitation was localized in the response selection stage as suggested by a shorter onset of the LRP. Both experiments indicate that familiarity and facial expression may not be independent processes. However, depending on the kind of decision different processing stages may be facilitated for happy familiar faces.

During reading, the parafoveal processing of an upcoming word n+1 can influence word recognition in two ways: It can affect fixation behavior during the preceding fixation on word n (parafovea-on-fovea effect, POF), and it can facilitate subsequent foveal processing once word n+1 is fixated (preview benefit). While preview benefits are established, evidence for POF effects is mixed. Recently, it has been suggested that POF effects exist, but have a delayed impact on saccade planning and thus coincide with preview benefits measured on word n+1. We combined eye movement and EEG recordings to investigate and separate neural correlates of POF and preview benefit effects. Participants read lists of nouns either in a boundary paradigm or the RSVP-with-flankers paradigm, while we recorded fixation- or event-related potentials (FRPs/ERPs), respectively. The validity and lexical frequency of the word shown as preview for the upcoming word n+1 were orthogonally manipulated. Analyses focused on the first fixation on word n+1. Preview validity (correct vs. incorrect preview) strongly modulated fixation times and electrophysiological N1 amplitudes, replicating previous findings. Importantly, gaze durations and FRPs measured on word n+1 were also affected by the frequency of the word shown as preview, with low-frequency previews eliciting a sustained, N400-like centroparietal negativity. Results support the idea that POF effects exist but affect word recognition with a delay. Lastly, once word n+1 was fixated, its frequency also modulated N1 amplitudes in ERPs and FRPs. Taken together, we separated immediate and delayed effects of parafoveal processing on brain correlates of word recognition.

Natural reading involves the preprocessing of upcoming words, resulting in shorter fixations on words visible in the parafovea during preceding fixations. While this preview benefit is established in behavior, its brain-electric correlates have only recently been investigated. Using fixation-related potentials, an attenuation of the occipitotemporal N1 component for words that were parafoveally visible during preceding fixations has been demonstrated. In contrast, another study, using an RSVP paradigm with parafoveal flanker words, observed no such general preview benefit in ERPs, but instead reported N400 effects triggered by semantically incongruous parafoveal words. To follow up on these discrepant findings and to extend them to a nonalphabetic writing system, we conducted two ERP experiments with Chinese readers using the RSVP-with-flankers paradigm and rigorous fixation control via eye tracking. We replicate robust parafoveal N400 semantic congruency effects in Chinese participants. Additionally, we found that, once a word was directly looked at, words after a valid preview elicited a smaller N1 and a weaker N400 than those after an invalid preview. Results underline the importance of considering parafoveal vision in ERP studies on reading.

Covert shifts of visuospatial attention are traditionally assumed to occur in the absence of oculomotor behavior. In contrast, recent behavioral studies have linked attentional cueing effects to the occurrence of microsaccades, small eye movements executed involuntarily during attempted fixation. Here we used a new type of electrophysiological marker to explore the attention–microsaccade relationship, the visual brain activity evoked by the microsaccade itself. By shifting the retinal image, microsaccades frequently elicit neural responses throughout the visual pathway, scalp-recordable in the human EEG as a microsaccade-related potential (mSRP). Although mSRPs contain similar signal components (P1/N1) as traditional visually-evoked potentials (VEPs), it is unknown whether they are also influenced by cognition. Based on established findings that VEPs are amplified for visual inputs at currently attended locations, we expected a selective gain-modulation also for mSRPs. Eye movements and EEG were coregistered in a classic spatial cueing task with an endogenous cue. Replicating behavioral findings, the direction of early microsaccades 200–400 ms after cue onset was biased towards the cued side. However, for microsaccades throughout the cue-target interval, mSRPs were systematically enhanced at occipital scalp sites contralateral to the cued hemifield. This attention effect resembled that in a control condition with VEPs and did not interact with the direction of the underlying microsaccade, suggesting that mSRPs reflect the focus of sustained visuospatial attention, which remains fixed at the cued location, despite microsaccades. Microsaccades are not merely an artifact source in the EEG; instead, they are followed by cognitively modulated brain potentials that can serve as non-intrusive electrophysiological probes of attention.

Reading requires the orchestration of vision, attention, and language processes, usually along with the programming of eye movements. The co-registration of eye movements and electroencephalography (EEG) during natural reading promises to deliver two complementary sets of information. Fixation durations reflect the difficulty of sublexical, lexical, and sentence-level properties, such as the frequency and predictability of the fixated word and (at least) its left and right neighbors; they reveal space-related constraints of distributed processing in the perceptual span. Co-registered fixation-related potentials (FRPs) yield time-related constraints of word recognition within fixation durations. Effects of word frequency and predictability in FRPs are shown to be qualitatively similar to those in event-related potentials (ERPs) measured during serial visual presentation of sentence words. However, FRPs and ERPs differ with respect to the timelines associated with frequency and predictability effects. It is anticipated that space-related and time-related constraints of word recognition during reading will guide the further development of computational models of eye-movement control during reading as well as their integration with computational models of (isolated) word recognition.

Microsaccades during fixation have been suggested to counteract visual fading. Recent experi- ments have also observed microsaccade-related neural responses from cellular record, scalp elec- troencephalogram (EEG) and functional magnetic resonance imaging (fMRI). The underlying mechanism, however, is not yet understood and highly debated. It has been proposed that the neural activity of primary visual cortex (V1) is a crucial component for counteracting visual adaptation. In this paper, we use computational modeling to investigate how short-term depres- sion (STD) in thalamocortical synapses might affect the neural responses of V1 in the presence of microsaccades. Our model not only gives a possible synaptic explanation for microsaccades in counteracting visual fading, but also reproduces several features in experimental findings. These modeling results suggest that STD in thalamocortical synapses plays an important role in microsaccade-related neural responses and the model may be useful for further investigation of behavioral properties and functional roles of microsaccades.

During natural reading, a parafoveal preview of the upcoming word facilitates its subsequent recognition (e.g., shorter fixation durations compared to masked preview) but nothing is known about the neural correlates of this so-called preview benefit. Furthermore, while the evidence is strong that readers preprocess orthographic features of upcoming words, it is controversial whether word meaning can also be accessed parafoveally. We investigated the timing, scope, and electrophysiological correlates of parafoveal information use in reading by simultaneously recording eye movements and fixation-related brain potentials (FRPs) while participants read word lists fluently from left to right. For one word-the target-(e.g., "blade") parafoveal information was manipulated by showing an identical ("blade"), semantically related ("knife"), or unrelated ("sugar") word as preview. In boundary trials, the preview was shown parafoveally but changed to the correct target word during the incoming saccade. Replicating classic findings, target words were fixated shorter after identical previews. In the EEG, this benefit was reflected in an occipitotemporal preview positivity between 200 and 280 ms. In contrast, there was no facilitation from related previews. In parafoveal-on-foveal trials, preview and target were embedded at neighboring list positions without a display change. Consecutive fixation of two related words produced N400 priming effects, but only shortly (160 ms) after the second word was directly fixated. Results demonstrate that neural responses to words are substantially altered by parafoveal preprocessing under normal reading conditions. We found no evidence that word meaning contributes to these effects. Saccade-contingent display manipulations can be combined with EEG recordings to study extrafoveal perception in vision.

Three ERP experiments examined the effect of word presentation rate (i.e., stimulus onset asynchrony, SOA) on the time course of word frequency and predictability effects in sentence reading. In Experiments 1 and 2, sentences were presented word-by-word in the screen center at an SOA of 700 and 490 ms, respectively. While these rates are typical for psycholinguistic ERP research, natural reading happens at a considerably faster pace. Accordingly, Experiment 3 employed a near-normal SOA of 280 ms, which approximated the rate of normal reading. Main results can be summarized as follows: (1) The onset latency of early frequency effects decreases gradually with increasing presentation rates. (2) An early interaction between top-down and bottom-up processing is observed only under a near-normal SOA. (3) N400 predictability effects occur later and are smaller at a near-normal (i.e., high) presentation rate than at the lower rates commonly used in ERP experiments. (4) ERP morphology is different at the shortest compared to longer SOAs. Together, the results point to a special role of a near-normal presentation rate for visual word recognition and therefore suggest that SOA should be taken into account in research of natural reading.

Brain-electric correlates of reading have traditionally been studied with word-by-word presentation, a condition that eliminates important aspects of the normal reading process and precludes direct comparisons between neural activity and oculomotor behavior. In the present study, we investigated effects of word predictability on eye movements (EM) and fixation-related brain potentials (FRPs) during natural sentence reading. Electroencephalogram (EEG) and EM (via video-based eye tracking) were recorded simultaneously while subjects read heterogeneous German sentences, moving their eyes freely over the text. FRPs were time-locked to first-pass reading fixations and analyzed according to the cloze probability of the currently fixated word. We replicated robust effects of word predictability on EMs and the N400 component in FRPs. The data were then used to model the relation among fixation duration, gaze duration, and N400 amplitude, and to trace the time course of EEG effects relative to effects in EM behavior. In an extended Methodological Discussion section, we review 4 technical and data-analytical problems that need to be addressed when FRPs are recorded in free-viewing situations (such as reading, visual search, or scene perception) and propose solutions. Results suggest that EEG recordings during normal vision are feasible and useful to consolidate findings from EEG and eye-tracking studies.

The development of theories and computational models of reading requires an understanding of processing constraints, in particular of timelines related to word recognition and oculomotor control. Timelines of word recognition are usually determined with event-related potentials (ERPs) recorded under conditions of serial visual presentation (SVP) of words; timelines of oculomotor control are derived from parameters of eye movements (EMs) during natural reading. We describe two strategies to integrate these approaches. One is to collect ERPs and EMs in separate SVP and natural reading experiments for the same experimental material (but different subjects). The other strategy is to co-register EMs and ERPs during natural reading from the same subjects. Both strategies yield data that allow us to determine how lexical properties influence ERPs (e.g., the N400 component) and EMs (e.g., fixation durations) across neighboring words. We review our recent research on the effects of frequency and predictability of words on both EM and ERP measures with reference to current models of eye-movement control during reading. Results are in support of the proposition that lexical access is distributed across several fixations and across brain-electric potentials measured on neighboring words.

Microsaccades are very small, involuntary flicks in eye position that occur on average once or twice per second during attempted visual fixation. Microsaccades give rise to EMG eye muscle spikes that can distort the spectrum of the scalp EEG and mimic increases in gamma band power. Here we demonstrate that microsaccades are also accompanied by genuine and sizeable cortical activity, manifested in the EEG. In three experiments, high-resolution eye movements were corecorded with the EEG: during sustained fixation of checkerboard and face stimuli and in a standard visual oddball task that required the counting of target stimuli. Results show that microsaccades as small as 0.15° generate a field potential over occipital cortex and midcentral scalp sites 100-140 ms after movement onset, which resembles the visual lambda response evoked by larger voluntary saccades. This challenges the standard assumption of human brain imaging studies that saccade-related brain activity is precluded by fixation, even when fully complied with. Instead, additional cortical potentials from microsaccades were present in 86% of the oddball task trials and of similar amplitude as the visual response to stimulus onset. Furthermore, microsaccade probability varied systematically according to the proportion of target stimuli in the oddball task, causing modulations of late stimulus-locked event-related potential (ERP) components. Microsaccades present an unrecognized source of visual brain signal that is of interest for vision research and may have influenced the data of many ERP and neuroimaging studies. Copyright © 2009 Society for Neuroscience.

About 15% of reading saccades move the eyes backwards in the text. To study the neurophysiological correlates of such regressions, we co-registered gaze position and ERPs of 54 subjects during natural, left-to-right reading. Sentences were grammatically diverse but contained no syntactic violations or local ambiguities. Accompanying the onset of long-range regressions, we observed a late centroparietal positivity, closely resembling the P600 component commonly observed for syntactic violations and garden-path sentences in traditional ERP experiments. This suggests that the P600 indexes individual comprehension difficulty or parsing problems even in the absence of syntactic ambiguity. Co-registration of eye movements and ERPs may help to differentiate between regressions caused by oculomotor overshoot, word identification failures, and syntactic parsing problems., The abstract of this poster was published in the Journal of Eye Movement Research, Vol. 1, Number 5, p. 129., {"references":["Vitu, F. (2005). Visual extraction processes and regressive saccades in reading. In G. Underwood (Ed.), Cognitive processes in eye guidance. Oxford, NY: Oxford University Press. pp. 1-32.","Dimigen, O., Hohlfeld, A., Sommer, W.; Jacobs, A., Engbert, R. & Kliegl, W. (2005). Measuring ERPs during left-to-right reading, Poster presented at the IX International Conference on Cognitive Neuroscience, Havana, Cuba","Dimigen, O., Sommer, W., Hohlfeld, A., Jacobs, Engbert, R., Kliegl, R. (2006). Concurrent recording of EEG and gaze position: Measuring effects of word predictability during left-to-right reading of normal sentences. Journal of Cognitive Neuroscience, Supplement 224","Frazier & Rayner (1982). Making and correcting errors during sentence comprehension: Eye movements in the analysis of structurally ambiguous sentences. Cognitive Psychology, 14, pp. 178-210","Engbert & Kliegl (2003). Microsaccades uncover the orientation of visual attention. Vision Research, 43, pp. 1035-1045"]}