Search

Compositionality is a primary feature of language, but graphics can also create combinatorial meaning, like with items above faces (e.g., lightbulbs to mean inspiration). We posit that these “upfixes” (i.e., upwards affixes) involve a productive schema enabling both stored and novel face–upfix dyads. In two experiments, participants viewed either conventional (e.g., lightbulb) or unconventional (e.g., clover-leaves) upfixes with faces which either matched (e.g., lightbulb/smile) or mismatched (e.g., lightbulb/frown). In Experiment 1, matching dyads sponsored higher comprehensibility ratings and faster response times, modulated by conventionality. In Experiment 2, event-related brain potentials (ERPs) revealed conventional upfixes, regardless of matching, evoked larger N250s, indicating perceptual expertise, but mismatching and unconventional dyads elicited larger semantic processing costs (N400) than conventional-matching dyads. Yet mismatches evoked a late negativity, suggesting congruent novel dyads remained construable compared with violations. These results support that combinatorial graphics involve a constrained productive schema, similar to the lexicon of language.

Available online 16 December 2021 We investigated how aging modulates lexico-semantic processes in the visual (seeing written items), auditory (hearing spoken items) and audiovisual (seeing written items while hearing congruent spoken items) modalities. Participants were young and older adults who performed a delayed lexical decision task (LDT) presented in blocks of visual, auditory, and audiovisual stimuli. Event-related potentials (ERPs) revealed differences between young and older adults despite older adults’ ability to identify words and pseudowords as accurately as young adults. The observed differences included more focalized lexico-semantic access in the N400 time window in older relative to young adults, stronger re-instantiation and/or more widespread activity of the lexicality effect at the time of responding, and stronger multimodal integration for older relative to young adults. Our results offer new insights into how functional neural differences in older adults can result in efficient access to lexico-semantic representations across the lifespan. This work was supported by a Marie Curie Individual Fellowship 702178 by the European Commission and a Juan de la Cierva Formacion Fellowship FJCI-2017-31782 from the Spanish Ministry of Science, Innovation and Universities to RALZ, NWO VENI grant 275-89-027 to MB, MINECO grant PSI 2017-82563-P from the Spanish Ministry of Economics and Competitiveness to AGS, Natural Sciences and Engineering Research Council of Canada Discovery Grant 502584 to BCA, and by the Basque Government through the BERC 2018–2021 program.

Available online 1 September 2018. There is considerable behavioral evidence that morphologically complex words such as ‘tax-able’ and ‘kiss-es’ are processed and represented combinatorially. In other words, they are decomposed into their constituents ‘tax’ and ‘-able’ during comprehension (reading or listening), and producing them might also involve on–the–spot combination of these constituents (especially for inflections). However, despite increasing amount of neurocognitive research, the neural mechanisms underlying these processes are still not fully understood. The purpose of this critical review is to offer a comprehensive overview on the state-of-the-art of the research on the neural mechanisms of morphological processing. In order to take into account all types of complex words, we include findings on inflected, derived, and compound words presented both visually and aurally. More specifically, we cover a wide range of electro- and magnetoencephalography (EEG and MEG, respectively) as well as structural/functional magnetic resonance imaging (s/fMRI) studies that focus on morphological processing. We present the findings with respect to the temporal course and localization of morphologically complex word processing. We summarize the observed findings, their interpretations with respect to current psycholinguistic models, and discuss methodological approaches as well as their possible limitations. This study was supported by Kone Foundation and Helsinki University Research Grants to Alina Leminen, by the Volkswagen Foundation, Grant FP 561/11 to Eva Smolka, PSI2015- 65689-P and SEV-2015-0490 from the Spanish Government (Jon Andoni Du~nabeitia).

Speech perception is influenced by vision through a process of audiovisual integration. This is demonstrated by the McGurk illusion where visual speech (for example /ga/) dubbed with incongruent auditory speech (such as /ba/) leads to a modified auditory percept (/da/). Recent studies have indicated that perception of the incongruent speech stimuli used in McGurk paradigms involves mechanisms of both general and audiovisual speech specific mismatch processing and that general mismatch processing modulates induced theta-band (4-8 Hz) oscillations. Here, we investigated whether the theta modulation merely reflects mismatch processing or, alternatively, audiovisual integration of speech. We used electroencephalographic recordings from two previously published studies using audiovisual sine-wave speech (SWS), a spectrally degraded speech signal sounding nonsensical to naive perceivers but perceived as speech by informed subjects. Earlier studies have shown that informed, but not naive subjects integrate SWS phonetically with visual speech. In an N1/P2 event-related potential paradigm, we found a significant difference in theta-band activity between informed and naive perceivers of audiovisual speech, suggesting that audiovisual integration modulates induced theta-band oscillations. In a McGurk mismatch negativity paradigm (MMN) where infrequent McGurk stimuli were embedded in a sequence of frequent audio-visually congruent stimuli we found no difference between congruent and McGurk stimuli. The infrequent stimuli in this paradigm are violating both the general prediction of stimulus content, and that of audiovisual congruence. Hence, we found no support for the hypothesis that audiovisual mismatch modulates induced theta-band oscillations. We also did not find any effects of audiovisual integration in the MMN paradigm, possibly due to the experimental design.

The cognitive view on deception proposes that lying comes with a cognitive cost. This view is supported by the finding that lying typically takes longer than truth telling. Event-related potentials (ERPs) provide a means to unravel the cognitive processes underlying this cost. Using a mock-crime design, the current study (n = 20) investigated the effects of deception on the Contingent Negative Variation (CNV), the Lateralized Readiness Potential (LRP), the Correct Response Negativity (CRN), and the stimulus-locked N200 and P300 components. In line with previous research, lying resulted in more errors, longer reaction times (RTs) and longer RT standard deviations compared to truthful responses. A marginally significant effect suggested a stronger CNV for the anticipation of lying compared to the anticipation of truth telling. There were no significant deception effects on the stimulus- and the response-locked LRPs. Unexpectedly, we found a significantly larger CRN for truth telling compared to lying. Additional analyses revealed an enhanced N200 and a decreased P300 for lying compared to truth telling. Our results support the cognitive load hypothesis for lying, yet are mixed regarding the response conflict hypothesis. Results are discussed with regard to the specific characteristics of our design and their theoretical and applied implications.

In traditional theories of language comprehension, syntactic and semantic processing are inextricably linked. This assumption has been challenged by the 'Semantic: Illusion Effect' found in studies using Event Related brain Potentials. Semantically anomalous sentences did not produce the expected increase in N400 amplitude but rather one in F'600 amplitude. To explain these findings, complex models have been devised in which an independent semantic processing stream can arrive at a sentence interpretation that may differ from the interpretation prescribed by the syntactic structure of the. sentence. We review five such multi-stream models and argue that they do not account for the full range of relevant results because they assume that the amplitude of the N400 indexes some form of semantic integration. Based on recent evidence we argue that N400 amplitude might reflect the retrieval of lexical information from memory. On this view, the absence of an N400-effect in Semantic Illusion sentences can be explained in terms cif priming. Furthermore, we suggest that semantic integration, which has previously been linked to the N400 component, might be reflected in the P600 instead. When combined, these functional interpretations result in a single-stream account of language processing that can explain all of the Semantic Illusion data. (C) 2012 Elsevier B.V. All rights reserved.

Recent studies have revealed a strong relationship between visual working memory and selective attention, such that attention is biased by what is currently on our mind. However, other data show that not all memorized items influence the deployment of attention, thus calling for a distinction within working memory: whereas active memory items function as an attentional template and directly affect perception, other, accessory items do not. We review recent evidence that items compete for the status of 'attentional template' that contains only one object at a time. Neurophysiological results provide insight into these different memory states by revealing a more intricate organization of working memory than was previously thought. © 2011 Elsevier Ltd.

We investigated whether it is possible to control the temporal window of attention used to rapidly integrate visual information. To study the underlying neural mechanisms, we recorded ERPs in an attentional blink task, known to elicit Lag-1 sparing. Lag-1 sparing fosters joint integration of the two targets, evidenced by increased order errors. Short versus long integration windows were induced by showing participants mostly fast or slow stimuli. Participants expecting slow speed used a longer integration window, increasing joint integration. Difference waves showed an early (200 ms post-T2) negative and a late positive modulation (390 ms) in the fast group, but not in the slow group. The modulations suggest the creation of a separate event for T2, which is not needed in the slow group, where targets were often jointly integrated. This suggests that attention can be guided by global expectations of presentation speed within tens of milliseconds.

Visual search and oddball paradigms were combined to investigate memory for to-be-ignored color changes in a group of 12 healthy participants. The onset of unexpected color change of an irrelevant stimulus evoked two reliable ERP effects: a component of the event-related potential (ERP), similar to the visual mismatch negativity response (vMMN), with a latency of 120-160 ms and a posterior distribution over the left hemisphere and Late Fronto-Central Negativity (LFCN) with a latency of 320-400 ms, apparent at fronto-central electrodes and some posterior sites. Color change of that irrelevant stimulus also slowed identification of a visual target, indicating distraction. The amplitude of this color-change vMMN, but not LFCN, indexed this distraction effect. That is, electrophysiological and behavioral measures were correlated. The interval between visual scenes approximated 1 s (611-1629 ms), indicating that the brain's sensory memory for the color of the preceding visual scenes must persist for at least 600 ms. Therefore, in the case of the neural code for color, durable memory representations are formed in an obligatory manner. (C) 2014 Elsevier B.V. All rights reserved.

Spoken word recognition and production require fast transformations between acoustic, phonological, and conceptual neural representations. Bilinguals perform these transformations in native and non-native languages, deriving unified semantic concepts from equivalent, but acoustically different words. Here we exploit this capacity of bilinguals to investigate input invariant semantic representations in the brain. We acquired EEG data while Dutch subjects, highly proficient in English listened to four monosyllabic and acoustically distinct animal words in both languages (e.g., "paard"-"horse"). Multivariate pattern analysis (MVPA) was applied to identify EEG response patterns that discriminate between individual words within one language (within-language discrimination) and generalize meaning across two languages (across-language generalization). Furthermore, employing two EEG feature selection approaches, we assessed the contribution of temporal and oscillatory EEG features to our classification results. MVPA revealed that within-language discrimination was possible in a broad time-window (~50-620 ms) after word onset probably reflecting acoustic-phonetic and semantic-conceptual differences between the words. Most interestingly, significant across-language generalization was possible around 550-600 ms, suggesting the activation of common semantic-conceptual representations from the Dutch and English nouns. Both types of classification, showed a strong contribution of oscillations below 12 Hz, indicating the importance of low frequency oscillations in the neural representation of individual words and concepts. This study demonstrates the feasibility of MVPA to decode individual spoken words from EEG responses and to assess the spectro-temporal dynamics of their language invariant semantic-conceptual representations. We discuss how this method and results could be relevant to track the neural mechanisms underlying conceptual encoding in comprehension and production.

Many perceptual decision making models posit that participants accumulate noisy evidence over time to improve the accuracy of their decisions, and that in free response tasks, participants respond when the accumulated evidence reaches a decision threshold. Research on the neural correlates of these models' components focuses primarily on evidence accumulation. Far less attention has been paid to the neural correlates of decision thresholds, reflecting the final commitment to a decision. Inspired by a model of bistable neural activity that implements a decision threshold, we reinterpret human lateralized readiness potentials (LRPs) as reflecting the crossing of a decision threshold. Interestingly, this threshold crossing preserves signatures of a drift-diffusion process of evidence accumulation that feeds in to the threshold mechanism. We show that, as our model predicts, LRP amplitudes and growth rates recorded while participants performed a motion discrimination task correlate with individual differences in behaviorally-estimated prior beliefs, decision thresholds and evidence accumulation rates. As such LRPs provide a useful measure to test dynamical models of both evidence accumulation and decision commitment processes non-invasively. © 2014 van Vugt et al.

During sentence production, linguistic information (semantics, syntax, phonology) of words is retrieved and assembled into a meaningful utterance. There is still debate on how we assemble single words into more complex syntactic structures such as noun phrases or sentences. In the present study, event-related potentials (ERPs) were used to investigate the time course of syntactic planning. Thirty-three volunteers described visually animated scenes using naming formats varying in syntactic complexity: from simple words ('W', e.g., "triangle", "red", "square", "green", "to fly towards"), to noun phrases ('NP', e.g., "the red triangle", "the green square", "to fly towards"), to a sentence ('S', e.g., "The red triangle flies towards the green square."). Behaviourally, we observed an increase in errors and corrections with increasing syntactic complexity, indicating a successful experimental manipulation. In the ERPs following scene onset, syntactic complexity variations were found in a P300-like component ('S'/'NP'>'W') and a fronto-central negativity (linear increase with syntactic complexity). In addition, the scene could display two actions - unpredictable for the participant, as the disambiguation occurred only later in the animation. Time-locked to the moment of visual disambiguation of the action and thus the verb, we observed another P300 component ('S'>'NP'/'W'). The data show for the first time evidence of sensitivity to syntactic planning within the P300 time window, time-locked to visual events critical of syntactic planning. We discuss the findings in the light of current syntactic planning views. © 2013 Timmers et al.

We investigated the neural correlates induced by prenatal exposure to melodies using brains' event-related potentials (ERPs). During the last trimester of pregnancy, the mothers in the learning group played the ‘Twinkle twinkle little star’ -melody 5 times per week. After birth and again at the age of 4 months, we played the infants a modified melody in which some of the notes were changed while ERPs to unchanged and changed notes were recorded. The ERPs were also recorded from a control group, who received no prenatal stimulation. Both at birth and at the age of 4 months, infants in the learning group had stronger ERPs to the unchanged notes than the control group. Furthermore, the ERP amplitudes to the changed and unchanged notes at birth were correlated with the amount of prenatal exposure. Our results show that extensive prenatal exposure to a melody induces neural representations that last for several months.

Patients with classic galactosemia, an inborn error of metabolism, have speech and language production impairments. Past research primarily focused on speech (motor) problems, but these cannot solely explain the language impairments. Which specific deficits contribute to the impairments in language production is not yet known. Deficits in semantic and syntactic planning are plausible and require further investigation. In the present study, we examined syntactic encoding while patients and matched controls overtly described scenes of moving objects using either separate words (minimal syntactic planning) or sentences (sentence-level syntactic planning). The design of the paradigm also allowed tapping into local noun phrase- and more global sentence-level syntactic planning. Simultaneously, we recorded event-related potentials (ERPs). The patients needed more time to prepare and finish the utterances and made more errors. The patient ERPs had a very similar morphology to that of healthy controls, indicating overall comparable neural processing. Most importantly, the ERPs diverged from those of controls in several functionally informative time windows, ranging from very early (90-150 ms post scene onset) to relatively late (1820-2020 ms post scene onset). These time windows can be associated with different linguistic encoding stages. The ERP results form the first neuroscientific evidence for language production impairments in patients with galactosemia in lexical and syntactic planning stages, i.e., prior to the linguistic output phase. These findings hence shed new light on the language impairments in this disease.

Converging electrophysiological and brain-imaging results show that sensory processing in V1 can be modulated by attention. In this study, we tested the prediction that this early filtering effect depends on the current affective state of the participant. We recorded visual evoked potentials (VEPs) to visual peripheral distractors while participants performed a demanding task at fixation, whose perceptual load was manipulated in a parametric fashion. Crucially, levels of negative affect were either increased or decreased independently of changes in perceptual load. Concurrent psychophysiological measurements and self-report scales confirmed that changes in emotional state were effective. In the control condition, ERP results showed that the C1 component generated in response to the exact same peripheral distractors systematically varied in amplitude with the amount of perceptual load imposed at fixation, being larger when perceptual load decreased. However, this early modulatory effect in V1 was disrupted when participants transiently experienced increased state anxiety, resulting in a decreased C1 amplitude even though task load at fixation remained low. These results suggest that early bottom-up processing in V1 is not only influenced by the amount of attention resources available, but also by the current internal state of the participant.

Increasing evidence suggests that working memory and perceptual processes are dynamically interrelated due to modulating activity in overlapping brain networks. However, the direct influence of working memory on the spatio-temporal brain dynamics of behaviorally relevant intervening information remains unclear. To investigate this issue, subjects performed a visual proximity grid perception task under three different visual-spatial working memory (VSWM) load conditions. VSWM load was manipulated by asking subjects to memorize the spatial locations of 6 or 3 disks. The grid was always presented between the encoding and recognition of the disk pattern. As a baseline condition, grid stimuli were presented without a VSWM context. VSWM load altered both perceptual performance and neural networks active during intervening grid encoding. Participants performed faster and more accurately on a challenging perceptual task under high VSWM load as compared to the low load and the baseline condition. Visual evoked potential (VEP) analyses identified changes in the configuration of the underlying sources in one particular period occurring 160-190 ms post-stimulus onset. Source analyses further showed an occipito-parietal down-regulation concurrent to the increased involvement of temporal and frontal resources in the high VSWM context. Together, these data suggest that cognitive control mechanisms supporting working memory may selectively enhance concurrent visual processing related to an independent goal. More broadly, our findings are in line with theoretical models implicating the engagement of frontal regions in synchronizing and optimizing mnemonic and perceptual resources towards multiple goals.