Search

Compositionality is a primary feature of language, but graphics can also create combinatorial meaning, like with items above faces (e.g., lightbulbs to mean inspiration). We posit that these “upfixes” (i.e., upwards affixes) involve a productive schema enabling both stored and novel face–upfix dyads. In two experiments, participants viewed either conventional (e.g., lightbulb) or unconventional (e.g., clover-leaves) upfixes with faces which either matched (e.g., lightbulb/smile) or mismatched (e.g., lightbulb/frown). In Experiment 1, matching dyads sponsored higher comprehensibility ratings and faster response times, modulated by conventionality. In Experiment 2, event-related brain potentials (ERPs) revealed conventional upfixes, regardless of matching, evoked larger N250s, indicating perceptual expertise, but mismatching and unconventional dyads elicited larger semantic processing costs (N400) than conventional-matching dyads. Yet mismatches evoked a late negativity, suggesting congruent novel dyads remained construable compared with violations. These results support that combinatorial graphics involve a constrained productive schema, similar to the lexicon of language.

Available online 16 December 2021 We investigated how aging modulates lexico-semantic processes in the visual (seeing written items), auditory (hearing spoken items) and audiovisual (seeing written items while hearing congruent spoken items) modalities. Participants were young and older adults who performed a delayed lexical decision task (LDT) presented in blocks of visual, auditory, and audiovisual stimuli. Event-related potentials (ERPs) revealed differences between young and older adults despite older adults’ ability to identify words and pseudowords as accurately as young adults. The observed differences included more focalized lexico-semantic access in the N400 time window in older relative to young adults, stronger re-instantiation and/or more widespread activity of the lexicality effect at the time of responding, and stronger multimodal integration for older relative to young adults. Our results offer new insights into how functional neural differences in older adults can result in efficient access to lexico-semantic representations across the lifespan. This work was supported by a Marie Curie Individual Fellowship 702178 by the European Commission and a Juan de la Cierva Formacion Fellowship FJCI-2017-31782 from the Spanish Ministry of Science, Innovation and Universities to RALZ, NWO VENI grant 275-89-027 to MB, MINECO grant PSI 2017-82563-P from the Spanish Ministry of Economics and Competitiveness to AGS, Natural Sciences and Engineering Research Council of Canada Discovery Grant 502584 to BCA, and by the Basque Government through the BERC 2018–2021 program.

Article first published online: January 13, 2020 Humans quickly adapt to variations in the speech signal. Adaptation may surface as recalibration, a learning effect driven by error-minimisation between a visual face and an ambiguous auditory speech signal, or as selective adaptation, a contrastive aftereffect driven by the acoustic clarity of the sound. Here, we examined whether these aftereffects occur for vowel identity and voice gender. Participants were exposed to male, female, or androgynous tokens of speakers pronouncing /e/, /ø/, (embedded in words with a consonant-vowel-consonant structure), or an ambiguous vowel halfway between /e/ and /ø/ dubbed onto the video of a male or female speaker pronouncing /e/ or /ø/. For both voice gender and vowel identity, we found assimilative aftereffects after exposure to auditory ambiguous adapter sounds, and contrastive aftereffects after exposure to auditory clear adapter sounds. This demonstrates that similar principles for adaptation in these dimensions are at play. The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by Gravitation Grant 024.001.006 of the Language in Interaction Consortium from Netherlands Organization for Scientific Research. The third author was supported by The Netherlands Organization for Scientific Research (NWO: VENI Grant 275-89-027).

Lip-reading is crucial for understanding speech in challenging conditions. But how the brain extracts meaning from--silent--visual speech is still under debate. Lip-reading in silence activates the auditory cortices, but it is not known whether such activation reflects immediate synthesis of the corresponding auditory stimulus or imagery of unrelated sounds.To disentangle these possibilities, we used magnetoencephalography to evaluate how cortical activity in 28 healthy adults humans (17 females) entrained to the auditory speech envelope and lip movements (mouth opening) when listening to a spoken story without visual input (audio-only), and when seeing a silent video of a speaker articulating another story (video-only).In video-only, auditory cortical activity entrained to the absent auditory signal at frequencies below 1 Hz more than to the seen lip movements. This entrainment process was characterized by an auditory-speech-to-brain delay of ∼70 ms in the left hemisphere, compared to ∼20 ms in audio-only Entrainment to mouth opening was found in the right angular gyrus at below 1 Hz, and in early visual cortices at 1-8 Hz.These findings demonstrate that the brain can use a silent lip-read signal to synthesize a coarse-grained auditory speech representation in early auditory cortices. Our data indicate the following underlying oscillatory mechanism: Seeing lip movements first modulates neuronal activity in early visual cortices at frequencies that match articulatory lip movements; the right angular gyrus then extracts slower features of lip movements, mapping them onto the corresponding speech sound features; this information is fed to auditory cortices, most likely facilitating speech parsing.SIGNIFICANCE STATEMENTLip-reading consists in decoding speech based on visual information derived from observation of a speaker's articulatory facial gestures. Lip reading is known to improve auditory speech understanding, especially when speech is degraded. Interestingly, lip-reading in silence still activates the auditory cortices, even when participants do not know what the absent auditory signal should be. However, it was uncertain what such activation reflected. Here, using magnetoencephalographic recordings, we demonstrate it reflects fast synthesis of the auditory stimulus rather than mental imagery of unrelated--speech or non-speech--sounds. Our results also shed light on the oscillatory dynamics underlying lip-reading., info:eu-repo/semantics/published

Speech perception is influenced by vision through a process of audiovisual integration. This is demonstrated by the McGurk illusion where visual speech (for example /ga/) dubbed with incongruent auditory speech (such as /ba/) leads to a modified auditory percept (/da/). Recent studies have indicated that perception of the incongruent speech stimuli used in McGurk paradigms involves mechanisms of both general and audiovisual speech specific mismatch processing and that general mismatch processing modulates induced theta-band (4-8 Hz) oscillations. Here, we investigated whether the theta modulation merely reflects mismatch processing or, alternatively, audiovisual integration of speech. We used electroencephalographic recordings from two previously published studies using audiovisual sine-wave speech (SWS), a spectrally degraded speech signal sounding nonsensical to naive perceivers but perceived as speech by informed subjects. Earlier studies have shown that informed, but not naive subjects integrate SWS phonetically with visual speech. In an N1/P2 event-related potential paradigm, we found a significant difference in theta-band activity between informed and naive perceivers of audiovisual speech, suggesting that audiovisual integration modulates induced theta-band oscillations. In a McGurk mismatch negativity paradigm (MMN) where infrequent McGurk stimuli were embedded in a sequence of frequent audio-visually congruent stimuli we found no difference between congruent and McGurk stimuli. The infrequent stimuli in this paradigm are violating both the general prediction of stimulus content, and that of audiovisual congruence. Hence, we found no support for the hypothesis that audiovisual mismatch modulates induced theta-band oscillations. We also did not find any effects of audiovisual integration in the MMN paradigm, possibly due to the experimental design.

Cognitive flexibility has been studied in two separate research traditions. Neuropsychologists typically rely on rather complex assessment tools such as the Wisconsin Card Sorting Test (WCST). In contrast, task-switching paradigms are used in experimental psychology to obtain more specific measures of cognitive flexibility. We aim to contribute to the integration of these research traditions by examining the role of the key factor that differs between the WCST and experimental task-switching paradigms: rule uncertainty. In two experimental studies, we manipulated the degree of rule uncertainty after rule switches in a computerized version of the WCST. Across a variety of task parameters, reducing rule uncertainty consistently impaired the speed and accuracy of responses when the rule designated to be more likely turned out to be incorrect. Other performance measures such as the number of perseverative errors were not significantly affected by rule uncertainty. We conclude that a fine-grained analysis of WCST performance can dissociate behavioural indicators that are affected vs. unaffected by rule uncertainty. By this means, it is possible to integrate WCST results and findings obtained from task-switching paradigms that do not involve rule uncertainty. ispartof: ACTA PSYCHOLOGICA vol:190 pages:53-64 ispartof: location:Netherlands status: published

The cognitive view on deception proposes that lying comes with a cognitive cost. This view is supported by the finding that lying typically takes longer than truth telling. Event-related potentials (ERPs) provide a means to unravel the cognitive processes underlying this cost. Using a mock-crime design, the current study (n = 20) investigated the effects of deception on the Contingent Negative Variation (CNV), the Lateralized Readiness Potential (LRP), the Correct Response Negativity (CRN), and the stimulus-locked N200 and P300 components. In line with previous research, lying resulted in more errors, longer reaction times (RTs) and longer RT standard deviations compared to truthful responses. A marginally significant effect suggested a stronger CNV for the anticipation of lying compared to the anticipation of truth telling. There were no significant deception effects on the stimulus- and the response-locked LRPs. Unexpectedly, we found a significantly larger CRN for truth telling compared to lying. Additional analyses revealed an enhanced N200 and a decreased P300 for lying compared to truth telling. Our results support the cognitive load hypothesis for lying, yet are mixed regarding the response conflict hypothesis. Results are discussed with regard to the specific characteristics of our design and their theoretical and applied implications.

Leber inherited optic neuropathy (LHON) is characterized by subacute bilateral loss of central vision due to dysfunction and loss of retinal ganglion cells (RGCs). Comprehensive visual electrophysiological investigations (including pattern reversal visual evoked potentials, pattern electroretinography and the photopic negative response) performed on 13 patients with acute and chronic LHON indicate early impairment of RGC cell body function and severe axonal dysfunction. Temporal, spatial and chromatic psychophysical tests performed on 7 patients with acute LHON and 4 patients with chronic LHON suggest severe involvement or loss of the midget, parasol and bistratified RGCs associated with all three principal visual pathways.

We investigated whether it is possible to control the temporal window of attention used to rapidly integrate visual information. To study the underlying neural mechanisms, we recorded ERPs in an attentional blink task, known to elicit Lag-1 sparing. Lag-1 sparing fosters joint integration of the two targets, evidenced by increased order errors. Short versus long integration windows were induced by showing participants mostly fast or slow stimuli. Participants expecting slow speed used a longer integration window, increasing joint integration. Difference waves showed an early (200 ms post-T2) negative and a late positive modulation (390 ms) in the fast group, but not in the slow group. The modulations suggest the creation of a separate event for T2, which is not needed in the slow group, where targets were often jointly integrated. This suggests that attention can be guided by global expectations of presentation speed within tens of milliseconds.

Visual search and oddball paradigms were combined to investigate memory for to-be-ignored color changes in a group of 12 healthy participants. The onset of unexpected color change of an irrelevant stimulus evoked two reliable ERP effects: a component of the event-related potential (ERP), similar to the visual mismatch negativity response (vMMN), with a latency of 120-160 ms and a posterior distribution over the left hemisphere and Late Fronto-Central Negativity (LFCN) with a latency of 320-400 ms, apparent at fronto-central electrodes and some posterior sites. Color change of that irrelevant stimulus also slowed identification of a visual target, indicating distraction. The amplitude of this color-change vMMN, but not LFCN, indexed this distraction effect. That is, electrophysiological and behavioral measures were correlated. The interval between visual scenes approximated 1 s (611-1629 ms), indicating that the brain's sensory memory for the color of the preceding visual scenes must persist for at least 600 ms. Therefore, in the case of the neural code for color, durable memory representations are formed in an obligatory manner. (C) 2014 Elsevier B.V. All rights reserved.

Contains fulltext : 55621.pdf (Publisher’s version ) (Closed access) This study examined auditory processing in 2-month-old infants at genetic risk for dyslexia and in controls. Manipulated natural speech stimuli (/bAk/ and /dAk/), at either side of the phoneme boundary, were presented to these infants and their automatic cortical deviance responses were recorded. Control infants showed two distinct mismatch responses, thus extending similar findings reported with kindergartners in terms of topographical distribution and cortical sources. The absence of such mismatch responses in the infants at risk supports the hypothesis of basic auditory (temporal) processing impairments in the disorder. The results suggest that these early signs of deficient auditory processing may point to problematic categorical perception at a later age. 6 p.

The ERP experiment reported here addresses some outstanding questions regarding word processing in sentential contexts: (1) Does only the 'message-level' representation (the representation of sentence meaning combining lexico-semantic and syntactic constraints) affect the processing of the incoming word [J. Exp. Psychol.: Learn. Mem. Cogn. 20 (1994) 92]? (2) Is lexically specified semantic relatedness between multiple words the primary factor instead [J. Exp. Psychol.: Learn. Mem. Cogn. 15 (1989) 791]? (3) Alternatively, do word and sentence level information interact during sentence comprehension? Volunteers read sentences (e.g. Dutch sentences resembling The javelin was by the athletes...) in which the (passive) syntactic structure and the semantic content of the lexical items together created a strong expectation of a specific final word (e.g., thrown), but also sentences in which the syntactic structure was changed from passive to active (e.g. Dutch sentences resembling The javelin has the athletes...), which altered the message level constraint substantially and strongly reduced. the expectation of any particular completion. Half of the sentences ended in a final word with a good lexico-semantic fit relative to the preceding content words (e.g. thrown, fitting well with the preceding javelin and athletes). This creates very plausible sentences in the strong constraint context but semantically anomalous ones in the weakly constraining context (e.g., The javelin has the athletes thrown). In the other half the final word had a poor lexico-semantic fit (e.g., summarized that does not fit at all with javelin and athletes). Good lexico-semantic fit endings showed no difference in N400 amplitude in the strong and weak message-level constraint sentences, despite the fact that the latter were semantically anomalous. This result suggests that lexico-semantic fit can be more important for word processing than the meaning of the sentence as determined by the syntactic structure, at least initially. These conditions did differ, however, in the region of the P600 where the anomalous weak constraint version was much more positive, a pattern usually seen with ungrammatical sentences. The processing of poor lexicosemantic fit words showed a quite different pattern; in both strong and weak constraint sentences they elicited a substantial N400 effect, but N400-amplitude was significantly more negative following strong constraint contexts, even though both sentence contexts were equivalently anomalous. Taken together, these findings provide evidence for the importance of both message-level and lexico-semantic information during sentence comprehension. The implications for theories of sentence interpretation are discussed and an extension of the message-based hypothesis will be proposed. (C) 2003 Elsevier B.V. All rights reserved.

Many perceptual decision making models posit that participants accumulate noisy evidence over time to improve the accuracy of their decisions, and that in free response tasks, participants respond when the accumulated evidence reaches a decision threshold. Research on the neural correlates of these models' components focuses primarily on evidence accumulation. Far less attention has been paid to the neural correlates of decision thresholds, reflecting the final commitment to a decision. Inspired by a model of bistable neural activity that implements a decision threshold, we reinterpret human lateralized readiness potentials (LRPs) as reflecting the crossing of a decision threshold. Interestingly, this threshold crossing preserves signatures of a drift-diffusion process of evidence accumulation that feeds in to the threshold mechanism. We show that, as our model predicts, LRP amplitudes and growth rates recorded while participants performed a motion discrimination task correlate with individual differences in behaviorally-estimated prior beliefs, decision thresholds and evidence accumulation rates. As such LRPs provide a useful measure to test dynamical models of both evidence accumulation and decision commitment processes non-invasively. © 2014 van Vugt et al.

During sentence production, linguistic information (semantics, syntax, phonology) of words is retrieved and assembled into a meaningful utterance. There is still debate on how we assemble single words into more complex syntactic structures such as noun phrases or sentences. In the present study, event-related potentials (ERPs) were used to investigate the time course of syntactic planning. Thirty-three volunteers described visually animated scenes using naming formats varying in syntactic complexity: from simple words ('W', e.g., "triangle", "red", "square", "green", "to fly towards"), to noun phrases ('NP', e.g., "the red triangle", "the green square", "to fly towards"), to a sentence ('S', e.g., "The red triangle flies towards the green square."). Behaviourally, we observed an increase in errors and corrections with increasing syntactic complexity, indicating a successful experimental manipulation. In the ERPs following scene onset, syntactic complexity variations were found in a P300-like component ('S'/'NP'>'W') and a fronto-central negativity (linear increase with syntactic complexity). In addition, the scene could display two actions - unpredictable for the participant, as the disambiguation occurred only later in the animation. Time-locked to the moment of visual disambiguation of the action and thus the verb, we observed another P300 component ('S'>'NP'/'W'). The data show for the first time evidence of sensitivity to syntactic planning within the P300 time window, time-locked to visual events critical of syntactic planning. We discuss the findings in the light of current syntactic planning views. © 2013 Timmers et al.

Patients with classic galactosemia, an inborn error of metabolism, have speech and language production impairments. Past research primarily focused on speech (motor) problems, but these cannot solely explain the language impairments. Which specific deficits contribute to the impairments in language production is not yet known. Deficits in semantic and syntactic planning are plausible and require further investigation. In the present study, we examined syntactic encoding while patients and matched controls overtly described scenes of moving objects using either separate words (minimal syntactic planning) or sentences (sentence-level syntactic planning). The design of the paradigm also allowed tapping into local noun phrase- and more global sentence-level syntactic planning. Simultaneously, we recorded event-related potentials (ERPs). The patients needed more time to prepare and finish the utterances and made more errors. The patient ERPs had a very similar morphology to that of healthy controls, indicating overall comparable neural processing. Most importantly, the ERPs diverged from those of controls in several functionally informative time windows, ranging from very early (90-150 ms post scene onset) to relatively late (1820-2020 ms post scene onset). These time windows can be associated with different linguistic encoding stages. The ERP results form the first neuroscientific evidence for language production impairments in patients with galactosemia in lexical and syntactic planning stages, i.e., prior to the linguistic output phase. These findings hence shed new light on the language impairments in this disease.

Converging electrophysiological and brain-imaging results show that sensory processing in V1 can be modulated by attention. In this study, we tested the prediction that this early filtering effect depends on the current affective state of the participant. We recorded visual evoked potentials (VEPs) to visual peripheral distractors while participants performed a demanding task at fixation, whose perceptual load was manipulated in a parametric fashion. Crucially, levels of negative affect were either increased or decreased independently of changes in perceptual load. Concurrent psychophysiological measurements and self-report scales confirmed that changes in emotional state were effective. In the control condition, ERP results showed that the C1 component generated in response to the exact same peripheral distractors systematically varied in amplitude with the amount of perceptual load imposed at fixation, being larger when perceptual load decreased. However, this early modulatory effect in V1 was disrupted when participants transiently experienced increased state anxiety, resulting in a decreased C1 amplitude even though task load at fixation remained low. These results suggest that early bottom-up processing in V1 is not only influenced by the amount of attention resources available, but also by the current internal state of the participant.

Increasing evidence suggests that working memory and perceptual processes are dynamically interrelated due to modulating activity in overlapping brain networks. However, the direct influence of working memory on the spatio-temporal brain dynamics of behaviorally relevant intervening information remains unclear. To investigate this issue, subjects performed a visual proximity grid perception task under three different visual-spatial working memory (VSWM) load conditions. VSWM load was manipulated by asking subjects to memorize the spatial locations of 6 or 3 disks. The grid was always presented between the encoding and recognition of the disk pattern. As a baseline condition, grid stimuli were presented without a VSWM context. VSWM load altered both perceptual performance and neural networks active during intervening grid encoding. Participants performed faster and more accurately on a challenging perceptual task under high VSWM load as compared to the low load and the baseline condition. Visual evoked potential (VEP) analyses identified changes in the configuration of the underlying sources in one particular period occurring 160-190 ms post-stimulus onset. Source analyses further showed an occipito-parietal down-regulation concurrent to the increased involvement of temporal and frontal resources in the high VSWM context. Together, these data suggest that cognitive control mechanisms supporting working memory may selectively enhance concurrent visual processing related to an independent goal. More broadly, our findings are in line with theoretical models implicating the engagement of frontal regions in synchronizing and optimizing mnemonic and perceptual resources towards multiple goals.

Name agreement is the extent to which different people agree on a name for a particular picture. Previous studies have found that it takes longer to name low name agreement pictures than high name agreement pictures. To examine the effect of name agreement in the online process of picture naming, we compared event-related potentials (ERPs) recorded whilst 19 healthy, native English speakers silently named pictures which had either high or low name agreement. A series of ERP components was examined: P1 approximately 120 ms from picture onset, N1 around 170 ms, P2 around 220 ms, N2 around 290 ms, and P3 around 400 ms. Additionally, a late time window from 800 to 900 ms was considered. Name agreement had an early effect, starting at P1 and possibly resulting from uncertainty of picture identity, and continuing into N2, possibly resulting from alternative names for pictures. These results support the idea that name agreement affects two consecutive processes: first, object recognition, and second, lexical selection and/or phonological encoding. (C) 2010 Elsevier B.V. All rights reserved.