Search

A crucial ability of the human brain is to learn and exploit probabilistic associations between stimuli to facilitate perception and behavior by predicting future events. While studies have shown how perceptual relationships are used to predict sensory inputs, relational knowledge is often between concepts rather than percepts (e.g., we learned to associate cats with dogs, rather than specific images of cats and dogs). Here we asked if and how sensory responses to visual input may be modulated by predictions derived from conceptual associations. To this end we exposed participants of both sexes to arbitrary word-word pairs (e.g., car – dog) repeatedly, creating an expectation of the second word, conditional on the occurrence of the first. In a subsequent session, we exposed participants to novel word-picture pairs, while measuring fMRI BOLD responses. All word-picture pairs were equally likely, but half of the pairs conformed to the previously formed conceptual (word-word) associations, whereas the other half violated this association. Results showed suppressed sensory responses throughout the ventral visual stream, including early visual cortex, to pictures that corresponded to the previously expected words compared to unexpected words. This suggests that the learned conceptual associations were used to generate sensory predictions that modulated processing of the picture stimuli. Moreover, these modulations were tuning-specific, selectively suppressing neural populations tuned towards the expected input. Combined, our results suggest that recently acquired conceptual priors are generalized across domains and used by the sensory brain to generate category specific predictions, facilitating processing of expected visual input.SIGNIFICANCE STATEMENT:Perceptual predictions play a crucial role in facilitating perception and the integration of sensory information. However, little is known about whether and how the brain uses more abstract, conceptual priors to form sensory predictions. In our pre-registered study, we show that priors derived from recently acquired arbitrary conceptual associations result in category specific predictions that modulate perceptual processing throughout the ventral visual hierarchy, including early visual cortex. These results suggest that the predictive brain uses prior knowledge across various domains to modulate perception, thereby extending our understanding of the extensive role predictions play in perception.

Contains fulltext : 284051.pdf (Publisher’s version ) (Open Access) In our connected era, we spend significant time and effort satisfying our curiosity. Often, we choose which information we seek, but sometimes the selection is made for us. We hypothesized that humans exhibit enhanced curiosity in the context of choice. We designed a task in which healthy participants saw two lotteries on each trial. On some trials, participants chose which lottery to play. On other trials, the lottery was selected for them. Participants then indicated their curiosity about the outcome of the to-be-played lottery via self-report ratings (Experiment 1, N = 34) or willingness-to-wait decisions (Experiment 2, N = 34). We found that participants exhibited higher curiosity ratings and greater willingness to wait for the outcome of lotteries they had chosen than for lotteries that had been selected for them (controlling for initial preference). This demonstrates that choice boosts curiosity, which may have implications for boosting learning, memory, and motivation. 12 p.

It has been proposed that autistic perception may be marked by a reduced influence of temporal context. Under this hypothesis, prior exposure to a stimulus should lead to a weaker or absent alteration of the behavioral and neural response to the stimulus in autism, compared to a typical population. To examine this hypothesis, we recruited two samples of human volunteers: a student sample (N=26), which we used to establish our analysis pipeline, and an adolescent sample (N=36), which consisted of a group of autistic (N=18) and a group of non-autistic (N=18) participants. All participants were presented with visual stimulus streams consisting of novel and familiar image pairs, while they attentively monitored each stream. We recorded task performance and used magnetoencephalography (MEG) to measure neural responses, and to compare the responses to familiar and novel images. We found behavioral facilitation as well as a reduction of event-related field (ERF) amplitude for familiar, compared to novel, images in both samples. Crucially, we found statistical evidence against between-group effects of familiarity on both behavioral and neural responses in the adolescent sample, suggesting that the influence of familiarity is comparable between autistic and non-autistic adolescents. These findings challenge the notion that perception in autism is marked by a reduced influence of prior exposure.

Contains fulltext : 292390.pdf (Publisher’s version ) (Closed access) The human brain is capable of using statistical regularities to predict future inputs. In the real world, such inputs typically comprise a collection of objects (e.g. a forest constitutes numerous trees). The present study aimed to investigate whether perceptual anticipation relies on lower-level or higher-level information. Specifically, we examined whether the human brain anticipates each object in a scene individually or anticipates the scene as a whole. To explore this issue, we first trained participants to associate co-occurring objects within fixed spatial arrangements. Meanwhile, participants implicitly learned temporal regularities between these displays. We then tested how spatial and temporal violations of the structure modulated behavior and neural activity in the visual system using fMRI. We found that participants only showed a behavioral advantage of temporal regularities when the displays conformed to their previously learned spatial structure, demonstrating that humans form configuration-specific temporal expectations instead of predicting individual objects. Similarly, we found suppression of neural responses for temporally expected compared with temporally unexpected objects in lateral occipital cortex only when the objects were embedded within expected configurations. Overall, our findings indicate that humans form expectations about object configurations, demonstrating the prioritization of higher-level over lower-level information in temporal expectation. 12 p.

Contains fulltext : 227546.pdf (Publisher’s version ) (Open Access) We predict how our actions will influence the world around us. Prevailing models in the action control literature propose that we use these predictions to suppress or "cancel" perception of expected action outcomes, to highlight more informative surprising events. However, contrasting normative Bayesian models in sensory cognition suggest that we are more, not less, likely to perceive what we expect - given that what we expect is more likely to occur. Here we adjudicated between these models by investigating how expectations influence perceptual decisions about action outcomes in a signal detection paradigm. Across three experiments, participants performed one of two manual actions that were sometimes accompanied by brief presentation of expected or unexpected visual outcomes. Contrary to dominant cancellation models but consistent with Bayesian accounts, we found that observers were biased to report the presence of expected action outcomes. There were no effects of expectation on sensitivity. Computational modeling revealed that the action-induced bias reflected a sensory bias in how evidence was accumulated rather than a baseline shift in decision circuits. Expectation effects remained in Experiments 2 and 3 when orthogonal cues indicated which finger was more likely to be probed (i.e. task-relevant). These biases toward perceiving expected action outcomes are suggestive of a mechanism that would enable generation of largely veridical representations of our actions and their consequences in an inherently uncertain sensory world. 12 p.

Contains fulltext : 285196.pdf (Publisher’s version ) (Open Access) Expectations, derived from previous experience, can help in making perception faster, more reliable and informative. A key neural signature of perceptual expectations is expectation suppression, an attenuated neural response to expected compared with unexpected stimuli. While expectation suppression has been reported using a variety of paradigms and recording methods, it remains unclear what neural modulation underlies this response attenuation. Sharpening models propose that neural populations tuned away from an expected stimulus are particularly suppressed by expectations, thereby resulting in an attenuated, but sharper population response. In contrast, dampening models suggest that neural populations tuned toward the expected stimulus are most suppressed, thus resulting in a dampened, less redundant population response. Empirical support is divided, with some studies favoring sharpening, while others support dampening. A key limitation of previous neuroimaging studies is the ability to draw inferences about neural-level modulations based on population (e.g. voxel) level signals. Indeed, recent simulations of repetition suppression showed that opposite neural modulations can lead to comparable population-level modulations. Forward models provide one solution to this inference limitation. Here, we used forward models to implement sharpening and dampening models, mapping neural modulations to voxel-level data. We show that a feature-specific gain modulation, suppressing neurons tuned toward the expected stimulus, best explains the empirical fMRI data. Thus, our results support the dampening account of expectation suppression, suggesting that expectations reduce redundancy in sensory cortex, and thereby promote updating of internal models on the basis of surprising information. 14 p.

Contains fulltext : 292546.pdf (Publisher’s version ) (Open Access) The relationship between conscious experience and brain activity has intrigued scientists and philosophers for centuries. In the last decades, several theories have suggested different accounts for these relationships. These theories have developed in parallel, with little to no cross-talk among them. To advance research on consciousness, we established an adversarial collaboration between proponents of two of the major theories in the field, Global Neuronal Workspace and Integrated Information Theory. Together, we devised and preregistered two experiments that test contrasting predictions of these theories concerning the location and timing of correlates of visual consciousness, which have been endorsed by the theories’ proponents. Predicted outcomes should either support, refute, or challenge these theories. Six theory-impartial laboratories will follow the study protocol specified here, using three complementary methods: Functional Magnetic Resonance Imaging (fMRI), Magneto-Electroencephalography (M-EEG), and intracranial electroencephalography (iEEG). The study protocol will include built-in replications, both between labs and within datasets. Through this ambitious undertaking, we hope to provide decisive evidence in favor or against the two theories and clarify the footprints of conscious visual perception in the human brain, while also providing an innovative model of large-scale, collaborative, and open science practice. 28 p.

What is selected when attention is directed to a specific location of the visual field? Theories of object-based attention have suggested that when spatial attention is directed to part of an object, attention does not simply enhance the attended location but automatically spreads to enhance all locations that comprise the object. Here, we tested this hypothesis by reconstructing the distribution of attention from primary visual cortex (V1) population neuronal activity patterns in 24 human adults (17 female) using functional magnetic resonance imaging (fMRI) and population-based receptive field (prf) mapping. We find that attention spreads from a spatially cued location to the underlying object, and enhances all spatial locations that comprise the object. Importantly, this spreading was also evident when the object was not task relevant. These data suggest that attentional selection automatically operates at an object level, facilitating the reconstruction of coherent objects from fragmented representations in early visual cortex.SIGNIFICANCE STATEMENTObject perception is an astonishing feat of the visual system. When visual information about orientation, shape, and color enters through our eyes, it has yet to be integrated into a coherent representation of an object. But which visual features constitute a single object and which features belong to the background? The brain mechanisms underpinning object perception are yet to be understood. We now demonstrate that one candidate mechanism, the successive activation of all parts of an object, occurs in early visual cortex and results in a detailed representation of the object following Gestalt principles. Furthermore, our results suggest that object selection occurs automatically, without involving voluntary control.

Human agents build models of their environment, which enable them to anticipate and plan upcoming events. However, little is known about the properties of such predictive models. Recently, it has been proposed that hippocampal representations take the form of a predictive map-like structure, the so-called successor representation. Here we used human fMRI to probe whether activity in the early visual cortex (V1) and hippocampus adhere to the postulated properties of the successor representation after visual sequence learning. Participants were exposed to an arbitrary spatiotemporal sequence consisting of four items (A-B-C-D). We found that after repeated exposure to the sequence, merely presenting single sequence items (e.g., - B - -) resulted in V1 activation at the successor locations of the full sequence (e.g., C-D), but not at the predecessor locations (e.g., A). This highlights that visual representations are skewed toward future states, in line with the successor representation. Similar results were also found in the hippocampus. Moreover, the hippocampus developed a tuning profile that showed sensitivity to the temporal distance in sequence-space, with fading representations for sequence events in the more distant past and future. V1, in contrast, showed a tuning profile that was only sensitive to spatial distance in stimulus-space. Together, these results provide empirical evidence for the proposition that both visual and hippocampal cortex represent a predictive map of the visual world akin to the successor representation.

Prediction errors have a prominent role in many forms of learning. For example, in reinforcement learning, agents learn by updating the association between states and outcomes as a function of the prediction error elicited by the event. One paradigm often used to study error-driven learning is blocking. In forward blocking, participants are first presented with stimulus A, followed by outcome X (A→X). In the second phase, A and B are presented together, followed by X (AB→X). Here, A→X blocks the formation of B→X, given that X is already fully predicted by A. In backward blocking, the order of phases is reversed. Here, the association between B and X that is formed during the first learning phase of AB→X is weakened when participants learn exclusively A→X in the second phase. The present study asked the question whether forward and backward blocking occur during visual statistical learning, the largely automatic and incidental learning of the statistical structure of the environment. In a series of studies, using both forward and backward blocking, we observed robust statistical learning of temporal associations among pairs of images. While we found no compelling evidence for forward blocking, we observed reliable backward blocking in visual statistical learning.

The ability to recognize and predict future spatiotemporal sequences is vital for perception. It has been proposed that the brain makes ‘intelligent guesses’ about future inputs by forward replaying these events. However, it is unknown whether and how this mechanism incorporates the probabilistic structure that is inherent to naturalistic environments. Here we tested forward replay in human V1 and hippocampus using a probabilistic cueing paradigm. Participants were exposed to two visual moving dot sequences (A and B) that shared the same starting point. Each stimulus sequence was paired with either a high or a low tone that predicted which sequence would follow with 80% cue validity (probabilistic context) or 50% cue validity (random context). We found that after exposure, the auditory cue together with the starting point triggered simultaneous forward replay of both the likely (A) and the less likely (B) stimulus sequence. Crucially, forward replay preserved the probabilistic relationship of the environment, such that the likely sequence was associated with greater anticipatory V1 activity compared to the less likely stimulus sequence. Analogous to V1, forward replay in hippocampus was also found to preserve the probabilistic cue-sequence relationship. Further, the anterior hippocampus was found to represent the predicted stimulus sequence, irrespective of the input, while the posterior hippocampus revealed a prediction error-like signal that was only observed when predictions were violated. These findings show how mnemonic and sensory areas coordinate predictive representations in probabilistic contexts to improve perceptual processing.

Sounds enhance the detection of visual stimuli while concurrently biasing an observer’s decisions. To investigate the neural mechanisms that underlie such multisensory interactions, we decoded time-resolved signal detection theory (SDT) sensitivity and criterion parameters from neural activity using magnetoencalography, while participants performed a visual detection task. Using temporal generalization analysis, we found that sounds improve visual detection by enhancing the maintenance of the most informative perceptual samples over time. In parallel, criterion decoding analyses revealed that sounds evoke patterns of activity that resembled the patterns evoked by an actual visual stimulus. These two complementary mechanisms of audiovisual interaction differed in terms of their automaticity: Whereas the sound-induced enhancement in visual information maintenance depended on participants being actively engaged in a detection task, sounds evoked visual activity patterns in the visual cortex in a bottom-up fashion, challenging the classical assumption that sound- induced reductions in criterion correspond to decision-level biases.

Item does not contain fulltext Predictive processing has become an influential framework in cognitive neuroscience. However, it often lacks specificity and direct empirical support. How can we probe the nature and limits of the predictive brain? We highlight the potential of recent advances in artificial intelligence (AI) for providing a richer and more computationally explicit test of this theory of cortical function. 2 p.

Contains fulltext : 252589.pdf (Publisher’s version ) (Open Access) Theorists propose that the brain constantly generates implicit predictions that guide information processing. During language comprehension, such predictions have indeed been observed, but it remains disputed under which conditions and at which processing level these predictions occur. Here, we address both questions by analyzing brain recordings of participants listening to audiobooks, and using a deep neural network to quantify the predictions evoked by the story. We find that brain responses are continuously modulated by linguistic predictions. We observe predictions at the level of meaning, grammar, words, and speech sounds, and find that high-level predictions can inform low-level ones. These results establish the predictive nature of language processing, demonstrating that the brain spontaneously predicts upcoming language at multiple levels of abstraction. Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction to guide the interpretation of incoming input. However, the role of prediction in language processing remains disputed, with disagreement about both the ubiquity and representational nature of predictions. Here, we address both issues by analyzing brain recordings of participants listening to audiobooks, and using a deep neural network (GPT-2) to precisely quantify contextual predictions. First, we establish that brain responses to words are modulated by ubiquitous predictions. Next, we disentangle model-based predictions into distinct dimensions, revealing dissociable neural signatures of predictions about syntactic category (parts of speech), phonemes, and semantics. Finally, we show that high-level (word) predictions inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore the ubiquity of prediction in language processing, showing that the brain spontaneously predicts upcoming language at multiple levels of abstraction. 12 p.

Contains fulltext : 285204.pdf (Publisher’s version ) (Open Access) Expectations shape our experience of music. However, the internal model upon which listeners form melodic expectations is still debated. Do expectations stem from Gestalt-like principles or statistical learning? If the latter, does long-term experience play an important role, or are short-term regularities sufficient? And finally, what length of context informs contextual expectations? To answer these questions, we presented human listeners with diverse naturalistic compositions from Western classical music, while recording neural activity using MEG. We quantified note-level melodic surprise and uncertainty using various computational models of music, including a state-of-the-art transformer neural network. A time-resolved regression analysis revealed that neural activity over fronto-temporal sensors tracked melodic surprise particularly around 200 ms and 300-500 ms after note onset. This neural surprise response was dissociated from sensory-acoustic and adaptation effects. Neural surprise was best predicted by computational models that incorporated long-term statistical learning - rather than by simple, Gestalt-like principles. Yet, intriguingly, the surprise reflected primarily short-range musical contexts of less than ten notes. We present a full replication of our novel MEG results in an openly available EEG dataset. Together, these results elucidate the internal model that shapes melodic predictions during naturalistic music listening. 31 p.

Visual scene context is well-known to facilitate the recognition of scene-congruent objects. Interestingly, however, according to the influential theory of predictive coding, scene congruency should lead to reduced (rather than enhanced) processing of congruent objects, compared to incongruent ones, since congruent objects elicit reduced prediction error responses. We tested this counterintuitive hypothesis in two online behavioural experiments with human participants (N = 300). We found clear evidence for impaired perception of congruent objects, both in a change detection task measuring response times as well as in a bias-free object discrimination task measuring accuracy. Congruency costs were related to independent subjective congruency ratings. Finally, we show that the reported effects cannot be explained by low-level stimulus confounds, response biases, or top-down strategy. These results provide convincing evidence for perceptual congruency costs during scene viewing, in line with predictive coding theory.Statement of RelevanceThe theory of the ‘Bayesian brain’, the idea that our brain is a hypothesis-testing machine, has become very influential over the past decades. A particularly influential formulation is the theory of predictive coding. This theory entails that stimuli that are expected, for instance because of the context in which they appear, generate a weaker neural response than unexpected stimuli. Scene context correctly ‘predicts’ congruent scene elements, which should result in lower prediction error. Our study tests this important, counterintuitive, and hitherto not fully tested, hypothesis. We find clear evidence in favour of it, and demonstrate that these ‘congruency costs’ are indeed evident in perception, and not limited to one particular task setting or stimulus set. Since perception in the real world is never of isolated objects, but always of entire scenes, these findings are important not just for the Bayesian brain hypothesis, but for our understanding of real-world visual perception in general.

Contains fulltext : 236285.pdf (Publisher’s version ) (Open Access) Being able to anticipate events before they happen facilitates stimulus processing. The anticipation of the contents of events is thought to be implemented by the elicitation of prestimulus templates in sensory cortex. In contrast, the anticipation of the timing of events is typically associated with entrainment of neural oscillations. It is so far unknown whether and in which conditions temporal expectations interact with feature-based expectations, and, consequently, whether entrainment modulates the generation of content-specific sensory templates. In this study, we investigated the role of temporal expectations in a sensory discrimination task. We presented participants with rhythmically interleaved visual and auditory streams of relevant and irrelevant stimuli while measuring neural activity using magnetoencephalography. We found no evidence that rhythmic stimulation induced prestimulus feature templates. However, we did observe clear anticipatory rhythmic preactivation of the relevant sensory cortices. This oscillatory activity peaked at behaviourally relevant, in-phase, intervals. Our results suggest that temporal expectations about stimulus features do not behave similarly to explicitly cued, nonrhythmic, expectations, yet elicit a distinct form of modality-specific preactivation. 16 p.