Search

Deep Convolutional Neural Networks (CNNs) have been one of the most influential recent developments in computer vision, particularly for categorization [20]. The promise of CNNs is at least two-fold. First, they represent the best engineering solution to successfully tackle the foundational task of visual categorization with a performance level that even exceeds that of humans [19, 27]. Second, for computational neuroscience, CNNs provide a testable modelling platform for visual categorizations inspired by the multi-layered organization of visual cortex [7]. Here, we used a 3D generative model to control the variance of information learned to identify 2,000 face identities in one CNN architecture (10-layer ResNet [9]). We generated 25M face images to train the network by randomly sampling intrinsic (i.e. face morphology, gender, age, expression and ethnicity) and extrinsic factors of face variance (i.e. 3D pose, illumination, scale and 2D translation). At testing, the network performed with 99% generalization accuracy for face identity across variations of intrinsic and extrinsic factors. State-of-the-art information mapping techniques from psychophysics (i.e. Representational Similarity Analysis [18] and Bubbles [8]) revealed respectively the network layer at which factors of variance are resolved and the face features that are used for identity. By explicitly controlling the generative factors of face information, we provide an alternative framework based on human psychophysics to understand information processing in CNNs.

Vrouwen vrijen met de man van wie ze houden, en mannen houden van de vrouw met wie ze vrijen. Mannen hebben altijd zin, en vrouwen (veel te) weinig. Mannen kunnen veel makkelijker klaarkomen dan vrouwen. Vrouwen zijn van nature monogaam en mannen niet. Al deze bij de meeste mensen bekende cliché’s over mannelijke en vrouwelijke seksualiteit hebben gemeen dat mannen sterker seksueel gemotiveerd lijken dan vrouwen. Is dat werkelijk zo? En als dat zo is, komt dat doordat mannen nu eenmaal gevoeliger zijn voor seksuele prikkels dan vrouwen, of laten vrouwen seksuele gevoelens minder toe, laten ze hun gedrag er minder door bepalen, met andere woorden: zetten vrouwen eerder de rem op seksuele gevoelens dan mannen? In dit artikel wordt aan de hand van een aantal experimenten geprobeerd een antwoord te geven op deze vragen.

Smoking is a major heritable and modifiable risk factor for many diseases, including cancer, common respiratory disorders and cardiovascular diseases. Fourteen genetic loci have previously been associated with smoking behaviour-related traits. We tested up to 235,116 single nucleotide variants (SNVs) on the exome-array for association with smoking initiation, cigarettes per day, pack-years, and smoking cessation in a fixed effects meta-analysis of up to 61 studies (up to 346,813 participants). In a subset of 112,811 participants, a further one million SNVs were also genotyped and tested for association with the four smoking behaviour traits. SNV-trait associations with P < 5 × 10-8 in either analysis were taken forward for replication in up to 275,596 independent participants from UK Biobank. Lastly, a meta-analysis of the discovery and replication studies was performed. Sixteen SNVs were associated with at least one of the smoking behaviour traits (P < 5 × 10-8) in the discovery samples. Ten novel SNVs, including rs12616219 near TMEM182, were followed-up and five of them (rs462779 in REV3L, rs12780116 in CNNM2, rs1190736 in GPR101, rs11539157 in PJA1, and rs12616219 near TMEM182) replicated at a Bonferroni significance threshold (P < 4.5 × 10-3) with consistent direction of effect. A further 35 SNVs were associated with smoking behaviour traits in the discovery plus replication meta-analysis (up to 622,409 participants) including a rare SNV, rs150493199, in CCDC141 and two low-frequency SNVs in CEP350 and HDGFRP2. Functional follow-up implied that decreased expression of REV3L may lower the probability of smoking initiation. The novel loci will facilitate understanding the genetic aetiology of smoking behaviour and may lead to the identification of potential drug targets for smoking prevention and/or cessation.

Smoking is a major heritable and modifiable risk factor for many diseases, including cancer, common respiratory disorders and cardiovascular diseases. Fourteen genetic loci have previously been associated with smoking behaviour-related traits. We tested up to 235,116 single nucleotide variants (SNVs) on the exome-array for association with smoking initiation, cigarettes per day, pack-years, and smoking cessation in a fixed effects meta-analysis of up to 61 studies (up to 346,813 participants). In a subset of 112,811 participants, a further one million SNVs were also genotyped and tested for association with the four smoking behaviour traits. SNV-trait associations with P < 5 × 10−8 in either analysis were taken forward for replication in up to 275,596 independent participants from UK Biobank. Lastly, a meta-analysis of the discovery and replication studies was performed. Sixteen SNVs were associated with at least one of the smoking behaviour traits (P < 5 × 10−8) in the discovery samples. Ten novel SNVs, including rs12616219 near TMEM182, were followed-up and five of them (rs462779 in REV3L, rs12780116 in CNNM2, rs1190736 in GPR101, rs11539157 in PJA1, and rs12616219 near TMEM182) replicated at a Bonferroni significance threshold (P < 4.5 × 10−3) with consistent direction of effect. A further 35 SNVs were associated with smoking behaviour traits in the discovery plus replication meta-analysis (up to 622,409 participants) including a rare SNV, rs150493199, in CCDC141 and two low-frequency SNVs in CEP350 and HDGFRP2. Functional follow-up implied that decreased expression of REV3L may lower the probability of smoking initiation. The novel loci will facilitate understanding the genetic aetiology of smoking behaviour and may lead to the identification of potential drug targets for smoking prevention and/or cessation.

Can deep convolutional neural networks (DCNNs) be used as models of visual cognition to capture the interaction between sensation and cognition during object recognition? In this thesis, I evaluated this question for three different modulations of object recognition: arousal state, spatial attention, and previous visual experience, while building on extensive prior research on object recognition in the visual ventral stream and deep neural networks. For each of these factors, the leading goal was to formulate the interaction between object recognition and cognitive modulation in an image-computable, mechanistic model that reproduces key characteristics of human behaviour. In Chapter 2 of this thesis, I used this approach to investigate how a global gain modulation — previously linked to changes in arousal state — may affect visual processing and recognition performance on a series of tasks with varying difficulty. In Chapter 3, I assessed the efficacy of different attention mechanisms at implementing selective processing during object recognition within naturalistic scenes. In Chapter 4 and 5, I delved into the consequences of dynamic sensory experience for object recognition, examining how different neural mechanisms, such as lateral recurrence and sensory adaptation, may support our ability to recognize objects across a wide range of timescales and during various dynamic tasks. Chapter 6 and 7 summarize and discuss the opportunities and challenges afforded by DCNNs as models of visual cognition. Overall, the mechanistic insights presented in this thesis provide encouraging examples of the usage of DCNNs as a linking framework between sensation and cognitive modulation.

In this thesis, I explore a different, complementary approach to the traditional methodology of hypothesis testing used in psychology and cognitive neuroscience research. Although this alternative approach has deep roots in psychology, the version I advocate and have used in this thesis extends it with ideas and techniques from machine learning. Chapter 2 describes a study in which we used predictive models applied to fMRI data, also known as “decoding models”, to investigate the shared neural basis of emotion experience and emotion understanding. To remedy the interpretational difficulties inherent to decoding analyses (and predictive models in general), chapter 3 outlines a method we developed to adjust for confounds in decoding analyses which helps to rule out alternative explanations. Moving away from the focus on predictive models, chapter 4 is the result from our effort to publish the “Amsterdam Open MRI Collection” (AOMIC), a set of three multimodal MRI datasets, and chapter 5 describes a confirmatory, fully pre-registered neuroimaging study on a psychological phenomenon called “morbid curiosity”. Finally, the last two chapters return to the use of predictive models, this time in the context of facial expression perception. Chapter 6 outlines a method we developed to formalize verbal hypotheses as quantitative predictive models, which we apply to a specific set of hypotheses about how facial movements relate to categorical emotions. At last, chapter 7 concludes this thesis with a study that compares predictive models of affective face perception based on facial morphology and facial movements, which shows that people integrate both sources of information in their affective inferences and experiences.

We see the world in scenes, where objects are embedded and often partially occluded in rich and complex surroundings containing other objects. How does the brain extract and transform low-level visual features into richer representations that facilitate recognition, while there are so many factors that affect the appearance of natural object categories? This thesis explores how natural scene properties influence visual processing during object recognition. A first question concerned whether low-level natural scene complexity, as indexed by two biologically plausible image statistics, influences perceptual decision-making. A second question concerned whether the human brain adjusts its processing based on the complexity or the amount of ‘informative’ (congruent) information in a scene. Using human behavior, brain measurements obtained with electroencephalography (EEG), and deep convolutional neural networks, we evaluated how different neural computations or functional architectures (feed-forward vs. recurrent) extract information from objects and their backgrounds. Finally, we examined visual processing in a brain-injured patient who perceives the visual world and sees objects but does not recognize them. Overall, results suggest that how object recognition is resolved depends on the context in which the object appears: for objects presented in simple environments, recognition can likely be solved within the first feed-forward sweep of visual information processing, based on an unbound collection of image features. For more complex scenes or in more challenging situations, additional processing (in the form of recurrent computations) appears to be necessary to group the elements that belong to the object and segregate them from the background.

Dyslexia is generally considered to be a disorder of accurate and/or fluent word recognition and spelling and decoding abilities. However, theories about what causes dyslexia differ to a large extent which prevents international agreement about how dyslexia should be diagnosed. In this thesis, various aspects of diagnostic methods were investigated, and new methods were proposed. It was found that existing methods can be improved by using item scores instead of sum scores, by applying multiple classifications, and by carefully evaluating criteria of dyslexia. Furthermore, it was found that self-report statements provide more reliable diagnoses than test results, mainly because self-report statements do not depend on general intelligence and level of schooling. Additionally, a classification accuracy of 80% was found using anatomical brain imaging techniques. Some findings of this thesis are relevant for the interpretation of theoretical perspectives about dyslexia. First, a severity score of dyslexia showed two separate normal distributions for people with and without dyslexia. Second, it was found that dyslexia is characterised by at least six cognitive variables. Third, some of these variables showed significant correlations with various areas in the brain. Fourth, support was found for the idea that anatomical brain differences are mainly the result of individual differences in training. Based on the findings in this thesis, it was proposed that dyslexia may not be a disorder, but a perceptual variation, originating in the subcortex and with widespread effects on various areas in the cortex. Especially processes of inhibition may be impaired in people with dyslexia.

Both computer vision and human visual system target the same goal: to accomplish visual tasks easily via a set of representations. In this thesis, we study to what extent representations from computer vision models align to human visual representations. To study this research question we used an interdisciplinary approach, integrating methods from psychology, neuroscience and computer vision. Such an approach is aimed to provide new insight in the understanding of human visual representations. In the four chapters of the thesis, we tested computer vision models against brain data obtained with electro-encephalography (EEG) and functional magnetic resonance imaging (fMRI). The main findings can be summarized as follows; 1) computer vision models with one or two computational stages correlate to visual representations of intermediate complexity in the human brain, 2) models with multiple computational stages correlate best to the hierarchy of representations in the human visual system, 3) computer vision models do not align one-to-one to the temporal hierarchy of representations in the visual cortex and 4) not only visual but also semantic representations correlate to representations in the human visual system.

While much of what we learn will be forgotten over time, fear memory appears to be particularly resilient to forgetting. Our understanding of how fearful events are transformed into durable memory, and how this memory subsequently influences the processing of (novel) stimuli, is limited. Studying fear memory is complicated by the fact that we cannot directly observe memory, but have to infer it from the different ways it is expressed. Yet, only a small portion of what is concurrently happening in our brain is translated into observable actions and peripheral physiology, or is verbally accessible. Hence, there is a notable discrepancy between the expression of fear during learning and the formation and expression of a fear memory. In this thesis, we used a variety of methods to identify indices that capture the complexity of human fear memory, combining several experimental procedures (Pavlovian fear conditioning, a pharmacological manipulation, and a perceptual judgment task) with behavioral and neural indices of fear (BOLD-MRI, pupil dilation, fear potentiated startle, and subjective reports). To specifically gauge the dynamic nature of fear (un)learning and memory, we developed a single-trial application of multi-voxel pattern analysis (MVPA), a technique that evaluates distributed patterns of BOLD-MRI. We disentangled learning and memory processes by testing on different days and varied contextual cues to (de)activate specific memories. Our findings provide new insights into the different levels at which fear associations emerge, reside and become activated, offering a framework to examine the conditions under which fear learning results in long-lasting fear memory.

How and where does our brain integrated the information that we get into our eyes into a unifying percept and into a conscious experience? Although different neural correlates of consciousness (NCC) have been proposed, depending on the kind of neural signals recorded, the type of manipulation used, and the interpretation of behavioral results, they all seem to agree that consciousness requires integration of information by means of recurrent processing (RP). But the role of neurotransmitters in the NCC has not received much attention, while neurotransmitters are essential in neural communication. Moreover, many studies investigating the NCC manipulated either the content of consciousness or the state of consciousness. Therefore, in this thesis, I have investigated how these are related. I manipulated both neurotransmitter systems (via e.g. pharmacological interventions) - and hence the state of consciousness - as well as the content of consciousness (via e.g. masking, bistable illusions etc.) while measuring neural activity with EEG and fMRI to assess RP. Our results show that the NMDA receptor is important for integrating information of different brain areas. But, inhibition (via the neurostransmitter GABA) guides this process and is required for the selection process to determine the outcome of the competition for conscious access. Combining manipulations of conscious state and content thus give more direct insights into the neural and pharmacological underpinnings of consciousness. This may open the way for a molecular biological understanding of consciousness and towards a neurochemical correlate of consciousness.

The starting point of this thesis is a neural definition of consciousness, independent of attention or other cognitive functions that are required for a behavioral report. It is defined as recurrent processing (RP) - which is needed for integration of information - as opposed to unconscious feedforward processing. As RP is mediated by NMDA receptors, it is hypothesized that consciousness (according to the neural definition) plays a role in learning. Using EEG, I showed that stimuli that evoked RP yet were unattended (and therefore unreportable) are learned at the perceptual level, whereas attended yet masked stimuli (for which RP was blocked) do not show any learning effects. Interestingly, the behavioral learning effect only became apparent when performance feedback was given. This suggests that the memory trace formed during inattention was latent until reactivated by behavioral feedback. A follow-up fMRI experiment revealed that performance feedback boosts neural learning effects as well. In addition, when NMDA receptors are manipulated with ketamine (thereby reducing RP), visual feature integration is diminished and perceptual learning precluded. Together, these results provide a first step in understanding the benefit of defining the distinction between unconscious and conscious processing as the divide between feedforward and recurrent processing respectively. It allows us to understand consciousness on a more fundamental level, by understanding its function: consciousness is needed for learning. It would be interesting to investigate how far the current findings extend beyond the perceptual learning domain, and to further elucidate the role of feedback in reconsolidating latent memory traces.

How does the brain represent our visual environment? Research has revealed brain areas that respond to specific information such as faces and objects, but how a representation of an entire visual scene is formed is still unclear. This thesis explores the idea that scene statistics play an important role in the formation of neural representations of real-world scenes. Scene statistics are statistical regularities resulting from the physical laws that govern our environment. The main hypothesis is that the brain uses these statistical regularities to rapidly categorize scenes. To test this hypothesis, we used an interdisciplinary approach, integrating methods from psychology, neuroscience and informatics (computer vision). In five chapters, we describe a total of 10 experiments in which we tested computational models of scene statistics against human behavioral and brain data obtained with electro-encephalography (EEG) and functional magnetic resonance imaging (fMRI). Our results show that the visual system exhibits strong sensitivity to scene statistics. Focusing in particular on the temporal dynamics of this neural sensitivity, we examined in depth how scene statistics shape neural representations. We demonstrate a potential role for scene statistics in perceived similarity of naturalistic textures, scene gist perception and object recognition. Overall, the results suggest that visual cortex might compute scene statistics in order to estimate the degree of coherence, i.e. the amount of organized structure vs. chaos present in the visual input, and use this information for rapid scene categorization, or to dynamically adjust its processing of other visual information, such as objects.

Onze hersenen kunnen razendsnel binnenkomende visuele informatie tot een behoorlijk complex niveau verwerken. Deze eerste informatiestroom kan zelfs zeer complex cognitief gedrag beïnvloeden en de hersenen tot in de prefrontale cortex activeren. Echter, deze eerste snelle informatiestroom blijft onbewust. Voor het verkrijgen van een complete (bewuste) visuele ervaring zijn feedbackprocessen in de hersenen nodig. Deze feedbackprocessen zorgen er waarschijnlijk voor dat informatie geïntegreerd wordt in onze hersenen. De integratie van informatie is op zijn beurt cruciaal voor de constructie van visuele perceptie en mogelijk voor het ontstaan van (visueel) bewustzijn. Martijn Wokke onderzocht op welke manier onze hersenen (bewuste) visuele perceptie produceren. De manier waarop onbewuste informatie ons gedrag beïnvloedt, kan implicaties hebben voor hoe bijvoorbeeld informatie aangeboden kan worden in het verkeer of met reclame. Om uit te vinden hoe ons visueel systeem werkt en hoe dit uiteindelijk tot een complete visuele ervaring leidt, kan ook gevolgen hebben voor bijvoorbeeld beeldherkenningssoftware of leiden tot een beter begrip van visuele stoornissen.

Waarschijnlijk heb je het zelf wel eens meegemaakt dat je gestoord werd tijdens je favoriete tv-programma met een vraag van een ouder, kind of echtgenoot. Gelukkig ging je zo op in het programma dat je de vraag niet eens hebt ‘gehoord’. Als diezelfde persoon vervolgens aan je vraagt: ‘Hoorde je me niet, of zo!?’, dan weet je toch de vraag te herhalen en antwoord te geven. Dit voorbeeld toont aan dat informatie die geen aandacht heeft gekregen nog enige tijd rondsluimert in ons sensorisch geheugen. Informatie waar we aandacht voor hebben (gehad), komt in ons werkgeheugen terecht. Ilja Sligte vindt dat deze discrete, twee-stadiumbeschrijving van korte-termijngeheugen een beetje simplistisch is. Korte-termijngeheugen lijkt volgens hem eerder een continu proces te zijn. De capaciteit en eigenschappen van korte-termijngeheugen hangen samen met de diepte waarmee sensorische input doordringt in het brein. In de primaire sensorische gebieden houden loops van hersenactiviteit heel kort een heleboel simpele informatie vast. Iets hoger in het brein, houden deze loops langere tijd geheugensporen in stand die al wat complexere eigenschappen hebben. Tot slot selecteren aandachtsgebieden maximaal vier stukjes informatie om in abstracte vorm te representeren en zolang aandacht beschikbaar blijft, leidt dit tot loops die een groot deel van het brein omvatten.