Search

People often have good intentions but fail to adhere to them. Implementation intentions, a form of strategic planning, can help people to close this intention–behavior gap. Their effectiveness has been proposed to depend on the mental formation of a stimulus–response association between a trigger and target behavior, thereby creating an “instant habit.” If implementation intentions do indeed lead to reliance on habitual control, then this may come at the cost of reduced behavioral flexibility. Furthermore, we would expect a shift from recruitment of corticostriatal brain regions implicated in goal-directed control toward habit regions. To test these ideas, we performed a fMRI study in which participants received instrumental training supported by either implementation or goal intentions, followed by an outcome revaluation to test reliance on habitual versus goal-directed control. We found that implementation intentions led to increased efficiency early in training, as reflected by higher accuracy, faster RTs, and decreased anterior caudate engagement. However, implementation intentions did not reduce behavioral flexibility when goals changed during the test phase, nor did it affect the underlying corticostriatal pathways. In addition, this study showed that “slips of action” toward devalued outcomes are associated with reduced activity in brain regions implicated in goal-directed control (ventromedial prefrontal cortex and lateral orbitofrontal cortex) and increased activity of the fronto-parietal salience network (including the insula, dorsal anterior cingulate cortex, and SMA). In conclusion, our behavioral and neuroimaging findings suggest that strategic if–then planning does not lead to a shift from goal-directed toward habitual control.

The value and uncertainty associated with choice alternatives constitute critical features along which decisions are made. While the neural substrates supporting reward and risk processing have been investigated, the temporal organization by which these computations are encoded remains elusive. Here we leverage the high spatiotemporal precision of intracranial electroencephalography (iEEG) to uncover how representations of decision-related computations unfold in time. We present evidence of locally distributed representations of reward and risk variables that are temporally organized across multiple regions of interest. Reward outcome representations across wide-spread regions follow a temporally cascading order along the anteroposterior axis of the brain. In contrast, expected value can be decoded from multiple regions at the same time, and error signals in both reward and risk domains reflect a mixture of sequential and parallel encoding. We highlight the role of the anterior insula in generalizing between reward prediction error (RePE) and risk prediction error (RiPE), within which the encoding of RePE in the distributed iEEG signal predicts RiPE. Together our results emphasize the utility of uncovering temporal dynamics in the human brain for understanding how computational processes critical for value-based decisions under uncertainty unfold.

Adaptive behavior in real-world environments demands that choices integrate over several variables, including the novelty of the options under consideration, their expected value, and uncertainty in value estimation. We recorded neurons from the human pre-supplementary motor area (preSMA), ventromedial prefrontal cortex (vmPFC) and dorsal anterior cingulate to probe how integration over decision variables occurs during decision-making. In contrast to the other areas, preSMA neurons not only represented separate pre-decision variables for each choice option, but also encoded an integrated utility signal and, subsequently, the decision itself. Conversely, post-decision related encoding of variables for the chosen option was more widely distributed and especially prominent in vmPFC. Our findings position the human preSMA as central to the implementation of value-based decisions.

Pavlovian conditioning is thought to involve the formation of learned associations between stimuli and values, and between stimuli and specific features of outcomes. Here we leveraged human single neuron recordings in ventromedial prefrontal, dorsomedial frontal, hippocampus and amygdala neurons while patients performed a sequential Pavlovian conditioning task containing both stimulus-value and stimulus-stimulus associations. Neurons in the ventromedial prefrontal cortex encoded predictive value along with the amygdala, but also encoded predictions about the identity of stimuli that would subsequently be presented, suggesting a role for neurons in this region in encoding predictive information beyond value. Unsigned error signals were found in dorsomedial prefrontal areas and hippocampus, potentially supporting learning of non-value related outcome features. Our findings implicate distinct human prefrontal and medial temporal neuronal populations in mediating predictive associations which could partially support model-based mechanisms during Pavlovian conditioning.Significance statementPavlovian conditioning is a fundamental form of learning, allowing organisms to associate stimuli and outcomes. Recent Pavlovian work suggests that phenomena such as devaluation sensitivity and sensory preconditioning can be explained by a model-based learning framework. How human neurons perform model-based learning during Pavlovian conditioning is still an open question. We recorded single neurons from epilepsy patients during a two-step Pavlovian conditioning task and found that ventromedial prefrontal neurons encoded expected rewards along with amygdala neurons, but also predicted the identity of upcoming stimuli as required for model-based cognition. Additionally, medial frontal neurons were found to encode error signals that could be used for stimulus-outcome learning. This is the first study mapping model-based computations during Pavlovian conditioning in human neurons.

The dual-process theory of action control postulates that there are two competitive and complementary mechanisms that control our behavior: a goal-directed system that executes deliberate actions, explicitly aimed toward a particular outcome, and a habitual system that autonomously execute well-learned actions, typically following an encounter with a previously associated cue. In line with dual-process theory, animal studies have provided convincing evidence for dissociable neural mechanisms, mainly manifested in cortico-striatal regions, involved in goal-directed and habitual action control. While substantial progress has been made in characterizing the neural mechanism underlying habit learning in animals, we still lack knowledge on how habits are formed and maintained in the human brain. Thus far only one study, conducted more than a decade ago by Tricomi et al. (2009), has been able to induce habitual behavior in humans via extensive training. This study also implicated the posterior putamen in the process, using functional magnetic resonance imaging (fMRI). However, recent attempts to replicate the behavioral results of this study were not successful. This leaves the research of human habits, and particularly the research of their formation through extensive repetition, as well as their neural basis, limited and far behind the animal research in the field. This motivated us to (1) attempt to replicate the behavioral and imaging main findings of Tricomi et al., (2) identify further functional and microstructural neural modifications associated with habit formation and manifestation, and (3) investigate the relationships between functional and structural plasticity and individual differences in habit expression. To this end, in this registered report we used Tricomi et al.’s free-operant task along with multi-modal MRI methods in a well-powered sample (n=123). In this task participants’ sensitivity to outcome devaluation (an index of goal-directed/habitual action control) is tested following either short or extensive training. In contrast to our hypothesis, we were not able to demonstrate habit formation as a function of training duration nor were we able to relate any functional or microstructural plasticity in the putamen with individual habit expression. We found that a pattern of increased activations in the left head of caudate that re-occurred across each day’s training is associated with goal directed behavior and that increased processing of devalued cues in low-level visual regions was indicative of goal-directed behavior (and vice versa). In a follow-up exploratory analysis comparing habitual and goal-directed subgroups within each experimental group, we found that elevated activations in frontoparietal regions during early stages of training, as well as increased reactivity towards still-valued cues in somatosensory and superior parietal regions were found in individuals that were more inclined to perform goal-directed behavior (compared with more habitual individuals). Taken together, regions commonly implicated in goal-directed behavior were most predictive of individual habit expression. Finally, we also found that differential patterns of training-related microstructural plasticity, as measured with diffusion MRI, in midbrain dopaminergic regions were associated with habit expression. This work provides new insights into the neural dynamics involved in individual habit formation/expression and encourages the development and testing of new, more sensitive, procedures for experimental habit induction in humans.

We review the current state of knowledge on the computational and neural mechanisms of reinforcement-learning with a particular focus on fronto-striatal circuits. We divide the literature in this area into five broad research themes: the target of the learning-whether it be learning about the value of stimuli or about the value of actions; the nature and complexity of the algorithm used to drive the learning and inference process; how learned values get converted into choices and associated actions; the nature of state representations, and of other cognitive machinery that support the implementation of various reinforcement-learning operations. An emerging fifth area focuses on how the brain allocates or arbitrates control over different reinforcement-learning sub-systems or "experts". We will outline what is known about the role of the prefrontal cortex and striatum in implementing each of these functions. We then conclude by arguing that it will be necessary to build bridges from algorithmic level descriptions of computational reinforcement-learning to implementational level models to better understand how reinforcement-learning emerges from multiple distributed neural networks in the brain.

It is an open question whether preferences for visual art can be lawfully predicted from the basic constituent elements of a visual image. Here, we developed and tested a computational framework to investigate how aesthetic values are formed. We show that it is possible to explain human preferences for a visual art piece based on a mixture of low- and high-level features of the image. Subjective value ratings could be predicted not only within but also across individuals, with a regression model with the same set of interpretable features. We also show that the features which predict aesthetic preference can emerge hierarchically within a deep convolutional neural network trained only for object recognition. Our findings suggest that human preferences for art can be explained at least in part as a systematic integration over the underlying visual features of an image.

The model-free algorithms of “reinforcement learning” (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This “generalized reinforcement learning” (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.

Here we argue that the assignment of subjective value to potential outcomes at the time of decision-making is an active process, in which individual features of a potential outcome of varying degrees of abstraction are represented hierarchically and integrated in a weighted fashion to produce an overall value judgment. We implicate the lateral orbital and medial prefrontal cortex in this function, situating these areas more broadly within a hierarchical integration process that takes place throughout the cortex for the ultimate purpose of valuing options to guide decisions.

Among the most challenging questions in the field of neuroaesthetics concerns how a piece of art comes to be liked in the first place. That is, how can the brain rapidly process a stimulus to form an aesthetic judgment even for stimuli never before encountered? In the article under discussion in this chapter, by leveraging computational methods in combination with behavioral and neuroimaging experiments the authors show that the brain does this by breaking a visual stimulus down into underlying features or attributes. These features are shared across objects, and weights over these features are integrated over to produce aesthetic judgments. This process is structured hierarchically in which elementary statistical properties of an image are combined to generate higher level features which in turn yield aesthetic value. Neuroimaging supports the implementation of this hierarchical integration along a gradient from early to higher order visual cortex extending into association cortex and ultimately converging in the anterior medial prefrontal cortex.

Faces are a highly privileged class of stimuli in humans, and facial attractiveness is a particularly salient attribute of faces that can exert considerable influence on the behavior of others. In the 2003 paper discussed in this chapter, the authors aimed to investigate the neural correlates of facial attractiveness in the brain, hypothesizing that attractive faces would recruit basic reward circuits, especially the orbitofrontal cortex. Consistent with their hypothesis, they found robust engagement of the orbitofrontal cortex to attractive faces and, moreover, that this response was enhanced if a face exhibited a smiling expression. Taken together, these results suggest that facial attractiveness and cues signaling positive social feedback can robustly recruit the brain’s reward circuitry, positioning attractive faces alongside other basic rewards while also aligning attractive faces with other aesthetically pleasing stimuli that engage similar circuits such as works of art.

Obsessions and compulsions are central components of obsessive-compulsive disorder (OCD) and obsessive-compulsive related disorders such as body dysmorphic disorder (BDD). Compulsive behaviours may result from an imbalance of habitual and goal-directed decision-making strategies. The relationship between these symptoms and the neural circuitry underlying habitual and goal-directed decision-making, and the arbitration between these strategies, remains unknown. This study examined resting state effective connectivity between nodes of these systems in two cohorts with obsessions and compulsions, each compared with their own corresponding healthy controls: OCD (nOCD=43; nhealthy=24) and BDD (nBDD=21; nhealthy=16). In individuals with OCD, the left ventrolateral prefrontal cortex, a node of the arbitration system, exhibited more inhibitory causal influence over the left posterolateral putamen, a node of the habitual system, compared with controls. Inhibitory causal influence in this connection showed a trend for a similar pattern in individuals with BDD compared with controls. Those with stronger negative connectivity had lower obsession and compulsion severity in both those with OCD and those with BDD. These relationships were not evident within the habitual or goal-directed circuits, nor were they associated with depressive or anxious symptomatology. These results suggest that abnormalities in the arbitration system may represent a shared neural phenotype across these two related disorders that is specific to obsessive-compulsive symptoms. In addition to nosological implications, these results identify potential targets for novel, circuit-specific treatments.

Both novelty and uncertainty are potent features guiding exploration; however, they are often experimentally conflated, and an understanding of how they interact to regulate the balance between exploration and exploitation has proved elusive. Using a task designed to decouple the influence of novelty and uncertainty, we identify separable mechanisms through which exploration is directed. We show that uncertainty-directed exploration is sensitive to the prospective benefit offered by new information, whereas novelty-directed exploration is maintained regardless of its potential advantage. Using a computational framework in conjunction with fMRI, we show that uncertainty-directed choice is rooted in an adaptive bias indexing the prospective utility of exploration. In contrast, novelty persistently promotes exploration by optimistically inflating reward expectations while simultaneously dampening uncertainty signals. Our results identify separable neural substrates charged with balancing the explore/exploit trade-off to foster a manageable decomposition of an otherwise intractable problem.

Diminished motivation to pursue and obtain primary and secondary rewards has been demonstrated in anorexia nervosa (AN). However, the neurobehavioral mechanisms underlying the behavioral activation component of aberrant reward motivation remains incompletely understood. This work aims to explore this underexplored facet of reward motivation in AN. We recruited female adolescents with AN, restricting type (n = 32) and a healthy control group (n = 28). All participants underwent functional magnetic resonance imaging (fMRI) while performing a monetary reward task. Diffusion MRI data was also collected to examine the reward motivation circuit's structural connectivity. Behavioral results demonstrated slower speed of reward-seeking behavior in those with AN compared with controls. Accompanying this was lower functional connectivity and reduced white matter structural integrity of the connection between the ventral tegmental area/substantia nigra pars compacta and the nucleus accumbens within the mesolimbic circuit. Further, there was evidence of neurobehavioral decoupling in AN between reward-seeking behavior and mesolimbic regional activation and functional connectivity. Aberrant activity of the bed nucleus of the stria terminalis (BNST) and its connectivity with the mesolimbic system was also evident in AN during the reward motivation period. Our findings suggest functional and structural dysconnectivity within a mesolimbic reward circuit, neurofunctional decoupling from reward-seeking behavior, and abnormal BNST function and circuit interaction with the mesolimbic system. These results show behavioral indicators of aberrant reward motivation in AN, particularly in its activational component. This is mediated neuronally by mesolimbic reward circuit functional and structural dysconnectivity as well as neurobehavioral decoupling. Based on these findings, we suggest a novel circuit-based mechanism of impaired reward processing in AN, with the potential for translation to developing more targeted and effective treatments in this difficult-to-treat psychiatric condition.

Recent evidence suggests that both novelty and uncertainty act as potent features guiding exploration. However, these variables are often conflated with each other experimentally, and an understanding of how these attributes interact to regulate the balance between exploration and exploitation has proved elusive. Using a novel task designed to decouple stimulus novelty and estimation uncertainty, we identify separable behavioral and neural mechanisms by which exploration is colored. We show that uncertainty was avoided except when the information gained through exploration could be reliably exploited in the future. In contrast, and contrary to existing theory, novel options grew increasingly attractive relative to familiar counterparts irrespective of the opportunity to leverage their consequences and despite the uncertainty inherent to novel options. These findings led us to develop a formal computational framework in which uncertainty directed choice adapts to the prospective utility of exploration, while novel stimuli persistently draw favor as a result of inflated reward expectations biasing an exploitative strategy. Crucially, novelty is proposed to actively modulate uncertainty processing, effectively blunting the influence of uncertainty in shaping the subjective utility ascribed to novel stimuli. Both behavioral data and fMRI activity sampled from the ventromedial prefrontal cortex, frontopolar cortex and ventral striatum validate this model, thereby establishing a computational account that can not only explain behavior but also shed light on the functional contribution of these key brain regions to the exploration/exploitation trade-off. Our results point to multiple strategies and neural substrates charged with balancing the explore/exploit dilemma, with each targeting distinct aspects of the decision problem to foster a manageable decomposition of an otherwise intractable task.

Contains fulltext : 233041.pdf (Publisher’s version ) (Open Access) Over the past three decades, functional MRI (fMRI) has become key to study how cognitive processes are implemented in the human brain. However, the question of whether participants recruited into fMRI studies differ from participants recruited into other study contexts has received little to no attention. This is particularly pertinent when effects fail to generalize across study contexts: for example, a behavioural effect discovered in a non-imaging context not replicating in a neuroimaging environment. Here, we tested the hypothesis, motivated by preliminary findings (n = 272), that fMRI participants differ from behaviour-only participants on one fundamental individual difference variable: trait anxiety. Analysing trait anxiety scores and possible confounding variables from healthy volunteers across multiple institutions (n = 3317), we found robust support for lower trait anxiety in fMRI study participants, consistent with a sampling or self-selection bias. The bias was larger in studies that relied on phone screening (compared to full in-person psychiatric screening), recruited at least partly from convenience samples (compared to community samples), and in pharmacology studies. Our findings highlight the need for surveying trait anxiety at recruitment and for appropriate screening procedures or sampling strategies to mitigate this bias. 14 p.

We review progress and highlight open questions in neuroaesthetics. We argue that computational methods can provide mechanistic insight into how aesthetic judgments are formed, while advocating for deeper collaboration between neuroscientists studying aesthetics and those in the arts and humanities.

Midbrain dopamine neurons are known to encode reward prediction errors (RPE) used to update value predictions. Here, we examine whether RPE signals coded by midbrain dopamine neurons are modulated by the cost paid to obtain rewards, by recording from dopamine neurons in awake behaving monkeys during performance of an effortful saccade task. Dopamine neuron responses to cues predicting reward and to the delivery of rewards were increased after the performance of a costly action compared to a less costly action, suggesting that RPEs are enhanced following the performance of a costly action. At the behavioral level, stimulus-reward associations are learned faster after performing a costly action compared to a less costly action. Thus, information about action cost is processed in the dopamine reward system in a manner that amplifies the following dopamine RPE signal, which in turn promotes more rapid learning under situations of high cost., Rewards that require high effort tend to be preferred over those that require low effort. Here, the authors show how the effort of obtaining rewards affects reward-related activity of dopamine neurons, and in turn the speed of learning stimulus-reward associations.

It has been suggested that there are two distinct and parallel mechanisms for controlling instrumental behavior in mammals: goal-directed actions and habits. To gain an understanding of how these two systems interact to control behavior, it is essential to characterize the mechanisms by which the balance between these systems is influenced by experience. Studies in rodents have shown that the amount of training governs the relative expression of these two systems: behavior is goal-directed following moderate training, but the more extensively an instrumental action is trained, the more it becomes habitual. It is less clear whether humans exhibit similar training effects on the expression of goal-directed and habitual behavior, as human studies have reported contradictory findings. To tackle these contradictory findings, we formed a consortium, where four laboratories undertook a pre-registered experimental induction of habits by manipulating the amount of training. There was no statistical evidence for a main effect of the amount of training on the formation and expression of habits. However, exploratory analyses suggest a moderating effect of the affective component of stress on the impact of training over habit expression. Participants who were lower in affective stress appeared to be initially goal-directed, but became habitual with increased training, whereas participants who were high in affective stress were already habitual even after moderate training, thereby manifesting insensitivity to overtraining effects. Our findings highlight the importance of the role of moderating variables such as individual differences in stress and anxiety when studying the experimental induction of habits in humans.

It has been suggested that there are two distinct and parallel mechanisms for controlling instrumental behavior in mammals: goal-directed actions and habits. To gain an understanding of how these two systems interact to control behavior, it is essential to characterize the mechanisms by which the balance between these systems is influenced by experience. Studies in rodents have shown that the amount of training governs the relative expression of these two systems: Behavior is goal-directed following moderate training, but the more extensively an instrumental action is trained, the more it becomes habitual. It is less clear whether humans exhibit similar training effects on the expression of goal-directed and habitual behavior, as human studies have reported contradictory findings. To tackle these contradictory findings, we formed a consortium, where four laboratories undertook a preregistered experimental induction of habits by manipulating the amount of training. There was no statistical evidence for a main effect of the amount of training on the formation and expression of habits. However, exploratory analyses suggest a moderating effect of the affective component of stress on the impact of training over habit expression. Participants who were lower in affective stress appeared to be initially goal-directed, but became habitual with increased training, whereas participants who were high in affective stress were already habitual even after moderate training, thereby manifesting insensitivity to overtraining effects. Our findings highlight the importance of the role of moderating variables such as individual differences in stress and anxiety when studying the experimental induction of habits in humans.

Humans, like many other animals, pre-empt danger by moving to locations that maximize their success at escaping future threats. Here, we test the idea that spatial margin of safety (MOS) decisions, a form of prospective avoidance, result in participants placing themselves closer to safer locations when facing more unpredictable threats. Using multivariate pattern analysis on fMRI data collected while subjects engaged in MOS decisions with varying attack location predictability, we show that while the hippocampus encodes MOS decisions across all types of threat, a vmPFC anterior-posterior gradient tracked threat predictability. The posterior vmPFC encoded for more unpredictable threat and showed functional coupling with the amygdala and hippocampus. Conversely, the anterior vmPFC was more active for the more predictable attacks and showed coupling with the striatum. Our findings suggest that when pre-empting danger, the anterior vmPFC may provide a safety signal, possibly via predictable positive outcomes, while the posterior vmPFC drives prospective danger signals.

In order to make decisions, we often seek and integrate information coming from other people, while at times also keeping track of the knowledge other people acquire from observing our own actions. In this chapter, we examine the computational mechanisms and the involvement of mentalizing when we learn from observing other people and when we engage in strategic social interactions in which two agents recursively represent each other’s beliefs and intentions. We shed light on evidence that regions of the brain’s mentalizing system play a key role in implementing these social learning computations, by representing (1) the mental states of other agents, (2) how these mental states are dynamically updated over time and (3) how other agents represent our own beliefs and intentions. We argue in favor of using a neuro-computational approach to study these processes, combining computational modelling to identify specific variables predicting behavior and model-based neuroimaging to understand how and where these variables are represented in the brain. We conclude by highlighting some open questions that remain to be addressed to provide a fully integrated account of mentalizing computations during social learning.

The midbrain dopamine (DA) neuron plays a key role in reward processing and codes signals associated with the reward prediction error (RPE) to update the value of options. Here, we examined whether these RPE signals are modulated by the cost paid to obtain the reward. After focussing a fixation point, two macaque monkeys were required to make a saccade to a condition cue, then a target appeared. In the high cost condition, long fixation to the target was required. In the low-cost condition, only a short fixation was required. After fixation on the target, the subjects made a saccade to the reward cue. Choice trials between condition cues and between reward cues were inserted randomly to test if the subjects showed a preference. Free reward and free air-puff trials were inserted randomly to determine whether each DA neuron was of a salience or motivation subtype. A cue signaling a costly action to be performed triggered less response in DA neurons with respect to a cue signaling a less costly action, but DA neuron responses to cues predicting reward and to the delivery of rewards were found to be enhanced after the monkey had performed a costly action compared to a less costly action. These findings suggest that DA neurons incorporate the cost of performing an action into the prediction error signal, and that RPEs are enhanced following the performance of a costly action. This finding suggested that monkeys would be faster to learn stimulus-reward associations after performing a costly action compared to a less costly action. A subsequent behavioral experiment confirmed this hypothesis. Information about action cost is processed in the DA reward system in a manner that amplifies the DA RPE signal, thereby producing more rapid learning under situations of high cost.

Humans possess an exceptional aptitude to efficiently make decisions from high-dimensional sensory observations. However, it is unknown how the brain compactly represents the current state of the environment to guide this process. The deep Q-network (DQN) achieves this by capturing highly nonlinear mappings from multivariate inputs to the values of potential actions. We deployed DQN as a model of brain activity and behavior in participants playing three Atari video games during fMRI. Hidden layers of DQN exhibited a striking resemblance to voxel activity in a distributed sensorimotor network, extending throughout the dorsal visual pathway into posterior parietal cortex. Neural state-space representations emerged from nonlinear transformations of the pixel space bridging perception to action and reward. These transformations reshape axes to reflect relevant high-level features and strip away information about task-irrelevant sensory features. Our findings shed light on the neural encoding of task representations for decision-making in real-world situations.

It has long been suggested that human behavior reflects the contributions of multiple systems that cooperate or compete for behavioral control. Here we propose that the brain acts as a “Mixture of Experts” in which different expert systems propose strategies for action. It will be argued that the brain determines which experts should control behavior at any one moment in time by keeping track of the reliability of the predictions within each system, and by allocating control over behavior in a manner that depends on the relative reliabilities across experts. fMRI and neurostimulation studies suggest a specific contribution of the anterior prefrontal cortex in this process. Further, such a mechanism also takes into consideration the complexity of the expert, favoring simpler over more cognitively complex experts. Results from the study of different expert systems in both experiential and social-learning domains hint at the possibility that this reliability-based control mechanism is domain general, exerting control over many different expert systems simultaneously in order to produce sophisticated behavior.

Hippocampal-midbrain circuit boosts the utility of anticipation in the prefrontal cortex and drives information-seeking behavior., Having something to look forward to is a keystone of well-being. Anticipation of future reward, such as an upcoming vacation, can often be more gratifying than the experience itself. Theories suggest the utility of anticipation underpins various behaviors, ranging from beneficial information-seeking to harmful addiction. However, how neural systems compute anticipatory utility remains unclear. We analyzed the brain activity of human participants as they performed a task involving choosing whether to receive information predictive of future pleasant outcomes. Using a computational model, we show three brain regions orchestrate anticipatory utility. Specifically, ventromedial prefrontal cortex tracks the value of anticipatory utility, dopaminergic midbrain correlates with information that enhances anticipation, while sustained hippocampal activity mediates a functional coupling between these regions. Our findings suggest a previously unidentified neural underpinning for anticipation’s influence over decision-making and unify a range of phenomena associated with risk and time-delay preference.

Most of our waking time as human beings is spent interacting with other individuals. In order to make good decisions in this social milieu, it is often necessary to make inferences about the internal states, traits and intentions of others. Recently, some progress has been made to uncover the neural computations underlying human social decision-making by combining functional magnetic resonance neuroimaging (fMRI) with computational modeling of behavior. Modeling of behavioral data allows us to identify key computations necessary for decision-making and how these computations are integrated. Furthermore, by correlating these computational variables against neuroimaging data, it has become possible to elucidate where in the brain various computational variables are implemented during social decision making. Here we review the current state of knowledge in the domain of social computational neuroscience. Findings to date have emphasized that social decisions are driven by multiple computations that are conducted in parallel and which are implemented in distinct brain regions. We suggest that further progress is going to depend on identifying how and where such variables get integrated in order to yield a coherent behavioral output.

It is an open question whether preferences for visual art can be lawfully predicted from the basic constituent elements of a visual image. Moreover, little is known about how such preferences are actually constructed in the brain. Here we developed and tested a computational framework to gain an understanding of how the human brain constructs aesthetic value. We show that it is possible to explain human preferences for a piece of art based on an analysis of features present in the image. This was achieved by analyzing the visual properties of drawings and photographs by multiple means, ranging from image statistics extracted by computer vision tools, subjective human ratings about attributes, to a deep convolutional neural network. Crucially, it is possible to predict subjective value ratings not only within but also across individuals, speaking to the possibility that much of the variance in human visual preference is shared across individuals. Neuroimaging data revealed that preference computations occur in the brain by means of a graded hierarchical representation of lower and higher level features in the visual system. These features are in turn integrated to compute an overall subjective preference in the parietal and prefrontal cortex. Our findings suggest that rather than being idiosyncratic, human preferences for art can be explained at least in part as a product of a systematic neural integration over underlying visual features of an image. This work not only advances our understanding of the brain-wide computations underlying value construction but also brings new mechanistic insights to the study of visual aesthetics and art appreciation.

It has been observed that the performing for high stakes can, paradoxically, lead to uncharacteristically poor performance. Here we investigate a novel approach to attenuating such ‘choking under pressure’ by instructing participants performing a demanding motor task that rewards successful performance with a monetary gain, to reappraise this incentive as a monetary loss for unsuccessful performance. We show that when participants applied this simple strategy, choking was significantly reduced. This strategy also influenced participants’ neural and physiological activity. When participants reappraised the incentive as a potential monetary loss, the representation of the magnitude of the incentive in the ventral striatum Blood Oxygenation Level Dependent (BOLD) signal was attenuated. In addition, individual differences in the degree of attenuation of the neural response to incentive predicted the effectiveness of the reappraisal strategy in reducing choking. Furthermore, participants’ skin conductance changed in proportion to the magnitude of the incentive being played for, and was exaggerated on high incentive trials on which participants failed. Reappraisal of the incentive abolished this exaggerated skin conductance response. This represents the first experimental association of sympathetic arousal with choking. Taken together, these results suggest that reappraisal of the incentive is indeed a promising intervention for attenuating choking under pressure.

The amygdala plays an important role in many aspects of social-cognition and reward-learning. Here we aimed to determine whether human amygdala neurons are involved in the computations necessary to implement learning through observation. We performed single-neuron recordings from the amygdalae of human neurosurgical patients (male and female) while they learned about the value of stimuli through observing the outcomes experienced by another agent interacting with those stimuli. We used a detailed computational modeling approach to describe patients’ behavior in the task. Then, using both population-level decoding and single neuron analyses we found evidence to implicate amygdala neurons in two key computations relevant for observational-learning: tracking the expected future reward associated with a given stimulus, and in tracking outcome values received by oneself or other agents. Encoding and decoding analyses suggested observational value coding in amygdala neurons occurred in a different subset of neurons than experiential value coding. Collectively, these findings support a key role for the human amygdala in the computations underlying the capacity for learning through observation.Significance statementSingle neuron studies of the human brain provide a unique window into the computational mechanisms of cognition. In this study, epilepsy patients implanted intracranially with depth microelectrodes performed an observational learning task. We measured activity bilaterally in the amygdala and found a representation for observational rewards as well as observational expected reward values. Additionally, the representation of self-experienced and observational values was performed by distinct subsets of amygdala neurons. This study thus provides a rare glimpse into the role of human amygdala neurons in social cognition.

It has previously been shown that the relative reliability of model-based and model-free reinforcement-learning (RL) systems plays a role in the allocation of behavioral control between them. However, the role of task complexity in the arbitration between these two strategies remains largely unknown. Here, using a combination of novel task design, computational modelling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process. Participants tended to increase model-based RL control in response to increasing task complexity. However, they resorted to model-free RL when both uncertainty and task complexity were high, suggesting that these two variables interact during the arbitration process. Computational fMRI revealed that task complexity interacts with neural representations of the reliability of the two systems in the inferior prefrontal cortex., The brain dynamically arbitrates between two model-based and model-free reinforcement learning (RL). Here, the authors show that participants tended to increase model-based control in response to increasing task complexity, but resorted to model-free when both uncertainty and task complexity were high.