Search

Actions can lead to an immediate reward or punishment and a complex set of delayed outcomes. Adaptive choice necessitates the brain track and integrate both of these potential consequences. Here, we designed a sequential task whereby the decision to exploit or forego an available offer was contingent on comparing immediate value and a state-dependent future cost of expending a limited resource. Crucially, the dynamics of the task demanded frequent switches in policy based on an online computation of changing delayed consequences. We found that human subjects choose on the basis of a near-optimal integration of immediate reward and delayed consequences, with the latter computed in a prefrontal network. Within this network, anterior cingulate cortex (ACC) was dynamically coupled to ventromedial prefrontal cortex (vmPFC) when adaptive switches in choice were required. Our results suggest a choice architecture whereby interactions between ACC and vmPFC underpin an integration of immediate and delayed components of value to support flexible policy switching that accommodates the potential delayed consequences of an action.

Action inhibition can globally prevent all motor output or selectively cancel specific actions during concurrent motor output. Here we examine the behavioral and neural basis of selective inhibition focusing on the role of preparation. In 18 healthy human participants we manipulated the extent to which they could prepare for selective inhibition by providing or withholding information on what actions might need to be stopped. We show that, on average, information improves both speed and selectivity of inhibition. Functional magnetic resonance imaging data show that preparation for selective inhibition engages the inferior frontal gyrus, supplementary motor area, and striatum. Examining interindividual differences, we find the benefit of proactive control to speed and selectivity of inhibition trade off against each other, such that an improvement in stopping speed leads to a deterioration of selectivity of inhibition, and vice versa. This trade-off is implemented through engagement of the dorsolateral prefrontal cortex and putamen. Our results suggest proactive selective inhibition is implemented within frontostriatal structures, and we provide evidence that a speed-selectivity trade-off might underlie a range of findings reported previously.

Pavlovian biases influence learning and decision making by intricately coupling reward seeking with action invigoration and punishment avoidance with action suppression. This bias is not always adaptive-it can often interfere with instrumental requirements. The prefrontal cortex is thought to help resolve such conflict between motivational systems, but the nature of this control process remains unknown. EEG recordings of midfrontal theta band power are sensitive to conflict and predictive of adaptive control over behavior, but it is not clear whether this signal reflects control over conflict between motivational systems. Here we used a task that orthogonalized action requirements and outcome valence while recording concurrent EEG in human participants. By applying a computational model of task performance, we derived parameters reflective of the latent influence of Pavlovian bias and how it was modulated by midfrontal theta power during motivational conflict. Between subjects, those who performed better under Pavlovian conflict exhibited higher midfrontal theta power. Within subjects, trial-to-trial variance in theta power was predictive of ability to overcome the influence of the Pavlovian bias, and this effect was most pronounced in subjects with higher midfrontal theta to conflict. These findings demonstrate that midfrontal theta is not only a sensitive index of prefrontal control, but it can also reflect the application of top-down control over instrumental processes.

Neural representations of the effort deployed in performing actions, and the valence of the outcomes they yield, form the foundation of action choice. To discover whether brain areas represent effort and outcome valence together or if they represent one but not the other, we examined these variables in an explicitly orthogonal way. We did this by asking human subjects to exert one of two levels of effort to improve their chances of either winning or avoiding the loss of money. Subjects responded faster both when exerting greater effort and when exerting effort in anticipation of winning money. Using fMRI, we inspected BOLD responses during anticipation (before any action was executed) and when the outcome was delivered. In this way, we indexed BOLD signals associated with an anticipated need to exert effort and its affective consequences, as well as the effect of executed effort on the representation of outcomes. Anterior cingulate cortex and dorsal striatum (dorsal putamen) signaled the anticipation of effort independently of the prospect of winning or losing. Activity in ventral striatum (ventral putamen) was greater for better-than-expected outcomes compared with worse-than-expected outcomes, an effect attenuated in the context of having exerted greater effort. Our findings provide evidence that neural representations of anticipated actions are sensitive to the expected demands, but not to the expected value of their consequence, whereas representations of outcome value are discounted by exertion, commensurate with an integration of cost and benefit so as to approximate net value.

Optimal decision making requires that we integrate mnemonic information regarding previous decisions with value signals that entail likely rewards and punishments. The fact that memory and value signals appear to be coded by segregated brain regions, the hippocampus in the case of memory and sectors of prefrontal cortex in the case of value, raises the question as to how they are integrated during human decision making. Using magnetoencephalography to study healthy human participants, we show increased theta oscillations over frontal and temporal sensors during nonspatial decisions based on memories from previous trials. Using source reconstruction we found that the medial temporal lobe (MTL), in a location compatible with the anterior hippocampus, and the anterior cingulate cortex in the medial wall of the frontal lobe are the source of this increased theta power. Moreover, we observed a correlation between theta power in the MTL source and behavioral performance in decision making, supporting a role for MTL theta oscillations in decision-making performance. These MTL theta oscillations were synchronized with several prefrontal sources, including lateral superior frontal gyrus, dorsal anterior cingulate gyrus, and medial frontopolar cortex. There was no relationship between the strength of synchronization and the expected value of choices. Our results indicate a mnemonic guidance of human decision making, beyond anticipation of expected reward, is supported by hippocampal-prefrontal theta synchronization.

Activation of the hippocampus is required to encode memories for new events (or episodes). Observations from animal studies suggest that, for these memories to persist beyond 4-6 h, a release of dopamine generated by strong hippocampal activation is needed. This predicts that dopaminergic enhancement should improve human episodic memory persistence also for events encoded with weak hippocampal activation. Here, using pharmacological functional MRI (fMRI) in an elderly population in which there is a loss of dopamine neurons as part of normal aging, we show this very effect. The dopamine precursor levodopa led to a dose-dependent (inverted U-shape) persistent episodic memory benefit for images of scenes when tested after 6 h, independent of whether encoding-related hippocampal fMRI activity was weak or strong (U-shaped dose-response relationship). This lasting improvement even for weakly encoded events supports a role for dopamine in human episodic memory consolidation, albeit operating within a narrow dose range.

We investigated how rapidly the reward-predicting properties of visual cues are signaled in the human brain and the extent these reward prediction signals are contextually modifiable. In a magnetoencephalography study, we presented participants with fractal visual cues that predicted monetary rewards with different probabilities. These cues were presented in the temporal context of a preceding novel or familiar image of a natural scene. Starting at ∼100 ms after cue onset, reward probability was signaled in the event-related fields (ERFs) over temporo-occipital sensors and in the power of theta (5-8 Hz) and beta (20-30 Hz) band oscillations over frontal sensors. While theta decreased with reward probability beta power showed the opposite effect. Thus, in humans anticipatory reward responses are generated rapidly, within 100 ms after the onset of reward-predicting cues, which is similar to the timing established in non-human primates. Contextual novelty enhanced the reward anticipation responses in both ERFs and in beta oscillations starting at ∼100 ms after cue onset. This very early context effect is compatible with a physiological model that invokes the mediation of a hippocampal-VTA loop according to which novelty modulates neural response properties within the reward circuitry. We conclude that the neural processing of cues that predict future rewards is temporally highly efficient and contextually modifiable.

The acquisition of reward and the avoidance of punishment could logically be contingent on either emitting or withholding particular actions. However, the separate pathways in the striatum for go and no-go appear to violate this independence, instead coupling affect and effect. Respect for this interdependence has biased many studies of reward and punishment, so potential action-outcome valence interactions during anticipatory phases remain unexplored. In a functional magnetic resonance imaging study with healthy human volunteers, we manipulated subjects' requirement to emit or withhold an action independent from subsequent receipt of reward or avoidance of punishment. During anticipation, in the striatum and a lateral region within the substantia nigra/ventral tegmental area (SN/VTA), action representations dominated over valence representations. Moreover, we did not observe any representation associated with different state values through accumulation of outcomes, challenging a conventional and dominant association between these areas and state value representations. In contrast, a more medial sector of the SN/VTA responded preferentially to valence, with opposite signs depending on whether action was anticipated to be emitted or withheld. This dominant influence of action requires an enriched notion of opponency between reward and punishment.

Reward representation in ventral striatum is boosted by perceptual novelty, although the mechanism of this effect remains elusive. Animal studies indicate a functional loop (Lisman and Grace, 2005) that includes hippocampus, ventral striatum, and midbrain as being important in regulating salience attribution within the context of novel stimuli. According to this model, reward responses in ventral striatum or midbrain should be enhanced in the context of novelty even if reward and novelty constitute unrelated, independent events. Using fMRI, we show that trials with reward-predictive cues and subsequent outcomes elicit higher responses in the striatum if preceded by an unrelated novel picture, indicating that reward representation is enhanced in the context of novelty. Notably, this effect was observed solely when reward occurrence, and hence reward-related salience, was low. These findings support a view that contextual novelty enhances neural responses underlying reward representation in the striatum and concur with the effects of novelty processing as predicted by the model of Lisman and Grace (2005).