Search

Objective: Diabetes is a well-known risk factor for dementia. We investigated the association between (pre)diabetes and older brain age and whether this can be attenuated by modifiable lifestyle behaviors., Research Design and Methods: The study included 31,229 dementia-free adults from the UK Biobank between the ages of 40 and 70 years. Glycemic status (normoglycemia, prediabetes, or diabetes) was ascertained based on medical history, medication use, and HbA1c measured at baseline. Information on cardiometabolic risk factors (obesity, hypertension, low HDL, and high triglycerides) and lifestyle behaviors (smoking, drinking, and physical activity) was also collected at baseline. Participants underwent up to two brain MRI scans over 11 years of follow-up. Brain age was estimated using a machine learning model based on 1,079 brain MRI phenotypes and used to calculate brain age gap (BAG; i.e., brain age minus chronological age)., Results: At baseline, 13,518 participants (43.3%) had prediabetes and 1,149 (3.7%) had diabetes. Prediabetes (β = 0.22 [95% CI 0.10, 0.34]) and diabetes (2.01 [1.70, 2.32]) were both associated with significantly higher BAG, and diabetes was further associated with significant increase in BAG over time (0.27 [0.01, 0.53]). The association between (pre)diabetes and higher BAG was more pronounced in men and in people with two or more cardiometabolic risk factors. In joint exposure analysis, having a healthy lifestyle (i.e., no smoking, no heavy drinking, and high physical activity) significantly attenuated the diabetes-BAG association., Conclusions: Diabetes and even prediabetes are associated with accelerated brain aging, especially among men and people with poor cardiometabolic health. However, a healthy lifestyle may counteract this., (© 2024 by the American Diabetes Association.)

Background: Cognitive reserve (CR) has been linked to dementia, yet its influence on the risk of depression and related outcomes remains unknown. We aimed to examine the association of CR with depression and subsequent dementia or death, and to assess the extent to which CR is related to depression-free survival., Methods: Within the UK Biobank, 436,232 participants free of depression and dementia were followed. A comprehensive CR indicator (low, moderate, and high) was created using latent class analysis based on information on education, occupation, mentally passive sedentary behavior, social connection, confiding with others, and leisure activities. Depression, dementia, and survival status were ascertained through self-reported medical history and/or linkages to medical records. Data were analyzed using multi-state Markov model and Laplace regression., Results: Over a median follow-up of 12.96 years, 16,560 individuals developed depression (including 617 with subsequent dementia) and 28,655 died. In multivariable multi-state models, compared with low CR, high CR was associated with lower risk of depression (hazard ratio 0.53 [95% confidence interval 0.51-0.56]) and lower risk of post-depression dementia (0.55 [0.34-0.88]) or death (0.69 [0.55-0.88]) in middle-aged adults (aged <60 years). In Laplace regression, the depression-free survival time was prolonged by 2.77 (2.58-2.96) years in participants with high compared to low CR., Conclusions: High CR is associated with lower risks of depression and subsequent transitions to dementia and death, particularly in middle age. High CR may prolong depression-free survival. Our findings highlight the importance of enhancing CR in the prevention and prognosis of depression.

We argue that a diverse and dynamic pool of agents mitigates proxy failure. Proxy modularity plays a key role in the ongoing production of diversity. We review examples from a range of scales.

The dopaminergic system is firmly implicated in reversal learning but human measurements of dopamine release as a correlate of reversal learning success are lacking. Dopamine release and hemodynamic brain activity in response to unexpected changes in action-outcome probabilities are here explored using simultaneous dynamic [11C]Raclopride PET-fMRI and computational modelling of behavior. When participants encounter reversed reward probabilities during a card guessing game, dopamine release is observed in associative striatum. Individual differences in absolute reward prediction error and sensitivity to errors are associated with peak dopamine receptor occupancy. The fMRI response to perseverance errors at the onset of a reversal spatially overlap with the site of dopamine release. Trial-by-trial fMRI correlates of absolute prediction errors show a response in striatum and association cortices, closely overlapping with the location of dopamine release, and separable from a valence signal in ventral striatum. The results converge to implicate striatal dopamine release in associative striatum as a central component of reversal learning, possibly signifying the need for increased cognitive control when new stimuli-responses should be learned., (© 2024. The Author(s).)

Unavoidable stress can lead to perceived lack of control and learned helplessness, a risk factor for depression. Avoiding punishment and gaining rewards involve updating the values of actions based on experience. Such updating is however useful only if action values are sufficiently stable, something that a lack of control may impair. We examined whether self-reported stress uncontrollability during the first wave of the COVID-19 pandemic predicted impaired reward-learning. In a preregistered study during the first-wave of the COVID-19 pandemic, we used self-reported measures of depression, anxiety, uncontrollable stress, and COVID-19 risk from 427 online participants to predict performance in a three-armed-bandit probabilistic reward learning task. As hypothesised, uncontrollable stress predicted impaired learning, and a greater proportion of probabilistic errors following negative feedback for correct choices, an effect mediated by state anxiety. A parameter from the best-fitting hidden Markov model that estimates expected beliefs that the identity of the optimal choice will shift across images, mediated effects of state anxiety on probabilistic errors and learning deficits. Our findings show that following uncontrollable stress, anxiety promotes an overly volatile representation of the reward-structure of uncertain environments, impairing reward attainment, which is a potential path to anhedonia in depression., (© 2023. The Author(s).)

Age-related alterations in D1-like dopamine receptor (D1DR) have distinct implications for human cognition and behavior during development and aging, but the timing of these periods remains undefined. Enabled by a large sample of in vivo assessments (n = 180, age 20 to 80 years of age, 50% female), we discover that age-related D1DR differences pivot at approximately 40 years of age in several brain regions. Focusing on the most age-sensitive dopamine-rich region, we observe opposing pre- and post-forties interrelations among caudate D1DR, cortico-striatal functional connectivity, and memory. Finally, particularly caudate D1DR differences in midlife and beyond, but not in early adulthood, associate with manifestation of white matter lesions. The present results support a model by which excessive dopamine modulation in early adulthood and insufficient modulation in aging are deleterious to brain function and cognition, thus challenging a prevailing view of monotonic D1DR function across the adult lifespan., Competing Interests: Declaration of interests The authors declare no competing interests., (Copyright © 2023 The Author(s). Published by Elsevier Inc. All rights reserved.)

Introduction: Delay discounting (DD), the preference for smaller and sooner rewards over larger and later ones, is an important behavioural phenomenon for daily functioning of increasing interest within psychopathology. The neurobiological mechanisms behind DD are not well understood and the literature on structural correlates of DD shows inconsistencies., Methods: Here we leveraged a large openly available dataset (n = 1196) to investigate associations with memory performance and gray and white matter correlates of DD using linked independent component analysis., Results: Greater DD was related to smaller anterior temporal gray matter volume. Associations of DD with total cortical volume, subcortical volumes, markers of white matter microscopic organization, working memory, and episodic memory scores were not significant after controlling for education and income., Conclusion: Effects of size comparable to the one we identified would be unlikely to be replicated with sample sizes common in many previous studies in this domain, which may explain the incongruities in the literature. The paucity and small size of the effects detected in our data underscore the importance of using large samples together with methods that accommodate their statistical structure and appropriate control for confounders, as well as the need to devise paradigms with improved task parameter reliability in studies relating brain structure and cognitive abilities with DD., (© The Author(s) 2022. Published by Oxford University Press.)

Vigor reflects how motivated people are to respond to stimuli. We previously showed that, on average, humans are more vigorous when a higher rate of reward is available, and that this relationship is modulated by the dopamine precursor levodopa. Dopamine signaling and probabilistic reward learning deteriorate across the adult life span, and thus, the relationship between vigor and reward may also change in aging. We tested this assertion and assessed whether it correlates with D1 dopamine receptor availability, measured using Positron Emission Tomography. We registered response times of 30 older and 30 younger participants during an oddball discrimination task where rewards varied systematically between trials. The average reward rate had a similar impact on vigor in both age groups. There was a weak positive association between ventral striatal dopamine receptor availability and the effect of average reward rate on response time. Overall, the effect of reward on response vigor was similar in younger and older adults, and weakly correlated with dopamine D1 receptor availability., Competing Interests: Declaration of Competing Interest None., (Copyright © 2022 The Author(s). Published by Elsevier Inc. All rights reserved.)

The prevalence of depressive symptoms decreases from late adolescence to middle age adulthood. Furthermore, despite significant losses in motor and cognitive functioning, overall emotional well-being tends to increase with age, and a bias to positive information has been observed multiple times. Several causes have been discussed for this age-related development, such as improvement in emotion regulation, less regret, and higher socioeconomic status. Here, we explore a further explanation. Our minds host mental models that generate predictions about forthcoming events to successfully interact with our physical and social environment. To keep these models faithful, the difference between the predicted and the actual event, that is, the prediction error, is computed. We argue that prediction errors are attenuated in the middle age and older mind, which, in turn, may translate to less negative affect, lower susceptibility to affective disorders, and possibly, to a bias to positive information. Our proposal is primarily linked to perceptual inferences, but may hold as well for higher-level, cognitive, and emotional forms of error processing., Competing Interests: Conflict of interestThe authors declare no conflicts of interest., (© The Author(s) 2022.)

Psychosis is associated with distorted perceptions and deficient bottom-up learning such as classical fear conditioning. This has been interpreted as reflecting imprecise priors in low-level predictive coding systems. Paradoxically, overly strong beliefs, such as overvalued beliefs and delusions, are also present in psychosis-associated states. In line with this, research has suggested that patients with psychosis and associated phenotypes rely more on high-order priors to interpret perceptual input. In this behavioural and fMRI study we studied two types of fear learning , i.e., instructed fear learning mediated by verbal suggestions about fear contingencies and classical fear conditioning mediated by low level associative learning, in delusion proneness-a trait in healthy individuals linked to psychotic disorders. Subjects were shown four faces out of which two were coupled with an aversive stimulation (CS+) while two were not (CS-) in a fear conditioning procedure. Before the conditioning, subjects were informed about the contingencies for two of the faces of each type, while no information was given for the two other faces. We could thereby study the effect of both classical fear conditioning and instructed fear learning. Our main outcome variable was evaluative rating of the faces. Simultaneously, fMRI-measurements were performed to study underlying mechanisms. We postulated that instructed fear learning, measured with evaluative ratings, is stronger in psychosis-related phenotypes, in contrast to classical fear conditioning that has repeatedly been shown to be weaker in these groups. In line with our hypothesis, we observed significantly larger instructed fear learning on a behavioural level in delusion-prone individuals ( n = 20) compared to non-delusion-prone subjects ( n = 23; n = 20 in fMRI study). Instructed fear learning was associated with a bilateral activation of lateral orbitofrontal cortex that did not differ significantly between groups. However, delusion-prone subjects showed a stronger functional connectivity between right lateral orbitofrontal cortex and regions processing fear and pain. Our results suggest that psychosis-related states are associated with a strong instructed fear learning in addition to previously reported weak classical fear conditioning. Given the similarity between nocebo paradigms and instructed fear learning, our results also have an impact on understanding why nocebo effects differ between individuals., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Louzolo, Almeida, Guitart-Masip, Björnsdotter, Lebedev, Ingvar, Olsson and Petrovic.)

Dopaminergic neurotransmission plays a pivotal role in appetitively motivated behavior in mammals, including humans. Notably, action and valence are not independent in motivated tasks, and it is particularly difficult for humans to learn the inhibition of an action to obtain a reward. We have previously observed that the carriers of the DRD2/ANKK1 TaqIA A1 allele, that has been associated with reduced striatal dopamine D2 receptor expression, showed a diminished learning performance when required to learn response inhibition to obtain rewards, a finding that was replicated in two independent cohorts. With our present study, we followed two aims: first, we aimed to replicate our finding on the DRD2/ANKK1 TaqIA polymorphism in a third independent cohort (N = 99) and to investigate the nature of the genetic effects more closely using trial-by-trial behavioral analysis and computational modeling in the combined dataset (N = 281). Second, we aimed to assess a potentially modulatory role of prefrontal dopamine availability, using the widely studied COMT Val108/158Met polymorphism as a proxy. We first report a replication of the above mentioned finding. Interestingly, after combining all three cohorts, exploratory analyses regarding the COMT Val108/158Met polymorphism suggest that homozygotes for the Met allele, which has been linked to higher prefrontal dopaminergic tone, show a lower learning bias. Our results corroborate the importance of genetic variability of the dopaminergic system in individual learning differences of action-valence interaction and, furthermore, suggest that motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function., (© 2021. The Author(s).)

Decision-making is a cognitive process of central importance for the quality of our lives. Here, we ask whether a common factor underpins our diverse decision-making abilities. We obtained 32 decision-making measures from 830 young people and identified a common factor that we call "decision acuity," which was distinct from IQ and reflected a generic decision-making ability. Decision acuity was decreased in those with aberrant thinking and low general social functioning. Crucially, decision acuity and IQ had dissociable brain signatures, in terms of their associated neural networks of resting-state functional connectivity. Decision acuity was reliably measured, and its relationship with functional connectivity was also stable when measured in the same individuals 18 months later. Thus, our behavioral and brain data identify a new cognitive construct that underpins decision-making ability across multiple domains. This construct may be important for understanding mental health, particularly regarding poor social function and aberrant thought patterns., Competing Interests: Declaration of interests The authors declare no competing interests., (Copyright © 2021 The Authors. Published by Elsevier Inc. All rights reserved.)

With increasing age, functional connectomes become dissimilar across normal individuals, reflecting heterogenous aging effects on functional connectivity (FC). We investigated the distribution of these effects across the connectome and their relationship with age-related differences in dopamine (DA) D1 receptor availability and gray matter density (GMD). With this aim, we determined aging effects on mean and interindividual variance of FC using fMRI in 30 younger and 30 older healthy subjects and mapped the contribution of each connection to the patterns of age-related similarity loss. Aging effects on mean FC accounted mainly for the dissimilarity between connectomes of younger and older adults, and were related, across brain regions, to aging effects on DA D1 receptor availability. Aging effects on the variance of FC indicated a global increase in variance with advancing age, explained connectome dissimilarity among older subjects and were related to aging effects on variance of GMD. The relationship between aging and the similarity of connectomes can thus be partly explained by age differences in DA modulation and gray matter structure.

Pavlovian associations drive approach towards reward-predictive cues, and avoidance of punishment-predictive cues. These associations "misbehave" when they conflict with correct instrumental behavior. This raises the question of how Pavlovian and instrumental influences on behavior are arbitrated. We test a computational theory according to which Pavlovian influence will be stronger when inferred controllability of outcomes is low. Using a model-based analysis of a Go/NoGo task with human subjects, we show that theta-band oscillatory power in frontal cortex tracks inferred controllability, and that these inferences predict Pavlovian action biases. Functional MRI data revealed an inferior frontal gyrus correlate of action probability and a ventromedial prefrontal correlate of outcome valence, both of which were modulated by inferred controllability., Competing Interests: The authors have declared that no competing interests exist.

Action is invigorated in the presence of reward-predicting stimuli and inhibited in the presence of punishment-predicting stimuli. Although valuable as a heuristic, this Pavlovian bias can also lead to maladaptive behaviour and is implicated in addiction. Here we explore whether Pavlovian bias can be overcome through training. Across five experiments, we find that Pavlovian bias is resistant to unlearning under most task configurations. However, we demonstrate that when subjects engage in instrumental learning in a verbal semantic space, as opposed to a motoric space, not only do they exhibit the typical Pavlovian bias, but this Pavlovian bias diminishes with training. Our results suggest that learning within the semantic space is necessary, but not sufficient, for subjects to unlearn their Pavlovian bias, and that other task features, such as gamification and spaced stimulus presentation may also be necessary. In summary, we show that Pavlovian bias, whilst robust, is susceptible to change with experience, but only under specific environmental conditions.

Learning to act to receive reward and to withhold to avoid punishment has been found to be easier than learning the opposite contingencies in young adults. To what extent this type of behavioral adaptation might develop during childhood and adolescence and differ during aging remains unclear. We therefore tested 247 healthy individuals across the human life span (7-80 years) with an orthogonalized valenced go/no-go learning task. Computational modeling revealed that peak performance in young adults was attributable to greater sensitivity to both reward and punishment. However, in children and adolescents, we observed an increased bias toward action but not reward sensitivity. By contrast, reduced learning in midlife and older adults was accompanied by decreased reward sensitivity and especially punishment sensitivity along with an age-related increase in the Pavlovian bias. These findings reveal distinct motivation-dependent learning capabilities across the human life span, which cannot be probed using conventional go/reward no-go/punishment style paradigms that have important implications in lifelong education., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Probabilistic reward learning reflects the ability to adapt choices based on probabilistic feedback. The dopaminergically innervated corticostriatal circuit in the brain plays an important role in supporting successful probabilistic reward learning. Several components of the corticostriatal circuit deteriorate with age, as it does probabilistic reward learning. We showed previously that D1 receptor availability in NAcc predicts the strength of anticipatory value signaling in vmPFC, a neural correlate of probabilistic learning that is attenuated in older participants and predicts probabilistic reward learning performance. We investigated how white matter integrity in the pathway between nucleus accumbens (NAcc) and ventromedial prefrontal cortex (vmPFC) relates to the strength of anticipatory value signaling in vmPFC in younger and older participants. We found that in a sample of 22 old and 23 young participants, fractional anisotropy in the pathway between NAcc and vmPFC predicted the strength of value signaling in vmPFC independently from D1 receptor availability in NAcc. These findings provide tentative evidence that integrity in the dopaminergic and white matter pathways of corticostriatal circuitry supports the expression of value signaling in vmPFC which supports reward learning, however, the limited sample size calls for independent replication. These and future findings could add to the improved understanding of how corticostriatal integrity contributes to reward learning ability., (© The Author(s) 2020. Published by Oxford University Press.)

Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans., (© The Author(s) 2020. Published by Oxford University Press.)

Pavlovian biases influence instrumental learning by coupling reward seeking with action invigoration and punishment avoidance with action suppression. Using a probabilistic go/no-go task designed to orthogonalize action (go/no-go) and valence (reward/punishment), recent studies have shown that the interaction between the two is dependent on the striatum and its key neuromodulator dopamine. Using this task, we sought to identify how structural and neuromodulatory age-related differences in the striatum may influence Pavlovian biases and instrumental learning in 25 young and 31 older adults. Computational modeling revealed a significant age-related reduction in reward and punishment sensitivity and marked (albeit not significant) reduction in learning rate and lapse rate (irreducible noise). Voxel-based morphometry analysis using 7 Tesla MRI images showed that individual differences in learning rate in older adults were related to the volume of the caudate nucleus. In contrast, dopamine synthesis capacity in the dorsal striatum, assessed using [18F]-DOPA positron emission tomography in 22 of these older adults, was not associated with learning performance and did not moderate the relationship between caudate volume and learning rate. This multiparametric approach suggests that age-related differences in striatal volume may influence learning proficiency in old age., (© The Author(s) 2020. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permission@oup.com.)

Learning to act to obtain reward and inhibit to avoid punishment is easier compared with learning the opposite contingencies. This coupling of action and valence is often thought of as a Pavlovian bias, although recent research has shown it may also emerge through instrumental mechanisms. We measured this learning bias with a rewarded go/no-go task in 60 adults of different ages. Using computational modeling, we characterized the bias as being instrumental. To assess the role of endogenous dopamine (DA) in the expression of this bias, we quantified DA D1 receptor availability using positron emission tomography (PET) with the radioligand [ 11 C]SCH23390. Using principal-component analysis on the binding potentials in a number of cortical and striatal regions of interest, we demonstrated that cortical, dorsal striatal, and ventral striatal areas provide independent sources of variance in DA D1 receptor availability. Interindividual variation in the dorsal striatal component was related to the strength of the instrumental bias during learning. These data suggest at least three anatomical sources of variance in DA D1 receptor availability separable using PET in humans, and we provide evidence that human dorsal striatal DA D1 receptors are involved in the modulation of instrumental learning biases., Competing Interests: The authors declare no conflict of interest., (Copyright © 2019 the Author(s). Published by PNAS.)

Evidence suggests that associations between the neurotransmitter dopamine and cognition are nonmonotonic and open to modulation by various other factors. The functional implications of a given level of dopamine may therefore differ from person to person. By applying latent-profile analysis to a large (n = 181) sample of adults aged 64-68 years, we probabilistically identified 3 subgroups that explain the multivariate associations between dopamine D2/3R availability (probed with 11C-raclopride-PET, in cortical, striatal, and hippocampal regions) and cognitive performance (episodic memory, working memory, and perceptual speed). Generally, greater receptor availability was associated with better cognitive performance. However, we discovered a subgroup of individuals for which high availability, particularly in striatum, was associated with poor performance, especially for working memory. Relative to the rest of the sample, this subgroup also had lower education, higher body-mass index, and lower resting-state connectivity between caudate nucleus and dorsolateral prefrontal cortex. We conclude that a smaller subset of individuals induces a multivariate non-linear association between dopamine D2/3R availability and cognitive performance in this group of older adults, and discuss potential reasons for these differences that await further empirical scrutiny.

Introduction: Reduced motivation is an important symptom of major depression, thought to impair recovery by reducing opportunities for rewarding experiences. We characterized motivation for monetary outcomes in depressed outpatients (N = 39, 22 female) and controls (N = 22, 11 female) in terms of their effectiveness in seeking rewards and avoiding losses. We assessed motivational function during learning of associations between stimuli and actions, as well as when learning was complete. We compared the activity within neural circuits underpinning these behaviors between depressed patients and controls., Methods: We used a Go/No-Go task that assessed subjects' abilities in learning to emit or withhold actions to obtain monetary rewards or avoid losses. We derived motivation-relevant parameters of behavior (learning rate, Pavlovian bias, and motivational influence of gains and losses). After learning, participants performed the task during functional magnetic resonance imaging (fMRI). We compared neural activation during anticipation of action emission vs. action inhibition, and for actions performed to obtain rewards compared to actions that avoid losses., Results: Depressed patients showed a similar Pavlovian bias to controls and were equivalent in terms of withholding action to gain rewards and emitting action to avoid losses, behaviors that conflict with well-described Pavlovian tendencies to approach rewards and avoid losses. Patients were not impaired in overall performance or learning and showed no abnormal neural responses, for example in bilateral midbrain or striatum. We conclude that basic mechanisms subserving motivated learning are thus intact in moderate depression., Implications: Therapeutically, the intact mechanisms identified here suggest that learning-based interventions may be particularly effective in encouraging recovery. Etiologically, our results suggest that the severe motivational deficits clinically observed in depression are likely to have complex origins, possibly related to an impairment in the representation of future states necessary for long-term planning., Competing Interests: The authors have declared that no competing interests exist.

The psychogenetically selected Roman high- (RHA) and low-avoidance (RLA) rats are being proposed as a valuable animal model of individual vulnerability to the two distinct neurobiological mechanisms of behavioral sensitization to psychostimulants, namely induction and expression. Most hallmarks of their divergent phenotypes are also found in the inbred RHA (RHA-I) and RLA (RLA-I) strains. For instance, they differ in the expression of sensitization to amphetamine. However, the pattern of spontaneous activity of the inbred rats seems to differ from that of outbred Roman strains. The present work shows the relevance of analyzing spontaneous activity as a covariant in order to determine the significance of day effect in the induction of behavioral sensitization to amphetamine (regime: 11 days, 1 mg/kg, i.p.) in the inbred strains and, for comparison, the standard low activity Sprague-Dawley (SD) strain. Our results also confirm that, in parallel to the outbred strains, only inbred RHA rats showed sensitization during the induction phase, here detectable from day 9 of treatment, while RLA-I and SD strains did not. Inbred RLA rats provide an interesting model to study individual resistance to sensitization, with nuances due to their underlying high spontaneous activity phenotype., (Copyright © 2018 Elsevier B.V. All rights reserved.)

Probabilistic reward learning is characterised by individual differences that become acute in aging. This may be due to age-related dopamine (DA) decline affecting neural processing in striatum, prefrontal cortex, or both. We examined this by administering a probabilistic reward learning task to younger and older adults, and combining computational modelling of behaviour, fMRI and PET measurements of DA D1 availability. We found that anticipatory value signals in ventromedial prefrontal cortex (vmPFC) were attenuated in older adults. The strength of this signal predicted performance beyond age and was modulated by D1 availability in nucleus accumbens. These results uncover that a value-anticipation mechanism in vmPFC declines in aging, and that this mechanism is associated with DA D1 receptor availability.

The hippocampus plays a central role in the approach-avoidance conflict that is central to the genesis of anxiety. However, its exact functional contribution has yet to be identified. We designed a novel gambling task that generated approach-avoidance conflict while controlling for spatial processing. We fit subjects' behavior using a model that quantified the subjective values of choice options, and recorded neural signals using functional magnetic resonance imaging (fMRI). Distinct functional signals were observed in anterior hippocampus, with inferior hippocampus selectively recruited when subjects rejected a gamble, to a degree that covaried with individual differences in anxiety. The superior anterior hippocampus, in contrast, uniquely demonstrated value signals that were potentiated in the context of approach-avoidance conflict. These results implicate the anterior hippocampus in behavioral avoidance and choice monitoring, in a manner relevant to understanding its role in anxiety. Our findings highlight interactions between subregions of the hippocampus as an important focus for future study., (© The Author 2016. Published by Oxford University Press.)

Dopamine (DA) losses are associated with various aging-related cognitive deficits. Typically, higher moment-to-moment brain signal variability in large-scale patterns of voxels in neocortical regions is linked to better cognitive performance and younger adult age, yet the physiological mechanisms regulating brain signal variability are unknown. We explored the relationship among adult age, DA availability, and blood oxygen level-dependent (BOLD) signal variability, while younger and older participants performed a spatial working memory (SWM) task. We quantified striatal and extrastriatal DA D1 receptor density with [(11)C]SCH23390 and positron emission tomography in all participants. We found that BOLD variability in a neocortical region was negatively related to age and positively related to SWM performance. In contrast, BOLD variability in subcortical regions and bilateral hippocampus was positively related to age and slower responses, and negatively related to D1 density in caudate and dorsolateral prefrontal cortex. Furthermore, BOLD variability in neocortical regions was positively associated with task-related disengagement of the default-mode network, a network whose activation needs to be suppressed for efficient SWM processing. Our results show that age-related DA losses contribute to changes in brain signal variability in subcortical regions and suggest a potential mechanism, by which neocortical BOLD variability supports cognitive performance., (© The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.)

The expectation of reward is known to enhance a consolidation of long-term memory for events. We tested whether this effect is driven by positive valence or action requirements tied to expected reward. Using a functional magnetic resonance imaging (fMRI) paradigm in young adults, novel images predicted gain or loss outcomes, which in turn were either obtained or avoided by action or inaction. After 24 h, memory for these images reflected a benefit of action as well as a congruence of action requirements and valence, namely, action for reward and inaction for avoidance. fMRI responses in the hippocampus, a region known to be critical for long-term memory function, reflected the anticipation of inaction. In contrast, activity in the putamen mirrored the congruence of action requirement and valence, whereas other basal ganglia regions mirrored overall action benefits on long-lasting memory. The findings indicate a novel type of functional division between the hippocampus and the basal ganglia in the motivational regulation of long-term memory consolidation, which favors remembering events that are worth acting for., (© The Author 2015. Published by Oxford University Press.)

Model-based and model-free reinforcement learning (RL) have been suggested as algorithmic realizations of goal-directed and habitual action strategies. Model-based RL is more flexible than model-free but requires sophisticated calculations using a learnt model of the world. This has led model-based RL to be identified with slow, deliberative processing, and model-free RL with fast, automatic processing. In support of this distinction, it has recently been shown that model-based reasoning is impaired by placing subjects under cognitive load--a hallmark of non-automaticity. Here, using the same task, we show that cognitive load does not impair model-based reasoning if subjects receive prior training on the task. This finding is replicated across two studies and a variety of analysis methods. Thus, task familiarity permits use of model-based reasoning in parallel with other cognitive demands. The ability to deploy model-based reasoning in an automatic, parallelizable fashion has widespread theoretical implications, particularly for the learning and execution of complex behaviors. It also suggests a range of important failure modes in psychiatric disorders.

Deep brain stimulation (DBS) of the subthalamic nucleus (STN) is an effective treatment for motor impairments in Parkinson's disease (PD) but its effect on the motivational regulation of action control is still not fully understood. We investigated whether DBS of the STN influences the ability of PD patients to act for anticipated reward or loss, or whether DBS improves action execution independent of motivational valence. 16 PD patients (12 male, mean age = 58.5 ± 10.17 years) treated with bilateral STN-DBS and an age- and gender-matched group of healthy controls (HC) performed a go/no-go task whose contingencies explicitly decouple valence and action. Patients were tested with (ON) and without (OFF) active STN stimulation. For HC, there was a benefit in performing rewarded actions when compared to actions that avoided punishment. PD patients showed such a benefit reliably only when STN stimulation was ON. In fact, the relative behavioral benefit for go for reward over go to avoid losing was stronger in the PD patients under DBS ON than in HC. In PD patients, rather than generally improving motor functions independent of motivational valence, modulation of the STN by DBS improves action execution specifically when rewards are anticipated. Thus, STN-DBS establishes a reliable congruency between action and reward ("Pavlovian congruency") and remarkably enhances it over the level observed in HC.

Motivational salience plays an important role in shaping human behavior, but recent studies demonstrate that human performance is not uniformly improved by motivation. Instead, action has been shown to dominate valence in motivated tasks, and it is particularly difficult for humans to learn the inhibition of an action to obtain a reward, but the neural mechanism behind this behavioral specificity is yet unclear. In all mammals, including humans, the monoamine neurotransmitter dopamine is particularly important in the neural manifestation of appetitively motivated behavior, and the human dopamine system is subject to considerable genetic variability. The well-studied TaqIA restriction fragment length polymorphism (rs1800497) has previously been shown to affect striatal dopamine metabolism. In this study we investigated a potential effect of this genetic variation on motivated action/inhibition learning. Two independent cohorts consisting of 87 and 95 healthy participants, respectively, were tested using the previously described valenced go/no-go learning paradigm in which participants learned the reward-associated no-go condition significantly worse than all other conditions. This effect was modulated by the TaqIA polymorphism, with carriers of the A1 allele showing a diminished learning-related performance enhancement in the rewarded no-go condition compared to the A2 homozygotes. This result highlights a modulatory role for genetic variability of the dopaminergic system in individual learning differences of action-valence interaction.

Repeated processing of the same information is associated with decreased neuronal responses, termed repetition suppression (RS). Although RS effects (i.e., the difference in activity between novel and repeated stimuli) have been demonstrated within several brain regions, such as the medial temporal lobe, their precise neural mechanisms still remain unclear. Here, we used functional magnetic resonance imaging together with psychopharmacology in 48 healthy human subjects, demonstrating that RS effects within the mesolimbic system are differentially modulated by cholinergic and dopaminergic stimulation. The dopamine precursor levodopa (100 mg) attenuated RS within the hippocampus, parahippocampal cortex, and substantia nigra/ventral tegmental area, and the degree of this reduction correlated with recognition memory performance 24 h later. The acetylcholinesterase inhibitor galantamine (8 mg), in contrast, reversed RS into repetition enhancement, showing no relationship to subsequent recognition memory. This suggests that novelty sensitive neural populations of the mesolimbic system can dynamically shift their responses depending on the balance of cholinergic and dopaminergic neurotransmission, and these shifts can influence memory retention.

The selection of actions, and the vigor with which they are executed, are influenced by the affective valence of predicted outcomes. This interaction between action and valence significantly influences appropriate and inappropriate choices and is implicated in the expression of psychiatric and neurological abnormalities, including impulsivity and addiction. We review a series of recent human behavioral, neuroimaging, and pharmacological studies whose key design feature is an orthogonal manipulation of action and valence. These studies find that the interaction between the two is subject to the critical influence of dopamine. They also challenge existing views that neural representations in the striatum focus on valence, showing instead a dominance of the anticipation of action., (Copyright © 2014 Elsevier Ltd. All rights reserved.)

Rationale: Decision-making involves two fundamental axes of control namely valence, spanning reward and punishment, and action, spanning invigoration and inhibition. We recently exploited a go/no-go task whose contingencies explicitly decouple valence and action to show that these axes are inextricably coupled during learning. This results in a disadvantage in learning to go to avoid punishment and in learning to no-go to obtain a reward. The neuromodulators dopamine and serotonin are likely to play a role in these asymmetries: Dopamine signals anticipation of future rewards and is also involved in an invigoration of motor responses leading to reward, but it also arbitrates between different forms of control. Conversely, serotonin is implicated in motor inhibition and punishment processing., Objective: To investigate the role of dopamine and serotonin in the interaction between action and valence during learning.Methods We combined computational modeling with pharmacological manipulation in 90 healthy human volunteers, using levodopa and citalopram to affect dopamine and serotonin, respectively., Results: We found that, after administration of levodopa,action learning was less affected by outcome valence when compared with the placebo and citalopram groups. This highlights in this context a predominant effect of levodopa in controlling the balance between different forms of control.Citalopram had distinct effects, increasing participants'tendency to perform active responses independent of outcome valence, consistent with a role in decreasing motor inhibition., Conclusions: Our findings highlight the rich complexities of the roles played by dopamine and serotonin during instrumental learning.

Actions can lead to an immediate reward or punishment and a complex set of delayed outcomes. Adaptive choice necessitates the brain track and integrate both of these potential consequences. Here, we designed a sequential task whereby the decision to exploit or forego an available offer was contingent on comparing immediate value and a state-dependent future cost of expending a limited resource. Crucially, the dynamics of the task demanded frequent switches in policy based on an online computation of changing delayed consequences. We found that human subjects choose on the basis of a near-optimal integration of immediate reward and delayed consequences, with the latter computed in a prefrontal network. Within this network, anterior cingulate cortex (ACC) was dynamically coupled to ventromedial prefrontal cortex (vmPFC) when adaptive switches in choice were required. Our results suggest a choice architecture whereby interactions between ACC and vmPFC underpin an integration of immediate and delayed components of value to support flexible policy switching that accommodates the potential delayed consequences of an action.

Economic choices are strongly influenced by whether potential outcomes entail gains or losses. We examined this influence of outcome valence in an economic risk task. We employed three experiments based on our task, each of which provided novel findings, and which together better characterize and explain how outcome valence influences risky choice. First, we found that valence perturbed an individual's choices around that individual's base-level of risk-taking, a base-level consistent across time, and context. Second, this perturbation by valence was highly context dependent, emerging when valence was introduced as a dimension within a decision-making setting, and being reversed by a change in task format (causing more gambling for gains than losses and the reverse). Third, we show this perturbation by valence is explicable by low-level approach-avoidance processes, an hypothesis not previously tested by a causal manipulation. We revealed such an effect, where individuals were less disposed to choose a riskier option with losses when they had to approach (go) as opposed to avoid (nogo) that option. Our data show valence perturbs an individual's choices independently of the impact of risk, and causally implicate approach-avoidance processes as important in shaping economic choice.

Action inhibition can globally prevent all motor output or selectively cancel specific actions during concurrent motor output. Here we examine the behavioral and neural basis of selective inhibition focusing on the role of preparation. In 18 healthy human participants we manipulated the extent to which they could prepare for selective inhibition by providing or withholding information on what actions might need to be stopped. We show that, on average, information improves both speed and selectivity of inhibition. Functional magnetic resonance imaging data show that preparation for selective inhibition engages the inferior frontal gyrus, supplementary motor area, and striatum. Examining interindividual differences, we find the benefit of proactive control to speed and selectivity of inhibition trade off against each other, such that an improvement in stopping speed leads to a deterioration of selectivity of inhibition, and vice versa. This trade-off is implemented through engagement of the dorsolateral prefrontal cortex and putamen. Our results suggest proactive selective inhibition is implemented within frontostriatal structures, and we provide evidence that a speed-selectivity trade-off might underlie a range of findings reported previously.

Flexible instrumental learning is required to harness the appropriate behaviors to obtain rewards and to avoid punishments. The precise contribution of dopaminergic midbrain regions (substantia nigra/ventral tegmental area [SN/VTA]) to this form of behavioral adaptation remains unclear. Normal aging is associated with a variable loss of dopamine neurons in the SN/VTA. We therefore tested the relationship between flexible instrumental learning and midbrain structural integrity. We compared task performance on a probabilistic monetary go/no-go task, involving trial and error learning of: "go to win," "no-go to win," "go to avoid losing," and "no-go to avoid losing" in 42 healthy older adults to previous behavioral data from 47 younger adults. Quantitative structural magnetization transfer images were obtained to index regional structural integrity. On average, both some younger and some older participants demonstrated a behavioral asymmetry whereby they were better at learning to act for reward ("go to win" > "no-go to win"), but better at learning not to act to avoid punishment ("no-go to avoid losing" > "go to avoid losing"). Older, but not younger, participants with greater structural integrity of the SN/VTA and the adjacent subthalamic nucleus could overcome this asymmetry. We show that interindividual variability among healthy older adults of the structural integrity within the SN/VTA and subthalamic nucleus relates to effective acquisition of competing instrumental responses., (Copyright © 2013 Elsevier Inc. All rights reserved.)

Psychological studies have emphasized that motivation is regulated by the anticipation of the emotional impact from the possible occurrence of unexpected rewarding events. Here, we scrutinized the existence of a corresponding neural signal by means of event-related potentials (ERPs) and computational modeling. In the first experiment, we designed a task that manipulated the probability of gaining a monetary reward and measured ERPs during anticipation and at reward delivery. A sustained frontocentral neural activity (i.e., the stimulus preceding negativity, SPN) was evidenced during the anticipation period. Critically, the SPN was found to increase in amplitude as the reward became more unexpected. Changes in the SPN were found to be predictive of individual differences in risk seeking, suggesting that a greater risk attitude involved a greater motivational state for receiving an improbable reward. In the second experiment, SPN results associated with unexpected monetary gains were replicated in a condition in which participants avoided monetary losses and the occurrence of unexpected rewards was also associated with an increase in the amount of self-reported pleasure. These findings support the existence of a neural ERP signature that encodes the process of tuning our motivation to the possibility of receiving a desirable but improbable rewarding outcome., (Copyright © 2013 Elsevier Inc. All rights reserved.)

A report by Payzan-LeNestour et al. (2013) in this issue of Neuron shows that the human locus coeruleus, a brain stem nucleus containing cell bodies of noradrenergic neurons, dynamically tracks the level of uncertainty about knowledge of the environment while making decisions., (Copyright © 2013 Elsevier Inc. All rights reserved.)

Subjects routinely control the vigor with which they emit motoric responses. However, the bulk of formal treatments of decision-making ignores this dimension of choice. A recent theoretical study suggested that action vigor should be influenced by experienced average reward rate and that this rate is encoded by tonic dopamine in the brain. We previously examined how average reward rate modulates vigor as exemplified by response times and found a measure of agreement with the first suggestion. In the current study, we examined the second suggestion, namely the potential influence of dopamine signaling on vigor. Ninety healthy subjects participated in a double-blind experiment in which they received one of the following: placebo, L-DOPA (which increases dopamine levels in the brain), or citalopram (which has a selective, if complex, effect on serotonin levels). Subjects performed multiple trials of a rewarded odd-ball discrimination task in which we varied the potential reward over time in order to exercise the putative link between vigor and average reward rate. Replicating our previous findings, we found that a significant fraction of the variance in subjects' responses could be explained by our experimentally manipulated changes in average reward rate. Crucially, this relationship was significantly stronger under L-Dopa than under Placebo, suggesting that the impact of average reward levels on action vigor is indeed subject to a dopaminergic influence.