Search

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goaltracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states. [ABSTRACT FROM AUTHOR]

Relapse to maladaptive eating habits during dieting is often provoked by stress and there is evidence for a role of ovarian hormones in stress responses and feeding. We studied the role of these hormones in stress-induced reinstatement of food seeking and medial prefrontal cortex (mPFC) neuronal activation in c-fos-GFP transgenic female rats, which express GFP in strongly activated neurons. Food-restricted ovariectomized or sham-operated c-fos-GFP rats were trained to lever-press for palatable food pellets. Subsequently, lever-pressing was extinguished and reinstatement of food seeking and mPFC neuronal activation was assessed after injections of the pharmacological stressor yohimbine (0.5-2 mg/kg) or pellet priming (1- 4 noncontingent pellets). Estrous cycle effects on reinstatement were also assessed in wild-type rats. Yohimbine- and pellet-priming-induced reinstatement was associated with Fos and GFP induction in mPFC; both reinstatement and neuronal activation were minimally affected by ovarian hormones in both c-fos-GFP and wild-type rats. c-fos-GFP transgenic rats were then used to assess glutamatergic synaptic alterations within activated GFP-positive and nonactivated GFP-negative mPFC neurons following yohimbine-induced reinstatement of food seeking. This reinstatement was associated with reduced AMPA receptor/NMDA receptor current ratios and increased paired-pulse facilitation in activated GFP-positive but not GFP-negative neurons. While ovarian hormones do not appear to play a role in stress-induced relapse of food seeking in our rat model, this reinstatement was associated with unique synaptic alterations in strongly activated mPFC neurons. Our paper introduces the c-fos-GFP transgenic rat as a new tool to study unique synaptic changes in activated neurons during behavior. [ABSTRACT FROM AUTHOR]

The amygdala is critical for associating predictive cues with primary rewarding and aversive outcomes. This is particularly evident in tasks in which information about expected outcomes is required for normal responding. Here we used a pavlovian overexpectation task to test whether outcome signaling by amygdala might also be necessary for changing those representations in the face of unexpected outcomes. Rats were trained to associate several different cues with a food reward. After learning, two of the cues were presented together, in compound, followed by the same reward. Before each compound training session, rats received infusions of 2,3-dioxo-6-nitro-1,2,3,4- tetrahydrobenzo[f]quinoxaline-7-sulfonamide or saline into either the basolateral (ABL) or central nucleus (CeN) of amygdala.Wefound that infusions into CeN abolished the normal decline in responding to the compounded cue in a later probe test, whereas infusions into ABL had no effect. These results are inconsistent with the proposal that signaling of information about expected outcomes by ABL contributes to learning, at least in this setting, and instead implicate the CeN in this process, perhaps attributable to the hypothesized involvement of this area in attention and variations in stimulus processing. [ABSTRACT FROM AUTHOR]

The discovery that dopamine neurons signal errors in reward prediction has demonstrated that concepts empirically derived from the study of animal behavior can be used to understand the neural implementation of reward learning. Yet the learning theory models linked to phasic dopamine activity treat attention to events such as cues and rewards as static quantities; other models, such as Pearce-Hall, propose that learning might be influenced by variations in processing of these events. A key feature of these accounts is that event processing is modulated by unsigned rather than signed reward prediction errors. Here we tested whether neural activity in rat basolateral amygdala conforms to this pattern by recording single units in a behavioral task in which rewards were unexpectedly delivered or omitted. We report that neural activity at the time of reward is providing an unsigned error signal with characteristics consistent with those postulated by these models. This neural signal increased immediately after a change in reward, and stronger firing was evident whether the value of the reward increased or decreased. Further, as predicted by these models, the change in firing developed over several trials as expectations for reward were repeatedly violated. This neural signal was correlated with faster orienting to predictive cues after changes in reward, and abolition of the signal by inactivation of basolateral amygdala disrupted this change in orienting and retarded learning in response to changes in reward. These results suggest that basolateral amygdala serves a critical function in attention for learning. [ABSTRACT FROM AUTHOR]

The article explores the role of glial cell line-derived neurotrophic factor (GDNF) in the reversal of withdrawal-induced decreases in nucleus accumbens dopamine and attenuation of alcohol reward. Dopamine-depletion hypothesis and related opponent process theories are being referred. It is concluded that further research is necessary to determine the mechanistic connection between GDNF and alcohol reward.

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states. SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms., (Copyright © 2023 the authors.)

Relapse to maladaptive eating habits during dieting is often provoked by stress. Recently, we identified a role of dorsal medial prefrontal cortex (mPFC) neurons in stress-induced reinstatement of palatable food seeking in male rats. It is unknown whether endogenous neural activity in dorsal mPFC drives stress-induced reinstatement in female rats. Here, we used an optogenetic approach, in which female rats received bilateral dorsal mPFC microinjections of viral constructs coding light-sensitive eNpHR3.0-eYFP or control eYFP protein and intracranial fiber optic implants. Rats were food restricted and trained to lever press for palatable food pellets. Subsequently, pellets were removed, and lever pressing was extinguished; then the effect of bilateral dorsal mPFC light delivery on reinstatement of food seeking was assessed after injections of the pharmacological stressor yohimbine (an α-2 andrenoceptor antagonist) or pellet priming, a manipulation known to provoke food seeking in hungry rats. Dorsal mPFC light delivery attenuated yohimbine-induced reinstatement of food seeking in eNpHR3.0-injected but not eYFP-injected rats. This optical manipulation had no effect on pellet-priming-induced reinstatement or ongoing food-reinforced responding. Dorsal mPFC light delivery attenuated yohimbine-induced Fos immunoreactivity and disrupted neural activity during in vivo electrophysiological recording in awake rats. Optical stimulation caused significant outward currents and blocked electrically evoked action potentials in eNpHR3.0-injected but not eYFP-injected mPFC hemispheres. Light delivery alone caused no significant inflammatory response in mPFC. These findings indicate that intracranial light delivery in eNpHR3.0 rats disrupts endogenous dorsal mPFC neural activity that plays a role in stress-induced relapse to food seeking in female rats.