Search

Direct replication of an eye-tracking study investigating the effect of action requirements on attention to rewards and punishments, and vice versa the effect of attention to rewards and punishments on action execution

We will assess whether and how the kinematics of reaching towards an intrinsically neutral stimulus changes, after this has acquired threat-related value following Pavlovian learning. The experimental task will include four consecutive phases controlled by the OpenSesame software (Mathôt et al., 2012). 1. Pavlovian threat learning. Participants will learn to identify a specific stimulus color as dangerous (pink or yellow, counterbalanced among participants), representing the conditioned stimulus (i.e. CS+), such that its presentation will terminate with an aversive shock. The other color will serve as within-subject control stimulus (i.e. CS-), never being paired with shock. The shock will be generated by a Digitimer Stimulator (Digitimer Ltd., UK) and delivered to participants’ non-dominant hand. The intensity of shock will be calibrated for each participant, to a level deemed ‘highly unpleasant, but not painful’, using an ascending staircase procedure. Participants will not make any motor response but only observe the screen. Skin conductance, subjective reports of CS valence and CS-US contingency awareness will be recorded. 2. Test 1 (no threat of shock). Participants will make reaching movements with the computer mouse towards three target locations (i.e. low, medium or high). Participants will start the trial by clicking on a start button at the bottom of the screen. Then, a colored circle (i.e. pink or yellow) will appear briefly to signal the target location. Participants will then reach the target location with the mouse as quickly and accurately as possible and click on it, terminating the trial. This phase will be conducted without any shock electrodes attached to the wrist. 3. Test 2 (under threat of shock). Participants will repeat the test phase, but this time they will have the shock electrodes attached to the wrist, although no shock will ever be delivered. 4. Test 3 (no threat of shock). Participants will repeat the test phase, without the shock electrodes attached to the wrist.

Systematic Literature Review of Human Pavlovian-Instrumental Transfer

The modern obesogenic environment contains an abundance of food cues (e.g., sight, smell of food) as well cues that are associated with food through learning and memory processes. Food cue exposure can lead to food seeking and excessive consumption in otherwise food-sated individuals, and a high level of food cue responsivity is a risk factor for overweight and obesity. Similar food cue responses are observed in experimental rodent models, and these models are therefore useful for mechanistically identifying the neural circuits mediating food cue responsivity. This review draws from both experimental rodent models and human data to characterize the behavioral and biological processes through which food-associated stimuli contribute to overeating and weight gain. Two rodent models are emphasized – cue-potentiated feeding and Pavlovian-instrumental transfer – that provide insight in the neural circuits and peptide systems underlying food cue responsivity. Data from humans are highlighted that reveal physiological, psychological, and neural mechanisms that connect food cue responsivity with overeating and weight gain. The collective literature identifies connections between heightened food cue responsivity and obesity in both rodents and humans, and identifies underlying brain regions (nucleus accumbens, amygdala, orbitofrontal cortex, hippocampus) and endocrine systems (ghrelin) that regulate food cue responsivity in both species. These species similarities are encouraging for the possibility of mechanistic rodent model research and further human research leading to novel treatments for excessive food cue responsivity in humans.

Dopamine neurotransmission has been ascribed multiple functions with respect to both motivational and associative processes in reward-based learning, though these have proven difficult to tease apart. In order to better describe the role of dopamine in associative learning, this series of experiments examined the potential of dopamine D1- and D2-receptor antagonism (or combined antagonism) to influence the ability of rats to learn neutral valence stimulus-stimulus associations. Using a sensory preconditioning task, rats were first exposed to pairings of two neutral stimuli (S2-S1). Subsequently, S1 was paired with a mild foot-shock and resulting fear to both S1 (directly conditioned) and S2 (preconditioned) was examined. Initial experiments demonstrated the validity of the procedure in that measures of sensory preconditioning were shown to be contingent on pairings of the two sensory stimuli. Subsequent experiments indicated that systemic administration of dopamine D1- or D2-receptor antagonists attenuated learning when administered prior to S2-S1 pairings. However, the administration of a more generic D1R/D2R antagonist was without effect. These effects remained constant regardless of the affective valence of the conditioning environment and did not differ between male and female rats. The results are discussed in the context of recent suggestions that dopaminergic systems encode more than a simple reward prediction error, and provide potential avenues for future investigation.

Structural and functional abnormalities of the orbitofrontal cortex (OFC) have been implicated in affective disorders that manifest anxiety-related symptoms. However, research into the functions of primate OFC has predominantly focused on reward-oriented rather than threat-oriented responses. To redress this imbalance, the present study performed a comprehensive analysis of the independent role of 2 distinct subregions of the central OFC (anterior area 11; aOFC and posterior area 13; pOFC) in the processing of distal and proximal threat. Temporary inactivation of both aOFC and pOFC heightened responses to distal threat in the form of an unknown human, but not to proximal threat assessed in a discriminative Pavlovian conditioning task. Inactivation of the aOFC, however, did unexpectedly blunt conditioned threat responses, although the effect was not valence-specific, as conditioned appetitive responses were similarly blunted and appeared restricted to a discriminative version of the task (when both CS− and CS+ are present within a session). Inactivation of the pOFC did not affect conditioned responses to either proximal threat or reward and basal cardiovascular activity was unaffected by manipulations of activity in either subregion. The results highlight the contribution of aOFC and pOFC to regulation of responses to more distal uncertain but not proximal, certain threat and reveal their opposing contribution to that of the immediately adjacent medial OFC, area 14.

The dysfunction and structural abnormalities of the orbitofrontal cortex have been reported in a number of affective disorders, including depression and several anxiety disorders. However, research has largely centred around reward-guided decision-making and economic choice, as opposed to how it may influence the expression and regulation of positive and negative emotion. Moreover, most studies fail to recognise the precise anatomical sub-divisions of the primate orbitofrontal cortex, in particular the anterior (area 11, antOFC) and posterior (area 13, pOFC) sub-divisions of the central orbitofrontal cortex. Evidence from cytoarchitecture, connectivity and function supports the idea that the two sub-regions may have distinct functional contributions. Overall, the purpose of the research in this thesis was to explore the possible contributions of the anterior and posterior orbitofrontal cortex to emotional regulation in the common marmoset (Callithrix jacchus). Past research in the lab has shown that excitotoxic lesions of the anterior orbitofrontal cortex heighten anxiety-like behaviour on the human intruder test, a paradigm testing the responses to a distal threat. The first experimental chapter confirmed that this effect is also present with an acute temporary inactivation of the antOFC. Inactivation of the pOFC produced a trend towards a similar anxiogenic-like effect. To elaborate on these findings, the effects of separate antOFC and pOFC inactivation on discriminative conditioned responses to an aversive stimulus were also examined, as a means of studying responses to a more proximal and imminent threat. While inactivation of the pOFC did not produce any effects, inactivations of the antOFC surprisingly reduced conditioned behavioural responses to proximal threat. These findings from threat conditioning were also for the first time directly compared to a corresponding task of discriminative conditioning to reward, revealing considerable similarities: pOFC inactivation produced no effects and antOFC inactivation caused mild blunting of conditioned responses to reward. Finally, the thesis also presents work on the development of a novel touchscreen task designed to study the relative contributions of positive and negative feedback to learning in the marmoset. The final chapter includes the preliminary data, showing the effects of inactivation as well as the blockade of serotonin reuptake, and discusses the future prospects of the task. Together, the research presented here supports past data indicating that the central OFC, and in particular the antOFC, may have an important function in regulating responses to a distal threat. However, neither the antOFC nor the pOFC appear to play a role in downregulating responses to more proximal threats. The studies examining conditioning to reward and threat also importantly highlight that the separate inactivations of the antOFC and pOFC can produce distinct results, and should be treated as functionally distinct units. The data presented here offers a basis for future research elaborating on the nuances of antOFC and pOFC contributions to emotional regulation., MRC DTP funding (Grant Number N013433-1) Pinsent Darwin Award Sackler Fund for Medical Sciences Supported by Prof. Roberts' MRC Grant (Grant Number MR/M023990/1)

The orbitofrontal cortex (OFC) is a critical structure in the flexible control of value-based behaviors. OFC dysfunction is typically only detected when task or environmental contingencies change, against a backdrop of apparently intact initial acquisition and behavior. While intact acquisition following OFC lesions in simple Pavlovian cue-outcome conditioning is often predicted by models of OFC function, this predicted null effect has not been thoroughly investigated. Here, we test the effects of lesions and temporary muscimol inactivation of the rodent lateral OFC on the acquisition of a simple single cue-outcome relationship. Surprisingly, pretraining lesions significantly enhanced acquisition after overtraining, whereas post-training lesions and inactivation significantly impaired acquisition. This impaired acquisition to the cue reflects a disruption of behavioral control and not learning since the cue could also act as an effective blocking stimulus in an associative blocking procedure. These findings suggest that even simple cue-outcome representations acquired in the absence of OFC function are impoverished. Therefore, while OFC function is often associated with flexible behavioral control in complex environments, it is also involved in very simple Pavlovian acquisition where complex cue-outcome relationships are irrelevant to task performance.

Rationale Checking is a functional behaviour that provides information to guide behaviour. However, in obsessive-compulsive disorder (OCD), checking may escalate to dysfunctional levels. The processes underpinning the transition from functional to dysfunctional checking are unclear but may be associated with individual differences that support the development of maladaptive behaviour. We examined one such predisposition, sign-tracking to a pavlovian conditioned stimulus, which we previously found associated with dysfunctional checking. How sign-tracking interacts with another treatment with emerging translational validity for OCD-like checking, chronic administration of the dopamine D2 receptor agonist quinpirole, is unknown. Objectives We tested how functional and dysfunctional checking in the rat observing response task (ORT) was affected by chronic quinpirole administration in non-autoshaped controls and autoshaped animals classified as sign-trackers or goal-trackers. Methods Sign-trackers or goal-trackers were trained on the ORT before the effects of chronic quinpirole administration on checking were assessed. Subsequently, the effects on checking of different behavioural challenges, including reward omission and the use of unpredictable reinforcement schedules, were tested. Results Prior autoshaping increased checking. Sign-trackers and goal-trackers responded differently to quinpirole sensitization, reward omission and reinforcement uncertainty. Sign-trackers showed greater elevations in dysfunctional checking, particularly during uncertainty. By contrast, goal-trackers predominantly increased functional checking responses, possibly in response to reduced discrimination accuracy in the absence of cues signalling which lever was currently active. Conclusions The results are discussed in terms of how pavlovian associations influence behaviour that becomes compulsive in OCD and how this may be dependent on striatal dopamine D2 receptors.

The dopamine system responds to reward-predictive cues to reflect a prospective estimation of reward value, although its role in encoding retrospective reward-related information is unclear. We report that cue-evoked dopamine release in the nucleus accumbens core encodes the time elapsed since the previous reward or rather the wait time. Specifically, a cue that always follows the preceding reward with a short wait time elicits a greater dopamine response relative to a distinct cue that always follows the preceding reward with a long wait time. Differences in the dopamine response between short wait and long wait cues were evident even when these cues were never experienced together within the same context. Conditioned responding updated accordingly with a change in cue-evoked dopamine release but was unrelated to a difference in the dopamine response between cues. Collectively, these findings illustrate that the cue-evoked dopamine response conveys a subjective estimation of the relative reward rate.

A recent study by Saunders, Richard, Margolis, and Janak (2018) shows that there is a great deal left to learn about what different mesotelencephalic dopamine circuits contribute to learning about the motivational significance of reward-related cues. Their findings suggest that nigrostriatal and mesolimbic dopamine pathways support distinct reinforcement processes that independently push and pull animals toward their goals.

The anterior cingulate cortex (ACC) responds to outcomes of a positive or negative valence, but past studies typically focus on one valence or the other, making it difficult to know how opposing valences are disambiguated. We recorded from ACC neurons as rats received tones followed by aversive, appetitive or null outcomes. The responses to the different tones/outcomes were highly inter-mixed at the single neuron level but combined to produce robust valence-specific representations at the ensemble level. The valence-specific patterns far outlasted the tones and outcomes, persisting throughout the long inter-trial intervals (ITIs) and even throughout trial blocks. When the trials were interleaved, the valence-specific patterns abruptly shifted at the start of each new trial. Overall the aversive trials had the greatest impact on the neurons. Thus within the ACC, valence-specificity is largely an emergent property of ensembles and valence-specific representations can appear quickly and persist long after the initiating event.

Studying aversive behaviour is critical for understanding negative emotions and associated psychopathologies. However a comprehensive picture of the mechanisms underlying aversion is lacking, with associative learning theories focusing on Pavlovian reactions and decision-making theoretic approaches on prospective functions. We propose a computational model of aversion that combines goal-directed and Pavlovian forms of control into a unifying framework in which their relative importance is regulated by factors such as threat distance and controllability. Using simulations, we test whether the model can reproduce available empirical findings and discuss its relevance to understanding factors underlying negative emotions such as fear and anxiety. Furthermore, the specific method used to construct the model permits a natural mapping from its components to brain structure and function. Our model provides a basis for a unifying account of aversion that can guide empirical and interventional study contexts.

New excavation undertaken in Oblazowa Cave, south Poland, yielded a fossil Conus shell, polished and incised artificially. The shell was found at the very bottom of the pit, beneath the point of the Aurignacian relics recovery. It has a stratigraphic and chronological meaning. The shell, as well as two others which had been found years before in layer VIII of the Oblazowa Cave, can be regarded as another trace of the Pavlovian people. It was probably the Pavolvians who destroyed the older layers of the cave. This raises a question on the relations between the Aurignacian and Pavlovian groups. Conus shells of the same kind were numerously found in the Pavlovian Moravian sites and in Lower Austria, in the vicinity of Grubgraben. Due to the fact that in the Aurignacian the Conus shells have not occurred, one can acknowledge them as a fingerprint of the Pavlovian.

Highlights • Pavlovian responses couple action and valence. • This coupling interferes with instrumental learning and performance. • Action dominates valence in the striatum and dopaminergic midbrain. • Boosting dopamine enhances the dominance of action over valence in the striatum. • Boosting dopamine decreases the extent of the behavioral coupling between action and valence., The selection of actions, and the vigor with which they are executed, are influenced by the affective valence of predicted outcomes. This interaction between action and valence significantly influences appropriate and inappropriate choices and is implicated in the expression of psychiatric and neurological abnormalities, including impulsivity and addiction. We review a series of recent human behavioral, neuroimaging, and pharmacological studies whose key design feature is an orthogonal manipulation of action and valence. These studies find that the interaction between the two is subject to the critical influence of dopamine. They also challenge existing views that neural representations in the striatum focus on valence, showing instead a dominance of the anticipation of action.

Reward-seeking actions can be guided by external cues that signal reward availability. For instance, when confronted with a stimulus that signals sugar, rats will prefer an action that produces sugar over a second action that produces grain pellets. Action selection is also sensitive to changes in the incentive value of potential rewards. Thus, rats that have been prefed a large meal of sucrose will prefer a grain-seeking action to a sucrose-seeking action. The current study investigated the dependence of these different aspects of action selection on cholinergic transmission. Hungry rats were given differential training with two unique stimulus-outcome (S1-O1 and S2-O2) and action-outcome (A1-O1 and A2-O2) contingencies during separate training phases. Rats were then given a series of Pavlovian-to-instrumental transfer tests, an assay of cue-triggered responding. Before each test, rats were injected with scopolamine (0, 0.03, or 0.1 mg/kg, intraperitoneally), a muscarinic receptor antagonist, or mecamylamine (0, 0.75, or 2.25 mg/kg, intraperitoneally), a nicotinic receptor antagonist. Although the reward-paired cues were capable of biasing action selection when rats were tested off-drug, both anticholinergic treatments were effective in disrupting this effect. During a subsequent round of outcome devaluation testing - used to assess the sensitivity of action selection to a change in reward value - we found no effect of either scopolamine or mecamylamine. These results reveal that cholinergic signaling at both muscarinic and nicotinic receptors mediates action selection based on Pavlovian reward expectations, but is not critical for flexibly selecting actions using current reward values. © 2014 American College of Neuropsychopharmacology.

In the service of their basic needs and desires, animals and humans can use information from their environment to guide their choice between actions. In the laboratory, the ability for reward-predictive cues to control action selection is studied through outcome-specific Pavlovian-instrumental transfer (PIT-S), in which a stimulus associated with a particular outcomes biases choice between actions towards the response that earned that same outcome. This thesis investigates the determinants of PIT-S within the nucleus accumbens shell (NAc-S), which is selectively recruited to mediate PIT-S, and is not involved in encoding Pavlovian or instrumental associations. Previous work indicated that delta-opioid receptor (DOR) accumulation at the membrane of cholinergic interneurons (CIN-m) in the NAc-S is triggered during Pavlovian learning, and is positively correlated with PIT-S performance. We took three approaches to investigating the role of DOR accumulation: behavioural and pharmacological manipulation of established receptor accumulation, and chemogenetic manipulation of CINs and the afferents that likely utilize DOR accumulation within the NAc-S to mediate PIT-S. Manipulations of the predictive status of Pavlovian cues that abolished PIT-S failed to reverse established DOR accumulation, suggesting that a region that encodes this information controls the use of DOR accumulation to drive PIT-S. Specific pharmacological internalisation of DOR transiently reduced receptor expression on CIN-m in the NAc-S, indicating that CINs have `memory' for DOR accumulation. PIT-S performance was impaired during the period of DOR reduction, indicating that DOR accumulation on NAc-S CIN-m is necessary, but not sufficient, for PIT-S expression. Inactivation of CINs and BLA terminals within the NAc-S impaired and attenuated PIT-S, respectively. Both contralateral and ipsilateral BLA terminal disconnection from CINs in the NAc-S impaired PIT-S performance, despite ipsilateral BLA-NAc-S disconnection failing to affect PIT-S in previous studies, which complicates our ability to interpret this finding. These findings, considered together, suggest that DOR accumulation on CIN-m is necessary, but not sufficient, for PIT-S, and that a functional circuit between BLA terminals and CINs mediates PIT-S, via DOR accumulation on CIN-m.

Decision-making invokes two fundamental axes of control: affect or valence, spanning reward and punishment, and effect or action, spanning invigoration and inhibition. We studied the acquisition of instrumental responding in healthy human volunteers in a task in which we orthogonalized action requirements and outcome valence. Subjects were much more successful in learning active choices in rewarded conditions, and passive choices in punished conditions. Using computational reinforcement-learning models, we teased apart contributions from putatively instrumental and Pavlovian components in the generation of the observed asymmetry during learning. Moreover, using model-based fMRI, we showed that BOLD signals in striatum and substantia nigra/ventral tegmental area (SN/VTA) correlated with instrumentally learnt action values, but with opposite signs for go and no-go choices. Finally, we showed that successful instrumental learning depends on engagement of bilateral inferior frontal gyrus. Our behavioral and computational data showed that instrumental learning is contingent on overcoming inherent and plastic Pavlovian biases, while our neuronal data showed this learning is linked to unique patterns of brain activity in regions implicated in action and inhibition respectively., Highlights ► Expectation of valence interferes with action learning in human participants. ► Computational modeling disentangles influences of instrumental and Pavlovian systems. ► Striatum and SN/VTA track action values and bind them to the control of vigor. ► Successful control is associated with activity in the inferior prefrontal cortex.

Updated theoretical accounts of the role of serotonin (5-HT) in motivation propose that 5-HT operates at the intersection of aversion and inhibition, promoting withdrawal in the face of aversive predictions. However, the specific cognitive mechanisms through which 5-HT modulates withdrawal behavior remain poorly understood. Behavioral inhibition in response to punishments reflects at least two concurrent processes: instrumental aversive predictions linking stimuli, responses, and punishments, and Pavlovian aversive predictions linking stimuli and punishments irrespective of response. In the current study, we examined to what extent 5-HT modulates the impact of instrumental vs Pavlovian aversive predictions on behavioral inhibition. We used acute tryptophan depletion to lower central 5-HT levels in healthy volunteers, and observed behavior in a novel task designed to measure the influence of Pavlovian and instrumental aversive predictions on choice (response bias) and response vigor (response latencies). After placebo treatment, participants were biased against responding on the button that led to punishment, and they were slower to respond in a punished context, relative to a non-punished context. Specifically, participants slowed their responses in the presence of stimuli predictive of punishments. Tryptophan depletion removed the bias against responding on the punished button, and abolished slowing in the presence of punished stimuli, irrespective of response. We suggest that this set of results can be explained by a role for 5-HT in Pavlovian aversive predictions. These findings suggest additional specificity for the influence of 5-HT on aversively motivated behavioral inhibition and extend recent models of the role of 5-HT in aversive predictions.

We present an elemental model of associative learning that describes interactions between stimulus elements as a process of competitive normalization. Building on the assumptions laid out in Harris (2006), stimuli are represented as an array of elements that compete for attention according to the strength of their input. Elements form associations among each other according to temporal correlations in their activation but restricted by their connectivity. The model moves beyond its predecessor by specifying excitatory, inhibitory, and attention processes for each element in real time and describing their interaction as a form of suppressive gain control. Attention is formalized in this model as a network of mutually inhibitory units that moderate the activation of stimulus elements by controlling the level to which the elements are suppressed by their own inhibitory processes. The model is applied to a range of complex discriminations and related phenomena that have been taken as evidence for configural-learning processes. Australian Research Council: DP0771154

Two studies explored the role of the amygdala in response modulation by an aversive conditioned stimulus (CS) in rats. Experiment 1 investigated the role of amygdala circuitry in conditioned suppression using a paradigm in which licking for sucrose was inhibited by a tone CS that had been previously paired with footshock. Electrolytic lesions of the lateral amygdala (LA) impaired suppression relative to sham-operated animals, and produced the same pattern of results when applied to central amygdala. In addition, disconnection of the lateral and central amygdala, by unilateral lesion of each on opposite sides of the brain, also impaired suppression relative to control subjects that received lesions of both areas on the same side. In each case, lesions were placed following Pavlovian conditioning and instrumental training, but before testing. This procedure produced within-subjects measures of the effects of lesion on freezing and between-group comparisons for the effects on suppression. Experiment 2 extended this analysis to a task where an aversive CS suppressed shuttling responses that had been previously food reinforced and also found effects of bilateral lesions of the central amygdala in a pre-post design. Together, these studies demonstrate that connections between the lateral and central amygdala constitute a serial circuit involved in processing aversive Pavlovian stimuli, and add to a growing body of findings implicating central amygdala in the modulation of instrumental behavior.

Studying aversive behaviour is critical for understanding negative emotions and associated psychopathologies. However a comprehensive picture of the mechanisms underlying aversion is lacking, with associative learning theories focusing on Pavlovian reactions and decision-making theoretic approaches on prospective functions. We propose a computational model of aversion that combines goal-directed and Pavlovian forms of control into a unifying framework in which their relative importance is regulated by factors such as threat distance and controllability. Using simulations, we test whether the model can reproduce available empirical findings and discuss its relevance to understanding factors underlying negative emotions such as fear and anxiety. Furthermore, the specific method used to construct the model permits a natural mapping from its components to brain structure and function. Our model provides a basis for a unifying account of aversion that can guide empirical and interventional study contexts.

The present thesis examined the role of the rodent lateral orbitofrontal cortex (LO) in controlling performance in Pavlovian cue-outcome learning and placed the function of LO in the broader context of the entire orbitofrontal cortex (OFC) region. We found that functional inactivation of LO disrupted the retention of between- but not within-session extinction behaviour. This finding was not due to the disruption of post-session memory consolidation processes by the functional inactivation of LO or impaired acquisition of Pavlovian conditioned inhibition learning. These data confirmed that LO is necessary for the acquisition of Pavlovian extinction learning. In a second set of experiments we investigated the role of LO in guiding behaviour using the sensory specific properties of expected outcomes. Pre-training LO lesions were found to disrupt Pavlovian devaluation by taste aversion but not specific satiety or instrumental devaluation by conditioned taste aversion, or the specific Pavlovian-to-instrumental transfer effect. These findings rule out an explanation of LO function in simply representing the sensory specific properties of expected outcomes. The third set of experiments examined the role of LO in the acquisition of simple Pavlovian learning. Pre-training LO lesions increased conditioned responding whereas post-training lesions and functional inactivation of LO decreased conditioned responding. A blocking procedure revealed that decreased responding following LO inactivation did not affect the subsequent blocking of new learning at test. Furthermore, LO inactivation impaired the use of current relative value of outcomes to modulate behaviour. This suggested a role for LO in the expression of conditioned responding but not initial Pavlovian learning. A final set of experiments examined the role of LO in a novel Pavlovian differential outcomes expectancy task. Pre-training LO lesions were found to decrease whereas post-training lesions were found to increase response accurate responding. This pattern of results, in combination with the opposite effect of pre- and post-training lesions in simple Pavlovian conditioning demonstrated the importance of LO in guiding Pavlovian behaviour. Taken together, these data demonstrated a role for LO in the integration of sensory and motivational properties of expected outcomes to flexibly control Pavlovian behaviour.

Animals quickly learn to approach sources of food. Here, we report on a form of approach in which rats made volitional orofacial contact with inactive feeders between trials of a self-paced operant task. This extraneous feeder sampling (EFS) was never reinforced and therefore imposed an opportunity and effort cost. EFS decreased during initial training but persisted thereafter. The relative rate of EFS to operant responding increased with novel changes to the operant chamber, reward devaluation by prefeeding, or lesions to the dorsolateral striatum. We speculate that this may function to increase exploration when the task is uncertain (early in learning or introduction of novel apparatus components), when the opportunity cost is low, or when the learned sensorimotor solution is compromised. Moreover, EFS strongly affected subsequent choices by triggering a lose-shift response away from the sampled feeder, even though it occurred outside of the trial context. This indicates that at least some behaviors occurring between trials impact future behaviors and should be considered in decision-making studies.

Pavlovian conditioned stimuli (CSs) play an important role in the reinforcement and motivation of instrumental active avoidance (AA). Conditioned threats can also invigorate ongoing AA responding [aversive Pavlovian–instrumental transfer (PIT)]. The neural circuits mediating AA are poorly understood, although lesion studies suggest that lateral, basal, and central amygdala nuclei, as well as infralimbic prefrontal cortex, make key, and sometimes opposing, contributions. We recently completed an extensive analysis of brain c-Fos expression in good vs. poor avoiders following an AA test (Martinez et al., 2013, Learning and Memory). This analysis identified medial amygdala (MeA) as a potentially important region for Pavlovian motivation of instrumental actions. MeA is known to mediate defensive responding to innate threats as well as social behaviors, but its role in mediating aversive Pavlovian–instrumental interactions is unknown. We evaluated the effect of MeA lesions on Pavlovian conditioning, Sidman two-way AA conditioning (shuttling) and aversive PIT in rats. Mild footshocks served as the unconditioned stimulus in all conditioning phases. MeA lesions had no effect on AA but blocked the expression of aversive PIT and 22 kHz ultrasonic vocalizations in the AA context. Interestingly, MeA lesions failed to affect Pavlovian freezing to discrete threats but reduced freezing to contextual threats when assessed outside of the AA chamber. These findings differentiate MeA from lateral and central amygdala, as lesions of these nuclei disrupt Pavlovian freezing and aversive PIT, but have opposite effects on AA performance. Taken together, these results suggest that MeA plays a selective role in the motivation of instrumental avoidance by general or uncertain Pavlovian threats.

Pavlovian-to-instrumental transfer (PIT) is an effect whereby a classically conditioned stimulus (CS) enhances ongoing instrumental responding. PIT has been extensively studied with appetitive conditioning but barely at all with aversive conditioning. Although it's been argued that conditioned suppression is a form of aversive PIT, this effect is fundamentally different from appetitive PIT because the CS suppresses, instead of facilitates, responding. Five experiments investigated the importance of a variety of factors on aversive PIT in a rodent Sidman avoidance paradigm in which ongoing shuttling behavior (unsignaled active avoidance or USAA) was facilitated by an aversive CS. Experiment 1 demonstrated a basic PIT effect. Experiment 2 found that a moderate amount of USAA extinction produces the strongest PIT with shuttling rates best at around 2 responses per minute prior to the CS. Experiment 3 tested a protocol in which the USAA behavior was required to reach the 2-response per minute mark in order to trigger the CS presentation and found that this produced robust and reliable PIT. Experiment 4 found that the Pavlovian conditioning US intensity was not a major determinant of PIT strength. Experiment 5 demonstrated that if the CS and US were not explicitly paired during Pavlovian conditioning, PIT did not occur, showing that CS-US learning is required. Together, these studies demonstrate a robust, reliable and stable aversive PIT effect that is amenable to analysis of neural circuitry.

Rationale: Whilst Cannabidiol (CBD), a non-psychotomimetic cannabinoid, has been shown to enhance extinction learning in rats, its effects on fear memory in humans have not previously been studied. Objectives: We employed a Pavlovian fear-conditioning paradigm in order to assess the effects of CBD on extinction and consolidation. Method: Forty-eight participants were conditioned to a coloured box (CS) with electric shocks (UCS) in one context and were extinguished in a second context. Participants received 32 mg of CBD either following before or after extinction in a double-blind, placebo-controlled design. At recall, 48 h later, participants were exposed to CSs and conditioning contexts before (recall) and after (reinstatement) exposure to the UCS. Skin conductance and shock expectancy measures of conditioned responding were recorded throughout. Results: Successful conditioning, extinction and recall were found in all three treatment groups. CBD given post-extinction enhanced consolidation of extinction learning as assessed by shock expectancy. CBD administered at either time produced trend level reduction in reinstatement of autonomic contextual responding. No acute effects of CBD were found on extinction. Conclusions: These findings provide the first evidence that CBD can enhance consolidation of extinction learning in humans and suggest that CBD may have potential as an adjunct to extinction-based therapies for anxiety disorders.

Anxiety disorders are often treated using extinction-based exposure therapy, but relapse is common and can occur as a result of reinstatement, whereby an aversive trigger can reinstate extinguished fear. Animal models of reinstatement commonly utilize a Pavlovian fear conditioning procedure, in which subjects are first trained to fear a conditional stimulus (CS) by pairing it with an aversive unconditional stimulus (US), and then extinguished by repeated presentations of the CS alone. Reinstatement is typically induced by exposing subjects to an aversive US after extinction, but here we show that exposure to a non-extinguished CS can reinstate conditional fear responding to an extinguished CS, a phenomenon we refer to as conditional reinstatement. Rats were trained to fear two CSs (light and tone) and subsequently underwent extinction training to only one CS (counterbalanced). Presenting the unextinguished CS (but not a novel cue) immediately after extinction reinstated conditional fear responding to the extinguished CS in a test session given 24h later. These findings indicate that reinstatement of extinguished fear can be triggered by exposure to conditional as well as unconditional aversive stimuli, and this may help to explain why relapse is common following clinical extinction therapy in humans. Further study of conditional reinstatement using animal models may prove useful for developing refined extinction therapies that are more resistant to reinstatement.

This chapter considers associative solutions to “non‐linear” discrimination problems, such as negative patterning (A+ and B+ vs AB‐) and the biconditional discrimination (AB+ and CD+ vs AC‐ and BD‐). It is commonly assumed that the solution to these discriminations requires “configural” elements that are added to the compound of two stimuli. However, these discriminations can be solved by assuming that some elements of each stimulus are suppressed when two stimuli are presented in compound. Each of these approaches can solve patterning and biconditional discriminations because they allow some elements, as the arguments of associations, to have differential “presence” on reinforced versus non‐ reinforced trials, and thus differential associability and control over responding. The chapter then presents a more specific version of one of these models, describing how interactions between stimuli, particularly the competition for attention, provide a mechanism whereby some elements are more suppressed than others when stimuli are presented simultaneously as a compound.

Activation of N-methyl-D-aspartate receptors (NMDAr) in the basolateral complex of the amygdala (BLA) is crucial for the acquisition and extinction of Pavlovian first-order fear. It is unknown whether these substrates also mediate higher-order fear. Higher-order fear has been observed using two procedures. In second-order conditioning, pairings of a tone (S1) and shock are followed by pairings of a light (S2) and the tone (S1); in sensory preconditioning, pairings of a light (S2) and tone (S1) are followed by pairings of the tone (S1) and shock. In each procedure, test presentations of the light alone elicit fear responses which extinguish across such presentations. This thesis examined whether activation of NMDAr in the BLA is necessary for the acquisition and extinction of higher-order fear. Inactivation of the BLA via infusion of the GABAA agonist muscimol, systemic injection of the NMDAr antagonist MK-801, and BLA infusion of the NMDAr NR2B subunit selective antagonist ifenprodil, disrupted acquisition and extinction of second-order fear. Extinction of sensory preconditioned fear was impaired by BLA infusion of muscimol or ifenprodil and by systemic MK-801. Acquisition of the neutral S2-S1 association and extinction of this association before conditioning of S1 was impaired by MK-801 but not by BLA infusion of muscimol. The final experiment examined the role of the perirhinal cortex (PRh) in the acquisition of higher-order fear. Muscimol inactivation of the PRh prior to S2-S1 pairings blocked sensory preconditioning but not second-order conditioning. These findings can be summarised as follows. First, activation of BLA NMDAr is necessary for the acquisition and extinction of second-order fear. Second, the acquisition and pre-extinction of sensory preconditioning requires NMDAr activity but not BLA activation. Third, once the fear circuit is engaged, learning to inhibit sensory preconditioned fear becomes dependent on BLA NMDAr activation. Finally, the PRh is necessary for the acquisition of an association between two neutral stimuli but not between a neutral stimulus and a learned danger signal. These findings are consistent with current views of amygdala function. They are also consistent with the claim that the contents of second-order conditioning and sensory preconditioning differ, the former involving associations between S2 and the fear responses elicited by S1 and the latter consisting in associations between the sensory properties of S2 and S1.

Superstitions are beliefs that are not supported by empirical evidence, or beliefs that persist in the face of contradictory empirical evidence (Zebb & Moore, 2003). Both cultural and personal superstitions are pervasive in the general population (Vyse, 1997) and appear to influence not only the clinically ill, but healthy individuals as well as successful sportspeople, businesspeople, and politicians. The history of superstition research is traced from the Skinner's seminal (1948) explanation of superstitious learning as the result of accidental pairings of events, or adventitious conditioning. Contemporary human superstitious learning research (e.g. Matute, 1994, 1995) has demonstrated that non-contingent delivery of reinforcers can lead to illusion of control if the person is actively making instrumental responses to gain those reinforcers. Superstitious beliefs appear to play a role in clinical anxiety disorders such as OCD, social phobia, and PD. Specifically, avoidance of threatening situations and engaging in in-situation safety behaviours (e.g. Salkovskis, D. M. Clark & Gelder, 1996), prevent non-occurrence of the feared outcome from disconfirming the threat belief and eventual fear extinction. Quite the opposite, instrumental attempts to prevent imminent catastrophe lead non-occurrence of the feared outcome to be perceived as confirmation that the threat is real. A human avoidance learning paradigm (P. Lovibond, 2006; P. Lovibond et al., 2008) was adapted for a laboratory investigation of superstitious learning using a threat-relevant stimulus (mild electric shock). A series of five experiments using undergraduate participants demonstrated that participants‟ expectations of task structure modulated their tendency to develop maintained stereotyped responding. Approximately half the participants maintained a chosen instrumental response to a target cue and maintained higher threat beliefs towards that cue even though it had never been paired with shock. Participants who did not maintain an instrumental response had lower threat estimates for the target cue. While non-shocked presentations of the target cue should lead to fear reduction and eventual extinction, response maintenance protects from extinction fear to a threat cue. Reasons for the lower-than-expected rates of response maintenance were discussed, and future experiments were suggested in order to address this perceived shortcoming.

The experiments reported in this thesis studied the mechanisms of expression of overexpectation of conditioned fear, as measured by freezing. In Stage I, rats were conditioned to fear a tone and a flashing light conditioned stimulus (CS) through pairings with a 0.5 mA, 1 s shock. In Stage II, overexpectation was trained by the reinforcement of a compound of these CSs with a shock of the same magnitude. Two compound – shock pairings produced an overexpectation effect, as measured by freezing to presentations of the tone alone, while further Stage II training caused over-training of overexpectation. Expression of the overexpectation effect produced by two compound – shock pairings could be prevented by pre-test injection of the benzodiazepine partial inverse agonist FG7142. This effect was dose-dependent and not due to state-dependent memory. Control experiments suggested that it was also not due to any general effect of FG7142 on the Pavlovian freezing response. Freezing to a tone that had been conditioned, but not subjected to any decremental training procedures, was unaffected by administration of FG7142 before either the conditioning or test session. FG7142 also did not affect freezing to a tone that had been subjected to an associative blocking procedure. The hypothesis that overexpectation of conditioned fear may be context-dependent was also tested. However, renewal was not observed. Rats that received Stage II training in a context distinct from the Stage I training context showed equivalent expression of overexpectation regardless of whether testing was conducted in the Stage I or Stage II training context. These results are consistent with the hypothesis that overexpectation, like extinction, leads to the imposition of a GABAA receptor-mediated mask on the fear CR. Moreover, they suggest that this masking of fear is the specific consequence of negative predictive error.

The ability to assign safety to stimuli in the environment is integral to everyday functioning. A key brain region for this evaluation is the ventromedial prefrontal cortex (vmPFC). To investigate the importance of vmPFC safety signaling, we used neuroimaging of Pavlovian fear reversal, a paradigm that involves flexible updating when the contingencies for a threatening (CS+) and safe (CS–) stimulus reverse, in a prototypical disorder of inflexible behavior influenced by anxiety, Obsessive Compulsive Disorder (OCD). Skin conductance responses in OCD patients (n = 43) failed to differentiate during reversal compared with healthy controls (n = 35), although significant differentiation did occur during early conditioning and amygdala BOLD signaling was unaffected in these patients. Increased vmPFC activation (for CS+ > CS–) during early conditioning predicted the degree of generalization in OCD patients during reversal, whereas vmPFC safety signals were absent throughout learning in these patients. Regions of the salience network (dorsal anterior cingulate, insula, and thalamus) showed early learning task-related hyperconnectivity with the vmPFC in OCD, consistent with biased processing of the CS+. Our findings reveal an absence of vmPFC safety signaling in OCD, undermining flexible threat updating and explicit contingency knowledge. Although differential threat learning can occur to some extent in the absence of vmPFC safety signals, effective CS– signaling becomes crucial during conflicting threat and safety cues. These results promote further investigation of vmPFC safety signaling in other anxiety disorders, with potential implications for the development of exposure-based therapies, in which safety signaling is likely to play a key role., This work was funded by Wellcome Trust Senior Investigator Award 104631/z/14/z (to T.W.R.) and a joint award from the Medical Research Council and the Wellcome Trust supporting the Behavioural and Clinical Neuroscience Institute (G0001354).