Search

Multiple neural systems contribute to choice adaptation following reinforcement. Recent evidence suggests that the lateral habenula (LHb) plays a key role in such adaptations, particularly when reinforcements are worse than expected. Here, we investigated the effects of bilateral LHb lesions on responding in a binary choice task with no discriminatory cues. LHb lesions in rats decreased win-stay responses but surprisingly left lose-shift responses intact. This same dissociated effect was also observed after systemic administration of d-amphetamine in a separate cohort of animals. These results suggest that at least some behavioural responses triggered by reward omission do not depend on an intact LHb.

Animals tend to repeat actions that are associated with reward delivery, whereas they tend to shift responses to alternate choices following reward omission. These so-called win-stay and lose-shift responses are employed by a wide range of animals in a variety of decision-making scenarios, and depend on dissociated regions of the striatum. Specifically, lose-shift responding is impaired by extensive excitotoxic lesions of the lateral striatum. Here we used focal lesions to assess whether dorsal and ventral regions of the lateral striatum contribute differently to this effect. We found that damage to ventrolateral striatum reduced lose-shift responding without impairing win-stay, motoric, or motivational aspects of behaviour in the task, whereas lesions confined to the dorsolateral striatum significantly impaired the ability of rats to complete trials of the task. Moreover, lesions to the dorsomedial striatum had no effect on either lose-shift or win-stay responding. Together, these data suggest a novel role of the ventral portion of the lateral striatum in driving lose-shift decisions.

Psychostimulants such as d-amphetamine (AMPH) often have behavioral effects that appear paradoxical within the framework of optimal choice theory. AMPH typically increases task engagement and the effort animals exert for reward, despite decreasing reward valuation. We investigated neural correlates of this phenomenon in the anterior cingulate cortex (ACC), a brain structure implicated in signaling cost-benefit utility. AMPH decreased signaling of reward, but not effort, in the ACC of freely-moving rats. Ensembles of simultaneously recorded neurons generated task-specific trajectories of neural activity encoding past, present, and future events. Low-dose AMPH contracted these trajectories and reduced their variance, whereas high-dose AMPH expanded both. We propose that under low-dose AMPH, increased network stability balances moderately increased excitability, which promotes accelerated unfolding of a neural ‘script’ for task execution, despite reduced reward valuation. Noise from excessive excitability at high doses overcomes stability enhancement to drive frequent deviation from the script, impairing task execution.

Behavior provides important insights into neuronal processes. For example, analysis of reaching movements can give a reliable indication of the degree of impairment in neurological disorders such as stroke, Parkinson disease, or Huntington disease. The analysis of such movement abnormalities is notoriously difficult and requires a trained evaluator. Here, we show that a deep neural network is able to score behavioral impairments with expert accuracy in rodent models of stroke. The same network was also trained to successfully score movements in a variety of other behavioral tasks. The neural network also uncovered novel movement alterations related to stroke, which had higher predictive power of stroke volume than the movement components defined by human experts. Moreover, when the regression network was trained only on categorical information (control = 0; stroke = 1), it generated predictions with intermediate values between 0 and 1 that matched the human expert scores of stroke severity. The network thus offers a new data-driven approach to automatically derive ratings of motor impairments. Altogether, this network can provide a reliable neurological assessment and can assist the design of behavioral indices to diagnose and monitor neurological disorders., This study demonstrates that a state-of-the-art neural network can provide automated scoring of motor deficits with an accuracy equivalent to that of human experts and has the potential teach us to develop more-sensitive behavioral tests.

Although male and female rats appear to perform differently in some tasks, a clear picture of sex differences in decision-making has yet to develop. This is in part due to significant variability arising from differences in strains and tasks. The aim of this study was to characterize the effects of sex on specific response elements in a reinforcement learning task so as to help identify potential explanations for this variability. We found that the primary difference between sexes was the propensity to approach feeders out of the task context. This extraneous feeder sampling affects choice on subsequent trials in both sexes by promoting a lose-shift response away from the last feeder sampled. Female rats, however, were more likely to engage in this extraneous feeder sampling, and therefore exhibited a greater rate of this effect. Once trials following extraneous sampling were removed, there were no significant sex differences in any of the tested measures. These data suggest that feeder approach outside of the task context, which is often not recorded, could produce a confound in sex-based differences of reinforcement sensitivity in some tasks.

Amphetamine and other drugs of abuse have both short-term and long-lasting effects on brain function, and drug sensitization paradigms often result in chronic impairments in behavioral flexibility. Here we show that acute amphetamine administration temporarily renders rats less sensitive to reward omission, as revealed by a decrease in lose-shift responding during a binary choice task. Intracerebral infusions of amphetamine into the ventral striatum did not affect lose-shift responding but did increase impulsive behavior in which rats chose to check both reward feeders before beginning the next trial. In contrast to acute systemic and intracerebral infusions, sensitization through repeated exposure induced long-lasting increased sensitivity to reward omission. These treatments did not affect choices on trials following reward delivery (i.e. win-stay responding), and sensitization increased spine density in the sensorimotor striatum. The dichotomous effects of amphetamine on short-term and long-term loss sensitivity, and the null effect on win-stay responding, are consistent with a shift of behavioral control to the sensorimotor striatum after drug sensitization. These data provide a new demonstration of such a shift in a novel task unrelated to drug administration, and suggests that the dominance of sensorimotor control persists over many hundreds of trials after sensitization.

Inflammatory bowel disease (IBD) is characterized by relapsing periods of gut inflammation, and is comorbid with depression, anxiety, and cognitive deficits. Animal models of IBD that explore the behavioral consequences almost exclusively use acute models of gut inflammation, which fails to recapitulate the cyclic, chronic nature of IBD. This study sought to identify behavioral differences in digging, memory, and stress-coping strategies in mice exposed to one (acute) or three (chronic) cycles of gut inflammation, using the dextran sodium sulfate (DSS) model of colitis. Similar levels of gut pathology were observed between acute and chronically exposed mice, although mice in the chronic treatment had significantly shorter colons, suggesting more severe disease. Behavioral measures revealed an unexpected pattern in which chronic treatment evoked fewer deficits than acute treatment. Specifically, acutely-treated mice showed alterations in measures of object burying, novel object recognition, object location memory, and stress-coping (forced swim task). Chronically-treated animals, however, showed similar alterations in object burying, but not the other measures. These data suggest an adaptive or tolerizing effect of repeated cycles of peripheral gut inflammation on mnemonic function and stress-coping, whereas some other behaviors continue to be affected by gut inflammation. We speculate that the normalization of some functions may involve the reversion to the baseline state of the hypothalamic-pituitary-adrenal axis and/or levels of neuroinflammation, which are both activated by the first exposure to the colitic agent.

Decision-making in the mammalian brain typically involves multiple brain structures within the midbrain, thalamus, striatum, limbic system, and cortex. Although task specific contributions of each brain region have been identified, neurons responding to reinforcement have been found throughout these structures. We sought to determine if any brain area, or cluster of areas, are the source of information, and if the fidelity of information varies among the areas. We recorded simultaneous field potentials (FPs) in rats from seven brain regions as they completed a binary choice task. The FPs of a 0.5 s window following reinforcement were given as input to a classifier that attempted to predict whether or not the rat received reward on each trial. The classifier correctly categorized reward on 77% of trials. Any region-specific signal could be omitted without lowering accuracy. Frequencies above 40 Hz and signals recorded later than 0.25 s following reinforcement were necessary to achieve this accuracy. Further, the classifier was able to predict reinforcement outcome above chance levels when using FPs from any single recorded brain region. Some combinations of structures, however, were more predictive than others. Analysis of FPs prior to reward revealed most regions reflected the prior probability of reward. Lastly, analyses of information flow suggested reinforcement information does not originate within a single structure of the network, within the resolution afforded by FP recordings. These data suggest reward delivery information is rapidly distributed non-uniformly across the network, and there is no canonical flow of information about reward events in the recorded structures.

Oscillatory activity is a ubiquitous property of brain signals, and yet relatively few studies have investigated how the phase of such ongoing oscillations affects our cognition. One of the main findings in this field is that the phase of electroencephalography (EEG) in the alpha band can affect perception of milliseconds-long stimuli. However, the importance of the phase of EEG for processing more naturalistic stimuli, which have a much longer duration, is still not clear. To address this question here, we presented word-nonword pairs, each of which was visible for 5 s and measured the effect of EEG phase during stimulus onset on later memory recall. The task consisted of an encoding (learning) phase in which 20 novel word-nonword pairs were presented, followed by a test phase in which participants were shown one of the seen words with four target nonwords to choose from. We found that memory recall performance was higher when the words during encoding were presented at a descending phase of the theta oscillation. This effect was the strongest in the frontal cortex. These results suggest that the phase of ongoing cortical activity can affect memorization of seconds-long stimuli that are an integral part of many daily tasks.

Background The physical distance between predator and prey is a primary determinant of behavior, yet few paradigms exist to study this reliably in rodents. New method The utility of a robotically controlled laser for use in a predator–prey-like (PPL) paradigm was explored for use in rats. This involved the construction of a robotic two-dimensional gimbal to dynamically position a laser beam in a behavioral test chamber. Custom software was used to control the trajectory and final laser position in response to user input on a console. The software also detected the location of the laser beam and the rodent continuously so that the dynamics of the distance between them could be analyzed. When the animal or laser beam came within a fixed distance the animal would either be rewarded with electrical brain stimulation or shocked subcutaneously. Results Animals that received rewarding electrical brain stimulation could learn to chase the laser beam, while animals that received aversive subcutaneous shock learned to actively avoid the laser beam in the PPL paradigm. Mathematical computations are presented which describe the dynamic interaction of the laser and rodent. Comparison with existing methods The robotic laser offers a neutral stimulus to train rodents in an open field and is the first device to be versatile enough to assess distance between predator and prey in real time. Conclusions With ongoing behavioral testing this tool will permit the neurobiological investigation of predator/prey-like relationships in rodents, and may have future implications for prosthetic limb development through brain–machine interfaces.

Problem gambling is thought to be highly comorbid with attention-deficit hyperactivity disorder (ADHD). We propose that the neurobiological pathologies underlying problem gambling overlap with those in ADHD. In this study, we used a simplified computerized version of the Iowa Gambling Task (IGT) to assess differences in reinforcement-driven choice adaptation among participants with pathological gambling and/or ADHD. The task contained two choice options with different net payouts over the session; a good bet that resulted in a win of +50 points on 60% of trials (and -50 points on 40%), and a bad bet that resulted in +100 points on 40% of the trials (and -100 points on 60%). We quantified participants' preference for the good bet over the session and their sensitivity to reinforcement. Both the control subjects and medicated ADHD nongamblers significantly increased the proportion of good bets over the 400-trial session. Subjects with problem gambling performed worse than controls and ADHD nongamblers, but better than our limited sample of unmedicated ADHD gamblers. Control subjects, medicated ADHD nongamblers, and unmedicated ADHD nongamblers tended to tolerate losses following good bets, whereas unmedicated ADHD gamblers tended to tolerate losses following bad bets. These data reveal that ADHD, particularly when treated with medication, is not associated with poor choices on the IGT, but may exacerbate pathological choices in problem gamblers. It seems that stabilization of dopamine signaling that occurs when ADHD is treated is itself also a treatment for certain forms of problem gambling.

Gambling studies have described a "near-miss effect" wherein the experience of almost winning increases gambling persistence. The near-miss has been proposed to inflate the value of preceding actions through its perceptual similarity to wins. We demonstrate here, however, that it acts as a conditioned stimulus to positively or negatively influence valuation, dependent on reward expectation and cognitive engagement. When subjects are asked to choose between two simulated slot machines, near-misses increase valuation of machines with a low payout rate, whereas they decrease valuation of high payout machines. This contextual effect impairs decisions and persists regardless of manipulations to outcome feedback or financial incentive provided for good performance. It is consistent with proposals that near-misses cause frustration when wins are expected, and we propose that it increases choice stochasticity and overrides avoidance of low-valued options. Intriguingly, the near-miss effect disappears when subjects are required to explicitly value machines by placing bets, rather than choosing between them. We propose that this task increases cognitive engagement and recruits participation of brain regions involved in cognitive processing, causing inhibition of otherwise dominant systems of decision-making. Our results reveal that only implicit, rather than explicit strategies of decision-making are affected by near-misses, and that the brain can fluidly shift between these strategies according to task demands.

We used focal brain lesions in rats to examine how dorsomedial (DMS) and dorsolateral (DLS) regions of the striatum differently contribute to response adaptation driven by the delivery or omission of rewards. Rats performed a binary choice task under two modes: one in which responses were rewarded on half of the trials regardless of choice; and another 'competitive' one in which only unpredictable choices were rewarded. In both modes, control animals were more likely to use a predictable lose-switch strategy than animals with lesions of either DMS or DLS. Animals with lesions of DMS presumably relied more on DLS for behavioural control, and generated repetitive responses in the first mode. These animals then shifted to a random response strategy in the competitive mode, thereby performing better than controls or animals with DLS lesions. Analysis using computational models of reinforcement learning indicated that animals with striatal lesions, particularly of the DLS, had blunted reward sensitivity and less stochasticity in the choice mechanism. These results provide further evidence that the rodent DLS is involved in rapid response adaptation that is more sophisticated than that embodied by the classic notion of habit formation driven by gradual stimulus-response learning.

Experience Versus Models There is an ongoing debate over what the orbitofrontal cortex contributes to behavior, learning, and decision-making. Jones et al. (p. 953 ) found that the orbitofrontal cortex was important for value-based computations when value must be inferred from an associative model of the task but not when value estimates based on previous experience are sufficient. This result calls into question the assumption that this region simply signals economic value. However, it would be consistent with a concept of the orbitofrontal cortex as being important for constructing model-based representations of the world that are orthogonal to value.

Animals quickly learn to approach sources of food. Here, we report on a form of approach in which rats made volitional orofacial contact with inactive feeders between trials of a self-paced operant task. This extraneous feeder sampling (EFS) was never reinforced and therefore imposed an opportunity and effort cost. EFS decreased during initial training but persisted thereafter. The relative rate of EFS to operant responding increased with novel changes to the operant chamber, reward devaluation by prefeeding, or lesions to the dorsolateral striatum. We speculate that this may function to increase exploration when the task is uncertain (early in learning or introduction of novel apparatus components), when the opportunity cost is low, or when the learned sensorimotor solution is compromised. Moreover, EFS strongly affected subsequent choices by triggering a lose-shift response away from the sampled feeder, even though it occurred outside of the trial context. This indicates that at least some behaviors occurring between trials impact future behaviors and should be considered in decision-making studies.

Basal ganglia circuits are organized as parallel loops that have been proposed to compete in a winner-take-all fashion to determine the appropriate behavioral outcome. However, limited experimental support for strong lateral inhibition mechanisms within striatal regions questions this model. Here, stimulation of the prefrontal cortex (PFC) using naturally occurring bursty patterns inhibited firing in most nucleus accumbens (NA) projection neurons. When an excitatory response was observed for one stimulation site, neighboring PFC sites evoked inhibition in the same neuron. Furthermore, PFC stimulation activated interneurons, and PFC-evoked inhibition was blocked by GABAA antagonists in corticoaccumbens slice preparations. Thus bursting PFC activity recruits local inhibition in the NA, shaping responses of projection neurons with a topographical arrangement that allows inhibition among parallel corticoaccumbens channels. The data indicate a high order of information processing within striatal circuits that should be considered in models of basal ganglia function and disease.

Hippocampal inputs to the nucleus accumbens (NA) have been proposed to implement a gating mechanism by driving NA medium spiny neurons (MSNs) to depolarized up states that facilitate action potential firing in response to brief activation of the prefrontal cortex (PFC). Brief PFC stimulation alone, on the other hand, could not drive NA up states. As these studies were conducted using single-pulse PFC stimulation, it remains possible that PFC activation with naturalistic, bursty patterns can also drive up states in NA MSNs. Here, we assessed NA responses to PFC stimulation with a pattern similar to what is typically observed in awake animals during PFC-relevant behaviors. In vivo intracellular recordings from NA MSNs revealed that brief 20-50 Hz PFC stimulus trains evoked depolarizations that were similar to spontaneous up states in NA MSNs and were sustained beyond stimulus offset. Similar train stimulation of corticoaccumbens afferents in a parasagittal slice preparation evoked large amplitude depolarizations in NA MSNs that were sustained during stimulation but decayed rapidly following stimulation offset, suggesting that activation of cortical afferents can drive MSN depolarizations but other mechanisms may contribute to sustaining up states. These data suggest that NA MSNs integrate temporal features of PFC activation and that the NA gating model can be reformulated to include a PFC-driven gating mechanism during periods of high PFC firing, such as during cognitively demanding tasks. Synapse 63:173-180, 2009. (c) 2008 Wiley-Liss, Inc.

Prenatal stress (PNS) is a significant risk factor for the development of psychopathology in adulthood such as anxiety, depression, schizophrenia and addiction. Animal models of PNS resemble many of the effects of PNS on humans and provide a means to study the accumulated effects of PNS over several generations on brain function. Here, we examined how mild PNS delivered during the third week in utero over four consecutive generations affects behavioral flexibility and functional signaling among cortical and limbic structures. These multi-generational prenatally stressed (MGPNS) rats were not impaired on an odor-cued reversal learning task as compared to control animals. Unilateral field potential (FP) recordings from the medial prefrontal cortex, basolateral amygdala, ventral hippocampus, and striatal territories revealed widespread differences in brain signaling between these groups during the odor sampling phase of the task. The FP power was significantly lower in most structures across most frequency bands in MGPNS animals, and the relative increase in power from baseline during the task was lower for the beta band (12–30 Hz) in MGPNS animals as compared to controls. The coherence of FPs between brain regions, however, was much higher in MGPNS animals among all structures and for most frequency bands. We propose that this pattern of changes in brain signaling reflects a simplification of network processing, which is consistent with reports of reduced spine density and dendritic complexity in the brains of animals receiving PNS. Our data support the proposal that recurrent ancestral stress leads to adaptations in the brain, and that these may confer adaptive behavior in some circumstances as compared to single-generation PNS.

The propensity of animals to shift choices immediately after unexpectedly poor reinforcement outcomes is a pervasive strategy across species and tasks. We report here that the memory supporting such lose-shift responding in rats rapidly decays during the intertrial interval and persists throughout training and testing on a binary choice task, despite being a suboptimal strategy. Lose-shift responding is not positively correlated with the prevalence and temporal dependence of win-stay responding, and it is inconsistent with predictions of reinforcement learning on the task. These data provide further evidence that win-stay and lose-shift are mediated by dissociated neural mechanisms and indicate that lose-shift responding presents a potential confound for the study of choice in the many operant choice tasks with short intertrial intervals. We propose that this immediate lose-shift responding is an intrinsic feature of the brain’s choice mechanisms that is engaged as a choice reflex and works in parallel with reinforcement learning and other control mechanisms to guide action selection.

The prefrontal cortex (PFC) is critical for decision making, and it becomes dysfunctional in many neuropsychiatric disorders. Studies in schizophrenia patients and relevant animal models suggest loss of PFC inhibitory interneuron function. For instance, rats with a neonatal ventral hippocampal lesion (NVHL) show a deficient modulation of PFC interneurons by dopamine (DA). Whether the PFC becomes disinhibited in this model and alters decision making remains to be determined. Here, we recorded neural activity in the medial PFC of NVHL rats during a reward-discounting choice task that activated DA systems. Rats were trained to sample odors that instructed them to select one of two feeders that delivered unequal amounts of liquid. Putative pyramidal neurons in the PFC were hyperactive whereas task-related field potential oscillations were significantly reduced in NVHL rats, consistent with impaired interneuron activation by DA during odor sampling leading to disorganized processing. Cognitive flexibility was tested by examining response bias and errors after reversing reward outcomes. NVHL rats demonstrated impaired flexibility as they were less able to track changes in reward outcome and made more response errors than controls did. Reducing cortical excitability with the metabotropic glutamate receptor 2/3 agonist LY379268 (1 mg/kg, i.p.) improved behavioral flexibility in NVHL rats but not controls. Furthermore, D2 dopamine receptors were involved, as the antagonist eticlopride (0.02 mg/kg, i.p.) reduced the ability to switch only in control animals. We conclude that NVHL rats present PFC disinhibition, which affects neural information processing and the selection of appropriate behavioral responses.

Reward intake optimization requires a balance between exploiting known sources of rewards and exploring for new sources. The prefrontal cortex (PFC) and associated basal ganglia circuits are likely candidates as neural structures responsible for such balance, while the hippocampus may be responsible for spatial/contextual information. Although studies have assessed interactions between hippocampus and PFC, and between hippocampus and the nucleus accumbens (NA), it is not known whether 3-way interactions among these structures vary under different behavioral conditions. Here, we investigated these interactions with multichannel recordings while rats explored an operant chamber and while they performed a learned lever-pressing task for reward in the same chamber shortly afterward. Neural firing and local field potentials in the NA core synchronized with hippocampal activity during spatial exploration, but during lever pressing they instead synchronized more strongly with the PFC. The latter is likely due to transient drive of NA neurons by bursting prefrontal activation, as in vivo intracellular recordings in anesthetized rats revealed that NA up states can transiently synchronize with spontaneous PFC activity and PFC stimulation with a bursting pattern reliably evoked up states in NA neurons. Thus, the ability to switch synchronization in a task-dependent manner indicates that the NA core can dynamically select its inputs to suit environmental demands, thereby contributing to decision-making, a function that was thought to primarily depend on the PFC.