Search

Learning the causes of rewards is necessary for survival. Thus, it is critical to understand the mechanisms of such a vital biological process. Cue-reward learning is controlled by mesolimbic dopamine and improves with spacing of cue-reward pairings. However, whether a mathematical rule governs such improvements in learning rate, and if so, whether a unifying mechanism captures this rule and dopamine dynamics during learning remain unknown. Here, we investigate the behavioral, algorithmic, and dopaminergic mechanisms governing cue-reward learning rate. Across a range of conditions in mice, we show a strong, mathematically proportional relationship between both behavioral and dopaminergic learning rates and the duration between rewards. Due to this relationship, removing up to 19 out of 20 cue-reward pairings over a fixed duration has no influence on overall learning. These findings are explained by a dopamine-based model of retrospective learning, thereby providing a unified account of the biological mechanisms of learning.

Compulsive behaviors are a hallmark symptom of obsessive compulsive disorder (OCD). Striatal hyperactivity has been linked to compulsive behavior generation in correlative studies in humans and causal studies in rodents. However, the contribution of the two distinct striatal output populations to the generation and treatment of compulsive behavior is unknown. These populations of direct and indirect pathway-projecting spiny projection neurons (SPNs) have classically been thought to promote or suppress actions, respectively, leading to a long-held hypothesis that increased output of direct relative to indirect pathway promotes compulsive behavior. Contrary to this hypothesis, here we find that indirect pathway hyperactivity is associated with compulsive grooming in the Sapap3-knockout mouse model of OCD-relevant behavior. Furthermore, we show that suppression of indirect pathway activity using optogenetics or treatment with the first-line OCD pharmacotherapy fluoxetine is associated with reduced grooming in Sapap3-knockouts. Together, these findings highlight the striatal indirect pathway as a potential treatment target for compulsive behavior., (© 2024. The Author(s).)

Learning to predict rewards is essential for the sustained fitness of animals. Contemporary views suggest that such learning is driven by a reward prediction error (RPE)-the difference between received and predicted rewards. The magnitude of learning induced by an RPE is proportional to the product of the RPE and a learning rate. Here we demonstrate using two-photon calcium imaging and optogenetics in mice that certain functionally distinct subpopulations of ventral/medial orbitofrontal cortex (vmOFC) neurons signal learning rate control. Consistent with learning rate control, trial-by-trial fluctuations in vmOFC activity positively correlate with behavioral updating when the RPE is positive, and negatively correlates with behavioral updating when the RPE is negative. Learning rate is affected by many variables including the salience of a reward. We found that the average reward response of these neurons signals the relative salience of a reward, because it decreases after reward prediction learning or the introduction of another highly salient aversive stimulus. The relative salience signaling in vmOFC is sculpted by medial thalamic inputs. These results support emerging theoretical views that prefrontal cortex encodes and controls learning parameters., Competing Interests: Declaration of interests The authors declare no competing interests., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Brain circuits are thought to form a "cognitive map" to process and store statistical relationships in the environment. A cognitive map is commonly defined as a mental representation that describes environmental states (i.e., variables or events) and the relationship between these states. This process is commonly conceptualized as a prospective process, as it is based on the relationships between states in chronological order (e.g., does reward follow a given state?). In this perspective, we expand this concept on the basis of recent findings to postulate that in addition to a prospective map, the brain forms and uses a retrospective cognitive map (e.g., does a given state precede reward?). In doing so, we demonstrate that many neural signals and behaviors (e.g., habits) that seem inflexible and non-cognitive can result from retrospective cognitive maps. Together, we present a significant conceptual reframing of the neurobiological study of associative learning, memory, and decision making., (Copyright © 2021. Published by Elsevier Inc.)

The prefrontal cortex is a critical neuroanatomical hub for controlling motivated behaviours across mammalian species. In addition to intra-cortical connectivity, prefrontal projection neurons innervate subcortical structures that contribute to reward-seeking behaviours, such as the ventral striatum and midline thalamus. While connectivity among these structures contributes to appetitive behaviours, how projection-specific prefrontal neurons encode reward-relevant information to guide reward seeking is unknown. Here we use in vivo two-photon calcium imaging to monitor the activity of dorsomedial prefrontal neurons in mice during an appetitive Pavlovian conditioning task. At the population level, these neurons display diverse activity patterns during the presentation of reward-predictive cues. However, recordings from prefrontal neurons with resolved projection targets reveal that individual corticostriatal neurons show response tuning to reward-predictive cues, such that excitatory cue responses are amplified across learning. By contrast, corticothalamic neurons gradually develop new, primarily inhibitory responses to reward-predictive cues across learning. Furthermore, bidirectional optogenetic manipulation of these neurons reveals that stimulation of corticostriatal neurons promotes conditioned reward-seeking behaviour after learning, while activity in corticothalamic neurons suppresses both the acquisition and expression of conditioned reward seeking. These data show how prefrontal circuitry can dynamically control reward-seeking behaviour through the opposing activities of projection-specific cell populations.

Several neuropsychiatric conditions, such as addiction and schizophrenia, may arise in part from dysregulated activity of ventral tegmental area dopaminergic (TH VTA ) neurons, as well as from more global maladaptation in neurocircuit function. However, whether TH VTA activity affects large-scale brain-wide function remains unknown. Here we selectively activated TH VTA neurons in transgenic rats and measured resulting changes in whole-brain activity using stimulus-evoked functional magnetic resonance imaging. Applying a standard generalized linear model analysis approach, our results indicate that selective optogenetic stimulation of TH VTA neurons enhanced cerebral blood volume signals in striatal target regions in a dopamine receptor-dependent manner. However, brain-wide voxel-based principal component analysis of the same data set revealed that dopaminergic modulation activates several additional anatomically distinct regions throughout the brain, not typically associated with dopamine release events. Furthermore, explicit pairing of TH VTA neuronal activation with a forepaw stimulus of a particular frequency expanded the sensory representation of that stimulus, not exclusively within the somatosensory cortices, but brain-wide. These data suggest that modulation of TH VTA neurons can impact brain dynamics across many distributed anatomically distinct regions, even those that receive little to no direct TH VTA input.

The habenula is a tiny brain region the size of a pea in humans. This region is highly conserved across vertebrates and has been traditionally overlooked by neuroscientists. The name habenula is derived from the Latin word habena, meaning "little rein", because of its elongated shape. Originally its function was thought to be related to the regulation of the nearby pineal gland (which Rene Descartes described as the "principal seat of the soul"). More recent evidence, however, demonstrates that the habenula acts as a critical neuroanatomical hub that connects and regulates brain regions important for divergent motivational states and cognition. In this Primer, we will discuss the recent and converging evidence that points to the habenula as a key brain region for motivation and decision-making., (Copyright © 2016. Published by Elsevier Ltd.)

Understanding the exploration patterns of foragers in the wild provides fundamental insight into animal behavior. Recent experimental evidence has demonstrated that path lengths (distances between consecutive turns) taken by foragers are well fitted by a power law distribution. Numerous theoretical contributions have posited that "Lévy random walks"-which can produce power law path length distributions-are optimal for memoryless agents searching a sparse reward landscape. It is unclear, however, whether such a strategy is efficient for cognitively complex agents, from wild animals to humans. Here, we developed a model to explain the emergence of apparent power law path length distributions in animals that can learn about their environments. In our model, the agent's goal during search is to build an internal model of the distribution of rewards in space that takes into account the cost of time to reach distant locations (i.e., temporally discounting rewards). For an agent with such a goal, we find that an optimal model of exploration in fact produces hyperbolic path lengths, which are well approximated by power laws. We then provide support for our model by showing that humans in a laboratory spatial exploration task search space systematically and modify their search patterns under a cost of time. In addition, we find that path length distributions in a large dataset obtained from free-ranging marine vertebrates are well described by our hyperbolic model. Thus, we provide a general theoretical framework for understanding spatial exploration patterns of cognitively complex foragers.

The ability to time intervals confers organisms, including humans, with many remarkable capabilities. A common method for studying interval timing is classification, in which a subject must indicate whether a given probe duration is nearer a previously learned short or long reference interval. This task is designed to reveal the probe duration that is equally likely to be labeled as short or long, known as the temporal bisection point. Studies have found that this bisection point is influenced by a variety of factors including the ratio of the target intervals, the spacing of the probe durations, the modalities of the stimuli, the attentional load, and the inter-trial duration. While several of these factors are thought to be mediated by memory effects, the prototypical classification task affords no opportunity to measure these memory effects directly. Here, we present a novel bisection task, termed the "Bisection by Classification and Production" (BiCaP) task, in which classification trials are interleaved with trials in which subjects must produce either the short or long referents or their midpoint. Using this method, we found a significant correlation between the means of the remembered referents and the bisection points for both classification and production trials. We then cross-validated the bisection points for production and classification trials by showing that they were not statistically differentiable. In addition to these population-level effects, we found within-subject evidence for co-variation across a session between the production bisection points and the means of the remembered referents. Finally, by using two sets of referent durations, we showed that only memory bias-corrected measures were consistent with a previously reported effect in which the ratio of the referents affects the location of the bisection point. These results suggest that memory effects should be considered in temporal tasks.

Most behaviors are generated in three steps: sensing the external world, processing that information to instruct decision-making, and producing a motor action. Sensory areas, especially primary sensory cortices, have long been held to be involved only in the first step of this sequence. Here, we develop a visually cued interval timing task that requires rats to decide when to perform an action following a brief visual stimulus. Using single-unit recordings and optogenetics in this task, we show that activity generated by the primary visual cortex (V1) embodies the target interval and may instruct the decision to time the action on a trial-by-trial basis. A spiking neuronal model of local recurrent connections in V1 produces neural responses that predict and drive the timing of future actions, rationalizing our observations. Our data demonstrate that the primary visual cortex may contribute to the instruction of visually cued timed actions., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Weber's law-the observation that the ability to perceive changes in magnitudes of stimuli is proportional to the magnitude-is a widely observed psychophysical phenomenon. It is also believed to underlie the perception of reward magnitudes and the passage of time. Since many ecological theories state that animals attempt to maximize reward rates, errors in the perception of reward magnitudes and delays must affect decision-making. Using an ecological theory of decision-making (TIMERR), we analyze the effect of multiple sources of noise (sensory noise, time estimation noise, and integration noise) on reward magnitude and subjective value perception. We show that the precision of reward magnitude perception is correlated with the precision of time perception and that Weber's law in time estimation can lead to Weber's law in value perception. The strength of this correlation is predicted to depend on the reward history of the animal. Subsequently, we show that sensory integration noise (either alone or in combination with time estimation noise) also leads to Weber's law in reward magnitude perception in an accumulator model, if it has balanced Poisson feedback. We then demonstrate that the noise in subjective value of a delayed reward, due to the combined effect of noise in both the perception of reward magnitude and delay, also abides by Weber's law. Thus, in our theory we prove analytically that the perception of reward magnitude, time, and subjective value change all approximately obey Weber's law.

Animals and humans make decisions based on their expected outcomes. Since relevant outcomes are often delayed, perceiving delays and choosing between earlier vs. later rewards (intertemporal decision-making) is an essential component of animal behavior. The myriad observations made in experiments studying intertemporal decision-making and time perception have not yet been rationalized within a single theory. Here we present a theory-Training-Integrated Maximized Estimation of Reinforcement Rate (TIMERR)-that explains a wide variety of behavioral observations made in intertemporal decision-making and the perception of time. Our theory postulates that animals make intertemporal choices to optimize expected reward rates over a limited temporal window which includes a past integration interval-over which experienced reward rate is estimated-as well as the expected delay to future reward. Using this theory, we derive mathematical expressions for both the subjective value of a delayed reward and the subjective representation of the delay. A unique contribution of our work is in finding that the past integration interval directly determines the steepness of temporal discounting and the non-linearity of time perception. In so doing, our theory provides a single framework to understand both intertemporal decision-making and time perception.

Xenopus NO38 is an abundant nucleolar chaperone and a member of the nucleoplasmin (Np) family. Here, we report high-resolution crystal structures of the N-terminal domain of NO38, as a pentamer and a decamer. As expected, NO38 shares the Np family fold. In addition, NO38- and Np-core pentamers each use highly conserved residues and numerous waters to form their respective decamers. Further studies show that NO38 and Np each bind equal amounts of the four core histones. However, NO38 prefers the (H3-H4)(2) tetramer, while Np probably prefers H2A-H2B dimers. We also show that NO38 and Np will each bind noncognate histones when the preferred partner is absent. We suggest that these chaperones must form decamers in order to bind histones and differentiate between histone tetramers and dimers. When taken together, these data imply that NO38 may function as a histone chaperone in the nucleolus.

The nucleoplasmin-like protein from Drosophila (dNLP) functions as a chaperone for core histones and may remodel chromatin in embryos. We now report the crystal structure of a dNLP-core pentamer at 1.5 A resolution. The monomer has an eight-stranded, beta barrel topology that is similar to nucleoplasmin (Np). However, a signature beta hairpin is tucked in along the lateral surface of the dNLP-core pentamer, while it extends outward in the Np-core decamer. Drosophila NLP and Np both assemble histone octamers. This process may require each chaperone to form a decamer, which would create symmetric binding sites for the histones. Conformational differences between dNLP and Np may reflect their different oligomeric states, while a conserved, nonpolar subunit interface may allow conformational plasticity during histone binding.