Search

Learning the causes of rewards is necessary for survival. Thus, it is critical to understand the mechanisms of such a vital biological process. Cue-reward learning is controlled by mesolimbic dopamine and improves with spacing of cue-reward pairings. However, whether a mathematical rule governs such improvements in learning rate, and if so, whether a unifying mechanism captures this rule and dopamine dynamics during learning remain unknown. Here, we investigate the behavioral, algorithmic, and dopaminergic mechanisms governing cue-reward learning rate. Across a range of conditions in mice, we show a strong, mathematically proportional relationship between both behavioral and dopaminergic learning rates and the duration between rewards. Due to this relationship, removing up to 19 out of 20 cue-reward pairings over a fixed duration has no influence on overall learning. These findings are explained by a dopamine-based model of retrospective learning, thereby providing a unified account of the biological mechanisms of learning.

Compulsive behaviors are a hallmark symptom of obsessive compulsive disorder (OCD). Striatal hyperactivity has been linked to compulsive behavior generation in correlative studies in humans and causal studies in rodents. However, the contribution of the two distinct striatal output populations to the generation and treatment of compulsive behavior is unknown. These populations of direct and indirect pathway-projecting spiny projection neurons (SPNs) have classically been thought to promote or suppress actions, respectively, leading to a long-held hypothesis that increased output of direct relative to indirect pathway promotes compulsive behavior. Contrary to this hypothesis, here we find that indirect pathway hyperactivity is associated with compulsive grooming in the Sapap3-knockout mouse model of OCD-relevant behavior. Furthermore, we show that suppression of indirect pathway activity using optogenetics or treatment with the first-line OCD pharmacotherapy fluoxetine is associated with reduced grooming in Sapap3-knockouts. Together, these findings highlight the striatal indirect pathway as a potential treatment target for compulsive behavior., (© 2024. The Author(s).)

Learning to predict rewards is essential for the sustained fitness of animals. Contemporary views suggest that such learning is driven by a reward prediction error (RPE)-the difference between received and predicted rewards. The magnitude of learning induced by an RPE is proportional to the product of the RPE and a learning rate. Here we demonstrate using two-photon calcium imaging and optogenetics in mice that certain functionally distinct subpopulations of ventral/medial orbitofrontal cortex (vmOFC) neurons signal learning rate control. Consistent with learning rate control, trial-by-trial fluctuations in vmOFC activity positively correlate with behavioral updating when the RPE is positive, and negatively correlates with behavioral updating when the RPE is negative. Learning rate is affected by many variables including the salience of a reward. We found that the average reward response of these neurons signals the relative salience of a reward, because it decreases after reward prediction learning or the introduction of another highly salient aversive stimulus. The relative salience signaling in vmOFC is sculpted by medial thalamic inputs. These results support emerging theoretical views that prefrontal cortex encodes and controls learning parameters., Competing Interests: Declaration of interests The authors declare no competing interests., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Brain circuits are thought to form a "cognitive map" to process and store statistical relationships in the environment. A cognitive map is commonly defined as a mental representation that describes environmental states (i.e., variables or events) and the relationship between these states. This process is commonly conceptualized as a prospective process, as it is based on the relationships between states in chronological order (e.g., does reward follow a given state?). In this perspective, we expand this concept on the basis of recent findings to postulate that in addition to a prospective map, the brain forms and uses a retrospective cognitive map (e.g., does a given state precede reward?). In doing so, we demonstrate that many neural signals and behaviors (e.g., habits) that seem inflexible and non-cognitive can result from retrospective cognitive maps. Together, we present a significant conceptual reframing of the neurobiological study of associative learning, memory, and decision making., (Copyright © 2021. Published by Elsevier Inc.)

The prefrontal cortex is a critical neuroanatomical hub for controlling motivated behaviours across mammalian species. In addition to intra-cortical connectivity, prefrontal projection neurons innervate subcortical structures that contribute to reward-seeking behaviours, such as the ventral striatum and midline thalamus. While connectivity among these structures contributes to appetitive behaviours, how projection-specific prefrontal neurons encode reward-relevant information to guide reward seeking is unknown. Here we use in vivo two-photon calcium imaging to monitor the activity of dorsomedial prefrontal neurons in mice during an appetitive Pavlovian conditioning task. At the population level, these neurons display diverse activity patterns during the presentation of reward-predictive cues. However, recordings from prefrontal neurons with resolved projection targets reveal that individual corticostriatal neurons show response tuning to reward-predictive cues, such that excitatory cue responses are amplified across learning. By contrast, corticothalamic neurons gradually develop new, primarily inhibitory responses to reward-predictive cues across learning. Furthermore, bidirectional optogenetic manipulation of these neurons reveals that stimulation of corticostriatal neurons promotes conditioned reward-seeking behaviour after learning, while activity in corticothalamic neurons suppresses both the acquisition and expression of conditioned reward seeking. These data show how prefrontal circuitry can dynamically control reward-seeking behaviour through the opposing activities of projection-specific cell populations.

Several neuropsychiatric conditions, such as addiction and schizophrenia, may arise in part from dysregulated activity of ventral tegmental area dopaminergic (TH VTA ) neurons, as well as from more global maladaptation in neurocircuit function. However, whether TH VTA activity affects large-scale brain-wide function remains unknown. Here we selectively activated TH VTA neurons in transgenic rats and measured resulting changes in whole-brain activity using stimulus-evoked functional magnetic resonance imaging. Applying a standard generalized linear model analysis approach, our results indicate that selective optogenetic stimulation of TH VTA neurons enhanced cerebral blood volume signals in striatal target regions in a dopamine receptor-dependent manner. However, brain-wide voxel-based principal component analysis of the same data set revealed that dopaminergic modulation activates several additional anatomically distinct regions throughout the brain, not typically associated with dopamine release events. Furthermore, explicit pairing of TH VTA neuronal activation with a forepaw stimulus of a particular frequency expanded the sensory representation of that stimulus, not exclusively within the somatosensory cortices, but brain-wide. These data suggest that modulation of TH VTA neurons can impact brain dynamics across many distributed anatomically distinct regions, even those that receive little to no direct TH VTA input.

The habenula is a tiny brain region the size of a pea in humans. This region is highly conserved across vertebrates and has been traditionally overlooked by neuroscientists. The name habenula is derived from the Latin word habena, meaning "little rein", because of its elongated shape. Originally its function was thought to be related to the regulation of the nearby pineal gland (which Rene Descartes described as the "principal seat of the soul"). More recent evidence, however, demonstrates that the habenula acts as a critical neuroanatomical hub that connects and regulates brain regions important for divergent motivational states and cognition. In this Primer, we will discuss the recent and converging evidence that points to the habenula as a key brain region for motivation and decision-making., (Copyright © 2016. Published by Elsevier Ltd.)

Making decisions that factor the cost of time is fundamental to survival. Yet, while it is readily appreciated that our perception of time is intimately involved in this process, theories regarding intertemporal decision-making and theories regarding time perception are treated, largely, independently. Even within these respective domains, models providing good fits to data fail to provide insight as to why, from a normative sense, those fits should take their apparent form. Conversely, normative models that proffer a rationalization for why an agent should weigh options in a particular way, or to perceive time in a particular way, fail to account for the full body of well-established experimental evidence. Here we review select, yet key advances in our understanding, identifying conceptual breakthroughs in the fields of intertemporal decision-making and in time perception, as well as their limits and failings in the face of hard-won experimental observation. On this background of accrued knowledge, a new conception unifying the domains of decision-making and time perception is put forward (Training-Integrated Maximization of Reinforcement Rate, TIMERR) to provide a better fit to observations and a more parsimonious reckoning of why we make choices, and thereby perceive time, the way we do., (Copyright © 2016 Elsevier Ltd. All rights reserved.)

Understanding the exploration patterns of foragers in the wild provides fundamental insight into animal behavior. Recent experimental evidence has demonstrated that path lengths (distances between consecutive turns) taken by foragers are well fitted by a power law distribution. Numerous theoretical contributions have posited that "Lévy random walks"-which can produce power law path length distributions-are optimal for memoryless agents searching a sparse reward landscape. It is unclear, however, whether such a strategy is efficient for cognitively complex agents, from wild animals to humans. Here, we developed a model to explain the emergence of apparent power law path length distributions in animals that can learn about their environments. In our model, the agent's goal during search is to build an internal model of the distribution of rewards in space that takes into account the cost of time to reach distant locations (i.e., temporally discounting rewards). For an agent with such a goal, we find that an optimal model of exploration in fact produces hyperbolic path lengths, which are well approximated by power laws. We then provide support for our model by showing that humans in a laboratory spatial exploration task search space systematically and modify their search patterns under a cost of time. In addition, we find that path length distributions in a large dataset obtained from free-ranging marine vertebrates are well described by our hyperbolic model. Thus, we provide a general theoretical framework for understanding spatial exploration patterns of cognitively complex foragers.

Genetically encoded calcium indicators for visualizing dynamic cellular activity have greatly expanded our understanding of the brain. However, owing to the light-scattering properties of the brain, as well as the size and rigidity of traditional imaging technology, in vivo calcium imaging has been limited to superficial brain structures during head-fixed behavioral tasks. These limitations can now be circumvented by using miniature, integrated microscopes in conjunction with an implantable microendoscopic lens to guide light into and out of the brain, thus permitting optical access to deep brain (or superficial) neural ensembles during naturalistic behaviors. Here we describe steps to conduct such imaging studies using mice. However, we anticipate that the protocol can be easily adapted for use in other small vertebrates. Successful completion of this protocol will permit cellular imaging of neuronal activity and the generation of data sets with sufficient statistical power to correlate neural activity with stimulus presentation, physiological state and other aspects of complex behavioral tasks. This protocol takes 6-11 weeks to complete., Competing Interests: Dr. G.D. Stuber is the recipient of a DECODE grant from Inscopix, Palo Alto, CA.

It has been previously shown (Namboodiri, Mihalas, Marton, & Hussain Shuler, 2014) that an evolutionary theory of decision making and time perception is capable of explaining numerous behavioral observations regarding how humans and animals decide between differently delayed rewards of differing magnitudes and how they perceive time. An implementation of this theory using a stochastic drift-diffusion accumulator model (Namboodiri, Mihalas, & Hussain Shuler, 2014a) showed that errors in time perception and decision making approximately obey Weber's law for a range of parameters. However, prior calculations did not have a clear mechanistic underpinning. Further, these calculations were only approximate, with the range of parameters being limited. In this letter, we provide a full analytical treatment of such an accumulator model, along with a mechanistic implementation, to calculate the expression of these errors for the entirety of the parameter space. In our mechanistic model, Weber's law results from synaptic facilitation and depression within the feedback synapses of the accumulator. Our theory also makes the prediction that the steepness of temporal discounting can be affected by requiring the precise timing of temporal intervals. Thus, by presenting exact quantitative calculations, this work provides falsifiable predictions for future experimental testing.

The ability to time intervals confers organisms, including humans, with many remarkable capabilities. A common method for studying interval timing is classification, in which a subject must indicate whether a given probe duration is nearer a previously learned short or long reference interval. This task is designed to reveal the probe duration that is equally likely to be labeled as short or long, known as the temporal bisection point. Studies have found that this bisection point is influenced by a variety of factors including the ratio of the target intervals, the spacing of the probe durations, the modalities of the stimuli, the attentional load, and the inter-trial duration. While several of these factors are thought to be mediated by memory effects, the prototypical classification task affords no opportunity to measure these memory effects directly. Here, we present a novel bisection task, termed the "Bisection by Classification and Production" (BiCaP) task, in which classification trials are interleaved with trials in which subjects must produce either the short or long referents or their midpoint. Using this method, we found a significant correlation between the means of the remembered referents and the bisection points for both classification and production trials. We then cross-validated the bisection points for production and classification trials by showing that they were not statistically differentiable. In addition to these population-level effects, we found within-subject evidence for co-variation across a session between the production bisection points and the means of the remembered referents. Finally, by using two sets of referent durations, we showed that only memory bias-corrected measures were consistent with a previously reported effect in which the ratio of the referents affects the location of the bisection point. These results suggest that memory effects should be considered in temporal tasks.

Most behaviors are generated in three steps: sensing the external world, processing that information to instruct decision-making, and producing a motor action. Sensory areas, especially primary sensory cortices, have long been held to be involved only in the first step of this sequence. Here, we develop a visually cued interval timing task that requires rats to decide when to perform an action following a brief visual stimulus. Using single-unit recordings and optogenetics in this task, we show that activity generated by the primary visual cortex (V1) embodies the target interval and may instruct the decision to time the action on a trial-by-trial basis. A spiking neuronal model of local recurrent connections in V1 produces neural responses that predict and drive the timing of future actions, rationalizing our observations. Our data demonstrate that the primary visual cortex may contribute to the instruction of visually cued timed actions., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Weber's law-the observation that the ability to perceive changes in magnitudes of stimuli is proportional to the magnitude-is a widely observed psychophysical phenomenon. It is also believed to underlie the perception of reward magnitudes and the passage of time. Since many ecological theories state that animals attempt to maximize reward rates, errors in the perception of reward magnitudes and delays must affect decision-making. Using an ecological theory of decision-making (TIMERR), we analyze the effect of multiple sources of noise (sensory noise, time estimation noise, and integration noise) on reward magnitude and subjective value perception. We show that the precision of reward magnitude perception is correlated with the precision of time perception and that Weber's law in time estimation can lead to Weber's law in value perception. The strength of this correlation is predicted to depend on the reward history of the animal. Subsequently, we show that sensory integration noise (either alone or in combination with time estimation noise) also leads to Weber's law in reward magnitude perception in an accumulator model, if it has balanced Poisson feedback. We then demonstrate that the noise in subjective value of a delayed reward, due to the combined effect of noise in both the perception of reward magnitude and delay, also abides by Weber's law. Thus, in our theory we prove analytically that the perception of reward magnitude, time, and subjective value change all approximately obey Weber's law.

Animals and humans make decisions based on their expected outcomes. Since relevant outcomes are often delayed, perceiving delays and choosing between earlier vs. later rewards (intertemporal decision-making) is an essential component of animal behavior. The myriad observations made in experiments studying intertemporal decision-making and time perception have not yet been rationalized within a single theory. Here we present a theory-Training-Integrated Maximized Estimation of Reinforcement Rate (TIMERR)-that explains a wide variety of behavioral observations made in intertemporal decision-making and the perception of time. Our theory postulates that animals make intertemporal choices to optimize expected reward rates over a limited temporal window which includes a past integration interval-over which experienced reward rate is estimated-as well as the expected delay to future reward. Using this theory, we derive mathematical expressions for both the subjective value of a delayed reward and the subjective representation of the delay. A unique contribution of our work is in finding that the past integration interval directly determines the steepness of temporal discounting and the non-linearity of time perception. In so doing, our theory provides a single framework to understand both intertemporal decision-making and time perception.

Xenopus NO38 is an abundant nucleolar chaperone and a member of the nucleoplasmin (Np) family. Here, we report high-resolution crystal structures of the N-terminal domain of NO38, as a pentamer and a decamer. As expected, NO38 shares the Np family fold. In addition, NO38- and Np-core pentamers each use highly conserved residues and numerous waters to form their respective decamers. Further studies show that NO38 and Np each bind equal amounts of the four core histones. However, NO38 prefers the (H3-H4)(2) tetramer, while Np probably prefers H2A-H2B dimers. We also show that NO38 and Np will each bind noncognate histones when the preferred partner is absent. We suggest that these chaperones must form decamers in order to bind histones and differentiate between histone tetramers and dimers. When taken together, these data imply that NO38 may function as a histone chaperone in the nucleolus.

NO38 is a multidomain protein that belongs to the nucleoplasmin (Np) family. Previous studies have suggested that acidic chaperones such as Np may function as histone-storage platforms. Here, the purification and crystallization of the N-terminal domain of NO38 in two crystal forms is reported. The C2 crystal form diffracts to 1.9 A and contains two pentamers in the asymmetric unit, while the P1 crystals diffract to 1.7 A and contain a non-crystallographic decamer with 522 symmetry. By analogy with Np, the NO38 decamer may represent the active form of this chaperone.

The nucleoplasmin-like protein from Drosophila (dNLP) functions as a chaperone for core histones and may remodel chromatin in embryos. We now report the crystal structure of a dNLP-core pentamer at 1.5 A resolution. The monomer has an eight-stranded, beta barrel topology that is similar to nucleoplasmin (Np). However, a signature beta hairpin is tucked in along the lateral surface of the dNLP-core pentamer, while it extends outward in the Np-core decamer. Drosophila NLP and Np both assemble histone octamers. This process may require each chaperone to form a decamer, which would create symmetric binding sites for the histones. Conformational differences between dNLP and Np may reflect their different oligomeric states, while a conserved, nonpolar subunit interface may allow conformational plasticity during histone binding.

A complete three dimensional model (RCSB000408; PDB code 1qaa) for the LexA repressor dimer bound to the recA operator site consistent with relevant biochemical and biophysical data for the repressor is proposed. A model of interaction of the N-terminal operator binding domain 1-72 with the operator was available. We have modelled residues 106-202 of LexA on the basis of the crystal structure of a homologous protein, UmuD'. Residues 70-105 have been modelled by us, residues 70-77 comprising the real hinge, followed by a beta-strand and an alpha-helix, both interacting with the rest of the C-domain. The preexponential Arrhenius factor for the LexA autocleavage is shown to be approximately 10(9) s(-1) at 298K whereas the exponential factor is approximately 2 x 10(-12), demanding that the autocleavage site is quite close to the catalytic site but reaction is slow due to an activation energy barrier. We propose that in the operator bound form, Ala 84- Gly 85 is about 7-10A from the catalytic groups, but the reaction does not occur as the geometry is not suitable for a nucleophilic attack from Ser 119 Ogamma, since Pro 87 is held in the cis conformation. When pH is elevated or under the action of activated RecA, cleavage may occur following a cis --> trans isomerization at Pro 87 and/or a rotation of the region beta9-beta10 about beta7-beta8 following the disruption of two hydrogen bonds. We show that the C-C interaction comprises the approach of two negatively charged surfaces neutralized by sodium ions, the C-domains of the monomers making a new beta barrel at the interface burying 710A2 of total surface area of each monomer.

An extracellular 1,3-specific lipase with molecular weight of 35.5 kDa and an isoelectric point of 4.4 from Aspergillus niger has been purified 50-fold by pH precipitation followed by a series of chromatographic steps with an overall yield of 10%. The enzyme was homogeneous as judged by denaturing polyacrylamide gel electrophoresis and size-exclusion fast-performance liquid chromatography. It contained 2.8% sugar which was completely removed by endoglycosidase F treatment, and the deglycosylated enzyme retained full activity. The native lipase showed optimal activity between temperatures 35 and 55 degrees C and pH 5.0 and 6.0. The amino acid composition and the N-terminal sequence were found to be different from lipases previously purified from A. niger. The enzyme was resistant to trypsin, chymotrypsin, endoprotease Glu-C, thrombin, and papain under native conditions but was susceptible to cleavage by the same proteases when heat-denatured.