Search

Abstract The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference learning (TD) learning, whereby certain units signal reward prediction errors (RPE). The TD algorithm has been traditionally mapped onto the dopaminergic system, as firing properties of dopamine neurons can resemble RPEs. However, certain predictions of TD learning are inconsistent with experimental results, and previous implementations of the algorithm have made unscalable assumptions regarding stimulus-specific fixed temporal bases. We propose an alternate framework to describe dopamine signaling in the brain, FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, dopamine release is similar, but not identical to RPE, leading to predictions that contrast to those of TD. While FLEX itself is a general theoretical framework, we describe a specific, biophysically plausible implementation, the results of which are consistent with a preponderance of both existing and reanalyzed experimental data.

Traditional synaptic plasticity experiments and models depend on tight temporal correlations between pre- and postsynaptic activity. These tight temporal correlations, on the order of tens of milliseconds, are incompatible with significantly longer behavioral time scales, and as such might not be able to account for plasticity induced by behavior. Indeed, recent findings in hippocampus suggest that rapid, bidirectional synaptic plasticity which modifies place fields in CA1 operates at behavioral time scales. These experimental results suggest that presynaptic activity generates synaptic eligibility traces both for potentiation and depression, which last on the order of seconds. These traces can be converted to changes in synaptic efficacies by the activation of an instructive signal that depends on naturally occurring or experimentally induced plateau potentials. We have developed a simple mathematical model that is consistent with these observations. This model can be fully analyzed to find the fixed points of induced place fields and how these fixed points depend on system parameters such as the size and shape of presynaptic place fields, the animal's velocity during induction, and the parameters of the plasticity rule. We also make predictions about the convergence time to these fixed points, both for induced and pre-existing place fields.

Multiple brain regions are able to learn and express temporal sequences, and this functionality is an essential component of learning and memory. We propose a substrate for such representations via a network model that learns and recalls discrete sequences of variable order and duration. The model consists of a network of spiking neurons placed in a modular microcolumn based architecture. Learning is performed via a biophysically realistic learning rule that depends on synaptic ‘eligibility traces’. Before training, the network contains no memory of any particular sequence. After training, presentation of only the first element in that sequence is sufficient for the network to recall an entire learned representation of the sequence. An extended version of the model also demonstrates the ability to successfully learn and recall non-Markovian sequences. This model provides a possible framework for biologically plausible sequence learning and memory, in agreement with recent experimental results.

Current models of word-production in Broca's area (i.e. left ventro-lateral prefrontal cortex, VLPFC) posit that sequential and staggered semantic, lexical, phonological and articulatory processes precede articulation. Using millisecond-resolution intra-cranial recordings, we evaluated spatiotemporal dynamics and high frequency functional interconnectivity between left VLPFC regions during single-word production. Through the systematic variation of retrieval, selection, and phonological loads, we identified specific activation profiles and functional coupling patterns between these regions that fit within current psycholinguistic theories of word production. However, network interactions underpinning these processes activate in parallel (not sequentially), while the processes themselves are indexed by specific changes in network state. We found evidence that suggests that pars orbitalis is coupled with pars triangularis during lexical retrieval, while lexical selection is terminated via coupled activity with M1 at articulation onset. Taken together, this work reveals that speech production relies on very specific inter-regional couplings in rapid sequence in the language dominant hemisphere.

The ability to maximize reward and avoid punishment is essential for animal survival. Reinforcement learning (RL) refers to the algorithms used by biological or artificial systems to learn how to maximize reward or avoid negative outcomes based on past experiences. While RL is also important in machine learning, the types of mechanistic constraints encountered by biological machinery might be different than those for artificial systems. Two major problems encountered by RL are how to relate a stimulus with a reinforcing signal that is delayed in time (temporal credit assignment), and how to stop learning once the target behaviors are attained (stopping rule). To address the first problem, synaptic eligibility traces were introduced, bridging the temporal gap between a stimulus and its reward. Although these were mere theoretical constructs, recent experiements have provided evidence of their existence. These experiments also reveal that the presence of specific neuromodulators converts the traces into changes in synaptic efficacy. A mechanistic implementation of the stopping rule usually assumes the inhibition of the reward nucleus; however, recent experimental results have shown that learning terminates at the appropriate network state even in setups where the reward cannot be inhibited. In an effort to describe a learning rule that solves the temporal credit assignment problem and implements a biologically plausible stopping rule, we proposed a model based on two separate synaptic eligibility traces, one for long-term potentiation (LTP) and one for long-term depression (LTD), each obeying different dynamics and having different effective magnitudes. The model has been shown to successfully generate stable learning in recurrent networks. Although the model assumes the presence of a single neuromodulator, evidence indicates that there are different neuromodulators for expressing the different traces. What could be the role of different neuromodulators for expressing the LTP and LTD traces? Here we expand on our previous model to include several neuromodulators, and illustrate through various examples how different neuromodulators contribute to learning reward-timing within a wide set of training paradigms and propose further roles that multiple neuromodulators can play in encoding additional information of the rewarding signal.

PKMζ is a persistently active PKC isoform proposed to maintain late-LTP and long-term memory. But late-LTP and memory are maintained without PKMζ in PKMζ-null mice. Two hypotheses can account for these findings. First, PKMζ is unimportant for LTP or memory. Second, PKMζ is essential for late-LTP and long-term memory in wild-type mice, and PKMζ-null mice recruit compensatory mechanisms. We find that whereas PKMζ persistently increases in LTP maintenance in wild-type mice, PKCι/λ, a gene-product closely related to PKMζ, persistently increases in LTP maintenance in PKMζ-null mice. Using a pharmacogenetic approach, we find PKMζ-antisense in hippocampus blocks late-LTP and spatial long-term memory in wild-type mice, but not in PKMζ-null mice without the target mRNA. Conversely, a PKCι/λ-antagonist disrupts late-LTP and spatial memory in PKMζ-null mice but not in wild-type mice. Thus, whereas PKMζ is essential for wild-type LTP and long-term memory, persistent PKCι/λ activation compensates for PKMζ loss in PKMζ-null mice.

Ziv Harel1,2, Ron Wald1,3, Jeff Perl1,3, Daniel Schwartz4, Chaim Bell2,31Division of Nephrology, St Michael's Hospital, University of Toronto, Toronto, Ontario, 2Department of Health Policy Management and Evaluation, University of Toronto, Toronto, Ontario, 3Department of Medicine and Keenan Research Centre in the Li Ka Shing Knowledge Institute of St Michael's Hospital, Toronto, Ontario, 4Division of Nephrology, University of British Columbia, Vancouver, British Columbia, CanadaBackground: Deficits in the transfer of information between inpatient and outpatient physicians are common and pose a patient safety risk. This is particularly the case for vulnerable populations such as patients with end-stage renal disease requiring dialysis. These patients have unique and complex health care needs that may not be effectively communicated on standard discharge summaries, which may result in potential medical errors and adverse events.Objective: To evaluate Canadian dialysis center directors' perceptions of deficiencies in the content and quality of hospital discharge summaries for dialysis patients.Methods: A web-based, cross-sectional survey of Canadian dialysis center directors was performed between September and November 2010. The survey consisted of three parts. The first part was designed to assess dialysis center directors' attitudes on the quality of discharge summaries they receive. The second part was designed to elicit respondents' preferences for discharge summary content, and the third part consisted of questions regarding demographic and practice information.Results: Of 79 dialysis center directors, 21 (27%) completed the survey. Sixty-two percent felt that current discharge summaries inadequately communicate dialysis-specific information. Receipt of antibiotics for line sepsis or peritonitis, modifications to vascular access, and changes in target weight/dialysis prescription were rated as essential dialysis-specific information to include in discharge summaries by respondents.Conclusion: Over three quarters of dialysis center directors find the current practice of transferring discharge information for hospitalized dialysis patients grossly inadequate. The inclusion of dialysis-specific information may improve the quality of discharge summaries for dialysis patients.Keywords: information transfer, dialysis patients, discharge information

Abstract Memory can last a lifetime, yet synaptic contacts that contribute to the storage of memory are composed of proteins that have much shorter lifetimes. A physiological model of memory formation, long‐term potentiation (LTP), has a late protein‐synthesis‐dependent phase (L‐LTP) that can last for many hours in slices or even for days in vivo. Could the activity‐dependent synthesis of new proteins account for the persistence of L‐LTP and memory? Here, we examine the proposal that a self‐sustaining regulation of translation can form a bistable switch that can persistently regulate the on‐site synthesis of plasticity‐related proteins. We show that an αCaMKII–CPEB1 molecular pair can operate as a bistable switch. Our results imply that L‐LTP should produce an increase in the total amount of αCaMKII at potentiated synapses. This study also proposes an explanation for why the application of protein synthesis and αCaMKII inhibitors at the induction and maintenance phases of L‐LTP result in very different outcomes.

The Scalar Timing Law, which is a temporal domain generalization of the well known Weber Law, states that the errors in estimating temporal intervals scale linearly with the durations of the intervals. Linear scaling has been studied extensively in human and animal models and holds over several orders of magnitude, though to date there is no agreed upon explanation for its physiological basis. Starting from the assumption that behavioral variability stems from neural variability, this work shows how to derive firing rate functions that are consistent with scalar timing. We show that firing rate functions with a log-power form, and a set of parameters that depend on spike count statistics, can account for scalar timing. Our derivation depends on a linear approximation, but we use simulations to validate the theory and show that log-power firing rate functions result in scalar timing over a large range of times and parameters.Simulation results also show that our theory as first posed exhibits a slight bias towards overestimation.We show that this bias can be corrected using a simple iterative approach to learn a decision threshold.

Spike timing dependent plasticity (STDP) is a phenomenon in which the precise timing of spikes affects the sign and magnitude of changes in synaptic strength. STDP is often interpreted as the comprehensive learning rule for a synapse - the “first law” of synaptic plasticity. This interpretation is made explicit in theoretical models in which the total plasticity produced by complex spike patterns results from a superposition of the effects of all spike pairs. Although such models are appealing for their simplicity, they can fail dramatically. For example, the measured single-spike learning rule between hippocampal CA3 and CA1 pyramidal neurons does not predict the existence of long-term potentiation. Layers of complexity have been added to the basic STDP model to repair predictive failures, but they have been outstripped by experimental data. We propose an alternate first law: neural activity triggers changes in key biochemical intermediates, which act as a more direct trigger of plasticity mechanisms. One particularly successful model uses intracellular calcium as the intermediate and can account for many observed properties of bidirectional plasticity. In this formulation, STDP is not itself the basis for explaining other forms of plasticity, but is instead a consequence of changes in the biochemical intermediate, calcium. Eventually a mechanism-based framework for learning rules should include other messengers, discrete change at individual synapses, spread of plasticity among neighboring synapses, and priming of hidden processes that change a synapse’s susceptibility to future change. Mechanism-based models provide a rich framework for the computational representation of synaptic plasticity.

BACKGROUND:Synaptic plasticity underlies many aspect of learning memory and development. The properties of synaptic plasticity can change as a function of previous plasticity and previous activation of synapses, a phenomenon called metaplasticity. Synaptic plasticity not only changes the functional connectivity between neurons but in some cases produces a structural change in synaptic spines; a change thought to form a basis for this observed plasticity. Here we examine to what extent structural plasticity of spines can be a cause for metaplasticity. This study is motivated by the observation that structural changes in spines are likely to affect the calcium dynamics in spines. Since calcium dynamics determine the sign and magnitude of synaptic plasticity, it is likely that structural plasticity will alter the properties of synaptic plasticity. METHODOLOGY/PRINCIPAL FINDINGS:In this study we address the question how spine geometry and alterations of N-methyl-D-aspartic acid (NMDA) receptors conductance may affect plasticity. Based on a simplified model of the spine in combination with a calcium-dependent plasticity rule, we demonstrated that after the induction phase of plasticity a shift of the long term potentiation (LTP) or long term depression (LTD) threshold takes place. This induces a refractory period for further LTP induction and promotes depotentiation as observed experimentally. That resembles the BCM metaplasticity rule but specific for the individual synapse. In the second phase, alteration of the NMDA response may bring the synapse to a state such that further synaptic weight alterations are feasible. We show that if the enhancement of the NMDA response is proportional to the area of the post synaptic density (PSD) the plasticity curves most likely return to the initial state. CONCLUSIONS/SIGNIFICANCE:Using simulations of calcium dynamics in synaptic spines, coupled with a biophysically motivated calcium-dependent plasticity rule, we find under what conditions structural plasticity can form the basis of synapse specific metaplasticity.

Memories that last a lifetime are thought to be stored, at least in part, as persistent enhancement of the strength of particular synapses. The synaptic mechanism of these persistent changes, late long-term potentiation (L-LTP), depends on the state and number of specific synaptic proteins. Synaptic proteins, however, have limited dwell times due to molecular turnover and diffusion, leading to a fundamental question: how can this transient molecular machinery store memories lasting a lifetime? Because the persistent changes in efficacy are synapse-specific, the underlying molecular mechanisms must to a degree reside locally in synapses. Extensive experimental evidence points to atypical protein kinase C (aPKC) isoforms as key components involved in memory maintenance. Furthermore, it is evident that establishing long-term memory requires new protein synthesis. However, a comprehensive model has not been developed describing how these components work to preserve synaptic efficacies over time. We propose a molecular model that can account for key empirical properties of L-LTP, including its protein synthesis dependence, dependence on aPKCs, and synapse-specificity. Simulations and empirical data suggest that either of the two aPKC subtypes in hippocampal neurons, PKM? and PKC?/?, can maintain L-LTP, making the system more robust. Given genetic compensation at the level of synthesis of these PKC subtypes as in knockout mice, this system is able to maintain L-LTP and memory when one of the pathways is eliminated.