Search

With the goal of understanding behavioral mechanisms of generalization, we analyzed the ability of neural networks to generalize across context. We modeled a behavioral task where the correct responses to a set of specific sensory stimuli varied systematically across different contexts. The correct response depended on the stimulus (A,B,C,D) and context quadrant (1,2,3,4). The possible 16 stimulus-context combinations were associated with one of two responses (X,Y), one of which was correct for half of the combinations. The correct responses varied symmetrically across contexts. This allowed responses to previously unseen stimuli (probe stimuli) to be generalized from stimuli that had been presented previously. By testing the simulation on two or more stimuli that the network had never seen in a particular context, we could test whether the correct response on the novel stimuli could be generated based on knowledge of the correct responses in other contexts. We tested this generalization capability with a Deep Belief Network (DBN), Multi-Layer Perceptron (MLP) network, and the combination of a DBN with a linear perceptron (LP). Overall, the combination of the DBN and LP had the highest success rate for generalization.

Since the discovery of grid cells in rat entorhinal cortex, many models of their hexagonally arrayed spatial firing fields have been suggested. We review the models and organize them according to the mechanisms they use to encode position, update the positional code, read it out in the spatial grid pattern, and learn any patterned synaptic connections needed. We mention biological implementations of the models, but focus on the models on Marr's algorithmic level, where they are not things to individually prove or disprove, but rather are a valuable collection of metaphors of the grid cell system for guiding research that are all likely true to some degree, with each simply emphasizing different aspects of the system. For the convenience of interested researchers, MATLAB implementations of the discussed grid cell models are provided at ModelDB accession 144006 or http://people.bu.edu/zilli/gridmodels.html.

One of the two primary classes of models of grid cell spatial firing uses interference between oscillators at dynamically modulated frequencies. Generally, these models are presented in terms of idealized oscillators (modeled as sinusoids), which differ from biological oscillators in multiple important ways. Here we show that two more realistic, noisy neural models (Izhikevich's simple model and a biophysical model of an entorhinal cortex stellate cell) can be successfully used as oscillators in a model of this type. When additive noise is included in the models such that uncoupled or sparsely coupled cells show realistic interspike interval variance, both synaptic and gap-junction coupling can synchronize networks of cells to produce comparatively less variable network-level oscillations. We show that the frequency of these oscillatory networks can be controlled sufficiently well to produce stable grid cell spatial firing on the order of at least 2-5 min, despite the high noise level. Our results suggest that the basic principles of oscillatory interference models work with more realistic models of noisy neurons. Nevertheless, a number of simplifications were still made and future work should examine increasingly realistic models.

Performance of memory tasks is impaired by lesions to either the medial prefrontal cortex (mPFC) or the hippocampus (HPC); although how these two areas contribute to successful performance is not well understood. mPFC unit activity is temporally affected by hippocampal-theta oscillations, with almost half the mPFC population entrained to theta in behaving animals, pointing to theta interactions as the mechanism enabling collaborations between these two areas. mPFC neurons respond to sensory stimuli and responses in working memory tasks, though the function of these correlated firing rate changes remains unclear because similar responses are reported during mPFC dependent and independent tasks. Using a DNMS task we compared error trials vs. correct trials and found almost all mPFC cells fired at similar rates during both error and correct trials (92%), however theta-entrainment of mPFC neurons declined during error performance as only 17% of cells were theta-entrained (during correct trials 46% of the population was theta-entrained). Across the population, error and correct trials did not differ in firing rate, but theta-entrainment was impaired. Periods of theta-entrainment and firing rate changes appeared to be independent variables, and only theta-entrainment was correlated with successful performance, indicating mPFC-HPC theta-range interactions are the key to successful DNMS performance.

Models of the hexagonally arrayed spatial activity pattern of grid cell firing in the literature generally fall into two main categories: continuous attractor models or oscillatory interference models. Burak and Fiete (2009, PLoS Comput Biol) recently examined noise in two continuous attractor models, but did not consider oscillatory interference models in detail. Here we analyze an oscillatory interference model to examine the effects of noise on its stability and spatial firing properties. We show analytically that the square of the drift in encoded position due to noise is proportional to time and inversely proportional to the number of oscillators. We also show there is a relatively fixed breakdown point, independent of many parameters of the model, past which noise overwhelms the spatial signal. Based on this result, we show that a pair of oscillators are expected to maintain a stable grid for approximately t = 5mu(3)/(4pisigma)(2) seconds where mu is the mean period of an oscillator in seconds and sigma(2) its variance in seconds(2). We apply this criterion to recordings of individual persistent spiking neurons in postsubiculum (dorsal presubiculum) and layers III and V of entorhinal cortex, to subthreshold membrane potential oscillation recordings in layer II stellate cells of medial entorhinal cortex and to values from the literature regarding medial septum theta bursting cells. All oscillators examined have expected stability times far below those seen in experimental recordings of grid cells, suggesting the examined biological oscillators are unfit as a substrate for current implementations of oscillatory interference models. However, oscillatory interference models can tolerate small amounts of noise, suggesting the utility of circuit level effects which might reduce oscillator variability. Further implications for grid cell models are discussed.

Neurophysiological data reveals intrinsic cellular properties that suggest how entorhinal cortical neurons could code memory by the phase of their firing. Potential cellular mechanisms for this phase coding in models of entorhinal function are reviewed. This mechanism for phase coding provides a substrate for modeling the responses of entorhinal grid cells, as well as the replay of neural spiking activity during waking and sleep. Efforts to implement these abstract models in more detailed biophysical compartmental simulations raise specific issues that could be addressed in larger scale population models incorporating mechanisms of inhibition.

Behavioral tasks are often used to study the different memory systems present in humans and animals. Such tasks are usually designed to isolate and measure some aspect of a single memory system. However, it is not necessarily clear that any given task actually does isolate a system or that the strategy used by a subject in the experiment is the one desired by the experimenter. We have previously shown that when tasks are written mathematically as a form of partially observable Markov decision processes, the structure of the tasks provide information regarding the possible utility of certain memory systems. These previous analyses dealt with the disambiguation problem: given a specific ambiguous observation of the environment, is there information provided by a given memory strategy that can disambiguate that observation to allow a correct decision? Here we extend this approach to cases where multiple memory systems can be strategically combined in different ways. Specifically, we analyze the disambiguation arising from three ways by which episodic-like memory retrieval might be cued (by another episodic-like memory, by a semantic association, or by working memory for some earlier observation). We also consider the disambiguation arising from holding earlier working memories, episodic-like memories or semantic associations in working memory. From these analyses we can begin to develop a quantitative hierarchy among memory systems in which stimulus-response memories and semantic associations provide no disambiguation while the episodic memory system provides the most flexible disambiguation, with working memory at an intermediate level.

Researchers use a variety of behavioral tasks to analyze the effect of biological manipulations on memory function. This research will benefit from a systematic mathematical method for analyzing memory demands in behavioral tasks. In the framework of reinforcement learning theory, these tasks can be mathematically described as partially-observable Markov decision processes. While a wealth of evidence collected over the past 15 years relates the basal ganglia to the reinforcement learning framework, only recently has much attention been paid to including psychological concepts such as working memory or episodic memory in these models. This paper presents an analysis that provides a quantitative description of memory states sufficient for correct choices at specific decision points. Using information from the mathematical structure of the task descriptions, we derive measures that indicate whether working memory (for one or more cues) or episodic memory can provide strategically useful information to an agent. In particular, the analysis determines which observed states must be maintained in or retrieved from memory to perform these specific tasks. We demonstrate the analysis on three simplified tasks as well as eight more complex memory tasks drawn from the animal and human literature (two alternation tasks, two sequence disambiguation tasks, two non-matching tasks, the 2-back task, and the 1-2-AX task). The results of these analyses agree with results from quantitative simulations of the task reported in previous publications and provide simple indications of the memory demands of the tasks which can require far less computation than a full simulation of the task. This may provide a basis for a quantitative behavioral stoichiometry of memory tasks.

Lesch-Nyhan syndrome is a rare genetic disorder, caused by a mutation in the gene coding for the enzyme hypoxanthine-guanine phosphoribosyltransferase, which is characterized by hyperuricemia and its associated symptoms along with motor disorders and compulsive self-mutilation. We show that the temporal difference learning algorithm that has been often used to interpret dopaminergic activity in the basal ganglia offers an explanation for the self-mutilation behaviors. We propose that a dysfunctional dopamine signal inadvertently reinforces early, accidental injurious behavior that is initially caused by clumsiness owing to the motor disorders. Simulations of this model reproduce findings on the results of behavioral treatments for dealing with self-mutilation behaviors.

The mechanisms of goal-directed behavior have been studied using reinforcement learning theory, but these theoretical techniques have not often been used to address the role of memory systems in performing behavioral tasks. This work addresses this shortcoming by providing a way in which working memory (WM) and episodic memory may be included in the reinforcement learning framework, then simulating the successful acquisition and performance of six behavioral tasks, drawn from or inspired by the rat experimental literature, that require WM or episodic memory for correct performance. With no delay imposed during the tasks, simulations with WM can solve all of the tasks at above the chance level. When a delay is imposed, simulations with both episodic memory and WM can solve all of the tasks except a disambiguation of odor sequences task., ((c) 2007 Wiley-Liss, Inc.)

Grid cells in layer II of rat entorhinal cortex fire to spatial locations in a repeating hexagonal grid, with smaller spacing between grid fields for neurons in more dorsal anatomical locations. Data from in vitro whole-cell patch recordings showed differences in frequency of subthreshold membrane potential oscillations in entorhinal neurons that correspond to different positions along the dorsal-to-ventral axis, supporting a model of physiological mechanisms for grid cell responses.

The hippocampal theta rhythm is a prominent oscillation in the field potential observed throughout the hippocampus as a rat investigates stimuli in the environment. A recent computational model [Hasselmo, M. E., Bodelon, C., & Wyble, B. P. (2002a). A proposed function for hippocampal theta rhythm: separate phases of encoding and retrieval enhance reversal of prior learning. Neural Computation, 14, 793-817. Neuromodulation, theta rhythm and rat spatial navigation. Neural Networks, 15, 689-707] suggested that the theta rhythm allows the hippocampal formation to alternate rapidly between conditions that promote memory encoding (strong synaptic input from entorhinal cortex to areas CA3 and CA1) and conditions that promote memory retrieval (strong synaptic input from CA3 to CA1). That model predicted that the preferred theta phase of CA1 spiking should differ for information being encoded versus information being retrieved. In the present study, the spiking activity of CA1 pyramidal cells was recorded while rats performed either an odor-cued delayed nonmatch-to-sample recognition memory test or an object recognition memory task based on the animal's spontaneous preference for novelty. In the test period of both tasks, the preferred theta phase exhibited by CA1 pyramidal cells differed between moments when the rat inspected repeated (match) and non-repeated (nonmatch) items. Also in the present study, additional modeling work extended the previous model to address the mean phase of CA1 spiking associated with stimuli inducing varying levels of retrieval relative to encoding, ranging from novel nonmatch stimuli with no retrieval to highly familiar repeated stimuli with extensive retrieval. The modeling results obtained here demonstrated that the experimentally observed phase differences are consistent with different levels of CA3 synaptic input to CA1 during recognition of repeated items.

Intracellular recording and computational modelling suggest that interactions of subthreshold membrane potential oscillation frequency in different dendritic branches of entorhinal cortex stellate cells could underlie the functional coding of continuous dimensions of space and time. Among other things, these interactions could underlie properties of grid cell field spacing. The relationship between experimental data on membrane potential oscillation frequency (f) and grid cell field spacing (G) indicates a constant scaling factor H = fG. This constant scaling factor between temporal oscillation frequency and spatial periodicity provides a starting constraint that is used to derive the model of Burgess et al. (Hippocampus, 2007). This model provides a consistent quantitative link between single cell physiological properties and properties of spiking units in awake behaving animals. Further properties and predictions of this model about single cell and network physiological properties are analyzed. In particular, the model makes quantitative predictions about the change in membrane potential, single cell oscillation frequency, and network oscillation frequency associated with speed of movement, about the independence of single cell properties from network theta rhythm oscillations, and about the effect of variations in initial oscillatory phase on the pattern of grid cell firing fields. These same mechanisms of subthreshold oscillations may play a more general role in memory function, by providing a method for learning arbitrary time intervals in memory sequences., ((c) 2007 Wiley-Liss, Inc.)

In a continuous T-maze alternation task, CA1 complex-spike neurons in the hippocampus differentially fire as the rat traverses overlapping segments of the maze (i.e., the stem) repeatedly via alternate routes. The temporal dynamics of this phenomenon were further investigated in the current study. Rats learned the alternation task from the first day of acquisition and the differential firing pattern in the stem was observed accordingly. More importantly, we report a phenomenon in which spatial correlates of CA1 neuronal ensembles gradually changed from their original firing locations, shifting toward prospective goal locations in the continuous T-maze alternation task. The relative locations of simultaneously recorded firing fields, however, were preserved within the ensemble spatial representation during this shifting. The within-session shifts in preferred firing locations in the absence of any changes in the environment suggest that certain cognitive factors can significantly alter the location-bound coding scheme of hippocampal neurons.

In a recent study, Manns et al. (2007) showed that CA1 population spiking activity differs in mean phase between conditions of exposure to stimuli of varying levels of familiarity. Here, we provide an analysis of a computational model of the hippocampus from Hasselmo et al. (2002) to examine how an animal's history of exposures to stimuli affects CA1 population activity. We show how the model can reproduce the major findings from the study by Manns et al. Specifically, we show that differences in direction and magnitude of the mean phase of population activity between two stimuli in the model depend on the order of presentation of the stimuli and the duration of exposure during each presentation. The analyses also reveal that the model's learning rule results in stability and normalization of synaptic weights and show how dysfunction of neuromodulatory systems might lead to epileptic or amnestic type conditions.

Both the hippocampus and the medial prefrontal cortex are essential for successful performance in learning- and memory-related tasks. Within the hippocampus the theta rhythm plays an integral role in the timing of action potentials of hippocampal neurons responding to elements of any given task. Medial prefrontal cortex (mPFC) neurons display firing rate changes to specific facets of behavioral tasks (Jung et al., 1998. Cereb Cortex 8:437--450). We recorded units in the mPFC and field potentials in the hippocampus to determine whether behaviorally correlated mPFC cells fired with phase relationships to the hippocampal theta rhythm. In two different behavioral tasks (running a linear track and foraging in two distinct environments) we found mPFC cells that alternated between theta entrained firing and nonphasic firing depending on the ongoing behavior, while other cells were modulated during all conditions in both tasks. The majority of the mPFC cells with a significant correlation of firing rate changes with behavior were entrained to hippocampal theta. Cells that fired to specific events during only one direction of running were predisposed to theta modulation only in that direction. mPFC neurons have the capability to respond to behaviorally relevant elements by dynamically alternating between hippocampal theta entrained and nonphasic firing., ((c) 2005 Wiley-Liss, Inc.)