Search

Temporal-difference (TD) algorithms have been proposed as models of reinforcement learning (RL). We examine two issues of distributed representation in these TD algorithms: distributed representations of belief and distributed discounting factors. Distributed representation of belief allows the believed state of the world to distribute across sets of equivalent states. Distributed exponential discounting factors produce hyperbolic discounting in the behavior of the agent itself. We examine these issues in the context of a TD RL model in which state-belief is distributed over a set of exponentially-discounting "micro-Agents", each of which has a separate discounting factor (gamma). Each microAgent maintains an independent hypothesis about the state of the world, and a separate value-estimate of taking actions within that hypothesized state. The overall agent thus instantiates a flexible representation of an evolving world-state. As with other TD models, the value-error (delta) signal within the model matches dopamine signals recorded from animals in standard conditioning reward-paradigms. The distributed representation of belief provides an explanation for the decrease in dopamine at the conditioned stimulus seen in overtrained animals, for the differences between trace and delay conditioning, and for transient bursts of dopamine seen at movement initiation. Because each microAgent also includes its own exponential discounting factor, the overall agent shows hyperbolic discounting, consistent with behavioral experiments.

This mini-symposium aims to integrate recent insights from anatomy, behavior, and neurophysiology, highlighting the anatomical organization, behavioral significance, and information-processing mechanisms of corticostriatal interactions. In this summary of topics, which is not meant to provide a comprehensive survey, we will first review the anatomy of corticostriatal circuits, comparing different ways by which "loops" of cortical-basal ganglia circuits communicate. Next, we will address the causal importance and systems-neurophysiological mechanisms of corticostriatal interactions for memory, emphasizing the communication between hippocampus and ventral striatum during contextual conditioning. Furthermore, ensemble recording techniques have been applied to compare information processing in the dorsal and ventral striatum to predictions from reinforcement learning theory. We will next discuss how neural activity develops in corticostriatal areas when habits are learned. Finally, we will evaluate the role of GABAergic interneurons in dynamically transforming cortical inputs into striatal output during learning and decision making.

Local field potential (LFP) oscillations in the brain reflect organization thought to be important for perception, attention, movement, and memory. In the basal ganglia, including dorsal striatum, dysfunctional LFP states are associated with Parkinson's disease, while in healthy subjects, dorsal striatal LFPs have been linked to decision-making processes. However, LFPs in ventral striatum have been less studied. We report that in rats running a spatial decision task, prominent gamma-50 (45-55 Hz) and gamma-80 (70-85 Hz) oscillations in ventral striatum had distinct relationships to behavior, task events, and spiking activity. Gamma-50 power increased sharply following reward delivery and before movement initiation, while in contrast, gamma-80 power ramped up gradually to reward locations. Gamma-50 power was low and contained little structure during early learning, but rapidly developed a stable pattern, while gamma-80 power was initially high before returning to a stable level within a similar timeframe. Putative fast-spiking interneurons (FSIs) showed phase, firing rate, and coherence relationships with gamma-50 and gamma-80, indicating that the observed LFP patterns are locally relevant. Furthermore, in a number of FSIs such relationships were specific to gamma-50 or gamma-80, suggesting that partially distinct FSI populations mediate the effects of gamma-50 and gamma-80.

Recordings of rat hippocampal place cells have provided information about how the hippocampus retrieves memory sequences. One line of evidence has to do with phase precession, a process organized by theta and gamma oscillations. This precession can be interpreted as the cued prediction of the sequence of upcoming positions. In support of this interpretation, experiments in two-dimensional environments and on a cue-rich linear track demonstrate that many cells represent a position ahead of the animal and that this position is the same irrespective of which direction the rat is coming from. Other lines of investigation have demonstrated that such predictive processes also occur in the non-spatial domain and that retrieval can be internally or externally cued. The mechanism of sequence retrieval and the usefulness of this retrieval to guide behaviour are discussed.

Flexible decision-making strategies (such as planning) are a key component of adaptive behavior, yet their neural mechanisms have remained resistant to experimental analysis. Theories of planning require prediction and evaluation of potential future rewards, suggesting that reward signals may covertly appear at decision points. To test this idea, we recorded ensembles of ventral striatal neurons on a spatial decision task, in which hippocampal ensembles are known to represent future possibilities at decision points. We found representations of reward which were not only activated at actual reward delivery sites, but also at a high-cost choice point and before error correction. This expectation-of-reward signal at decision points was apparent at both the single cell and the ensemble level, and vanished with behavioral automation. We conclude that ventral striatal representations of reward are more dynamic than suggested by previous reports of reward- and cue-responsive cells, and may provide the necessary signal for evaluation of internally generated possibilities considered during flexible decision-making.

Neural activity in the mammalian CNS is determined by both observable processes, such as sensory stimuli or motor output, and covert, internal cognitive processes that cannot be directly observed. We propose methods to identify these cognitive processes by examining the covert structure within the apparent 'noise' in spike trains. Contemporary analyses of neural codes include encoding (tuning curves derived from spike trains and behavioral, sensory or motor variables), decoding (reconstructing behavioral, sensory or motor variables from spike trains and hypothesized tuning curves) and generative models (predicting the spike trains from hypothesized encoding models and decoded variables). We review examples of each of these processes in hippocampal activity, and propose a general methodology to examine cognitive processes via the identification of dynamic changes in covert variables.

The understanding of decision-making systems has come together in recent years to form a unified theory of decision-making in the mammalian brain as arising from multiple, interacting systems (a planning system, a habit system, and a situation-recognition system). This unified decision-making system has multiple potential access points through which it can be driven to make maladaptive choices, particularly choices that entail seeking of certain drugs or behaviors. We identify 10 key vulnerabilities in the system: (1) moving away from homeostasis, (2) changing allostatic set points, (3) euphorigenic "reward-like" signals, (4) overvaluation in the planning system, (5) incorrect search of situation-action-outcome relationships, (6) misclassification of situations, (7) overvaluation in the habit system, (8) a mismatch in the balance of the two decision systems, (9) over-fast discounting processes, and (10) changed learning rates. These vulnerabilities provide a taxonomy of potential problems with decision-making systems. Although each vulnerability can drive an agent to return to the addictive choice, each vulnerability also implies a characteristic symptomology. Different drugs, different behaviors, and different individuals are likely to access different vulnerabilities. This has implications for an individual's susceptibility to addiction and the transition to addiction, for the potential for relapse, and for the potential for treatment.

The dorsal striatum plays a critical role in procedural learning and memory. Current models of basal ganglia assume that striatal neurons and circuitry are critical for the execution of overlearned, habitual sequences of action. However, less is known about how the striatum encodes task information that guides the performance of actions in procedural tasks. To explore the striatal encoding of task information, we compared the behavioral correlates of striatal neurons tested in two tasks: a multiple T-maze task in which reward delivery was entirely predictable based on spatial cues (the Multiple-T task), and a task in which rats ran on a rectangular track, but food delivery depended on the distance traveled on the track and was not dependent solely on spatial location (the Take-5 task). Striatal cells recorded on these tasks were divisible into three cell types: phasic-firing neurons (PFNs), tonically firing neurons (TFNs), and high-firing neurons (HFNs) and similar proportions of each cell type were found in each task. However, the behavioral correlates of each cell type were differentially sensitive to the type of task rats were performing. PFNs were responsive to specific task-parameters on each task. TFNs showed reliable burst-and-pause responses following food delivery and other events that were consistent with tonically active neurons (TANs) on the Take-5 (non-spatial) task but not on the Multiple-T (spatial) task. HFNs showed spatial oscillations on the Multiple-T (spatial) task but not the Take-5 (non-spatial) task. Reconstruction of the rats' position on the maze was highly accurate when using striatal ensembles recorded on the Multiple-T (spatial) task, but not when using ensembles recorded on the Take-5 (non-spatial) task. In contrast, reconstruction of time following food delivery was successful in both tasks. The results indicated a strong task dependency of the quality of the spatial, but not the reward-related, striatal representations on these tasks. These results suggest that striatal spatial representations depend on the degree to which spatial task-parameters can be unambiguously associated with goals.

Learning and memory and navigation literatures emphasize interactions between multiple memory systems: a flexible, planning-based system and a rigid, cached-value system. This has profound implications for decision-making. Recent conceptualizations of flexible decision-making employ prospection and projection arising from a network involving the hippocampus. Recent recordings from rodent hippocampus in decision-making situations have found transient forward-shifted representations. Evaluation of that prediction and subsequent action-selection probably occurs downstream (e.g. in orbitofrontal cortex, in ventral and dorsomedial striatum). Classically, striatum has been identified as a crucial component of the less-flexible, incremental system. Current evidence, however, suggests that striatum is involved in both flexible and stimulus-response decision-making, with dorsolateral striatum involved in stimulus-response strategies and ventral and dorsomedial striatum involved in goal-directed strategies.

Neural ensembles were recorded from the CA3 region of rats running on T-based decision tasks. Examination of neural representations of space at fast time scales revealed a transient but repeatable phenomenon as rats made a decision: the location reconstructed from the neural ensemble swept forward, first down one path and then the other. Estimated representations were coherent and preferentially swept ahead of the animal rather than behind the animal, implying it represented future possibilities rather than recently traveled paths. Similar phenomena occurred at other important decisions (such as in recovery from an error). Local field potentials from these sites contained pronounced theta and gamma frequencies, but no sharp wave frequencies. Forward-shifted spatial representations were influenced by task demands and experience. These data suggest that the hippocampus does not represent space as a passive computation, but rather that hippocampal spatial processing is an active process likely regulated by cognitive mechanisms.

Because learned associations are quickly renewed following extinction, the extinction process must include processes other than unlearning. However, reinforcement learning models, such as the temporal difference reinforcement learning (TDRL) model, treat extinction as an unlearning of associated value and are thus unable to capture renewal. TDRL models are based on the hypothesis that dopamine carries a reward prediction error signal; these models predict reward by driving that reward error to zero. The authors construct a TDRL model that can accommodate extinction and renewal through two simple processes: (a) a TDRL process that learns the value of situation-action pairs and (b) a situation recognition process that categorizes the observed cues into situations. This model has implications for dysfunctional states, including relapse after addiction and problem gambling., (Copyright 2007 APA.)

If addictions and problematic behaviors arise from interactions between drugs, reward sequences, and natural learning sytems, then an explanation of clinically problematic conditions (such as the self-administration of drugs or problem gambling) requires an understanding of the neural systems that have evolved to allow an agent to make decisions. We hypothesize a unified decision-making system consisting of three components-a situation recognition system, a flexible, planning-capable system, and an inflexible, habit-like system. In this article, we present a model of the planning-capable system based on a planning process arising from experimentally observed look-ahead dynamics in the hippocampus enabling a forward search of possibilities and an evaluation process in the nucleus accumbens. Based on evidence that opioid signaling can provide hedonic evalutation of an achieved outcome, we hypothesize that similar opioid-signaling processes evaluate the value of expected outcomes. This leads to a model of craving, based on the recognition of a path to a high-value outcome, and obsession, based on a value-induced limitation of the search process. This theory can explain why opioid antagonists reduce both hedonic responses and craving.

The firing of place cells in the rodent hippocampus is reliable enough to infer the rodent's position to a high accuracy; however, hippocampal firing also reflects the stages of complex tasks. Theories have suggested that these task-stage responses may reflect changes in reference frame related to task-related subgoals. If the hippocampus represents an environment in multiple ways depending on a task's demands, then switching between these cell assemblies should be detectable as a switch in spatial maps or reference frames. Place cells exhibit extreme temporal variability or "overdispersion," which Fenton et al. suggest reflects changes in active cell-assemblies. If reference-frame switching exists, investigating the relationship of the single cell variability described by Fenton and colleagues to network level processes provides an entry point to understanding the relationship between cell-assembly-like mechanisms and an animal's behavior. We tested the cell-assembly explanation for overdispersion by recording hippocampal neural ensembles from rats running three tasks of varying spatial complexity: linear track (LT), cylinder-foraging (CF), and cylinder-goal (CG). Consistent with the reports by Fenton and colleagues, hippocampal place cells showed high variance in their firing rates across place field passes on the CF and CG tasks. The directional firing of hippocampal place cells on LT provided a test of the reference-frame hypothesis: ignoring direction produced overdispersion similar to the CF and CG tasks; taking direction into account produced a significant decrease in overdispersion. To directly examine the possibility of a network modulation of cell-assemblies, we clustered the firing patterns within each pixel and chained them together to construct whole-environment spatial firing maps. Maps were internally self-consistent, switching with mean rates of several hundred milliseconds. There were significant increases in map-switching rates following reward-related events on the LT and CG tasks, but not on the CF task. Our results link single cell variability with network-level processes and imply that hippocampal spatial representations are made up of multiple, continuous sub-maps, the selection of which depends on the animal's goals when reward is tied to the animal's spatial behavior., ((c) 2007 Wiley-Liss, Inc.)

Hippocampal firing patterns during behavior are reactivated during rest and subsequent slow-wave sleep. These reactivations occur during transient local field potential (LFP) events, termed sharp waves. Theories of hippocampal processing suggest that sharp waves arise from strengthened plasticity, and that the strengthened plasticity depends on repeated cofiring of pyramidal cells. We tested these predictions by recording neural ensembles and LFPs from rats running tasks requiring different levels of behavioral repetition. The number of sharp waves emitted increased during sessions with more regular behaviors. Reactivation became more similar to behavioral firing patterns across the session. This enhanced reactivation also depended on the regularity of the behavior. Additional studies in CA3 and CA1 found that the number of sharp waves emitted also increased in CA3 recordings as well as CA1, but that the time courses were different between the two structures.

This paper describes a novel technique to realize an integrated CMOS bio-potential amplifier with a feedforward DC cancellation topology. The amplifier is designed to provide substantial DC cancellation even while amplifying very low frequency signals. More than 80 dB offset rejection ratio is achieved without any external capacitors. The cancellation scheme is robust against process and temperature variations. The amplifier is fabricated through MOSIS AMI 1.5 microm technology (0.05 mm2 area). Measurement results show a gain of 43.5 dB in the pass band (<1 mHz-5 KHz), an input referred noise of 3.66 microVrms, and a current consumption of 22 microA.

This paper reports the design of a battery-operated, high bandwidth, multi-channel wireless medical telemetry system. The system is capable of transmitting 2.3 Mbps of raw streaming data using the IEEE 802.11b protocol. In a typical application, the system was used to collect data from micro-wire electrodes implanted in the ventral striatum of an awake and behaving rat. The complete system weighs 87 g (without battery) and consumes 2.7 W.

Transient coherent neural oscillations, as indicated by local field potentials, are thought to underlie key perceptual and cognitive events. We report a transient, state-dependent 50 Hz oscillation recorded from electrodes placed in the striatum of awake, behaving rats. These coherent oscillations, which we term gamma(50), occurred in brief (150 ms) events co-incident with the initiation of movement. On navigation tasks, the animal's speed increased dramatically at the precise moment of the gamma(50) event. This synchronous oscillation may provide a key to understanding striatal function, as well as basal ganglia pathology, which often impairs the control of voluntary movements.

The recording of neural ensembles in awake, behaving rats has been an extremely successful experimental paradigm, providing demonstrable scientific advances. Dynamic control of the position of the implanted electrodes is of key importance as mobile electrodes provide a better signal-to-noise ratio and a better cell/ electrode yield than nonmobile electrodes. Here we describe the use of low cost, soon to be commercially available dc motors to successfully control the depth of electrodes. The prototype designed is approximately 30 mm in diameter and 50 mm in length and weighed about 30 gms. This paper presents the results of linear displacements of electrodes achievable with this motorized microdrive.

Temporal difference reinforcement learning (TDRL) algorithms, hypothesized to partially explain basal ganglia functionality, learn more slowly than real animals. Modified TDRL algorithms (e.g. the Dyna-Q family) learn faster than standard TDRL by practicing experienced sequences offline. We suggest that the replay phenomenon, in which ensembles of hippocampal neurons replay previously experienced firing sequences during subsequent rest and sleep, may provide practice sequences to improve the speed of TDRL learning, even within a single session. We test the plausibility of this hypothesis in a computational model of a multiple-T choice-task. Rats show two learning rates on this task: a fast decrease in errors and a slow development of a stereotyped path. Adding developing replay to the model accelerates learning the correct path, but slows down the stereotyping of that path. These models provide testable predictions relating the effects of hippocampal inactivation as well as hippocampal replay on this task.

Head direction cells change their firing rates as a function of the orientation of an animal within an environment. Typically, these cells display a unimodal tuning curve with maximal firing at the cell's preferred direction. As different cells have different preferred directions, the population of cells has been hypothesized to represent the orientation of the animal within the environment. Previous research has shown that pairs of simultaneously recorded head direction cells respond similarly to cue manipulations, suggesting that a population of head direction cells acts in concert to represent the animal's orientation within its environment. Ensembles of head direction cells were recorded from the postsubiculum from rats foraging in an open field. Directional responses of each cell were quantified by the nonparametric Watson's U2 statistic, a measure which makes no explicit assumptions of tuning curve shape. Directionally responsive cells were then used to reconstruct each animal's orientation within the open field using population vector, optimal-linear estimator, and Bayesian methods. The results indicated that postsubiculum contained a complete representation of the animal's orientation. The internal consistency of a neural ensemble can be assessed by comparing the ensemble activity to the expected activity given the reconstructed orientation. This has been termed the "coherency" of the neural ensemble. Reconstruction error decreased as the coherency of the orientation representation increased, indicating that coherency could be used to measure a level of confidence in the representation quality. Because coherency is a linear measure dependent only on internal variables, coherency may be a behaviorally relevant measure used to ascertain the animal's confidence in its representation of orientation., (Copyright (c) 2004 Wiley-Liss, Inc.)

While the use of multi-channel electrodes (stereotrodes and tetrodes) has allowed for the simultaneous recording and identification of many neurons, quantitative measures of the quality of neurons in such recordings are lacking. In multi-channel recordings, each spike waveform is discriminated in a high-dimensional space, making traditional measures of unit quality inapplicable. We describe two measures of unit isolation quality, Lratio and Isolation Distance, and evaluate their performance using simulations and tetrode recordings. Both measures quantified how well separated the spikes of one cluster (putative neuron) were from other spikes recorded simultaneously on the same multi-channel electrode. In simulations and tetrode recordings, both Lratio and Isolation Distance discriminated well- and poorly-separated clusters. In data sets from the rodent hippocampus in which neurons were simultaneously recorded intracellularly and extracellularly, values of Isolation Distance and Lratio were related to the correct identification of spikes.

Addictive drugs have been hypothesized to access the same neurophysiological mechanisms as natural learning systems. These natural learning systems can be modeled through temporal-difference reinforcement learning (TDRL), which requires a reward-error signal that has been hypothesized to be carried by dopamine. TDRL learns to predict reward by driving that reward-error signal to zero. By adding a noncompensable drug-induced dopamine increase to a TDRL model, a computational model of addiction is constructed that over-selects actions leading to drug receipt. The model provides an explanation for important aspects of the addiction literature and provides a theoretic view-point with which to address other aspects.

Neural processes display rhythmic oscillations in local field potentials; identification of their characteristic frequencies is complicated due to their highly non-stationary nature. A simple technique, combining Fourier transforms and correlation coefficients yields unambiguous determinations of the frequencies without a priori filtering. This procedure also provides quantitative information concerning interactions between frequencies. Fundamental frequencies in local field potential data acquired from the hippocampus, cortex, and striatum from awake, behaving rats were calculated using this technique. Characteristic frequencies identified using this technique from hippocampus and cortex agreed with known oscillations. Application to dorsal striatal local field potentials identified a low-frequency theta component as well as a narrow gamma band oscillation at 50-55 Hz.

The striatum plays an important role in "habitual" learning and memory and has been hypothesized to implement a reinforcement-learning algorithm to select actions to perform given the current sensory input. Many experimental approaches to striatal activity have made use of temporally structured tasks, which imply that the striatal representation is temporal. To test this assumption, we recorded neurons in the dorsal striatum of rats running a sequential navigation task: the multiple T maze. Rats navigated a sequence of four T maze turns to receive food rewards delivered in two locations. The responses of neurons that fired phasically were examined. Task-responsive phasic neurons were active as rats ran on the maze (maze-responsive) or during reward receipt (reward-responsive). Neither maze- nor reward-responsive neurons encoded simple motor commands: maze-responses were not well correlated with the shape of the rat's path and most reward-responsive neurons did not fire at similar rates at both food-delivery sites. Maze-responsive neurons were active at one or more locations on the maze, but these responses did not cluster at spatial landmarks such as turns. Across sessions the activity of maze-responsive neurons was highly correlated when rats ran the same maze. Maze-responses encoded the location of the rat on the maze and imply a spatial representation in the striatum in a task with prominent spatial demands. Maze-responsive and reward-responsive neurons were two separate populations, suggesting a divergence in striatal information processing of navigation and reward.

Technological advances allowing simultaneous recording of neuronal ensembles have led to many developments in our understanding of how the brain performs neural computations. One key technique for extracting information from neural populations has been population reconstruction. While reconstruction is a powerful tool, it only provides a value and gives no indication of the quality of the representation itself. In this paper, we present a mathematically and statistically justified measure for assessing the quality of a representation in a neuronal ensemble. Using a simulated neural network, we show that this measure can distinguish between system states and identify moments of dynamical change within the system. While the examples used in this paper all derive from a standard network model, the measure itself is very general. It requires only a representational space, measured tuning curves, and neural ensembles.

The spatial selectivity of hippocampal neurons suggests that they contribute to an internal representation of current location. The activity of hippocampal pyramidal cells was recorded while adult (10-13 months old) and aged (24-28 months old) rats performed a task in which two spatial reference frames were put in conflict. Rats attempted to find an unmarked goal whose position was fixed relative to only one of the two reference frames. The ability of a rat's hippocampus to adjust to the conflicting information and use the 'correct' position estimate (hippocampal map 'realignment') was correlated with the rat's ability to find the hidden goal. In addition, aged rats were impaired relative to adult rats in both goal-finding accuracy and map realignment. Thus, changes in the effectiveness with which the hippocampal spatial representation is updated on the basis of external cues may contribute to both within-age-group spatial learning variability and age-related spatial learning deficits.

Humans and animals trained on sequential reaction tasks show decreases in reaction time and increases in anticipatory movements even long after they have ceased to make errors. Humans show these changes even when they do not explicitly recognize that they performed a repeating sequence. We have developed a task which rats learn to perform error-free quickly, but in which they continue to show path-refinement on a single day. This task may enable the study of performance strategy changes occurring within a single day.

For years, the debate has been: "Is the hippocampus the cognitive map?" or "Is the hippocampus the core of memory?" These two hypotheses derived their original power from two key experiments--the cognitive map theory from the remarkable spatial correlates seen in recordings of hippocampal pyramidal cells and the memory theory from the profound amnesias seen in the patient H.M. Both of these key experiments have been reinterpreted over the years: hippocampal cells are correlated with much more than place and H.M. is missing much more than just his hippocampus. However, both theories are still debated today. The hippocampus clearly plays a role in both navigation and memory processing. The question that must be addressed is rather: "What is the role played by the hippocampus in the navigation and memory systems?" By looking at the navigation system as a whole, one can identify the major role played by the hippocampus as correcting for accumulation errors that occur within idiothetic navigation systems. This is most clearly experimentally evident as reorientation when an animal is lost. Carrying this over to a more general process, this becomes a role of recalling a context, bridging a contextual gap, or, in other words, it becomes a form of recognition memory. I will review recent experimental data which seems to support this theory over the more general spatial or memory theories traditionally applied to hippocampus.

In neocortex, neighboring neurons frequently exhibit correlated encoding properties. There is conflicting evidence whether a similar phenomenon occurs in hippocampus. To assess this quantitatively, a comparison was made of the spatial and temporal firing correlations within and between local groups of hippocampal cells, spaced 350-1400 microm apart. No evidence of clustering was found in a sample of >3000 neurons. Moreover, cells active in two environments were uniformly interspersed at a scale of <100 microm, as assessed by the activity-induced gene Arc. Independence of encoding characteristics implies uncorrelated inputs, which could enhance the capacity of the hippocampus to store arbitrary associations.

Whether hippocampal map realignment is coupled more strongly to position or time was studied in rats trained to shuttle on a linear track. The rats were required to run from a start box and to pause at a goal location at a fixed location relative to stable distal cues (room-aligned coordinate frame). The origin of each lap was varied by shifting the start box and track as a unit (box-aligned coordinate frame) along the direction of travel. As observed by Gothard et al. (1996a), on each lap the hippocampal activity realigned from a representation that was box-aligned to one that was room-aligned. We studied the dynamics of this transition using a measure of how well the moment-by-moment ensemble activity matched the expected activity given the location of the animal in each coordinate frame. The coherency ratio, defined as the ratio of the matches for the two coordinate systems, provides a quantitative measure of the ensemble activity alignment and was used to compare four possible descriptions of the realignment process. The elapsed time since leaving the box provided a better predictor of the occurrence of the transition than any of the three spatial parameters investigated, suggesting that the shift between coordinate systems is at least partially governed by a stochastic, time-dependent process.

We suggest that the hippocampus plays two roles that allow rodents to solve the hidden-platform water maze: self-localization and route replay. When an animal explores an environment such as the water maze, the combination of place fields and correlational (Hebbian) long-term potentiation produces a weight matrix in the CA3 recurrent collaterals such that cells with overlapping place fields are more strongly interconnected than cells with nonoverlapping fields. When combined with global inhibition, this forms an attractor with coherent representations of position as stable states. When biased by local view information, this allows the animal to determine its position relative to the goal when it returns to the environment. We call this self-localization. When an animal traces specific routes within an environment, the weights in the CA3 recurrent collaterals become asymmetric. We show that this stores these routes in the recurrent collaterals. When primed with noise in the absence of sensory input, a coherent representation of position still forms in the CA3 population, but then that representation drifts, retracing a route. We show that these two mechanisms can coexist and form a basis for memory consolidation, explaining the anterograde and limited retrograde amnesia seen following hippocampal lesions.

We present a conceptual framework for the role of the hippocampus and its afferent and efferent structures in rodent navigation. Our proposal is compatible with the behavioral, neurophysiological, anatomical, and neuropharmacological literature, and suggests a number of practical experiments that could support or refute it. We begin with a review of place cells and how the place code for an environment might be aligned with sensory cues and updated by self-motion information. The existence of place fields in the dark suggests that location information is maintained by path integration, which requires an internal representation of direction of motion. This leads to a consideration of the organization of the rodent head direction system, and thence into a discussion of the computational structure and anatomical locus of the path integrator. If the place code is used in navigation, there must be a mechanism for selecting an action based on this information. We review evidence that the nucleus accumbens subserves this function. From there, we move to interactions between the hippocampal system and the environment, emphasizing mechanisms for learning novel environments and for aligning the various subsystems upon re-entry into familiar environments. We conclude with a discussion of the relationship between navigation and declarative memory.

We present a computational theory of navigation in rodents based on interacting representations of place, head direction, and local view. An associated computer model is able to replicate a variety of behavioral and neurophysiological results from the rodent navigation literature. The theory and model generate predictions that are testable with current technologies.

During a reaching task, the population vector is an encoding of direction based on cells with cosine response functions. Scaling the response by a magnitude factor produces a vector encoding, enabling vector arithmetic to be performed by the summation of firing rates. We show that the response properties of selected populations of cells in the primary motor cortex and area 5 can be explained in terms of arithmetic relationships among load, goal, and motor command vectors. Our computer simulations show good agreement with single-cell recording data.