Search

As rats learn to search for multiple sources of food or water in a complex environment, they generate increasingly efficient trajectories between reward sites. Such spatial navigation capacity involves the replay of hippocampal place-cells during awake states, generating small sequences of spatially related place-cell activity that we call “snippets”. These snippets occur primarily during sharp-wave-ripples (SWRs). Here we focus on the role of such replay events, as the animal is learning a traveling salesperson task (TSP) across multiple trials. We hypothesize that snippet replay generates synthetic data that can substantially expand and restructure the experience available and make learning more optimal. We developed a model of snippet generation that is modulated by reward, propagated in the forward and reverse directions. This implements a form of spatial credit assignment for reinforcement learning. We use a biologically motivated computational framework known as ‘reservoir computing’ to model prefrontal cortex (PFC) in sequence learning, in which large pools of prewired neural elements process information dynamically through reverberations. This PFC model consolidates snippets into larger spatial sequences that may be later recalled by subsets of the original sequences. Our simulation experiments provide neurophysiological explanations for two pertinent observations related to navigation. Reward modulation allows the system to reject non-optimal segments of experienced trajectories, and reverse replay allows the system to “learn” trajectories that it has not physically experienced, both of which significantly contribute to the TSP behavior., Author summary As rats search for multiple sources of food in a complex environment, they generate increasingly efficient trajectories between reward sites, across multiple trials. This spatial navigation optimization behavior can be measured in the laboratory using a traveling salesperson task (TSP). This likely involves the coordinated replay of place-cell “snippets” between successive trials. We hypothesize that “snippets” can be used by the prefrontal cortex (PFC) to implement a form of reward-modulated reinforcement learning. Our simulation experiments provide neurophysiological explanations for two pertinent observations related to navigation. Reward modulation allows the system to reject non-optimal segments of experienced trajectories, and reverse replay allows the system to “learn” trajectories that it has not physically experienced, both of which significantly contribute to the TSP behavior.

The hippocampus is involved in spatial navigation and memory in rodents and humans. Anatomically, the hippocampus extends along a longitudinal axis that shows a combination of graded and specific interconnections with neocortical and subcortical brain areas. Functionally, place cells are found all along the longitudinal axis and exhibit gradients of properties including an increasing dorsal-to-ventral place field size. We propose a view of hippocampal function in which fine-dorsal to coarse-ventral overlapping representations collaborate to form a multi-level representation of spatial and episodic memory that is dominant during navigation in large and complex environments or when encoding complex memories. This view is supported by the fact that the effects of ventral hippocampal damage are generally only found in larger laboratory-scale environments, and by the finding that human virtual navigation studies associate ventral hippocampal involvement with increased environmental complexity. Other mechanisms such as the ability of place cells to exhibit multiple fields and their ability to scale their fields with changes in environment size may be utilized when forming large-scale cognitive maps. Coarse-grained ventral representations may overlap with and provide multi-modal global contexts to finer-grained intermediate and dorsal representations, a mechanism that may support mnemonic hierarchies of autobiographical memory in humans.

Description of the Paper: Our paper outlines the history of spill response requirements in the U.S. Arctic and how incidents have shaped spill legislation. A review of the equipment utilized in spill response from the 1980’s and focuses on the exploration efforts in the Beaufort and Chukchi Seas. It also covers a review of the changing spill response requirements and their impact on the Shell exploration effort. The experiments and real life experience with spills in ice are reviewed and the equipment and tactics developed from this experience will be described. The limitations of each recovery option are explored and when each tactic would be effective in a spill event. Proposed spill response platforms for effective spill response operations are identified and how spill response equipment can be deployed in the field. Application: Spill response options are shown for working in the Arctic in the various ice conditions that may be experienced throughout the year. Results, Observations and Conclusions: The paper covers the changes in response equipment designed to operate in the Arctic and the tactics that are effective as well as touch on future research that is needed to improve recovery options. Significance of Subject Matter: The effectiveness of a spill mitigation plan is critical in securing the permits and social license to explore for hydrocarbons in the fragile Arctic environment.