Search

Hippocampus is critical for memory, imagination, and constructive reasoning. However, recent models have suggested that its neuronal responses can be well explained by state-spaces that model the transitions between experiences. How do we reconcile these two views? Here we show that if state-spaces are constructed compositionally from existing primitives, hippocampal responses can be interpreted as compositional memories, binding these primitives together. Critically, this enables agents to behave optimally in novel environments with no new learning, inferring behaviour directly from the composition. This provides natural interpretations of generalisation and latent learning. Hippocampal replay can build and consolidate these compositional memories, but importantly, due to their compositional nature, it can construct states it has never experienced - effectively building memories of the future. This enables new predictions of optimal replays for novel environments, or after structural changes. Together, these findings provide a framework for reasoning about several seemingly disparate functions of hippocampus.

Humans and other animals effortlessly generalize prior knowledge to solve novel problems, by abstracting common structure and mapping it onto new sensorimotor specifics. To investigate how the brain achieves this, in this study, we trained mice on a series of reversal learning problems that shared the same structure but had different physical implementations. Performance improved across problems, indicating transfer of knowledge. Neurons in medial prefrontal cortex (mPFC) maintained similar representations across problems despite their different sensorimotor correlates, whereas hippocampal (dCA1) representations were more strongly influenced by the specifics of each problem. This was true for both representations of the events that comprised each trial and those that integrated choices and outcomes over multiple trials to guide an animal’s decisions. These data suggest that prefrontal cortex and hippocampus play complementary roles in generalization of knowledge: PFC abstracts the common structure among related problems, and hippocampus maps this structure onto the specifics of the current situation.

Reinforcement learning models of the basal ganglia map the phasic dopamine signal to reward prediction errors (RPEs). Conventional models assert that, when a stimulus predicts a reward with fixed delay, dopamine activity during the delay should converge to baseline through learning. However, recent studies have found that dopamine ramps up before reward in certain conditions even after learning, thus challenging the conventional models. In this work, we show that sensory feedback causes an unbiased learner to produce RPE ramps. Our model predicts that, when feedback gradually decreases during a trial, dopamine activity should resemble a ‘bump,’ whose ramp-up phase should furthermore be greater than that of conditions where the feedback stays high. We trained mice on a virtual navigation task with varying brightness, and both predictions were empirically observed. In sum, our theoretical and experimental results reconcile the seemingly conflicting data on dopamine behaviors under the RPE hypothesis.

Memory for everyday experience shapes our representation of the structure of the world, while retrieval of these experiences is fundamental for informing our future decisions. The fine-grained neurophysiological mechanisms that support such retrieval are largely unknown. We studied participants who first experienced, without repetition, unique multi-component episodes. One day later, they engaged in cued retrieval of these episodes whilst undergoing magnetoencephalography (MEG). By decoding individual episode elements, we found that trial-by-trial successful retrieval was supported by sequential replay of episode elements, with a temporal compression factor greater than 60. The direction of replay supporting this retrieval, either backward or forward, depended on whether a participant’s goal was to retrieve elements of an episode that followed or preceded a retrieval cue, respectively. This sequential replay was weaker in very high performing participants, where instead we found evidence for simultaneous clustered reactivation. Our results demonstrate that memory-mediated decisions are supported by a rapid replay mechanism that can flexibly shift in direction in response to task requirements.One Sentence SummaryRecall of extended episodes of experience is supported by compressed replay of memory elements that flexibly changes direction depending on task temporal orientation.

There has been strong interest in preprint commenting and review activities in recent years. Public preprint feedback can bring benefits to authors, readers and others in scholarly communication, however, the level of public commenting on preprints is still low. This is likely due to cultural barriers, such as fear by authors that criticisms on their paper will bias readers, editors and evaluators, and concerns by commenters that posting a public critique on a preprint by a more senior colleague may lead to retribution. In order to help address these cultural barriers and foster positive and constructive participation in public preprint feedback, we have developed a set of 14 principles for creating, responding to, and interpreting preprint feedback. The principles are clustered around four broad themes: Focused, Appropriate, Specific, Transparent (FAST). We describe each of the FAST principles and designate which actors (authors, reviewers and the community) each of the principles applies to. We discuss the possible implementation of the FAST principles by different stakeholders in science communication, and explore what opportunities and challenges lie ahead in the path towards a thriving preprint feedback ecosystem.

We use our eyes to assess the value of objects around us and carefully fixate options that we are about to choose. Neurons in the prefrontal cortex reliably encode the value of fixated options, which is essential for decision making. Yet as a decision unfolds, it remains unclear how prefrontal regions determine which option should be fixated next. Here we show that anterior cingulate cortex (ACC) encodes the value of options in the periphery to guide subsequent fixations during economic choice. In an economic decision-making task involving four simultaneously presented cues, we found rhesus macaques evaluated cues using their peripheral vision. This served two distinct purposes: subjects were more likely to fixate valuable peripheral cues, and more likely to choose valuable options whose cues were never even fixated. ACC, orbitofrontal cortex, dorsolateral prefrontal cortex, and ventromedial prefrontal cortex neurons all encoded cue value post-fixation. ACC was unique, however, in also encoding the value of cues before fixation and even cues that were never fixated. This pre-saccadic value encoding by ACC predicted which cue was next fixated during the decision process. ACC therefore conducts simultaneous processing of peripheral information to guide information sampling and choice during decision making.

An emerging goal in neuroscience is tracking what information is represented in brain activity over time as a participant completes some task. Whilst EEG and MEG offer millisecond temporal resolution of how activity patterns emerge and evolve, standard decoding methods present significant barriers to interpretability as they obscure the underlying spatial and temporal activity patterns. We instead propose the use of a generative encoding model framework that simultaneously infers the multivariate spatial patterns of activity and the variable timing at which these patterns emerge on individual trials. An encoding model inversion allows predictions to be made about unseen test data in the same way as in standard decoding methodology. These SpatioTemporally Resolved MVPA (STRM) models can be flexibly applied to a wide variety of experimental paradigms, including classification and regression tasks. We show that these models provide insightful maps of the activity driving predictive accuracy metrics; demonstrate behaviourally meaningful variation in the timing of pattern emergence on individual trials; and achieve predictive accuracies that are either equivalent or surpass those achieved by more widely used methods. This provides a new avenue for investigating the brain’s representational dynamics and could ultimately support more flexible experimental designs in future.HIGHLIGHTSWe introduce SpatioTemporally Resolved MVPA (STRM), an approach that explicitly models how successive stages of stimulus processing are distributed in both space and time in M/EEG data.We show that STRM is broadly applicable to diverse types of M/EEG data and outputs meaningful and interpretable maps of how neural representations evolve in space and time at millisecond resolution.The trial-specific deviations in activity pattern timings identified by STRM are not random, but vary systematically with inter-trial differences in behavioural, cognitive and physiological variables.These methods result in predictive accuracy metrics that are mostly equivalent to, or a modest improvement on, conventional methods.

Prefrontal cortex is crucial for learning and decision-making. Classic reinforcement learning (RL) theories centre on learning the expectation of potential rewarding outcomes and explain a wealth of neural data in prefrontal cortex. Distributional RL, on the other hand, learns the full distribution of rewarding outcomes and better explains dopamine responses. Here we show distributional RL also better explains prefrontal cortical responses, suggesting it is a ubiquitous mechanism for reward-guided learning.

Research in many different areas of medicine will benefit from new approaches to peer review and publishing.

Understanding the visual world is a constructive process. Whilst a frontal-hippocampal circuit is known to be essential for this task, little is known about the associated neuronal computations. Visual understanding appears superficially distinct from other known functions of this circuit, such as spatial reasoning and model-based planning, but recent models suggest deeper computational similarities. Here, using fMRI, we show that representations of a simple visual scene in these brain regions are relational and compositional – key computational properties theorised to support rapid construction of hippocampal maps. Using MEG, we show that rapid sequences of representations, akin to replay in spatial navigation and planning problems, are also engaged in visual construction. Whilst these sequences have previously been proposed as mechanisms to plan possible futures or learn from the past, here they are used to understand the present. Replay sequences form constructive hypotheses about possible scene configurations. These hypotheses play out in an optimal order for relational inference, progressing from predictable to uncertain scene elements, gradually constraining possible configurations, and converging on the correct scene configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry, and a role for generative replay in constructive inference and hypothesis testing.

Replay supports planning Learning from direct experience is easy—we can always use trial and error—but how do we learn from nondirect (nonlocal) experiences? For this, we need additional mechanisms that bridge time and space. In rodents, hippocampal replay is hypothesized to promote this function. Liu et al. measured high-temporal-resolution brain signals using human magnetoencephalography combined with a new model-based, visually oriented, multipath reinforcement memory task. This task was designed to differentiate local versus nonlocal learning episodes within the subject. They found that reverse sequential replay in the human medial temporal lobe supports nonlocal reinforcement learning and is the underlying mechanism for solving complex credit assignment problems such as value learning. Science , abf1357, this issue p. eabf1357

There are rich structures in off-task neural activity which are hypothesized to reflect fundamental computations across a broad spectrum of cognitive functions. Here, we develop an analysis toolkit – temporal delayed linear modelling (TDLM) – for analysing such activity. TDLM is a domain-general method for finding neural sequences that respect a pre-specified transition graph. It combines nonlinear classification and linear temporal modelling to test for statistical regularities in sequences of task-related reactivations. TDLM is developed on the non-invasive neuroimaging data and is designed to take care of confounds and maximize sequence detection ability. Notably, as a linear framework, TDLM can be easily extended, without loss of generality, to capture rodent replay in electrophysiology, including in continuous spaces, as well as addressing second-order inference questions, for example, its temporal and spatial varying pattern. We hope TDLM will advance a deeper understanding of neural computation and promote a richer convergence between animal and human neuroscience.

Choices rely on a transformation of sensory inputs into motor responses. Using invasive single neuron recordings, the evolution of a choice process has been tracked by projecting population neural responses into state spaces. Here, we develop an approach that allows us to recover similar trajectories on a millisecond timescale in non-invasive human recordings. We selectively suppress activity related to three task-axes, relevant and irrelevant sensory inputs and response direction, in magnetoencephalography data acquired during context-dependent choices. Recordings from premotor cortex show a progression from processing sensory input to processing the response. In contrast to previous macaque recordings, information related to choice-irrelevant features is represented more weakly than choice-relevant sensory information. To test whether this mechanistic difference between species is caused by extensive over-training common in non-human primate studies, we trained humans on >20,000 trials of the task. Choice-irrelevant features were still weaker than relevant features in premotor cortex after over-training.

Summary Our brains at rest spontaneously replay recently acquired information, but how this process is orchestrated to avoid interference with ongoing cognition is an open question. Here we investigated whether replay coincided with spontaneous patterns of whole-brain activity. We found, in two separate datasets, that replay sequences were packaged into transient bursts occurring selectively during activation of the default mode network (DMN) and parietal alpha networks. These networks are believed to support inwardly oriented attention and inhibit bottom-up sensory processing and were characterized by widespread synchronized oscillations coupled to increases in high frequency power, mechanisms thought to coordinate information flow between disparate cortical areas. Our data reveal a tight correspondence between two widely studied phenomena in neural physiology and suggest that the DMN may coordinate replay bursts in a manner that minimizes interference with ongoing cognition., Highlights • Replay in humans coincides with activity in specific resting brain networks • Clusters of heightened default mode and alpha activity are linked to replay bursts • These networks are characterized by highly synchronized brain-wide oscillations • High-frequency power bursts are uniquely linked to default mode network activation, Our brains form stable memory traces by spontaneously replaying recently acquired information. Higgins et al. show that this process coincides with activity in specific resting-state brain networks, with bursts of replay events occurring selectively during clusters of heightened default mode and parietal alpha network activity.

Summary Knowledge of the structure of a problem, such as relationships between stimuli, enables rapid learning and flexible inference. Humans and other animals can abstract this structural knowledge and generalize it to solve new problems. For example, in spatial reasoning, shortest-path inferences are immediate in new environments. Spatial structural transfer is mediated by cells in entorhinal and (in humans) medial prefrontal cortices, which maintain their co-activation structure across different environments and behavioral states. Here, using fMRI, we show that entorhinal and ventromedial prefrontal cortex (vmPFC) representations perform a much broader role in generalizing the structure of problems. We introduce a task-remapping paradigm, where subjects solve multiple reinforcement learning (RL) problems differing in structural or sensory properties. We show that, as with space, entorhinal representations are preserved across different RL problems only if task structure is preserved. In vmPFC and ventral striatum, representations of prediction error also depend on task structure., Highlights • Entorhinal cortex represents the task structure (rules) of an RL task • As in physical space, this representation is divorced from sensory signals • Spatial patterns of prediction error signals depended on task structure • Such relational representations of task structure might underlie generalisation, How do we generalize knowledge between problems that are only loosely related but have a similar structure? Baram et al. take inspiration from the generalization properties of entorhinal cortex in rodent spatial tasks to show that entorhinal and ventromedial prefrontal cortices generalize the structure of reinforcement learning tasks in humans.

From July 2021 eLife will only review manuscripts already published as preprints, and will focus its editorial process on producing public reviews to be posted alongside the preprints.

To make effective decisions we need to consider the relationship between actions and outcomes. They are, however, often separated by time and space. The biological mechanism capable of spanning those gaps remains unknown. One promising, albeit hypothetical, mechanism involves neural replay of non-local experience. Using a novel task, that segregates direct from indirect learning, combined with magnetoencephalography (MEG), we tested the role of neural replay in non-local learning in humans. Following reward receipt, we found significant backward replay of non-local experience, with a 160 msec state-to-state time lag, and this replay facilitated learning of action values. This backward replay, combined with behavioural evidence of non-local learning, was more pronounced in experiences that were of greater benefit for future behavior, as predicted by theories of prioritization. These findings establish rationally targeted non-local replay as a neural mechanism for solving complex credit assignment problems during learning.One Sentence SummaryReverse sequential replay is found, for the first time, to support non-local reinforcement learning in humans and is prioritized according to utility.

Relations between task elements often follow hidden underlying structural forms such as periodicities or hierarchies, whose inferences fosters performance. However, transferring structural knowledge to novel environments requires flexible representations that are generalizable over particularities of the current environment, such as its stimuli and size. We suggest that humans represent structural forms as abstract basis sets and that in novel tasks, the structural form is inferred and the relevant basis set is transferred. Using a computational model, we show that such representation allows inference of the underlying structural form, important task states, effective behavioural policies and the existence of unobserved state-trajectories. In two experiments, participants learned three abstract graphs during two successive days. We tested how structural knowledge acquired on Day-1 affected Day-2 performance. In line with our model, participants who had a correct structural prior were able to infer the existence of unobserved state-trajectories and appropriate behavioural policies., Humans are able to exploit patterns or schemas when performing new tasks, but the mechanism for this ability is still unknown. Using graph-learning tasks, we show that humans are able to transfer abstract structural knowledge and suggest a computational mechanism by which such transfer can occur.

Our brains at rest spontaneously replay recently acquired information, but how this process is orchestrated to avoid interference with ongoing cognition is an open question. We investigated whether replay coincided with spontaneous patterns of whole brain activity. We found, in two separate datasets, that replay sequences were packaged into transient bursts occurring selectively during activation of the default mode network (DMN) and parietal alpha network. These networks were characterized by widespread synchronized oscillations coupled to increases in ripple band power, mechanisms that coordinate information flow between disparate cortical areas. Our data show a tight correspondence between two widely studied phenomena of neural physiology and suggest the DMN may coordinate replay bursts in a manner that minimizes interference with ongoing cognition.

Since its introduction, the reward prediction error (RPE) theory of dopamine has explained a wealth of empirical phenomena, providing a unifying framework for understanding the representation of reward and value in the brain(1–3). According to the now canonical theory, reward predictions are represented as a single scalar quantity, which supports learning about the expectation, or mean, of stochastic outcomes. In the present work, we propose a novel account of dopamine-based reinforcement learning. Inspired by recent artificial intelligence research on distributional reinforcement learning(4–6), we hypothesized that the brain represents possible future rewards not as a single mean, but instead as a probability distribution, effectively representing multiple future outcomes simultaneously and in parallel. This idea leads immediately to a set of empirical predictions, which we tested using single-unit recordings from mouse ventral tegmental area. Our findings provide strong evidence for a neural realization of distributional reinforcement learning.

SUMMARYThere are rich structures in off-task neural activity. For example, task related neural codes are thought to be reactivated in a systematic way during rest. This reactivation is hypothesised to reflect a fundamental computation that supports a variety of cognitive functions. Here, we introduce an analysis toolkit (TDLM) for analysing this activity. TDLM combines nonlinear classification and linear temporal modelling to testing for statistical regularities in sequences of neural representations. It is developed using non-invasive neuroimaging data and is designed to take care of confounds and maximize sequence detection ability. The method can be extended to rodent electrophysiological recordings. We outline how TDLM can successfully reveal human replay during rest, based upon non-invasive magnetoencephalography (MEG) measurements, with strong parallels to rodent hippocampal replay. TDLM can therefore advance our understanding of sequential computation and promote a richer convergence between animal and human neuroscience research.

Anatomical, neuroimaging and lesion studies indicate that prefrontal cortex (PFC) can be subdivided into different subregions supporting distinct aspects of decision making. However, explanations of neuronal computations within these subregions varies widely across studies. An integrated and mechanistic account of PFC function therefore remains elusive. Resolving these debates demands a rich dataset that directly contrasts neuronal activity across multiple PFC subregions within a single paradigm, whilst experimentally controlling factors such as the order, duration and frequency in which choice options are attended and compared. Here, we contrast neuronal population responses between macaque orbitofrontal (OFC), anterior cingulate (ACC) and dorsolateral prefrontal cortices (DLPFC) during sequential value-guided information search and choice. From the first fixation of choice-related stimuli, a strong triple dissociation of information encoding emerges in parallel across these PFC subregions. As further information is gathered, population responses in OFC reflect an attention-guided value comparison process. Meanwhile, parallel signals in ACC reflect belief updating in light of new evidence, integration of that evidence to a decision bound, and an emerging action plan for which option should be chosen. Our findings demonstrate the co-existence of multiple, distributed decision-related computations across PFC subregions during value-guided choice. They provide a synthesis of several competing accounts of PFC function.

The anterior limb of the internal capsule (ALIC) carries thalamic and brainstem fibers from prefrontal cortical regions that are associated with different aspects of emotion, motivation, cognition processing, and decision-making. This large fiber bundle is abnormal in several psychiatric illnesses and a major target for deep brain stimulation. Yet, we have very little information about where specific prefrontal fibers travel within the bundle. Using a combination of tracing studies and diffusion MRI in male nonhuman primates, as well as diffusion MRI in male and female human subjects, we segmented the human ALIC into five regions based on the positions of axons from different cortical regions within the capsule. Fractional anisotropy (FA) abnormalities in patients with bipolar disorder were detected when FA was averaged in the ALIC segment that carries ventrolateral prefrontal cortical connections. Together, the results set the stage for linking abnormalities within the ALIC to specific connections and demonstrate the utility of applying connectivity profiles of large white matter bundles based on animal anatomic studies to human connections and associating disease abnormalities in those pathways with specific connections. The ability to functionally segment large white matter bundles into their components begins a new era of refining how we think about white matter organization and use that information in understanding abnormalities.SIGNIFICANCE STATEMENTThe anterior limb of the internal capsule (ALIC) connects prefrontal cortex with the thalamus and brainstem and is abnormal in psychiatric illnesses. However, we know little about the location of specific prefrontal fibers within the bundle. Using a combination of animal tracing studies and diffusion MRI in animals and human subjects, we segmented the human ALIC into five regions based on the positions of axons from different cortical regions. We then demonstrated that differences in FA values between bipolar disorder patients and healthy control subjects were specific to a given segment. Together, the results set the stage for linking abnormalities within the ALIC to specific connections and for refining how we think about white matter organization in general.

Background: Neuronal loss and dopamine depletion alter motor signal processing between cortical motor areas, basal ganglia, and the thalamus, resulting in the motor manifestations of Parkinson's di ...

Summary The hippocampal-entorhinal system is important for spatial and relational memory tasks. We formally link these domains, provide a mechanistic understanding of the hippocampal role in generalization, and offer unifying principles underlying many entorhinal and hippocampal cell types. We propose medial entorhinal cells form a basis describing structural knowledge, and hippocampal cells link this basis with sensory representations. Adopting these principles, we introduce the Tolman-Eichenbaum machine (TEM). After learning, TEM entorhinal cells display diverse properties resembling apparently bespoke spatial responses, such as grid, band, border, and object-vector cells. TEM hippocampal cells include place and landmark cells that remap between environments. Crucially, TEM also aligns with empirically recorded representations in complex non-spatial tasks. TEM also generates predictions that hippocampal remapping is not random as previously believed; rather, structural knowledge is preserved across environments. We confirm this structural transfer over remapping in simultaneously recorded place and grid cells., Graphical Abstract, Highlights • Common principles for space and relational memory in the hippocampal formation • Explains hippocampal generalization in both spatial and non-spatial problems • Accounts for many reported hippocampal and entorhinal cell types from such tasks • Predicts how hippocampus remaps in both spatial and non-spatial tasks, The Tolman-Eichenbaum Machine, named in honor of Edward Chace Tolman and Howard Eichenbaum for their contributions to cognitive theory, provides a unifying framework for the hippocampal role in spatial and nonspatial generalization and unifying principles underlying many entorhinal and hippocampal cell types.

The hippocampal formation is critical for spatial and relational inference in navigation problems. The neural code underlying such inference is factorized in the entorhinal cortex (EC) and conjunctive in the hippocampus (HC). A factorized code implies a separate encoding of sensory and relational knowledge, which can be flexibly conjoined to an object representation that reflects both sensory and relational properties. We hypothesize that the same neural mechanisms are employed in complex decision-making and compositional planning, which requires the flexible generalization of knowledge to novel instances. We tested this hypothesis in a task where subjects had to construct novel visual objects based on a set of basic visual building blocks and relations. We found behavioral evidence that subjects form a hierarchical representation of this task that allows them to flexibly apply compositional knowledge to novel stimuli. Using fMRI adaption, we found evidence that the construction of novel objects depends on compositional neural representations in HC-EC and medial prefrontal cortex (mPFC). Further, we found that these structures also encoded purely relational information, indicative of a factorized representation. These results suggest that compositional neural representations in the hippocampal formation and prefrontal cortex enable the generalization of abstract knowledge to novel stimuli during visual construction.

It is proposed that a cognitive map encoding the relationships between entities in the world supports flexible behavior, but the majority of the neural evidence for such a system comes from studies of spatial navigation. Recent work describing neuronal parallels between spatial and non-spatial behaviors has rekindled the notion of a systematic organization of knowledge across multiple domains. We review experimental evidence and theoretical frameworks that point to principles unifying these apparently disparate functions. These principles describe how to learn and use abstract, generalizable knowledge and suggest that map-like representations observed in a spatial context may be an instance of general coding mechanisms capable of organizing knowledge of all kinds. We highlight how artificial agents endowed with such principles exhibit flexible behavior and learn map-like representations observed in the brain. Finally, we speculate on how these principles may offer insight into the extreme generalizations, abstractions, and inferences that characterize human cognition.

Contemporary reinforcement learning (RL) theory suggests that potential choices can be evaluated by strategies that may or may not be sensitive to the computational structure of tasks. A paradigmatic model-free (MF) strategy simply repeats actions that have been rewarded in the past; by contrast, model-sensitive (MS) strategies exploit richer information associated with knowledge of task dynamics. MF and MS strategies should typically be combined, because they have complementary statistical and computational strengths; however, this tradeoff between MF/MS RL has mostly only been demonstrated in humans, often with only modest numbers of trials. We trained rhesus monkeys to perform a two-stage decision task designed to elicit and discriminate the use of MF and MS methods. A descriptive analysis of choice behaviour revealed directly that the structure of the task (of MS importance) and the reward history (of MF and MS importance) significantly influenced both choice and response vigour. A detailed, trial-by-trial computational analysis confirmed that choices were made according to a combination of strategies, with a dominant influence of a particular form of model sensitivity that persisted over weeks of testing. The residuals from this model necessitated development of a new combined RL model which incorporates a particular credit assignment weighting procedure. Finally, response vigor exhibited a subtly different collection of MF and MS influences. These results provide new illumination onto RL behavioural processes in non-human primates., Author summary We routinely solve planning problems in which present decisions have consequences in the future. These pose complex computational and statistical problems and are addressed by multiple systems in the brain which use different solutions to these problems, and which may compete and cooperate. We trained two rhesus monkeys on a paradigmatic planning task that transparently reveals canonical aspects of different strategies. We performed a detailed behavioral analysis using methods of reinforcement learning on choice and reaction time to reveal conjoint influences and structural interactions of different sources of information. We show the strengths and limitations of these analyses, at the same time as we provide a novel perspective on how different learning systems interact for choice in non-human primates.

Summary Every day we make decisions critical for adaptation and survival. We repeat actions with known consequences. But we also draw on loosely related events to infer and imagine the outcome of entirely novel choices. These inferential decisions are thought to engage a number of brain regions; however, the underlying neuronal computation remains unknown. Here, we use a multi-day cross-species approach in humans and mice to report the functional anatomy and neuronal computation underlying inferential decisions. We show that during successful inference, the mammalian brain uses a hippocampal prospective code to forecast temporally structured learned associations. Moreover, during resting behavior, coactivation of hippocampal cells in sharp-wave/ripples represent inferred relationships that include reward, thereby “joining-the-dots” between events that have not been observed together but lead to profitable outcomes. Computing mnemonic links in this manner may provide an important mechanism to build a cognitive map that stretches beyond direct experience, thus supporting flexible behavior., Graphical Abstract, Highlights • Inferential decisions engage the hippocampus in humans and mice • During inference, a hippocampal prospective code draws on associative memories • This hippocampal prospective code preserves the learned temporal statistics • During rest, hippocampal ripples nest cognitive short-cuts for inferred relations, In humans and mice, the hippocampus supports inferential reasoning by computing a prospective code to predict upcoming events, before extracting logical links between discrete events during rest to form a mnemonic short cut for inferred relationships.

It is proposed that a cognitive map encoding the relationships between objects supports the ability to flexibly navigate the world. Place cells and grid cells provide evidence for such a map in a spatial context. Emerging evidence suggests analogous cells code for non-spatial information. Further, it has been shown that grid cells resemble the eigenvectors of the relationship between place cells and can be learnt from local inputs. Here we show that these locally-learnt eigenvectors contain not only local information but also global knowledge that can provide both distributions over future states as well as a global distance measure encoding approximate distances between every object in the world. By simply changing the weights in the grid cell population, it is possible to switch between computing these different measures. We demonstrate a simple algorithm can use these measures to globally navigate arbitrary topologies without searching more than one step ahead. We refer to this as intuitive planning.

When axonal fibres approach or leave the cortex, their trajectories tend to closely follow the cortical convolutions. To quantify this tendency, we propose a three-dimensional coordinate system based on the gyral geometry. For every voxel in the brain, we define a “radial” axis orthogonal to nearby surfaces, a “sulcal” axis along the sulcal depth gradient that preferentially points from deep white matter to the gyral crown, and a “gyral” axis aligned with the long axis of the gyrus.\ud \ud When compared with high-resolution, in-vivo diffusion MRI data from the Human Connectome Project, we find that in superficial white matter the apparent diffusion coefficient (at b = 1000) along the sulcal axis is on average 16% larger than along the gyral axis and twice as large as along the radial axis. This is reflected in the vast majority of observed fibre orientations lying close to the tangential plane (median angular offset

It is proposed that a cognitive map encoding the relationships between entities in the world supports flexible behaviour, but the majority of the neural evidence for such a system comes from studies of spatial navigation. Recent work describing neuronal parallels between spatial and non-spatial behaviours has rekindled the notion of a systematic organisation of knowledge across multiple domains. We review experimental evidence and theoretical frameworks that point to principles unifying these apparently disparate functions. These principles describe how to learn and use abstract, generalisable knowledge and suggest map-like representations observed in a spatial context may be an instance of general coding mechanisms capable of organising knowledge of all kinds. We highlight how artificial agents endowed with such principles exhibit flexible behaviour and learn map-like representations observed in the brain. Finally, we speculate on how these principles may offer insight into the extreme generalisations, abstractions and inferences that characterise human cognition.

The ventral intermediate nucleus (VIM) of the thalamus is an established surgical target for stereotactic ablation and deep brain stimulation (DBS) in the treatment of tremor in Parkinson's disease (PD) and essential tremor (ET). It is centrally placed on a cerebello-thalamo-cortical network connecting the primary motor cortex, to the dentate nucleus of the contralateral cerebellum through the dentato-rubro-thalamic tract (DRT). The VIM is not readily visible on conventional MR imaging, so identifying the surgical target traditionally involved indirect targeting that relies on atlas-defined coordinates. Unfortunately, this approach does not fully account for individual variability and requires surgery to be performed with the patient awake to allow for intraoperative targeting confirmation. The aim of this study is to identify the VIM and the DRT using probabilistic tractography in patients that will undergo thalamic DBS for tremor. Four male patients with tremor dominant PD and five patients (three female) with ET underwent high angular resolution diffusion imaging (HARDI) (128 diffusion directions, 1.5 mm isotropic voxels and b value = 1500) preoperatively. Patients received VIM-DBS using an MR image guided and MR image verified approach with indirect targeting. Postoperatively, using parallel Graphical Processing Unit (GPU) processing, thalamic areas with the highest diffusion connectivity to the primary motor area (M1), supplementary motor area (SMA), primary sensory area (S1) and contralateral dentate nucleus were identified. Additionally, volume of tissue activation (VTA) corresponding to active DBS contacts were modelled. Response to treatment was defined as 40% reduction in the total Fahn-Tolosa-Martin Tremor Rating Score (FTMTRS) with DBS-ON, one year from surgery. Three out of nine patients had a suboptimal, long-term response to treatment. The segmented thalamic areas corresponded well to anatomically known counterparts in the ventrolateral (VL) and ventroposterior (VP) thalamus. The dentate-thalamic area, lay within the M1-thalamic area in a ventral and lateral location. Streamlines corresponding to the DRT connected M1 to the contralateral dentate nucleus via the dentate-thalamic area, clearly crossing the midline in the mesencephalon. Good response was seen when the active contact VTA was in the thalamic area with highest connectivity to the contralateral dentate nucleus. Non-responders had active contact VTAs outside the dentate-thalamic area. We conclude that probabilistic tractography techniques can be used to segment the VL and VP thalamus based on cortical and cerebellar connectivity. The thalamic area, best representing the VIM, is connected to the contralateral dentate cerebellar nucleus. Connectivity based segmentation of the VIM can be achieved in individual patients in a clinically feasible timescale, using HARDI and high performance computing with parallel GPU processing. This same technique can map out the DRT tract with clear mesencephalic crossing., Highlights • The thalamic target for surgery for tremor is not readily visible on conventional MRI. • Probabilistic tractography is used to segment the thalamus based on connectivity. • The best target area is connected to the contralateral dentate cerebellar nucleus. • GPU processing is used to segment the thalamus in a clinically feasible timescale. • This technique can map out the DRT tract with clear mesencephalic crossing.

Statistical regularities in the causal structure of the environment enable us to predict the probable outcomes of our actions. Environments differ in the extent to which action-outcome contingencies are stable or volatile. Difficulty in being able to use this information to optimally update outcome predictions might contribute to the decision-making difficulties seen in anxiety. We tested this using an aversive learning task manipulating environmental volatility. Human participants low in trait anxiety matched updating of their outcome predictions to the volatility of the current environment, as predicted by a Bayesian model. Individuals with high trait anxiety showed less ability to adjust updating of outcome expectancies between stable and volatile environments. This was linked to reduced sensitivity of the pupil dilatory response to volatility, potentially indicative of altered norepinephrinergic responsivity to changes in this aspect of environmental information.

Item does not contain fulltext The degree to which genetic factors influence brain connectivity is beginning to be understood. Large-scale efforts are underway to map the profile of genetic effects in various brain regions. The NIH-funded Human Connectome Project (HCP) is providing data valuable for analyzing the degree of genetic influence underlying brain connectivity revealed by state-of-the-art neuroimaging methods. We calculated the heritability of the fractional anisotropy (FA) measure derived from diffusion tensor imaging (DTI) reconstruction in 481 HCP subjects (194/287 M/F) consisting of 57/60 pairs of mono- and dizygotic twins, and 246 siblings. FA measurements were derived using (Enhancing NeuroImaging Genetics through Meta-Analysis) ENIGMA DTI protocols and heritability estimates were calculated using the SOLAR-Eclipse imaging genetic analysis package. We compared heritability estimates derived from HCP data to those publicly available through the ENIGMA-DTI consortium, which were pooled together from five-family based studies across the US, Europe, and Australia. FA measurements from the HCP cohort for eleven major white matter tracts were highly heritable (h(2)=0.53-0.90, p

Objectives Firstly, to identify subthalamic region stimulation clusters that predict maximum improvement in rigidity, bradykinesia and tremor, or emergence of side-effects; and secondly, to map-out the cortical fingerprint, mediated by the hyperdirect pathways which predict maximum efficacy. Methods High angular resolution diffusion imaging in twenty patients with advanced Parkinson's disease was acquired prior to bilateral subthalamic nucleus deep brain stimulation. All contacts were screened one-year from surgery for efficacy and side-effects at different amplitudes. Voxel-based statistical analysis of volumes of tissue activated models was used to identify significant treatment clusters. Probabilistic tractography was employed to identify cortical connectivity patterns associated with treatment efficacy. Results All patients responded well to treatment (46% mean improvement off medication UPDRS-III [p < 0.0001]) without significant adverse events. Cluster corresponding to maximum improvement in tremor was in the posterior, superior and lateral portion of the nucleus. Clusters corresponding to improvement in bradykinesia and rigidity were nearer the superior border in a further medial and posterior location. The rigidity cluster extended beyond the superior border to the area of the zona incerta and Forel-H2 field. When the clusters where averaged, the coordinates of the area with maximum overall efficacy was X = -10(-9.5), Y = -13(-1) and Z = -7(-3) in MNI(AC-PC) space. Cortical connectivity to primary motor area was predictive of higher improvement in tremor; whilst that to supplementary motor area was predictive of improvement in bradykinesia and rigidity; and connectivity to prefrontal cortex was predictive of improvement in rigidity. Interpretation These findings support the presence of overlapping stimulation sites within the subthalamic nucleus and its superior border, with different cortical connectivity patterns, associated with maximum improvement in tremor, rigidity and bradykinesia.

SUMMARYNervous systems use excitatory cell assemblies or “perceptual engrams” to encode and represent sensory percepts. Similarly, synaptically connected cell assemblies or “memory engrams” are thought to represent memories of past experience. Multiple lines of recent evidence indicate that brain systems also create inhibitory replicas of excitatory engrams with important cognitive functions. Such matched inhibitory engrams may form through homeostatic potentiation of inhibition onto postsynaptic cells that show increased levels of excitation. Inhibitory engrams can reduce behavioral responses to familiar stimuli thereby resulting in behavioral habituation. In addition, by preventing inappropriate activation of excitatory memory engrams, inhibitory engrams can make memories quiescent, stored in a latent form that is available for contextrelevant activation. In neural networks with balanced excitatory and inhibitory engrams, the release of innate responses and recall of associative memories can occur through focussed disinhibition. Understanding mechanisms that regulate the formation and expression of inhibitory engramsin vivomay help not only to explain key features of cognition, but also to provide insight into transdiagnostic traits associated with psychiatric conditions such as autism, schizophrenia and post-traumatic stress disorder (PTSD).

The hippocampal–entorhinal system encodes a map of space that guides spatial navigation. Goal-directed behaviour outside of spatial navigation similarly requires a representation of abstract forms of relational knowledge. This information relies on the same neural system, but it is not known whether the organisational principles governing continuous maps may extend to the implicit encoding of discrete, non-spatial graphs. Here, we show that the human hippocampal–entorhinal system can represent relationships between objects using a metric that depends on associative strength. We reconstruct a map-like knowledge structure directly from a hippocampal–entorhinal functional magnetic resonance imaging adaptation signal in a situation where relationships are non-spatial rather than spatial, discrete rather than continuous, and unavailable to conscious awareness. Notably, the measure that best predicted a behavioural signature of implicit knowledge and blood oxygen level-dependent adaptation was a weighted sum of future states, akin to the successor representation that has been proposed to account for place and grid-cell firing patterns. DOI: http://dx.doi.org/10.7554/eLife.17086.001, eLife digest To help us navigate, the brain encodes information about the positions of landmarks in space in a series of maps. These maps are housed by two neighbouring brain regions called the hippocampus and entorhinal cortex. These regions also encode information about non-spatial relationships, for example, between two events that often occur close together in time. However, it was not known whether such non-spatial relationships may also be encoded as a map. To address this question, Garvert et al. showed volunteers a series of objects on a screen. Unbeknown to the volunteers, the order of the objects was not entirely random. Instead, each object could only follow certain others. The objects were thus connected to one another by a network of non-spatial relationships, broadly comparable to the spatial relationships that connect physical locations in the environment. The next day, the volunteers viewed some of the objects again, this time while lying inside a brain scanner. Although the volunteers still believed that the objects had been presented at random, the activity of their hippocampus and entorhinal cortex reflected the non-spatial relationships volunteers had experienced between the objects. The relationships were organised in an abstract map. This suggests that the brain organises knowledge about abstract non-spatial relationships into maps comparable to those used to represent spatial relationships. The brain can use these maps of non-spatial relationships to guide our behaviour, even though we have no conscious awareness of the information they contain. The maps may also enable us to make new inferences, just as we can use our spatial maps to find short cuts or navigate around obstacles. Future studies should investigate the mechanisms underlying our ability to create maps of non-spatial relationships and how we use them to guide decision making. DOI: http://dx.doi.org/10.7554/eLife.17086.002

Valuation is a key tenet of decision neuroscience, where it is generally assumed that different attributes of competing options are assimilated into unitary values. Such values are central to current neural models of choice. By contrast, psychological studies emphasize complex interactions between choice and valuation. Principles of neuronal selection also suggest that competitive inhibition may occur in early valuation stages, before option selection. We found that behavior in multi-attribute choice is best explained by a model involving competition at multiple levels of representation. This hierarchical model also explains neural signals in human brain regions previously linked to valuation, including striatum, parietal and prefrontal cortex, where activity represents within-attribute competition, competition between attributes and option selection. This multi-layered inhibition framework challenges the assumption that option values are computed before choice. Instead, our results suggest a canonical competition mechanism throughout all stages of a processing hierarchy, not simply at a final choice stage.