Search

To better understand neural processing during adaptive learning of stimulus‐response‐reward contingencies, we recorded synchrony of neuronal activity in anterior cingulate cortex (ACC) and hippocampal rhythms in male rats acquiring and switching between spatial and visual discrimination tasks in a Y‐maze. ACC population activity as well as single unit activity shifted shortly after task rule changes or just before the rats adopted different task strategies. Hippocampal theta oscillations (associated with memory encoding) modulated an elevated proportion of rule‐change responsive neurons (70%), but other neurons that were correlated with strategy‐change, strategy value and reward‐rate were not. However, hippocampal sharp wave‐ripples modulated significantly higher proportions of rule‐change, strategy‐change and reward‐rate responsive cells during post‐session sleep but not pre‐session sleep. This suggests an underestimated mechanism for hippocampal mismatch and contextual signals to facilitate ACC to detect contingency changes for cognitive flexibility, a function that is attenuated after it is damaged. [ABSTRACT FROM AUTHOR]

Minimizing negative impacts of Artificial Intelligent (AI) systems on human societies without human supervision requires them to be able to align with human values. However, most current work only addresses this issue from a technical point of view, e.g., improving current methods relying on reinforcement learning from human feedback, neglecting what it means and is required for alignment to occur. Here, we propose to distinguish strong and weak value alignment. Strong alignment requires cognitive abilities (either human-like or different from humans) such as understanding and reasoning about agents' intentions and their ability to causally produce desired effects. We argue that this is required for AI systems like large language models (LLMs) to be able to recognize situations presenting a risk that human values may be flouted. To illustrate this distinction, we present a series of prompts showing ChatGPT's, Gemini's and Copilot's failures to recognize some of these situations. We moreover analyze word embeddings to show that the nearest neighbors of some human values in LLMs differ from humans' semantic representations. We then propose a new thought experiment that we call "the Chinese room with a word transition dictionary", in extension of John Searle's famous proposal. We finally mention current promising research directions towards a weak alignment, which could produce statistically satisfying answers in a number of common situations, however so far without ensuring any truth value., (© 2024. The Author(s).)

In uncertain environments in which resources fluctuate continuously, animals must permanently decide whether to stabilise learning and exploit what they currently believe to be their best option, or instead explore potential alternatives and learn fast from new observations. While such a trade-off has been extensively studied in pretrained animals facing non-stationary decision-making tasks, it is yet unknown how they progressively tune it while learning the task structure during pretraining. Here, we compared the ability of different computational models to account for long-term changes in the behaviour of 24 rats while they learned to choose a rewarded lever in a three-armed bandit task across 24 days of pretraining. We found that the day-by-day evolution of rat performance and win-shift tendency revealed a progressive stabilisation of the way they regulated reinforcement learning parameters. We successfully captured these behavioural adaptations using a meta-learning model in which either the learning rate or the inverse temperature was controlled by the average reward rate., (© 2024 The Author(s). European Journal of Neuroscience published by Federation of European Neuroscience Societies and John Wiley & Sons Ltd.)

This study was conducted to examine the chemical composition of the essential oils (EOs) from six Tunisian Eucalyptus species and to evaluate their anti-enzymatic and antibiofilm activities. The EOs were obtained through hydro-distillation of dried leaves and subsequently analyzed using GC/MS. The main class of compounds was constituted by oxygenated monoterpenes, particularly prominent in E. brevifolia (75.7%), E. lehmannii (72.8%), and E. woollsiana (67%). Anti-enzymatic activities against cholinesterases, α-amylase, and α-glucosidase were evaluated using spectrophotometric methods. Notably, the E. brevifolia , E. extensa , E. leptophylla , E. patellaris , and E. woollsiana EOs displayed potent acetylcholinesterase (AChE) inhibition (IC 50 : 0.25-0.60 mg/mL), with E. lehmannii exhibiting lower activity (IC 50 : 1.2 mg/mL). E. leptophylla and E. brevifolia showed remarkable α-amylase inhibition (IC 50 : 0.88 mg/mL), while E. brevifolia and E. leptophylla significantly hindered α-glucosidase (IC 50 < 30 mg/mL), distinguishing them from other EOs with limited effects. Additionally, the EOs were assessed for their anti-biofilm properties of Gram-positive ( Staphylococcus aureus and Listeria monocytogenes ) and Gram-negative ( Acinetobacter baumannii , Pseudomonas aeruginosa and Escherichia coli ) bacterial strains. The E. extensa EO demonstrated the main antibiofilm effect against E. coli and L. monocytogenes with an inhibition > 80% at 10 mg/mL. These findings could represent a basis for possible further use of Eucalyptus EOs in the treatment of human microbial infections and/or as a coadjutant in preventing and treating Alzheimer's disease and/or diabetes mellitus.

In recent years, soft robots gain increasing attention as a result of their compliance when operating in unstructured environments, and their flexibility that ensures safety when interacting with humans. However, challenges lie on the difficulty to develop control algorithms due to various limitations induced by their soft structure. In this paper, we introduce a novel technique that aims to perform motion control of a modular bio-inspired soft-robotic arm, with the main focus lying on facilitating the qualitative reproduction of well-specified periodic trajectories. The introduced method combines the notion behind two previously developed methodologies both based on the Movement Primitive (MP) theory, by exploiting their capabilities while coping with their main drawbacks. Concretely, the requested actuation is initially computed using a Probabilistic MP (ProMP)-based method that considers the trajectory as a combination of simple movements previously learned and stored as a MP library. Subsequently, the key components of the resulting actuation are extracted and filtered in the frequency domain. These are eventually used as input to a Central Pattern Generator (CPG)-based model that takes over the generation of rhythmic patterns at the motor level. The proposed methodology is evaluated on a two-module soft arm. Results show that the first algorithmic component (ProMP) provides an immediate estimation of the requested actuation by avoiding time-consuming training, while the latter (CPG) further simplifies the execution by allowing its control through a low-dimensional parameterization. Altogether, these results open new avenues for the rapid acquisition of periodic movements in soft robots, and their compression into CPG parameters for long-term storage and execution., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2023 Oikonomou, Dometios, Khamassi and Tzafestas.)

Our brain is continuously challenged by daily experiences. Thus, how to avoid systematic erasing of previously encoded memories? While it has been proposed that a dual-learning system with 'slow' learning in the cortex and 'fast' learning in the hippocampus could protect previous knowledge from interference, this has never been observed in the living organism. Here, we report that increasing plasticity via the viral-induced overexpression of RGS14414 in the prelimbic cortex leads to better one-trial memory, but that this comes at the price of increased interference in semantic-like memory. Indeed, electrophysiological recordings showed that this manipulation also resulted in shorter NonREM-sleep bouts, smaller delta-waves and decreased neuronal firing rates. In contrast, hippocampal-cortical interactions in form of theta coherence during wake and REM-sleep as well as oscillatory coupling during NonREM-sleep were enhanced. Thus, we provide the first experimental evidence for the long-standing and unproven fundamental idea that high thresholds for plasticity in the cortex protect preexisting memories and modulating these thresholds affects both memory encoding and consolidation mechanisms., Competing Interests: IN, AA, AS, MB, SN, AR, AA, Jv, MK, ZK, LG No competing interests declared, (© 2023, Navarro Lobato et al.)

Healthcare professionals' statistical illiteracy can impair medical decision quality and compromise patient safety. Previous studies have documented clinicians' insufficient proficiency in statistics and a tendency in overconfidence. However, an underexplored aspect is clinicians' awareness of their lack of statistical knowledge that precludes any corrective intervention attempt. Here, we investigated physicians', residents' and medical students' alignment between subjective confidence judgments and objective accuracy in basic medical statistics. We also examined how gender, profile of experience and practice of research activity affect this alignment, and the influence of problem framing (conditional probabilities, CP vs. natural frequencies, NF). Eight hundred ninety-eight clinicians completed an online survey assessing skill and confidence on three topics: vaccine efficacy, p value and diagnostic test results interpretation. Results evidenced an overall consistent poor proficiency in statistics often combined with high confidence, even in incorrect answers. We also demonstrate that despite overconfidence bias, clinicians show a degree of metacognitive sensitivity, as their confidence judgments discriminate between their correct and incorrect answers. Finally, we confirm the positive impact of the more intuitive NF framing on accuracy. Together, our results pave the way for the development of teaching recommendations and pedagogical interventions such as promoting metacognition on basic knowledge and statistical reasoning as well as the use of NF to tackle statistical illiteracy in the medical context., (© 2023. The Author(s).)

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states. SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms., (Copyright © 2023 the authors.)

Experience replay is widely used in AI to bootstrap reinforcement learning (RL) by enabling an agent to remember and reuse past experiences. Classical techniques include shuffled-, reversed-ordered- and prioritized-memory buffers, which have different properties and advantages depending on the nature of the data and problem. Interestingly, recent computational neuroscience work has shown that these techniques are relevant to model hippocampal reactivations recorded during rodent navigation. Nevertheless, the brain mechanisms for orchestrating hippocampal replay are still unclear. In this paper, we present recent neurorobotics research aiming to endow a navigating robot with a neuro-inspired RL architecture (including different learning strategies, such as model-based (MB) and model-free (MF), and different replay techniques). We illustrate through a series of numerical simulations how the specificities of robotic experimentation (e.g., autonomous state decomposition by the robot, noisy perception, state transition uncertainty, non-stationarity) can shed new lights on which replay techniques turn out to be more efficient in different situations. Finally, we close the loop by raising new hypotheses for neuroscience from such robotic models of hippocampal replay., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Massi, Barthélemy, Mailly, Dromnelle, Canitrot, Poniatowski, Girard and Khamassi.)

The ability to attribute thoughts to others, also called theory of mind (TOM), has been extensively studied in humans; however, its evolutionary origins have been challenged. Computationally, the basis of TOM has been interpreted within the predictive coding framework and associated with activity in the temporoparietal junction (TPJ). Here, we revealed, using a nonlinguistic task and functional magnetic resonance imaging, that activity in a region of the macaque middle superior temporal cortex was specifically modulated by the predictability of social situations. As in human TPJ, this region could be distinguished from other temporal regions involved in face processing. Our result suggests the existence of a precursor for the TOM ability in the last common ancestor of human and Old World monkeys.

This special issue, commissioned after the 4th Quadrennial Meeting on Orbitofrontal Cortex Function held in Paris in November of 2019 (https://ofc2019.sciencesconf.org/), is intended to provide a snapshot of this ongoing transformation; we hope that the ideas presented herein will provide a foundation for the next stage in the evolution of our understanding of this magical brain region. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Our understanding of orbitofrontal cortex (OFC) function has progressed remarkably over the past decades in part due to theoretical advances in associative and reinforcement learning theories. These theoretical accounts of OFC function have implicated the region in progressively more psychologically refined processes from the value and sensory-specific properties of expected outcomes to the representation and inference over latent state representations in cognitive maps of task space. While these accounts have been successful at modeling many of the effects of causal manipulation of OFC function in both rodents and primates, recent findings suggest that further refinement of our current models are still required. Here, we briefly review how our understanding of OFC function has developed to understand two cardinal deficits following OFC dysfunction: Reversal learning and outcome devaluation. We then consider recent findings that OFC dysfunction also significantly affects initial acquisition learning, often assumed to be intact. To account for these findings, we consider a possible role for the OFC in the arbitration and exploration between model-free (MF) and model-based (MB) learning systems, offline updating of MB representations. While the function of the OFC as a whole is still likely to be integral to the formation and use of a cognitive map of task space, these refinements suggest a way in which distinct orbital subregions, such as the rodent lateral OFC, might contribute to this overall function. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The antibody molecule comprises a variable domain conferring antigen specificity and affinity distinct from the heavy chain constant (CH) domains dictating effector functions. We here interrogate this paradigm by evaluating the unique influence of the CH1α domain on epitope specificity and functions using two mucosal gp41-specific Fab-IgAs (FabA) derived from HIV-1 highly-exposed but persistently seronegative individuals (HESN). These HESN develop selectively affinity-matured HIV-1-specific mucosal IgA that target the gp41 viral envelope and might provide protection although by unclear mechanisms. Isotype-switching FabAs into Fab-IgGs (FabGs) results in a >10-fold loss in affinity for HIV-1 clade A, B, and C gp41, together with reduced neutralization of HIV-1 cross-clade. The FabA conformational epitopes map selectively on gp41 in 6-Helix bundle and pre-fusion conformations cross-clade, unlike FabGs. Finally, we designed in silico, a 12 amino-acid peptide recapitulating one FabA conformational epitope that inhibits the FabA binding to gp41 cross-clade and its neutralizing activity. Altogether, our results reveal that the CH1α domain shapes the antibody paratope through an allosteric effect, thereby strengthening the antibody specificity and functional activities. Further, they clarify the mechanisms by which these HESN IgAs might confer protection against HIV-1-sexual acquisition. The IgA-specific epitope we characterized by reverse vaccinology could help designing a mucosal HIV-1 vaccine., Competing Interests: The authors have declared that no competing interests exist.

Kleefstra syndrome is a disorder caused by a mutation in the EHMT1 gene characterized in humans by general developmental delay, mild to severe intellectual disability and autism. Here, we characterized cumulative memory in the Ehmt1 +/- mouse model using the Object Space Task. We combined conventional behavioral analysis with automated analysis by deep-learning networks, a session-based computational learning model, and a trial-based classifier. Ehmt1 +/- mice showed more anxiety-like features and generally explored objects less, but the difference decreased over time. Interestingly, when analyzing memory-specific exploration, Ehmt1 +/- show increased expression of cumulative memory, but a deficit in a more simple, control memory condition. Using our automatic classifier to differentiate between genotypes, we found that cumulative memory features are better suited for classification than general exploration differences. Thus, detailed behavioral classification with the Object Space Task produced a more detailed behavioral phenotype of the Ehmt1 +/- mouse model., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2020 The Author(s). Published by Elsevier Inc. All rights reserved.)

People and other animals learn the values of choices by observing the contingencies between them and their outcomes. However, decisions are not guided by choice-linked reward associations alone; macaques also maintain a memory of the general, average reward rate - the global reward state - in an environment. Remarkably, global reward state affects the way that each choice outcome is valued and influences future decisions so that the impact of both choice success and failure is different in rich and poor environments. Successful choices are more likely to be repeated but this is especially the case in rich environments. Unsuccessful choices are more likely to be abandoned but this is especially likely in poor environments. Functional magnetic resonance imaging (fMRI) revealed two distinct patterns of activity, one in anterior insula and one in the dorsal raphe nucleus, that track global reward state as well as specific outcome events.

Hippocampal offline reactivations during reward-based learning, usually categorized as replay events, have been found to be important for performance improvement over time and for memory consolidation. Recent computational work has linked these phenomena to the need to transform reward information into state-action values for decision making and to propagate it to all relevant states of the environment. Nevertheless, it is still unclear whether an integrated reinforcement learning mechanism could account for the variety of awake hippocampal reactivations, including variety in order (forward and reverse reactivated trajectories) and variety in the location where they occur (reward site or decision-point). Here, we present a model-based bidirectional search model which accounts for a variety of hippocampal reactivations. The model combines forward trajectory sampling from current position and backward sampling through prioritized sweeping from states associated with large reward prediction errors until the two trajectories connect. This is repeated until stabilization of state-action values (convergence), which could explain why hippocampal reactivations drastically diminish when the animal's performance stabilizes. Simulations in a multiple T-maze task show that forward reactivations are prominently found at decision-points while backward reactivations are exclusively generated at reward sites. Finally, the model can generate imaginary trajectories that are not allowed to the agent during task performance. We raise some experimental predictions and implications for future studies of the role of the hippocampo-prefronto-striatal network in learning.

In the context of Pavlovian conditioning, two types of behaviour may emerge within the population (Flagel et al. Nature, 469(7328): 53-57, 2011). Animals may choose to engage either with the conditioned stimulus (CS), a behaviour known as sign-tracking (ST) which is sensitive to dopamine inhibition for its acquisition, or with the food cup in which the reward or unconditioned stimulus (US) will eventually be delivered, a behaviour known as goal-tracking (GT) which is dependent on dopamine for its expression only. Previous work by Lesaint et al. (PLoS Comput Biol, 10(2), 2014) offered a computational explanation for these phenomena and led to the prediction that varying the duration of the inter-trial interval (ITI) would change the relative ST-GT proportion in the population as well as phasic dopamine responses. A recent study verified this prediction, but also found a rich variance of ST and GT behaviours within the trial which goes beyond the original computational model. In this paper, we provide a computational perspective on these novel results.

Declarative memory encompasses representations of specific events as well as knowledge extracted by accumulation over multiple episodes. To investigate how these different sorts of memories are created, we developed a new behavioral task in rodents. The task consists of 3 distinct conditions (stable, overlapping, and random). Rodents are exposed to multiple sample trials, in which they explore objects in specific spatial arrangements, with object identity changing from trial to trial. In the stable condition, the locations are constant during all sample trials even though the objects themselves change; in the test trial, 1 object's location is changed. In the random condition, object locations are presented in the sample phase without a specific spatial pattern. In the overlapping condition, 1 location is shared (overlapping) between all trials, while the other location changes during sample trials. We show that in the overlapping condition, instead of only remembering the last sample trial, rodents form a cumulative memory of the sample trials. Here, we could show that both mice and rats can accumulate information across multiple trials and express a long-term abstracted memory., Competing Interests: The authors have declared that no competing interests exist.

In a volatile environment where rewards are uncertain, successful performance requires a delicate balance between exploitation of the best option and exploration of alternative choices. It has theoretically been proposed that dopamine contributes to the control of this exploration-exploitation trade-off, specifically that the higher the level of tonic dopamine, the more exploitation is favored. We demonstrate here that there is a formal relationship between the rescaling of dopamine positive reward prediction errors and the exploration-exploitation trade-off in simple non-stationary multi-armed bandit tasks. We further show in rats performing such a task that systemically antagonizing dopamine receptors greatly increases the number of random choices without affecting learning capacities. Simulations and comparison of a set of different computational models (an extended Q-learning model, a directed exploration model, and a meta-learning model) fitted on each individual confirm that, independently of the model, decreasing dopaminergic activity does not affect learning rate but is equivalent to an increase in random exploration rate. This study shows that dopamine could adapt the exploration-exploitation trade-off in decision-making when facing changing environmental contingencies.

Accumulating evidence suggest that human behavior in trial-and-error learning tasks based on decisions between discrete actions may involve a combination of reinforcement learning (RL) and working-memory (WM). While the understanding of brain activity at stake in this type of tasks often involve the comparison with non-human primate neurophysiological results, it is not clear whether monkeys use similar combined RL and WM processes to solve these tasks. Here we analyzed the behavior of five monkeys with computational models combining RL and WM. Our model-based analysis approach enables to not only fit trial-by-trial choices but also transient slowdowns in reaction times, indicative of WM use. We found that the behavior of the five monkeys was better explained in terms of a combination of RL and WM despite inter-individual differences. The same coordination dynamics we used in a previous study in humans best explained the behavior of some monkeys while the behavior of others showed the opposite pattern, revealing a possible different dynamics of WM process. We further analyzed different variants of the tested models to open a discussion on how the long pretraining in these tasks may have favored particular coordination dynamics between RL and WM. This points towards either inter-species differences or protocol differences which could be further tested in humans., (Copyright © 2017 Elsevier B.V. All rights reserved.)

Multiple in vivo studies have shown that place cells from the hippocampus replay previously experienced trajectories. These replays are commonly considered to mainly reflect memory consolidation processes. Some data, however, have highlighted a functional link between replays and reinforcement learning (RL). This theory, extensively used in machine learning, has introduced efficient algorithms and can explain various behavioral and physiological measures from different brain regions. RL algorithms could constitute a mechanistic description of replays and explain how replays can reduce the number of iterations required to explore the environment during learning. We review the main findings concerning the different hippocampal replay types and the possible associated RL models (either model-based, model-free, or hybrid model types). We conclude by tying these frameworks together. We illustrate the link between data and RL through a series of model simulations. This review, at the frontier between informatics and biology, paves the way for future work on replays.

In economics and perceptual decision-making contextual effects are well documented, where decision weights are adjusted as a function of the distribution of stimuli. Yet, in reinforcement learning literature whether and how contextual information pertaining to decision states is integrated in learning algorithms has received comparably little attention. Here, we investigate reinforcement learning behavior and its computational substrates in a task where we orthogonally manipulate outcome valence and magnitude, resulting in systematic variations in state-values. Model comparison indicates that subjects' behavior is best accounted for by an algorithm which includes both reference point-dependence and range-adaptation-two crucial features of state-dependent valuation. In addition, we find that state-dependent outcome valuation progressively emerges, is favored by increasing outcome information and correlated with explicit understanding of the task structure. Finally, our data clearly show that, while being locally adaptive (for instance in negative valence and small magnitude contexts), state-dependent valuation comes at the cost of seemingly irrational choices, when options are extrapolated out from their original contexts.

Recent computational models of sign tracking (ST) and goal tracking (GT) have accounted for observations that dopamine (DA) is not necessary for all forms of learning and have provided a set of predictions to further their validity. Among these, a central prediction is that manipulating the intertrial interval (ITI) during autoshaping should change the relative ST-GT proportion as well as DA phasic responses. Here, we tested these predictions and found that lengthening the ITI increased ST, i.e., behavioral engagement with conditioned stimuli (CS) and cue-induced phasic DA release. Importantly, DA release was also present at the time of reward delivery, even after learning, and DA release was correlated with time spent in the food cup during the ITI. During conditioning with shorter ITIs, GT was prominent (i.e., engagement with food cup), and DA release responded to the CS while being absent at the time of reward delivery after learning. Hence, shorter ITIs restored the classical DA reward prediction error (RPE) pattern. These results validate the computational hypotheses, opening new perspectives on the understanding of individual differences in Pavlovian conditioning and DA signaling., Competing Interests: The authors have declared that no competing interests exist.

Despite major progress in Robotics and AI, robots are still basically "zombies" repeatedly achieving actions and tasks without understanding what they are doing. Deep-Learning AI programs classify tremendous amounts of data without grasping the meaning of their inputs or outputs. We still lack a genuine theory of the underlying principles and methods that would enable robots to understand their environment, to be cognizant of what they do, to take appropriate and timely initiatives, to learn from their own experience and to show that they know that they have learned and how. The rationale of this paper is that the understanding of its environment by an agent (the agent itself and its effects on the environment included) requires its self-awareness, which actually is itself emerging as a result of this understanding and the distinction that the agent is capable to make between its own mind-body and its environment. The paper develops along five issues: agent perception and interaction with the environment; learning actions; agent interaction with other agents-specifically humans; decision-making; and the cognitive architecture integrating these capacities., (Copyright © 2018 Chatila, Renaudo, Andries, Chavez-Garcia, Luce-Vayrac, Gottstein, Alami, Clodic, Devin, Girard and Khamassi.)

We present a computational model of spatial navigation comprising different learning mechanisms in mammals, i.e., associative, cognitive mapping and parallel systems. This model is able to reproduce a large number of experimental results in different variants of the Morris water maze task, including standard associative phenomena (spatial generalization gradient and blocking), as well as navigation based on cognitive mapping. Furthermore, we show that competitive and cooperative patterns between different navigation strategies in the model allow to explain previous apparently contradictory results supporting either associative or cognitive mechanisms for spatial learning. The key computational mechanism to reconcile experimental results showing different influences of distal and proximal cues on the behavior, different learning times, and different abilities of individuals to alternatively perform spatial and response strategies, relies in the dynamic coordination of navigation strategies, whose performance is evaluated online with a common currency through a modular approach. We provide a set of concrete experimental predictions to further test the computational model. Overall, this computational work sheds new light on inter-individual differences in navigation learning, and provides a formal and mechanistic approach to test various theories of spatial cognition in mammals.

The protective efficacy of human immunodeficiency virus-1 (HIV-1) antibodies (Abs) remains mostly correlated with their in vitro neutralizing activity engaging their Fab region. However, anti-HIV-1 Abs also mediate a broad array of Fc-mediated effector functions including Ab-dependent cellular cytotoxicity (ADCC), which depend primarily on the Ab isotype. While ADCC is commonly associated with HIV-1 gp120 envelope-specific IgGs, whether IgAs, especially those targeting the HIV-1 gp41 envelope, also mediate ADCC remains elusive. Therefore, to assess the capacity of IgA specific for HIV-1 to induce Fcα-mediated ADCC, we used the gp41 envelope-specific IgA transformed from the broadly neutralizing 2F5-IgG we have previously reported to induce ADCC. We demonstrate that 2F5-IgA engages FcαRI (CD89), expressed on human monocytes used as effector cells, to induce the lysis of HIV-1 Clade A- and B-infected target cells by ADCC. Furthermore, the 2F5-IgA and 2F5-IgG cooperate to enhance target cells lysis by ADCC. Cooperation in ADCC is also observed between 2F5-IgA and the broadly neutralizing 10E8-IgG. These results provide a new perspective for IgA in protection against HIV-1 acquisition or reservoir eradication and suggest that inducing IgA by vaccination, in particular when targeting gp41, in combination with IgG could strengthen protection by complementary and cooperative activities with IgG.

Computer science offers a large set of tools for prototyping, writing, running, testing, validating, sharing and reproducing results; however, computational science lags behind. In the best case, authors may provide their source code as a compressed archive and they may feel confident their research is reproducible. But this is not exactly true. James Buckheit and David Donoho proposed more than two decades ago that an article about computational results is advertising, not scholarship. The actual scholarship is the full software environment, code, and data that produced the result. This implies new workflows, in particular in peer-reviews. Existing journals have been slow to adapt: source codes are rarely requested and are hardly ever actually executed to check that they produce the results advertised in the article. ReScience is a peer-reviewed journal that targets computational research and encourages the explicit replication of already published research, promoting new and open-source implementations in order to ensure that the original research can be replicated from its description. To achieve this goal, the whole publishing chain is radically different from other traditional scientific journals. ReScience resides on GitHub where each new implementation of a computational study is made available together with comments, explanations, and software tests., Competing Interests: Federico Vaggi is an employee of Amazon, Inc., Roman Yurchak is an employee of Symerio, and C. Titus Brown and Nicolas P. Rougier are Academic Editors for PeerJ., (©2017 Rougier et al.)

To explain the high level of flexibility in primate decision-making, theoretical models often invoke reinforcement-based mechanisms, performance monitoring functions, and core neural features within frontal cortical regions. However, the underlying biological mechanisms remain unknown. In recent models, part of the regulation of behavioral control is based on meta-learning principles, for example, driving exploratory actions by varying a meta-parameter, the inverse temperature, which regulates the contrast between competing action probabilities. Here we investigate how complementary processes between lateral prefrontal cortex (LPFC) and dorsal anterior cingulate cortex (dACC) implement decision regulation during exploratory and exploitative behaviors. Model-based analyses of unit activity recorded in these 2 areas in monkeys first revealed that adaptation of the decision function is reflected in a covariation between LPFC neural activity and the control level estimated from the animal's behavior. Second, dACC more prominently encoded a reflection of outcome uncertainty useful for control regulation based on task monitoring. Model-based analyses also revealed higher information integration before feedback in LPFC, and after feedback in dACC. Overall the data support a role of dACC in integrating reinforcement-based information to regulate decision functions in LPFC. Our results thus provide biological evidence on how prefrontal cortical subregions may cooperate to regulate decision-making., (© The Author 2014. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.)

Current learning theory provides a comprehensive description of how humans and other animals learn, and places behavioral flexibility and automaticity at heart of adaptive behaviors. However, the computations supporting the interactions between goal-directed and habitual decision-making systems are still poorly understood. Previous functional magnetic resonance imaging (fMRI) results suggest that the brain hosts complementary computations that may differentially support goal-directed and habitual processes in the form of a dynamical interplay rather than a serial recruitment of strategies. To better elucidate the computations underlying flexible behavior, we develop a dual-system computational model that can predict both performance (i.e., participants' choices) and modulations in reaction times during learning of a stimulus-response association task. The habitual system is modeled with a simple Q-Learning algorithm (QL). For the goal-directed system, we propose a new Bayesian Working Memory (BWM) model that searches for information in the history of previous trials in order to minimize Shannon entropy. We propose a model for QL and BWM coordination such that the expensive memory manipulation is under control of, among others, the level of convergence of the habitual learning. We test the ability of QL or BWM alone to explain human behavior, and compare them with the performance of model combinations, to highlight the need for such combinations to explain behavior. Two of the tested combination models are derived from the literature, and the latter being our new proposal. In conclusion, all subjects were better explained by model combinations, and the majority of them are explained by our new coordination proposal.

Compared with reward seeking, punishment avoidance learning is less clearly understood at both the computational and neurobiological levels. Here we demonstrate, using computational modelling and fMRI in humans, that learning option values in a relative--context-dependent--scale offers a simple computational solution for avoidance learning. The context (or state) value sets the reference point to which an outcome should be compared before updating the option value. Consequently, in contexts with an overall negative expected value, successful punishment avoidance acquires a positive value, thus reinforcing the response. As revealed by post-learning assessment of options values, contextual influences are enhanced when subjects are informed about the result of the forgone alternative (counterfactual information). This is mirrored at the neural level by a shift in negative outcome encoding from the anterior insula to the ventral striatum, suggesting that value contextualization also limits the need to mobilize an opponent punishment learning system.

Gaining a better understanding of the biological mechanisms underlying the individual variation observed in response to rewards and reward cues could help to identify and treat individuals more prone to disorders of impulsive control, such as addiction. Variation in response to reward cues is captured in rats undergoing autoshaping experiments where the appearance of a lever precedes food delivery. Although no response is required for food to be delivered, some rats (goal-trackers) learn to approach and avidly engage the magazine until food delivery, whereas other rats (sign-trackers) come to approach and engage avidly the lever. The impulsive and often maladaptive characteristics of the latter response are reminiscent of addictive behaviour in humans. In a previous article, we developed a computational model accounting for a set of experimental data regarding sign-trackers and goal-trackers. Here we show new simulations of the model to draw experimental predictions that could help further validate or refute the model. In particular, we apply the model to new experimental protocols such as injecting flupentixol locally into the core of the nucleus accumbens rather than systemically, and lesioning of the core of the nucleus accumbens before or after conditioning. In addition, we discuss the possibility of removing the food magazine during the inter-trial interval. The predictions from this revised model will help us better understand the role of different brain regions in the behaviours expressed by sign-trackers and goal-trackers., (Copyright © 2014 Elsevier Ltd. All rights reserved.)

Animals, including Humans, are prone to develop persistent maladaptive and suboptimal behaviours. Some of these behaviours have been suggested to arise from interactions between brain systems of Pavlovian conditioning, the acquisition of responses to initially neutral stimuli previously paired with rewards, and instrumental conditioning, the acquisition of active behaviours leading to rewards. However the mechanics of these systems and their interactions are still unclear. While extensively studied independently, few models have been developed to account for these interactions. On some experiment, pigeons have been observed to display a maladaptive behaviour that some suggest to involve conflicts between Pavlovian and instrumental conditioning. In a procedure referred as negative automaintenance, a key light is paired with the subsequent delivery of food, however any peck towards the key light results in the omission of the reward. Studies showed that in such procedure some pigeons persisted in pecking to a substantial level despite its negative consequence, while others learned to refrain from pecking and maximized their cumulative rewards. Furthermore, the pigeons that were unable to refrain from pecking could nevertheless shift their pecks towards a harmless alternative key light. We confronted a computational model that combines dual-learning systems and factored representations, recently developed to account for sign-tracking and goal-tracking behaviours in rats, to these negative automaintenance experimental data. We show that it can explain the variability of the observed behaviours and the capacity of alternative key lights to distract pigeons from their detrimental behaviours. These results confirm the proposed model as an interesting tool to reproduce experiments that could involve interactions between Pavlovian and instrumental conditioning. The model allows us to draw predictions that may be experimentally verified, which could help further investigate the neural mechanisms underlying theses interactions.

Reinforcement Learning has greatly influenced models of conditioning, providing powerful explanations of acquired behaviour and underlying physiological observations. However, in recent autoshaping experiments in rats, variation in the form of Pavlovian conditioned responses (CRs) and associated dopamine activity, have questioned the classical hypothesis that phasic dopamine activity corresponds to a reward prediction error-like signal arising from a classical Model-Free system, necessary for Pavlovian conditioning. Over the course of Pavlovian conditioning using food as the unconditioned stimulus (US), some rats (sign-trackers) come to approach and engage the conditioned stimulus (CS) itself - a lever - more and more avidly, whereas other rats (goal-trackers) learn to approach the location of food delivery upon CS presentation. Importantly, although both sign-trackers and goal-trackers learn the CS-US association equally well, only in sign-trackers does phasic dopamine activity show classical reward prediction error-like bursts. Furthermore, neither the acquisition nor the expression of a goal-tracking CR is dopamine-dependent. Here we present a computational model that can account for such individual variations. We show that a combination of a Model-Based system and a revised Model-Free system can account for the development of distinct CRs in rats. Moreover, we show that revising a classical Model-Free system to individually process stimuli by using factored representations can explain why classical dopaminergic patterns may be observed for some rats and not for others depending on the CR they develop. In addition, the model can account for other behavioural and pharmacological results obtained using the same, or similar, autoshaping procedures. Finally, the model makes it possible to draw a set of experimental predictions that may be verified in a modified experimental protocol. We suggest that further investigation of factored representations in computational neuroscience studies may be useful.

Recent experiments showed that the bio-mechanical ease and end-point stability associated to reaching movements are predicted prior to movement onset, and that these factors exert a significant influence on the choice of movement. As an extension of these results, here we investigate whether the knowledge about biomechanical costs and their influence on decision-making are the result of an adaptation process taking place during each experimental session or whether this knowledge was learned at an earlier stage of development. Specifically, we analysed both the pattern of decision-making and its fluctuations during each session, of several human subjects making free choices between two reaching movements that varied in path distance (target relative distance), biomechanical cost, aiming accuracy and stopping requirement. Our main result shows that the effect of biomechanics is well established at the start of the session, and that, consequently, the learning of biomechanical costs in decision-making occurred at an earlier stage of development. As a means to characterise the dynamics of this learning process, we also developed a model-based reinforcement learning model, which generates a possible account of how biomechanics may be incorporated into the motor plan to select between reaching movements. Results obtained in simulation showed that, after some pre-training corresponding to a motor babbling phase, the model can reproduce the subjects' overall movement preferences. Although preliminary, this supports that the knowledge about biomechanical costs may have been learned in this manner, and supports the hypothesis that the fluctuations observed in the subjects' behaviour may adapt in a similar fashion., (Copyright © 2013 Elsevier Ltd. All rights reserved.)

Head direction (HD) neurons fire selectively according to head orientation in the yaw plane relative to environmental landmark cues. Head movements provoke optic field flow signals that enter the vestibular nuclei, indicating head velocity, and hence angular displacements. To test whether optic field flow alone affects the directional firing of HD neurons, rats walked about on a circular platform as a spot array was projected onto the surrounding floor-to-ceiling cylindrical black curtain. Directional responses in the anterodorsal thalamus of four rats remained stable as they moved about with the point field but in the absence of landmark cues. Then, the spherical projector was rotated about its yaw axis at 4.5°/s for ∼90 s. In 27 sessions the mean drift speed of the preferred directions (PDs) was 1.48°/s (SD=0.78°/s; range: 0.15 to 2.88°/s). Thus, optic flow stimulation entrained PDs, albeit at drift speeds slower than the field rotation. This could be due to conflicts with vestibular, motor command, and efferent copy signals. After field rotation ended, 20/27 PDs drifted back to within 45° of the initial values over several minutes, generally following the shortest path to return to the initial value. Poststimulation drifts could change speed and/or direction, with mean speeds of 0.68±0.64°/s (range 0 to 1.36°/s). Since the HD cell pathway (containing anterodorsal thalamus) is the only known projection of head direction information to entorhinal grid cells and hippocampal place cells, yaw plane optic flow signals likely influence representations in this spatial reference coordinate system for orientation and navigation.

Converging evidence suggest that the medial prefrontal cortex (MPFC) is involved in feedback categorization, performance monitoring, and task monitoring, and may contribute to the online regulation of reinforcement learning (RL) parameters that would affect decision-making processes in the lateral prefrontal cortex (LPFC). Previous neurophysiological experiments have shown MPFC activities encoding error likelihood, uncertainty, reward volatility, as well as neural responses categorizing different types of feedback, for instance, distinguishing between choice errors and execution errors. Rushworth and colleagues have proposed that the involvement of MPFC in tracking the volatility of the task could contribute to the regulation of one of RL parameters called the learning rate. We extend this hypothesis by proposing that MPFC could contribute to the regulation of other RL parameters such as the exploration rate and default action values in case of task shifts. Here, we analyze the sensitivity to RL parameters of behavioral performance in two monkey decision-making tasks, one with a deterministic reward schedule and the other with a stochastic one. We show that there exist optimal parameter values specific to each of these tasks, that need to be found for optimal performance and that are usually hand-tuned in computational models. In contrast, automatic online regulation of these parameters using some heuristics can help producing a good, although non-optimal, behavioral performance in each task. We finally describe our computational model of MPFC-LPFC interaction used for online regulation of the exploration rate and its application to a human-robot interaction scenario. There, unexpected uncertainties are produced by the human introducing cued task changes or by cheating. The model enables the robot to autonomously learn to reset exploration in response to such uncertain cues and events. The combined results provide concrete evidence specifying how prefrontal cortical subregions may cooperate to regulate RL parameters. It also shows how such neurophysiologically inspired mechanisms can control advanced robots in the real world. Finally, the model's learning mechanisms that were challenged in the last robotic scenario provide testable predictions on the way monkeys may learn the structure of the task during the pretraining phase of the previous laboratory experiments., (Copyright © 2013 Elsevier B.V. All rights reserved.)

Behavior in spatial navigation is often organized into map-based (place-driven) vs. map-free (cue-driven) strategies; behavior in operant conditioning research is often organized into goal-directed vs. habitual strategies. Here we attempt to unify the two. We review one powerful theory for distinct forms of learning during instrumental conditioning, namely model-based (maintaining a representation of the world) and model-free (reacting to immediate stimuli) learning algorithms. We extend these lines of argument to propose an alternative taxonomy for spatial navigation, showing how various previously identified strategies can be distinguished as "model-based" or "model-free" depending on the usage of information and not on the type of information (e.g., cue vs. place). We argue that identifying "model-free" learning with dorsolateral striatum and "model-based" learning with dorsomedial striatum could reconcile numerous conflicting results in the spatial navigation literature. From this perspective, we further propose that the ventral striatum plays key roles in the model-building process. We propose that the core of the ventral striatum is positioned to learn the probability of action selection for every transition between states of the world. We further review suggestions that the ventral striatal core and shell are positioned to act as "critics" contributing to the computation of a reward prediction error for model-free and model-based systems, respectively.

We continuously face the dilemma of choosing between actions that gather new information or actions that exploit existing knowledge. This "exploration-exploitation" trade-off depends on the environment: stability favors exploiting knowledge to maximize gains; volatility favors exploring new options and discovering new outcomes. Here we set out to reconcile recent evidence for dopamine's involvement in the exploration-exploitation trade-off with the existing evidence for basal ganglia control of action selection, by testing the hypothesis that tonic dopamine in the striatum, the basal ganglia's input nucleus, sets the current exploration-exploitation trade-off. We first advance the idea of interpreting the basal ganglia output as a probability distribution function for action selection. Using computational models of the full basal ganglia circuit, we showed that, under this interpretation, the actions of dopamine within the striatum change the basal ganglia's output to favor the level of exploration or exploitation encoded in the probability distribution. We also found that our models predict striatal dopamine controls the exploration-exploitation trade-off if we instead read-out the probability distribution from the target nuclei of the basal ganglia, where their inhibitory input shapes the cortical input to these nuclei. Finally, by integrating the basal ganglia within a reinforcement learning model, we showed how dopamine's effect on the exploration-exploitation trade-off could be measurable in a forced two-choice task. These simulations also showed how tonic dopamine can appear to affect learning while only directly altering the trade-off. Thus, our models support the hypothesis that changes in tonic dopamine within the striatum can alter the exploration-exploitation trade-off by modulating the output of the basal ganglia.

A major challenge in modern robotics is to liberate robots from controlled industrial settings, and allow them to interact with humans and changing environments in the real-world. The current research attempts to determine if a neurophysiologically motivated model of cortical function in the primate can help to address this challenge. Primates are endowed with cognitive systems that allow them to maximize the feedback from their environment by learning the values of actions in diverse situations and by adjusting their behavioral parameters (i.e., cognitive control) to accommodate unexpected events. In such contexts uncertainty can arise from at least two distinct sources - expected uncertainty resulting from noise during sensory-motor interaction in a known context, and unexpected uncertainty resulting from the changing probabilistic structure of the environment. However, it is not clear how neurophysiological mechanisms of reinforcement learning and cognitive control integrate in the brain to produce efficient behavior. Based on primate neuroanatomy and neurophysiology, we propose a novel computational model for the interaction between lateral prefrontal and anterior cingulate cortex reconciling previous models dedicated to these two functions. We deployed the model in two robots and demonstrate that, based on adaptive regulation of a meta-parameter β that controls the exploration rate, the model can robustly deal with the two kinds of uncertainties in the real-world. In addition the model could reproduce monkey behavioral performance and neurophysiological data in two problem-solving tasks. A last experiment extends this to human-robot interaction with the iCub humanoid, and novel sources of uncertainty corresponding to "cheating" by the human. The combined results provide concrete evidence for the ability of neurophysiologically inspired cognitive systems to control advanced robots in the real-world.

Simultaneous recordings of many single neurons reveals unique insights into network processing spanning the timescale from single spikes to global oscillations. Neurons dynamically self-organize in subgroups of coactivated elements referred to as cell assemblies. Furthermore, these cell assemblies are reactivated, or replayed, preferentially during subsequent rest or sleep episodes, a proposed mechanism for memory trace consolidation. Here we employ Principal Component Analysis to isolate such patterns of neural activity. In addition, a measure is developed to quantify the similarity of instantaneous activity with a template pattern, and we derive theoretical distributions for the null hypothesis of no correlation between spike trains, allowing one to evaluate the statistical significance of instantaneous coactivations. Hence, when applied in an epoch different from the one where the patterns were identified, (e.g. subsequent sleep) this measure allows to identify times and intensities of reactivation. The distribution of this measure provides information on the dynamics of reactivation events: in sleep these occur as transients rather than as a continuous process.

To study the interplay between hippocampus and medial prefrontal cortex (Pfc) and its importance for learning and memory consolidation, we measured the coherence in theta oscillations between these two structures in rats learning new rules on a Y maze. Coherence peaked at the choice point, most strongly after task rule acquisition. Simultaneously, Pfc pyramidal neurons reorganized their phase, concentrating at hippocampal theta trough, and synchronous cell assemblies emerged. This synchronous state may result from increased inhibition exerted by interneurons on pyramidal cells, as measured by cross-correlation, and could be modulated by dopamine: we found similar hippocampal-Pfc theta coherence increases and neuronal phase shifts following local administration of dopamine in Pfc of anesthetized rats. Pfc cell assemblies emerging during high coherence were preferentially replayed during subsequent sleep, concurrent with hippocampal sharp waves. Thus, hippocampal/prefrontal coherence could lead to synchronization of reward predicting activity in prefrontal networks, tagging it for subsequent memory consolidation.

During Slow Wave Sleep (SWS), cortical activity is dominated by endogenous processes modulated by slow oscillations (0.1-1 Hz): cell ensembles fluctuate between states of sustained activity (UP states) and silent epochs (DOWN states). We investigate here the temporal structure of ensemble activity during UP states by means of multiple single unit recordings in the prefrontal cortex of naturally sleeping rats. As previously shown, the firing rate of each PFC cell peaks at a distinct time lag after the DOWN/UP transition in a consistent order. We show here that, conversely, the latency of the first spike after the UP state onset depends primarily on the session-averaged firing rates of cells (which can be considered as an indirect measure of their intrinsic excitability). This latency can be explained by a simple homogeneous process (Poisson model) of cell firing, with sleep averaged firing rates employed as parameters. Thus, at DOWN/UP transitions, neurons are affected both by a slow process, possibly originating in the cortical network, modulating the time course of firing for each cell, and by a fast, relatively stereotyped reinstatement of activity, related mostly to global activity levels.

Slow-wave sleep (SWS) is important for memory consolidation. During sleep, neural patterns reflecting previously acquired information are replayed. One possible reason for this is that such replay exchanges information between hippocampus and neocortex, supporting consolidation. We recorded neuron ensembles in the rat medial prefrontal cortex (mPFC) to study memory trace reactivation during SWS following learning and execution of cross-modal strategy shifts. In general, reactivation of learning-related patterns occurred in distinct, highly synchronized transient bouts, mostly simultaneous with hippocampal sharp wave/ripple complexes (SPWRs), when hippocampal ensemble reactivation and cortico-hippocampal interaction is enhanced. During sleep following learning of a new rule, mPFC neural patterns that appeared during response selection replayed prominently, coincident with hippocampal SPWRs. This was learning dependent, as the patterns appeared only after rule acquisition. Therefore, learning, or the resulting reliable reward, influenced which patterns were most strongly encoded and successively reactivated in the hippocampal/prefrontal network.

It has been proposed that the striatum plays a crucial role in learning to select appropriate actions, optimizing rewards according to the principles of 'Actor-Critic' models of trial-and-error learning. The ventral striatum (VS), as Critic, would employ a temporal difference (TD) learning algorithm to predict rewards and drive dopaminergic neurons. This study examined this model's adequacy for VS responses to multiple rewards in rats. The respective arms of a plus-maze provided rewards of varying magnitudes; multiple rewards were provided at 1-s intervals while the rat stood still. Neurons discharged phasically prior to each reward, during both initial approach and immobile waiting, demonstrating that this signal is predictive and not simply motor-related. In different neurons, responses could be greater for early, middle or late droplets in the sequence. Strikingly, this activity often reappeared after the final reward, as if in anticipation of yet another. In contrast, previous TD learning models show decremental reward-prediction profiles during reward consumption due to a temporal-order signal introduced to reproduce accurate timing in dopaminergic reward-prediction error signals. To resolve this inconsistency in a biologically plausible manner, we adapted the TD learning model such that input information is nonhomogeneously distributed among different neurons. By suppressing reward temporal-order signals and varying richness of spatial and visual input information, the model reproduced the experimental data. This validates the feasibility of a TD-learning architecture where different groups of neurons participate in solving the task based on varied input information.

Head direction cells, which are functionally coupled to 'place' cells of the hippocampus, a structure critically involved in spatial cognition, are likely neural substrates for the sense of direction. Here we studied the mechanism by which head direction cells are principally anchored to background visual cues [M.B. Zugaro et al. (2001) J. Neurosci., 21, RC154,1-5]. Anterodorsal thalamic head direction cells were recorded while the rat foraged on a small elevated platform in a 3-m diameter cylindrical enclosure. A large card was placed in the background, near the curtain, and a smaller card was placed in the foreground, near the platform. The cards were identically marked, proportionally dimensioned, subtended the same visual angles from the central vantage point and separated by 90 degrees. The rat was then disoriented in darkness, the cards were rotated by 90 degrees in opposite directions about the center and the rat was returned. Preferred directions followed either the background card, foreground card or midpoint between the two cards. In continuous lighting, preferred directions shifted to follow the background cue in most cases (30 of the 53 experiments, Batschelet V-test, P < 0.01). Stroboscopic illumination, which perturbs dynamic visual signals (e.g. motion parallax), blocked this selectivity. Head direction cells remained equally anchored to the background card, foreground card or configuration of the two cards (Watson test, P > 0.1). This shows that dynamic visual signals are critical in distinguishing typically more stable background cues which govern spatial neuronal responses and orientation behaviors.