Search

To better understand neural processing during adaptive learning of stimulus‐response‐reward contingencies, we recorded synchrony of neuronal activity in anterior cingulate cortex (ACC) and hippocampal rhythms in male rats acquiring and switching between spatial and visual discrimination tasks in a Y‐maze. ACC population activity as well as single unit activity shifted shortly after task rule changes or just before the rats adopted different task strategies. Hippocampal theta oscillations (associated with memory encoding) modulated an elevated proportion of rule‐change responsive neurons (70%), but other neurons that were correlated with strategy‐change, strategy value and reward‐rate were not. However, hippocampal sharp wave‐ripples modulated significantly higher proportions of rule‐change, strategy‐change and reward‐rate responsive cells during post‐session sleep but not pre‐session sleep. This suggests an underestimated mechanism for hippocampal mismatch and contextual signals to facilitate ACC to detect contingency changes for cognitive flexibility, a function that is attenuated after it is damaged. [ABSTRACT FROM AUTHOR]

Minimizing negative impacts of Artificial Intelligent (AI) systems on human societies without human supervision requires them to be able to align with human values. However, most current work only addresses this issue from a technical point of view, e.g., improving current methods relying on reinforcement learning from human feedback, neglecting what it means and is required for alignment to occur. Here, we propose to distinguish strong and weak value alignment. Strong alignment requires cognitive abilities (either human-like or different from humans) such as understanding and reasoning about agents' intentions and their ability to causally produce desired effects. We argue that this is required for AI systems like large language models (LLMs) to be able to recognize situations presenting a risk that human values may be flouted. To illustrate this distinction, we present a series of prompts showing ChatGPT's, Gemini's and Copilot's failures to recognize some of these situations. We moreover analyze word embeddings to show that the nearest neighbors of some human values in LLMs differ from humans' semantic representations. We then propose a new thought experiment that we call "the Chinese room with a word transition dictionary", in extension of John Searle's famous proposal. We finally mention current promising research directions towards a weak alignment, which could produce statistically satisfying answers in a number of common situations, however so far without ensuring any truth value., (© 2024. The Author(s).)

In uncertain environments in which resources fluctuate continuously, animals must permanently decide whether to stabilise learning and exploit what they currently believe to be their best option, or instead explore potential alternatives and learn fast from new observations. While such a trade-off has been extensively studied in pretrained animals facing non-stationary decision-making tasks, it is yet unknown how they progressively tune it while learning the task structure during pretraining. Here, we compared the ability of different computational models to account for long-term changes in the behaviour of 24 rats while they learned to choose a rewarded lever in a three-armed bandit task across 24 days of pretraining. We found that the day-by-day evolution of rat performance and win-shift tendency revealed a progressive stabilisation of the way they regulated reinforcement learning parameters. We successfully captured these behavioural adaptations using a meta-learning model in which either the learning rate or the inverse temperature was controlled by the average reward rate., (© 2024 The Author(s). European Journal of Neuroscience published by Federation of European Neuroscience Societies and John Wiley & Sons Ltd.)

This study was conducted to examine the chemical composition of the essential oils (EOs) from six Tunisian Eucalyptus species and to evaluate their anti-enzymatic and antibiofilm activities. The EOs were obtained through hydro-distillation of dried leaves and subsequently analyzed using GC/MS. The main class of compounds was constituted by oxygenated monoterpenes, particularly prominent in E. brevifolia (75.7%), E. lehmannii (72.8%), and E. woollsiana (67%). Anti-enzymatic activities against cholinesterases, α-amylase, and α-glucosidase were evaluated using spectrophotometric methods. Notably, the E. brevifolia , E. extensa , E. leptophylla , E. patellaris , and E. woollsiana EOs displayed potent acetylcholinesterase (AChE) inhibition (IC 50 : 0.25-0.60 mg/mL), with E. lehmannii exhibiting lower activity (IC 50 : 1.2 mg/mL). E. leptophylla and E. brevifolia showed remarkable α-amylase inhibition (IC 50 : 0.88 mg/mL), while E. brevifolia and E. leptophylla significantly hindered α-glucosidase (IC 50 < 30 mg/mL), distinguishing them from other EOs with limited effects. Additionally, the EOs were assessed for their anti-biofilm properties of Gram-positive ( Staphylococcus aureus and Listeria monocytogenes ) and Gram-negative ( Acinetobacter baumannii , Pseudomonas aeruginosa and Escherichia coli ) bacterial strains. The E. extensa EO demonstrated the main antibiofilm effect against E. coli and L. monocytogenes with an inhibition > 80% at 10 mg/mL. These findings could represent a basis for possible further use of Eucalyptus EOs in the treatment of human microbial infections and/or as a coadjutant in preventing and treating Alzheimer's disease and/or diabetes mellitus.

In recent years, soft robots gain increasing attention as a result of their compliance when operating in unstructured environments, and their flexibility that ensures safety when interacting with humans. However, challenges lie on the difficulty to develop control algorithms due to various limitations induced by their soft structure. In this paper, we introduce a novel technique that aims to perform motion control of a modular bio-inspired soft-robotic arm, with the main focus lying on facilitating the qualitative reproduction of well-specified periodic trajectories. The introduced method combines the notion behind two previously developed methodologies both based on the Movement Primitive (MP) theory, by exploiting their capabilities while coping with their main drawbacks. Concretely, the requested actuation is initially computed using a Probabilistic MP (ProMP)-based method that considers the trajectory as a combination of simple movements previously learned and stored as a MP library. Subsequently, the key components of the resulting actuation are extracted and filtered in the frequency domain. These are eventually used as input to a Central Pattern Generator (CPG)-based model that takes over the generation of rhythmic patterns at the motor level. The proposed methodology is evaluated on a two-module soft arm. Results show that the first algorithmic component (ProMP) provides an immediate estimation of the requested actuation by avoiding time-consuming training, while the latter (CPG) further simplifies the execution by allowing its control through a low-dimensional parameterization. Altogether, these results open new avenues for the rapid acquisition of periodic movements in soft robots, and their compression into CPG parameters for long-term storage and execution., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2023 Oikonomou, Dometios, Khamassi and Tzafestas.)

Our brain is continuously challenged by daily experiences. Thus, how to avoid systematic erasing of previously encoded memories? While it has been proposed that a dual-learning system with 'slow' learning in the cortex and 'fast' learning in the hippocampus could protect previous knowledge from interference, this has never been observed in the living organism. Here, we report that increasing plasticity via the viral-induced overexpression of RGS14414 in the prelimbic cortex leads to better one-trial memory, but that this comes at the price of increased interference in semantic-like memory. Indeed, electrophysiological recordings showed that this manipulation also resulted in shorter NonREM-sleep bouts, smaller delta-waves and decreased neuronal firing rates. In contrast, hippocampal-cortical interactions in form of theta coherence during wake and REM-sleep as well as oscillatory coupling during NonREM-sleep were enhanced. Thus, we provide the first experimental evidence for the long-standing and unproven fundamental idea that high thresholds for plasticity in the cortex protect preexisting memories and modulating these thresholds affects both memory encoding and consolidation mechanisms., Competing Interests: IN, AA, AS, MB, SN, AR, AA, Jv, MK, ZK, LG No competing interests declared, (© 2023, Navarro Lobato et al.)

Healthcare professionals' statistical illiteracy can impair medical decision quality and compromise patient safety. Previous studies have documented clinicians' insufficient proficiency in statistics and a tendency in overconfidence. However, an underexplored aspect is clinicians' awareness of their lack of statistical knowledge that precludes any corrective intervention attempt. Here, we investigated physicians', residents' and medical students' alignment between subjective confidence judgments and objective accuracy in basic medical statistics. We also examined how gender, profile of experience and practice of research activity affect this alignment, and the influence of problem framing (conditional probabilities, CP vs. natural frequencies, NF). Eight hundred ninety-eight clinicians completed an online survey assessing skill and confidence on three topics: vaccine efficacy, p value and diagnostic test results interpretation. Results evidenced an overall consistent poor proficiency in statistics often combined with high confidence, even in incorrect answers. We also demonstrate that despite overconfidence bias, clinicians show a degree of metacognitive sensitivity, as their confidence judgments discriminate between their correct and incorrect answers. Finally, we confirm the positive impact of the more intuitive NF framing on accuracy. Together, our results pave the way for the development of teaching recommendations and pedagogical interventions such as promoting metacognition on basic knowledge and statistical reasoning as well as the use of NF to tackle statistical illiteracy in the medical context., (© 2023. The Author(s).)

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states. SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms., (Copyright © 2023 the authors.)

Experience replay is widely used in AI to bootstrap reinforcement learning (RL) by enabling an agent to remember and reuse past experiences. Classical techniques include shuffled-, reversed-ordered- and prioritized-memory buffers, which have different properties and advantages depending on the nature of the data and problem. Interestingly, recent computational neuroscience work has shown that these techniques are relevant to model hippocampal reactivations recorded during rodent navigation. Nevertheless, the brain mechanisms for orchestrating hippocampal replay are still unclear. In this paper, we present recent neurorobotics research aiming to endow a navigating robot with a neuro-inspired RL architecture (including different learning strategies, such as model-based (MB) and model-free (MF), and different replay techniques). We illustrate through a series of numerical simulations how the specificities of robotic experimentation (e.g., autonomous state decomposition by the robot, noisy perception, state transition uncertainty, non-stationarity) can shed new lights on which replay techniques turn out to be more efficient in different situations. Finally, we close the loop by raising new hypotheses for neuroscience from such robotic models of hippocampal replay., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Massi, Barthélemy, Mailly, Dromnelle, Canitrot, Poniatowski, Girard and Khamassi.)

The ability to attribute thoughts to others, also called theory of mind (TOM), has been extensively studied in humans; however, its evolutionary origins have been challenged. Computationally, the basis of TOM has been interpreted within the predictive coding framework and associated with activity in the temporoparietal junction (TPJ). Here, we revealed, using a nonlinguistic task and functional magnetic resonance imaging, that activity in a region of the macaque middle superior temporal cortex was specifically modulated by the predictability of social situations. As in human TPJ, this region could be distinguished from other temporal regions involved in face processing. Our result suggests the existence of a precursor for the TOM ability in the last common ancestor of human and Old World monkeys.

This special issue, commissioned after the 4th Quadrennial Meeting on Orbitofrontal Cortex Function held in Paris in November of 2019 (https://ofc2019.sciencesconf.org/), is intended to provide a snapshot of this ongoing transformation; we hope that the ideas presented herein will provide a foundation for the next stage in the evolution of our understanding of this magical brain region. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Our understanding of orbitofrontal cortex (OFC) function has progressed remarkably over the past decades in part due to theoretical advances in associative and reinforcement learning theories. These theoretical accounts of OFC function have implicated the region in progressively more psychologically refined processes from the value and sensory-specific properties of expected outcomes to the representation and inference over latent state representations in cognitive maps of task space. While these accounts have been successful at modeling many of the effects of causal manipulation of OFC function in both rodents and primates, recent findings suggest that further refinement of our current models are still required. Here, we briefly review how our understanding of OFC function has developed to understand two cardinal deficits following OFC dysfunction: Reversal learning and outcome devaluation. We then consider recent findings that OFC dysfunction also significantly affects initial acquisition learning, often assumed to be intact. To account for these findings, we consider a possible role for the OFC in the arbitration and exploration between model-free (MF) and model-based (MB) learning systems, offline updating of MB representations. While the function of the OFC as a whole is still likely to be integral to the formation and use of a cognitive map of task space, these refinements suggest a way in which distinct orbital subregions, such as the rodent lateral OFC, might contribute to this overall function. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The antibody molecule comprises a variable domain conferring antigen specificity and affinity distinct from the heavy chain constant (CH) domains dictating effector functions. We here interrogate this paradigm by evaluating the unique influence of the CH1α domain on epitope specificity and functions using two mucosal gp41-specific Fab-IgAs (FabA) derived from HIV-1 highly-exposed but persistently seronegative individuals (HESN). These HESN develop selectively affinity-matured HIV-1-specific mucosal IgA that target the gp41 viral envelope and might provide protection although by unclear mechanisms. Isotype-switching FabAs into Fab-IgGs (FabGs) results in a >10-fold loss in affinity for HIV-1 clade A, B, and C gp41, together with reduced neutralization of HIV-1 cross-clade. The FabA conformational epitopes map selectively on gp41 in 6-Helix bundle and pre-fusion conformations cross-clade, unlike FabGs. Finally, we designed in silico, a 12 amino-acid peptide recapitulating one FabA conformational epitope that inhibits the FabA binding to gp41 cross-clade and its neutralizing activity. Altogether, our results reveal that the CH1α domain shapes the antibody paratope through an allosteric effect, thereby strengthening the antibody specificity and functional activities. Further, they clarify the mechanisms by which these HESN IgAs might confer protection against HIV-1-sexual acquisition. The IgA-specific epitope we characterized by reverse vaccinology could help designing a mucosal HIV-1 vaccine., Competing Interests: The authors have declared that no competing interests exist.

Kleefstra syndrome is a disorder caused by a mutation in the EHMT1 gene characterized in humans by general developmental delay, mild to severe intellectual disability and autism. Here, we characterized cumulative memory in the Ehmt1 +/- mouse model using the Object Space Task. We combined conventional behavioral analysis with automated analysis by deep-learning networks, a session-based computational learning model, and a trial-based classifier. Ehmt1 +/- mice showed more anxiety-like features and generally explored objects less, but the difference decreased over time. Interestingly, when analyzing memory-specific exploration, Ehmt1 +/- show increased expression of cumulative memory, but a deficit in a more simple, control memory condition. Using our automatic classifier to differentiate between genotypes, we found that cumulative memory features are better suited for classification than general exploration differences. Thus, detailed behavioral classification with the Object Space Task produced a more detailed behavioral phenotype of the Ehmt1 +/- mouse model., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2020 The Author(s). Published by Elsevier Inc. All rights reserved.)

People and other animals learn the values of choices by observing the contingencies between them and their outcomes. However, decisions are not guided by choice-linked reward associations alone; macaques also maintain a memory of the general, average reward rate - the global reward state - in an environment. Remarkably, global reward state affects the way that each choice outcome is valued and influences future decisions so that the impact of both choice success and failure is different in rich and poor environments. Successful choices are more likely to be repeated but this is especially the case in rich environments. Unsuccessful choices are more likely to be abandoned but this is especially likely in poor environments. Functional magnetic resonance imaging (fMRI) revealed two distinct patterns of activity, one in anterior insula and one in the dorsal raphe nucleus, that track global reward state as well as specific outcome events.

Hippocampal offline reactivations during reward-based learning, usually categorized as replay events, have been found to be important for performance improvement over time and for memory consolidation. Recent computational work has linked these phenomena to the need to transform reward information into state-action values for decision making and to propagate it to all relevant states of the environment. Nevertheless, it is still unclear whether an integrated reinforcement learning mechanism could account for the variety of awake hippocampal reactivations, including variety in order (forward and reverse reactivated trajectories) and variety in the location where they occur (reward site or decision-point). Here, we present a model-based bidirectional search model which accounts for a variety of hippocampal reactivations. The model combines forward trajectory sampling from current position and backward sampling through prioritized sweeping from states associated with large reward prediction errors until the two trajectories connect. This is repeated until stabilization of state-action values (convergence), which could explain why hippocampal reactivations drastically diminish when the animal's performance stabilizes. Simulations in a multiple T-maze task show that forward reactivations are prominently found at decision-points while backward reactivations are exclusively generated at reward sites. Finally, the model can generate imaginary trajectories that are not allowed to the agent during task performance. We raise some experimental predictions and implications for future studies of the role of the hippocampo-prefronto-striatal network in learning.

In the context of Pavlovian conditioning, two types of behaviour may emerge within the population (Flagel et al. Nature, 469(7328): 53-57, 2011). Animals may choose to engage either with the conditioned stimulus (CS), a behaviour known as sign-tracking (ST) which is sensitive to dopamine inhibition for its acquisition, or with the food cup in which the reward or unconditioned stimulus (US) will eventually be delivered, a behaviour known as goal-tracking (GT) which is dependent on dopamine for its expression only. Previous work by Lesaint et al. (PLoS Comput Biol, 10(2), 2014) offered a computational explanation for these phenomena and led to the prediction that varying the duration of the inter-trial interval (ITI) would change the relative ST-GT proportion in the population as well as phasic dopamine responses. A recent study verified this prediction, but also found a rich variance of ST and GT behaviours within the trial which goes beyond the original computational model. In this paper, we provide a computational perspective on these novel results.

Declarative memory encompasses representations of specific events as well as knowledge extracted by accumulation over multiple episodes. To investigate how these different sorts of memories are created, we developed a new behavioral task in rodents. The task consists of 3 distinct conditions (stable, overlapping, and random). Rodents are exposed to multiple sample trials, in which they explore objects in specific spatial arrangements, with object identity changing from trial to trial. In the stable condition, the locations are constant during all sample trials even though the objects themselves change; in the test trial, 1 object's location is changed. In the random condition, object locations are presented in the sample phase without a specific spatial pattern. In the overlapping condition, 1 location is shared (overlapping) between all trials, while the other location changes during sample trials. We show that in the overlapping condition, instead of only remembering the last sample trial, rodents form a cumulative memory of the sample trials. Here, we could show that both mice and rats can accumulate information across multiple trials and express a long-term abstracted memory., Competing Interests: The authors have declared that no competing interests exist.

In a volatile environment where rewards are uncertain, successful performance requires a delicate balance between exploitation of the best option and exploration of alternative choices. It has theoretically been proposed that dopamine contributes to the control of this exploration-exploitation trade-off, specifically that the higher the level of tonic dopamine, the more exploitation is favored. We demonstrate here that there is a formal relationship between the rescaling of dopamine positive reward prediction errors and the exploration-exploitation trade-off in simple non-stationary multi-armed bandit tasks. We further show in rats performing such a task that systemically antagonizing dopamine receptors greatly increases the number of random choices without affecting learning capacities. Simulations and comparison of a set of different computational models (an extended Q-learning model, a directed exploration model, and a meta-learning model) fitted on each individual confirm that, independently of the model, decreasing dopaminergic activity does not affect learning rate but is equivalent to an increase in random exploration rate. This study shows that dopamine could adapt the exploration-exploitation trade-off in decision-making when facing changing environmental contingencies.

Accumulating evidence suggest that human behavior in trial-and-error learning tasks based on decisions between discrete actions may involve a combination of reinforcement learning (RL) and working-memory (WM). While the understanding of brain activity at stake in this type of tasks often involve the comparison with non-human primate neurophysiological results, it is not clear whether monkeys use similar combined RL and WM processes to solve these tasks. Here we analyzed the behavior of five monkeys with computational models combining RL and WM. Our model-based analysis approach enables to not only fit trial-by-trial choices but also transient slowdowns in reaction times, indicative of WM use. We found that the behavior of the five monkeys was better explained in terms of a combination of RL and WM despite inter-individual differences. The same coordination dynamics we used in a previous study in humans best explained the behavior of some monkeys while the behavior of others showed the opposite pattern, revealing a possible different dynamics of WM process. We further analyzed different variants of the tested models to open a discussion on how the long pretraining in these tasks may have favored particular coordination dynamics between RL and WM. This points towards either inter-species differences or protocol differences which could be further tested in humans., (Copyright © 2017 Elsevier B.V. All rights reserved.)

Multiple in vivo studies have shown that place cells from the hippocampus replay previously experienced trajectories. These replays are commonly considered to mainly reflect memory consolidation processes. Some data, however, have highlighted a functional link between replays and reinforcement learning (RL). This theory, extensively used in machine learning, has introduced efficient algorithms and can explain various behavioral and physiological measures from different brain regions. RL algorithms could constitute a mechanistic description of replays and explain how replays can reduce the number of iterations required to explore the environment during learning. We review the main findings concerning the different hippocampal replay types and the possible associated RL models (either model-based, model-free, or hybrid model types). We conclude by tying these frameworks together. We illustrate the link between data and RL through a series of model simulations. This review, at the frontier between informatics and biology, paves the way for future work on replays.

In economics and perceptual decision-making contextual effects are well documented, where decision weights are adjusted as a function of the distribution of stimuli. Yet, in reinforcement learning literature whether and how contextual information pertaining to decision states is integrated in learning algorithms has received comparably little attention. Here, we investigate reinforcement learning behavior and its computational substrates in a task where we orthogonally manipulate outcome valence and magnitude, resulting in systematic variations in state-values. Model comparison indicates that subjects' behavior is best accounted for by an algorithm which includes both reference point-dependence and range-adaptation-two crucial features of state-dependent valuation. In addition, we find that state-dependent outcome valuation progressively emerges, is favored by increasing outcome information and correlated with explicit understanding of the task structure. Finally, our data clearly show that, while being locally adaptive (for instance in negative valence and small magnitude contexts), state-dependent valuation comes at the cost of seemingly irrational choices, when options are extrapolated out from their original contexts.

Recent computational models of sign tracking (ST) and goal tracking (GT) have accounted for observations that dopamine (DA) is not necessary for all forms of learning and have provided a set of predictions to further their validity. Among these, a central prediction is that manipulating the intertrial interval (ITI) during autoshaping should change the relative ST-GT proportion as well as DA phasic responses. Here, we tested these predictions and found that lengthening the ITI increased ST, i.e., behavioral engagement with conditioned stimuli (CS) and cue-induced phasic DA release. Importantly, DA release was also present at the time of reward delivery, even after learning, and DA release was correlated with time spent in the food cup during the ITI. During conditioning with shorter ITIs, GT was prominent (i.e., engagement with food cup), and DA release responded to the CS while being absent at the time of reward delivery after learning. Hence, shorter ITIs restored the classical DA reward prediction error (RPE) pattern. These results validate the computational hypotheses, opening new perspectives on the understanding of individual differences in Pavlovian conditioning and DA signaling., Competing Interests: The authors have declared that no competing interests exist.

Despite major progress in Robotics and AI, robots are still basically "zombies" repeatedly achieving actions and tasks without understanding what they are doing. Deep-Learning AI programs classify tremendous amounts of data without grasping the meaning of their inputs or outputs. We still lack a genuine theory of the underlying principles and methods that would enable robots to understand their environment, to be cognizant of what they do, to take appropriate and timely initiatives, to learn from their own experience and to show that they know that they have learned and how. The rationale of this paper is that the understanding of its environment by an agent (the agent itself and its effects on the environment included) requires its self-awareness, which actually is itself emerging as a result of this understanding and the distinction that the agent is capable to make between its own mind-body and its environment. The paper develops along five issues: agent perception and interaction with the environment; learning actions; agent interaction with other agents-specifically humans; decision-making; and the cognitive architecture integrating these capacities., (Copyright © 2018 Chatila, Renaudo, Andries, Chavez-Garcia, Luce-Vayrac, Gottstein, Alami, Clodic, Devin, Girard and Khamassi.)

We present a computational model of spatial navigation comprising different learning mechanisms in mammals, i.e., associative, cognitive mapping and parallel systems. This model is able to reproduce a large number of experimental results in different variants of the Morris water maze task, including standard associative phenomena (spatial generalization gradient and blocking), as well as navigation based on cognitive mapping. Furthermore, we show that competitive and cooperative patterns between different navigation strategies in the model allow to explain previous apparently contradictory results supporting either associative or cognitive mechanisms for spatial learning. The key computational mechanism to reconcile experimental results showing different influences of distal and proximal cues on the behavior, different learning times, and different abilities of individuals to alternatively perform spatial and response strategies, relies in the dynamic coordination of navigation strategies, whose performance is evaluated online with a common currency through a modular approach. We provide a set of concrete experimental predictions to further test the computational model. Overall, this computational work sheds new light on inter-individual differences in navigation learning, and provides a formal and mechanistic approach to test various theories of spatial cognition in mammals.

The protective efficacy of human immunodeficiency virus-1 (HIV-1) antibodies (Abs) remains mostly correlated with their in vitro neutralizing activity engaging their Fab region. However, anti-HIV-1 Abs also mediate a broad array of Fc-mediated effector functions including Ab-dependent cellular cytotoxicity (ADCC), which depend primarily on the Ab isotype. While ADCC is commonly associated with HIV-1 gp120 envelope-specific IgGs, whether IgAs, especially those targeting the HIV-1 gp41 envelope, also mediate ADCC remains elusive. Therefore, to assess the capacity of IgA specific for HIV-1 to induce Fcα-mediated ADCC, we used the gp41 envelope-specific IgA transformed from the broadly neutralizing 2F5-IgG we have previously reported to induce ADCC. We demonstrate that 2F5-IgA engages FcαRI (CD89), expressed on human monocytes used as effector cells, to induce the lysis of HIV-1 Clade A- and B-infected target cells by ADCC. Furthermore, the 2F5-IgA and 2F5-IgG cooperate to enhance target cells lysis by ADCC. Cooperation in ADCC is also observed between 2F5-IgA and the broadly neutralizing 10E8-IgG. These results provide a new perspective for IgA in protection against HIV-1 acquisition or reservoir eradication and suggest that inducing IgA by vaccination, in particular when targeting gp41, in combination with IgG could strengthen protection by complementary and cooperative activities with IgG.

Computer science offers a large set of tools for prototyping, writing, running, testing, validating, sharing and reproducing results; however, computational science lags behind. In the best case, authors may provide their source code as a compressed archive and they may feel confident their research is reproducible. But this is not exactly true. James Buckheit and David Donoho proposed more than two decades ago that an article about computational results is advertising, not scholarship. The actual scholarship is the full software environment, code, and data that produced the result. This implies new workflows, in particular in peer-reviews. Existing journals have been slow to adapt: source codes are rarely requested and are hardly ever actually executed to check that they produce the results advertised in the article. ReScience is a peer-reviewed journal that targets computational research and encourages the explicit replication of already published research, promoting new and open-source implementations in order to ensure that the original research can be replicated from its description. To achieve this goal, the whole publishing chain is radically different from other traditional scientific journals. ReScience resides on GitHub where each new implementation of a computational study is made available together with comments, explanations, and software tests., Competing Interests: Federico Vaggi is an employee of Amazon, Inc., Roman Yurchak is an employee of Symerio, and C. Titus Brown and Nicolas P. Rougier are Academic Editors for PeerJ., (©2017 Rougier et al.)

To explain the high level of flexibility in primate decision-making, theoretical models often invoke reinforcement-based mechanisms, performance monitoring functions, and core neural features within frontal cortical regions. However, the underlying biological mechanisms remain unknown. In recent models, part of the regulation of behavioral control is based on meta-learning principles, for example, driving exploratory actions by varying a meta-parameter, the inverse temperature, which regulates the contrast between competing action probabilities. Here we investigate how complementary processes between lateral prefrontal cortex (LPFC) and dorsal anterior cingulate cortex (dACC) implement decision regulation during exploratory and exploitative behaviors. Model-based analyses of unit activity recorded in these 2 areas in monkeys first revealed that adaptation of the decision function is reflected in a covariation between LPFC neural activity and the control level estimated from the animal's behavior. Second, dACC more prominently encoded a reflection of outcome uncertainty useful for control regulation based on task monitoring. Model-based analyses also revealed higher information integration before feedback in LPFC, and after feedback in dACC. Overall the data support a role of dACC in integrating reinforcement-based information to regulate decision functions in LPFC. Our results thus provide biological evidence on how prefrontal cortical subregions may cooperate to regulate decision-making., (© The Author 2014. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.)

Current learning theory provides a comprehensive description of how humans and other animals learn, and places behavioral flexibility and automaticity at heart of adaptive behaviors. However, the computations supporting the interactions between goal-directed and habitual decision-making systems are still poorly understood. Previous functional magnetic resonance imaging (fMRI) results suggest that the brain hosts complementary computations that may differentially support goal-directed and habitual processes in the form of a dynamical interplay rather than a serial recruitment of strategies. To better elucidate the computations underlying flexible behavior, we develop a dual-system computational model that can predict both performance (i.e., participants' choices) and modulations in reaction times during learning of a stimulus-response association task. The habitual system is modeled with a simple Q-Learning algorithm (QL). For the goal-directed system, we propose a new Bayesian Working Memory (BWM) model that searches for information in the history of previous trials in order to minimize Shannon entropy. We propose a model for QL and BWM coordination such that the expensive memory manipulation is under control of, among others, the level of convergence of the habitual learning. We test the ability of QL or BWM alone to explain human behavior, and compare them with the performance of model combinations, to highlight the need for such combinations to explain behavior. Two of the tested combination models are derived from the literature, and the latter being our new proposal. In conclusion, all subjects were better explained by model combinations, and the majority of them are explained by our new coordination proposal.