Search

Reinforcement learning (RL) is widely regarded as divisible into two distinct computational strategies. Model-free learning is a simple RL process in which a value is associated with actions, whereas model-based learning relies on the formation of internal models of the environment to maximise reward. Recently, theoretical and animal work has suggested that such models might be used to train model-free behaviour, reducing the burden of costly forward planning. Here we devised a way to probe this possibility in human behaviour. We adapted a two-stage decision task and found evidence that model-based processes at the time of learning can alter model-free valuation in healthy individuals. We asked people to rate subjective value of an irrelevant feature that was seen at the time a model-based decision would have been made. These irrelevant feature value ratings were updated by rewards, but in a way that accounted for whether the selected action retrospectively ought to have been taken. This model-based influence on model-free value ratings was best accounted for by a reward prediction error that was calculated relative to the decision path that would most likely have led to the reward. This effect occurred independently of attention and was not present when participants were not explicitly told about the structure of the environment. These findings suggest that current conceptions of model-based and model-free learning require updating in favour of a more integrated approach. Our task provides an empirical handle for further study of the dialogue between these two learning systems in the future.

Apathy and impulsivity are debilitating conditions associated with many neuropsychiatric conditions, and expressed to variable degrees in healthy people. While some theories suggest that they lie at different ends of a continuum, others suggest their possible co-existence. Surprisingly little is known, however, about their empirical association in the general population. Here, gathering data from six large studies ($$n = 3755$$ n = 3755 ), we investigated the relationship between measures of apathy and impulsivity in young adults. The questionnaires included commonly used self-assessment tools—Apathy Evaluation Scale, Barratt Impulsiveness Scale (BIS-11) and UPPS-P Scale—as well as a more recent addition, the Apathy Motivation Index (AMI). Remarkably, across datasets and assessment tools, global measures of apathy and impulsivity correlated positively. However, analysis of sub-scale scores revealed a more complex relationship. Although most dimensions correlated positively with one another, there were two important exceptions revealed using the AMI scale. Social apathy was mostly negatively correlated with impulsive behaviour, and emotional apathy was orthogonal to all other sub-domains. These results suggest that at a global level, apathy and impulsivity do not exist at distinct ends of a continuum. Instead, paradoxically, they most often co-exist in young adults. Processes underlying social and emotional apathy, however, appear to be different and dissociable from behavioural apathy and impulsivity.

The Apolipoprotein-E (APOE) ε4 gene allele, the highest known genetic risk factor for Alzheimer’s disease, has paradoxically been well preserved in the human population. One possible explanation offered by evolutionary biology for survival of deleterious genes is antagonistic pleiotropy. This theory proposes that such genetic variants might confer an advantage, even earlier in life when humans are also reproductively fit. The results of some small-cohort studies have raised the possibility of such a pleiotropic effect for the ε4 allele in short-term memory (STM) but the findings have been inconsistent. Here, we tested STM performance in a large cohort of individuals (N = 1277); nine hundred and fifty-nine of which included carrier and non-carriers of the APOE ε4 gene, those at highest risk of developing Alzheimer’s disease. We first confirm that this task is sensitive to subtle deterioration in memory performance across ageing. Importantly, individuals carrying the APOE ε4 gene actually exhibited a significant memory advantage across all ages, specifically for brief retention periods but crucially not for longer durations. Together, these findings present the strongest evidence to date for a gene having an antagonistic pleiotropy effect on human cognitive function across a wide age range, and hence provide an explanation for the survival of the APOE ε4 allele in the gene pool.