Model-Based Reasoning in Humans Becomes Automatic with Training

Model-based and model-free reinforcement learning (RL) have been suggested as algorithmic realizations of goal-directed and habitual action strategies. Model-based RL is more flexible than model-free but requires sophisticated calculations using a learnt model of the world. This has led model-based RL to be identified with slow, deliberative processing, and model-free RL with fast, automatic processing. In support of this distinction, it has recently been shown that model-based reasoning is impaired by placing subjects under cognitive load—a hallmark of non-automaticity. Here, using the same task, we show that cognitive load does not impair model-based reasoning if subjects receive prior training on the task. This finding is replicated across two studies and a variety of analysis methods. Thus, task familiarity permits use of model-based reasoning in parallel with other cognitive demands. The ability to deploy model-based reasoning in an automatic, parallelizable fashion has widespread theoretical implications, particularly for the learning and execution of complex behaviors. It also suggests a range of important failure modes in psychiatric disorders.
Author Summary Automaticity develops with task familiarity. One possible explanation is that automaticity arises when performance of the task becomes habitual, or model-free. Here we asked whether goal-directed, or model-based, reasoning could also become automatic, or resistant to distraction. We used a well-characterized task that differentiates model-based from model-free action. We replicate previous findings that distraction strongly impairs model-based reasoning in task-naive subjects. However, in subjects with prior exposure to the task, distraction does not impair model-based reasoning. This suggests that humans can deploy sophisticated and flexible reasoning more extensively than previously thought.