The relation between reinforcement learning parameters and the influence of reinforcement history on choice behavior

Reinforcement learning (RL) models have been widely used to analyze the choice behavior of humans and other animals in a broad range of fields, including psychology and neuroscience. Linear regression-based models that explicitly represent how reward and choice history influences future choices have also been used to model choice behavior. While both approaches have been used independently, the relation between the two models has not been explicitly described. The aim of the present study is to describe this relation and investigate how the parameters in the RL model mediate the effects of reward and choice history on future choices. To achieve these aims, we performed analytical calculations and numerical simulations. First, we describe a special case in which the RL and regression models can provide equivalent predictions of future choices. The general properties of the RL model are discussed as a departure from this special case. We clarify the role of the RL-model parameters, specifically, the learning rate, inverse temperature, and outcome value (also referred to as the reward value, reward sensitivity, or motivational value), in the formation of history dependence.