Search

Animals rely on internal motivational states to make decisions. The role of motivational salience in decision making is in early stages of mathematical understanding. Here, we propose a reinforcement learning framework that relies on neural networks to learn optimal ongoing behavior for dynamically changing motivation values. First, we show that neural networks implementing Q-learning with motivational salience can navigate in environment with dynamic rewards without adjustments in synaptic strengths when the needs of an agent shift. In this setting, our networks may display elements of addictive behaviors. Second, we use a similar framework in hierarchical manager-agent system to implement a reinforcement learning algorithm with motivation that both infers motivational states and behaves. Finally, we show that, when trained in the Pavlovian conditioning setting, the responses of the neurons in our model resemble previously published neuronal recordings in the ventral pallidum, a basal ganglia structure involved in motivated behaviors. We conclude that motivation allows Q-learning networks to quickly adapt their behavior to conditions when expected reward is modulated by agent's dynamic needs. Our approach addresses the algorithmic rationale of motivation and makes a step toward better interpretability of behavioral data via inference of motivational dynamics in the brain., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2021 Shuvaev, Tran, Stephenson-Jones, Li and Koulakov.)

There is sufficient evidence to believe that variations in the error-related negativity (ERN) are linked to dispositional characteristics in individuals. However, explanations of individual differences in the amplitude of the ERN cannot be derived from functional theories of the ERN. The ERN has a counterpart that occurs after correct responses (correct-response negativity, CRN). Based on the assumption that ERN and CRN reflect an identical cognitive process, variations in CRN might be associated with dispositional characteristics as well. Higher CRN amplitudes have been found to reflect task engagement. In the present study, a simple-choice-reaction task was used to investigate ERN and CRN amplitudes in relation to their score on a conscientiousness scale. The task consisted of a simple rule that required pressing the left or right key when a circle or square appeared, respectively. During alternative conditions that occur infrequently, participants were instructed to violate or reverse the previously established response rules. Smaller ΔERN amplitudes (manifested in almost equal CRN and ERN amplitudes) and a tendency of better task performance from participants scoring high on the conscientiousness scale might indicate a greater focus on the task and higher motivation of responding correctly. In addition, higher Pc amplitudes directly following the CRN indicated that the response monitoring system of less conscientious participants showed a higher disengagement. The role of individual differences in CRN amplitude should be studied in future experiments on performance monitoring.

The study of food aversion in humans by the induction of illness is ethically unthinkable, and it is difficult to propose a type of food that is disgusting for everybody. However, although cheese is considered edible by most people, it can also be perceived as particularly disgusting to some individuals. As such, the perception of cheese constitutes a good model to study the cerebral processes of food disgust and aversion. In this study, we show that a higher percentage of people are disgusted by cheese than by other types of food. Functional magnetic resonance imaging then reveals that the internal and external globus pallidus and the substantia nigra belonging to the basal ganglia are more activated in participants who dislike or diswant to eat cheese (Anti) than in other participants who like to eat cheese, as revealed following stimulation with cheese odors and pictures. We suggest that the aforementioned basal ganglia structures commonly involved in reward are also involved in the aversive motivated behaviors. Our results further show that the ventral pallidum, a core structure of the reward circuit, is deactivated in Anti subjects stimulated by cheese in the wanting task, highlighting the suppression of motivation-related activation in subjects disgusted by cheese.

Basal forebrain (BF) is one of the largest cortically-projecting neuromodulatory systems in the mammalian brain, and plays a key role in attention, arousal, learning and memory. The cortically projecting BF neurons, comprised of mainly magnocellular cholinergic and GABAergic neurons, are widely distributed across several brain regions that spatially overlap with the ventral striatopallidal system at the ventral pallidum (VP). As a first step toward untangling the respective functions of spatially overlapping BF and VP systems, the goal of this study was to comprehensively characterize the behavioral correlates and physiological properties of heterogeneous neuronal populations in the BF region. We found that, while rats performed a reward-biased simple reaction time task, distinct neuronal populations encode either motivational salience or movement information. The motivational salience of attended stimuli is encoded by phasic bursting activity of a large population of slow-firing neurons that have large, broad, and complex action potential waveforms. In contrast, two other separate groups of neurons encode movement-related information, and respectively increase and decrease firing rates while rats maintained fixation. These two groups of neurons mostly have higher firing rates and small, narrow action potential waveforms. These results support the conclusion that multiple neurophysiologically distinct neuronal populations in the BF region operate independently of each other as parallel functional circuits. These observations also caution against interpreting neuronal activity in this region as a homogeneous population reflecting the function of either BF or VP alone. We suggest that salience- and movement-related neuronal populations likely correspond to BF corticopetal neurons and VP neurons, respectively.