Search

Humans can flexibly pursue goal-oriented behavior in the face of changes in the environment. Cognitive control refers to this set of processes allowing such adjustments, and is thought to rely on neural oscillations in the theta band (4-8Hz). First, theta amplitude increases when control is needed, and second, shifts of peak frequency in the theta band have been suggested to reliably balance task representation and gating of task-relevant sensory and action information. However, it remains unknown how these two characteristics of the control signal interact and how optimal configuration for task performance is achieved. To tackle this question, we developed a computational model that relies on reinforcement learning principles to find optimal control settings for task performance. Our simulations show that these different oscillatory characteristics play distinct roles in the flexible adjustment to task demands. This work opens new avenues for research on the mechanisms allowing cognitive flexibility.

Every day, people are exposed to vast amounts of information that can impact how they feel, think about, and act upon the world. Here, we extend the computational reinforcement learning framework to explain how such an impact can shape future decisions to either seek or avoid information. By simulating human behavioral data, we showed that agents are more likely to seek information after exposure to information with a positive net impact on the agent’s affect, cognition, and ability to make good decisions. The more the agent is exposed to this kind of information, the higher the probability that it will seek even more information in the future. On the contrary, decisions to remain ignorant are more likely to occur after repeated exposure to information with a negative net impact. Our model offers a novel computational framework within which maladaptive information-seeking and information-avoidance behaviors can be further investigated.

Recent studies and models have been successful in describing human search in choice-rich environments in terms of different generalization and exploration strategies. However, it remains an open question how and to what extent people adapt these strategies to different environments. Therefore, we used a within-subject manipulation to study human search in two classes of environments, where the reward distributions differed in the strength of spatial correlations. These environments were chosen through simulations designed to maximize the need for deploying different generalization strategies. We describe participant behavior with the parameters of a model using Gaussian Process function learning to generalize and an Upper Confidence Bound to sample from unexplored options. Our results show that while participants adapted their exploration strategies, their generalization remained unchanged across environments. Together, these findings offer important first insights into how people adapt decision-making strategies depending on environmental demands in vast decision spaces.

Recent theories propose that autism is characterized by an impairment in determining when to learn and when not. We investigated this by estimating learning rate in environments varying in volatility and uncertainty. Specifically, we correlated autistic traits (in 163 neurotypical participants) with modelled learning behaviour during probabilistic reward learning under the following three conditions: a Stationary Low Noise condition with stable reward contingencies, a Volatile condition with changing reward contingencies and a Stationary High Noise condition where reward probabilities for all options were 60%, resulting in an uncertain, noisy environment. Consistent with earlier findings, we found less optimal decision-making in the Volatile condition for participants with more autistic traits. However, we observed no correlations between underlying adjustments in learning rates and autistic traits, suggesting no impairment in updating learning rates in response to volatile versus noisy environments. Exploratory analyses indicated that impaired performance in the Volatile condition in participants with more autistic traits, was specific to trials with reward contingencies opposite to those initially learned, suggesting a primacy bias. We conclude that performance in volatile environments is lower in participants with more autistic traits, but this cannot be unambiguously attributed to difficulties with adjusting learning rates.

Human decision making is accompanied by a sense of confidence. According to Bayesian decision theory, confidence reflects the learned probability of making a correct response, given available data (e.g., accumulated stimulus evidence and response time). Although optimal, independently learning these probabilities for all possible data combinations is computationally intractable. Here, we describe a novel model of confidence implementing a low-dimensional approximation of this optimal yet intractable solution. This model allows efficient estimation of confidence, while at the same time accounting for idiosyncrasies, different kinds of biases and deviation from the optimal probability correct. Our model dissociates confidence biases resulting from the estimate of the reliability of evidence by individuals (captured by parameter α), from confidence biases resulting from general stimulus independent under and overconfidence (captured by parameter β). We provide empirical evidence that this model accurately fits both choice data (accuracy, response time) and trial-by-trial confidence ratings simultaneously. Finally, we test and empirically validate two novel predictions of the model, namely that 1) changes in confidence can be independent of performance and 2) selectively manipulating each parameter of our model leads to distinct patterns of confidence judgments. As a tractable and flexible account of the computation of confidence, our model offers a clear framework to interpret and further resolve different forms of confidence biases. Author summary: Mathematical and computational work has shown that in order to optimize decision making, humans and other adaptive agents must compute confidence in their perception and actions. Currently, it remains unknown how this confidence is computed. We demonstrate how humans can approximate confidence in a tractable manner. Our computational model makes novel predictions about when confidence will be biased (e.g., over- or underconfidence due to selective environmental feedback). We empirically tested these predictions in a novel experimental paradigm, by providing continuous model-based feedback. We observed that different feedback manipulations elicited distinct patterns of confidence judgments, in ways predicted by the model. Overall, we offer a framework to both interpret optimal confidence and resolve confidence biases that characterize several psychiatric disorders. [ABSTRACT FROM AUTHOR]

People often have to switch back and forth between different environments that come with different problems and volatilities. While volatile environments require fast learning (i.e., high learning rates), stable environments call for lower learning rates. Previous studies have shown that people adapt their learning rates, but it remains unclear whether they can also learn about environment-specific learning rates, and instantaneously retrieve them when revisiting environments. Here, using optimality simulations and hierarchical Bayesian analyses across three experiments, we show that people can learn to use different learning rates when switching back and forth between two different environments. We even observe a signature of these environment-specific learning rates when the volatility of both environments is suddenly the same. We conclude that humans can flexibly adapt and learn to associate different learning rates to different environments, offering important insights for developing theories of meta-learning and context-specific control. Author summary: People constantly have to make decisions, such as what to wear for the day, or which pizza to order at a restaurant. Fortunately, people can learn from past decisions to inform future ones. However, environments may be unstable: The best pizza today is not necessarily the best pizza tomorrow. The chef may have had a bad day, in which case no learning needs to take place, or the restaurant may have changed chefs, in which case learning needs to restart from scratch. For this reason, it pays off to learn the instabilities of different environments: Which pizza is best may change more often in one restaurant than in another (e.g., because chefs change more quickly in one restaurant than in another). Formally, environmental instability determines how strongly one should update the expected value of an option (e.g., a pizza) based on novel information, often referred to as the learning rate. We thus investigated if people can learn different learning rates for different environments. We demonstrated that they can: Participants randomly and quickly alternated between a stable and an unstable environment, and they learned to use higher learning rates in the unstable than in the stable environment. [ABSTRACT FROM AUTHOR]