Search

Predicting the actions and reactions of others is crucial to suc- cessful social interaction. When deciding whether to bluff in a game of poker, we consider the chances that the other players will fold or continue to play and unmask our bluff. When deciding whether to tell our boss that their plans are likely to have adverse effects, we consider a range of reac- tions, from being grateful for our honesty to being dismissed out of spite. Such predictions are highly uncertain and com- plex, not least because the other's (re)actions usually result from them making equally complex and uncertain inferences about us. Nevertheless, we are often remarkably successful – although sometimes utterly wrong – in our social inferences. How do we explain these successes and failures?

In social interactions, inferring the interaction partner's hidden mental state is crucial for predicting their actions and optimiz- ing our responses. Effective models for this inference must account for how these mental states evolve due to the interac- tion history and environmental changes. For example, recog- nizing someone's emotional state can help forecast their be- havior. Our study investigates how making these latent states visible influences decision-making in social interactions. Us- ing the repeated trust game paradigm, we show how to use hid- den Markov models (HMM) to formally represent latent state dependent strategies of the players. HMMs fitted to human dyadic play in the trust game are then used to specify adap- tive AI agents that simulate changes in mental dispositions of human players, such as the level of trust in the opponent, dur- ing a repeated interaction. Making these artificial HMM based agents take the role of the investor and interact with real hu- man trustees, we then explore how displaying “emotion” cues to the opponent's latent state affects people's actions. We find that the presence of cues was associated with more cooperative behavior from the human trustees, and that patterns of behav- ior that promote the maintenance of cooperation emerged in the presence of latent state cues and were transferred to set- tings where the cues were subsequently hidden.

In this study, we investigated human causal learning in a continuous time and space setting. Our experiments revealed that people are capable causal learners in such contexts, and that standard Bayesian updating of prior beliefs partially explains how priors impact their judgments. Specifically, individuals with strong prior beliefs consistent with the ground truth are less likely to misinterpret indirect effects as direct in structures like causal chains. When priors were inconsistent with the ground truth however, participants performed worse on average, choosing less informative interventions. Computational modelling revealed that to deal with the abundance of data in this setting, participants use interventions to mark informative data and focus on direct outgoing links from the variable they intervene on. This task-decomposition strategy, when paired with participants' interventions, achieves comparable accuracy to using the entirety of the dataset, despite ignoring almost half of the available data. These findings are in line with the resource rational framework, where discarding data outside of interventions saves computational costs and the full inference problem about the graph is broken down into sub-problems of inferring individual links between variables. Overall, our study reinforces the idea that humans are frugal and intuitive active learners who combine actions and inference to optimize learning while minimizing effort.

This article details a correction to: Röseler, L., Weber, L., Helgerth, K., Stich, E., Günther, M., Tegethoff, P., … Schütz, A. (2022). The Open Anchoring Quest Dataset: Anchored Estimates from 96 Studies on Anchoring Effects. Journal of Open Psychology Data, 10(1), 16. DOI: https://doi.org/10.5334/jopd.67

Human learning transfer takes advantage of important cognitive building blocks such as an abstract representation of concepts underlying tasks and causal models of the environment. One way to build abstract representations of the environment when the task involves interactions with others is to build a model of the opponent that may inform what actions they are likely to take next. In this study, we explore opponent modelling and its role in learning transfer by letting human participants play different games against the same computer agent, who possesses human-like theory of mind abilities with a limited degree of iterated reasoning. We find that participants deviate from Nash equilibrium play and learn to adapt to the opponent's strategy to exploit it. Moreover, we show that participants transfer their learning to new games and that this transfer is moderated by the level of sophistication of the opponent. Computational modelling shows that it is likely that players start each game using a model-based learning strategy that facilitates generalisation and opponent model transfer, but then switch to behaviour that is consistent with a model-free learning strategy in the later stages of the interaction.

To what extent do human reward learning and decision-making rely on the ability to represent and generate richly structured relationships between options? We provide evidence that structure learning and the principle of compositionality play crucial roles in human reinforcement learning. In a new multi-armed bandit paradigm, we found evidence that participants are able to learn representations of different reward structures and combine them to make correct generalizations about options in novel contexts. Moreover, we found substantial evidence that participants transferred knowledge of simpler reward structures to make compositional generalizations about rewards in complex contexts. This allowed participants to accumulate more rewards earlier, and to explore less whenever such knowledge transfer was possible. We also provide a computational model which is able to generalize and compose knowledge for complex reward structures. This model describes participant behaviour in the compositional generalization task better than various other models of decision-making and transfer learning.

How does time pressure influence attitudes towards uncer-tainty? When time is limited, do people engage in differentexploration strategies? We study human exploration in a rangeof four-armed bandit tasks with different reward distributionsand manipulate the available time for each decision (limitedvs. unlimited). Through multiple behavioral and model-basedanalyses, we show that reactions towards uncertainty are influ-enced by time pressure. Specifically, participants seek out un-certain options when time is unlimited, but avoid uncertaintyunder time pressure. Moreover, larger relative differences inuncertainty between options slowed down reaction times anddampened the drift rate of a linear ballistic accumulator model.These results shed new light on the differential effect of uncer-tainty and time pressure on human exploration.

Abstract How does time pressure influence exploration and decision-making? We investigated this question with several four-armed bandit tasks manipulating (within subjects) expected reward, uncertainty, and time pressure (limited vs. unlimited). With limited time, people have less opportunity to perform costly computations, thus shifting the cost-benefit balance of different exploration strategies. Through behavioral, reinforcement learning (RL), reaction time (RT), and evidence accumulation analyses, we show that time pressure changes how people explore and respond to uncertainty. Specifically, participants reduced their uncertainty-directed exploration under time pressure, were less value-directed, and repeated choices more often. Since our analyses relate uncertainty to slower responses and dampened evidence accumulation (i.e., drift rates), this demonstrates a resource-rational shift towards simpler, lower-cost strategies under time pressure. These results shed light on how people adapt their exploration and decision-making strategies to externally imposed cognitive constraints.

How do people plan ahead when searching for rewards? Weinvestigate planning in a foraging task in which participantssearch for rewards on an infinite two-dimensional grid. Ourresults show that their search is best-described by a modelwhich searches at least 3 steps ahead. Furthermore, partici-pants do not seem to update their beliefs during planning, butrather treat their initial beliefs as given, a strategy similar to aheuristic called root-sampling. This planning algorithm corre-sponds well with participants’ behavior in test problems withrestricted movement and varying degrees of information, out-performing more complex models. These results enrich ourunderstanding of adaptive planning in complex environments

The estimation of duration has been shown to follow Bayesianinference, where people use their prior belief to calibrate theestimation. This explains timing biases such as the range biaswhere a duration is reproduced as longer when previouslyencountered durations were longer than shorter. However, it isunclear whether prior belief is based on previously perceivedor reproduced durations. In 4 experiments, we show that therange bias occurs between short and long reproduction rangesbut not between short and long perception ranges. Furtheranalyses also show that the prior is updated by the most recentreproduced (but not perceived) duration. Together these resultssupport a task-oriented Bayesian inference account of timereproduction, where people use the perceived duration andtheir past reproduction experience to make an inference abouthow much time to reproduce.

How do people explore in order to gain rewards in uncer-tain dynamical systems? Within a reinforcement learningparadigm, control normally involves trading off between ex-ploration (i.e. trying out actions in order to gain more knowl-edge about the system) and exploitation (i.e. using currentknowledge of the system to maximize reward). We study anovel control task in which participants must steer a boat ona grid, aiming to follow a path of high reward whilst learninghow their actions affect the boat’s position. We find that partic-ipants explore strategically yet conservatively, exploring morewhen mistakes are less costly and practicing actions that willbe required later on.

We introduce the spatially correlated multi-armed banditas a task coupling function learning with the exploration-exploitation trade-off. Participants interacted with bi-variatereward functions on a two-dimensional grid, with the goal ofeither gaining the largest average score or finding the largestpayoff. By providing an opportunity to learn the underly-ing reward function through spatial correlations, we modelto what extent people form beliefs about unexplored payoffsand how that guides search behavior. Participants adapted toassigned payoff conditions, performed better in smooth thanin rough environments, and—surprisingly—sometimes per-formed equally well in short as in long search horizons. Ourmodeling results indicate a preference for local search options,which when accounted for, still suggests participants werebest-described as forming local inferences about unexploredregions, combined with a search strategy that directly tradedoff between exploiting high expected rewards and exploring toreduce uncertainty about the spatial structure of rewards.

Variance of the outcomes associated with an option oftenprovides a measure of the riskiness of that option. Hence, it isimportant for organisms are able to detect any sudden changesin outcome variance. In Experiment 1, we presented peoplewith graphs of share price time series or water level timeseries. In half the graphs, variance (financial or flooding risk)changed at some point. People were better at detectingincreases than decreases in risk - maybe because it is moreimportant to detect increases in danger than decreases in it.However, in Experiment 2, people were still better atdetecting increases than decreases in variance even whenthose changes did not reflect altered levels of risk. Ourfindings may reflect the fact that the actual change in varianceexceeds the change needed to identify a regime change invariance by a larger amount for upward than for downwardchanges.

How do people decide whether to try out novel options? We argue that they utilize contextual information toefficiently generalize from learned functional relations in order to decide between known or novel options. In a contextualmulti-armed bandit task, in which rewards are a noisy function of observable features, we assess participants’ preferences fornewly introduced options. We show that participants preferably choose a novel option if its features indicate high rewards,but shun the option if its features indicate low rewards, a behavior that can only be explained by functional generalization.Moreover, we assess people’s preferences for novel options that have medium rewards to test whether they prefer options lesssimilar to experienced options, consistent with choices guided by uncertainty. Given that novel options normally come withobservable features, we argue that contextual learning is a parsimonious yet powerful explanation of behavior in the face ofnovelty.

Warning labels can be considered as descriptions added to repeated decisions-from-experience. Limited researchso far has looked at the theoretical integration of decisions from descriptions and decisions from experience when the two areavailable concurrently. We explore how the presence and timing of such warning labels influence behaviour. We expected theprovision of warning labels to subsequently reduce risk taking, and that more prior experience before the appearance of suchlabels would lead to stronger habit formation and reduce their behavioural impact. Instead, we show how the appearance ofdescriptions warning against risks can have a perverse effect of increasing risk taking. And counter-intuitively, we also observethat the amount of previous experience prior to the appearance of descriptions does not impact behaviour. Briefly presentedwarning labels also have the same effect as constantly present ones. All of these findings have strong implications on the designof effective warning labels.

Exploration-exploitation of functions, that is learningand optimizing a mapping between inputs and expectedoutputs, is ubiquitous to many real world situations.These situations sometimes require us to avoid certainoutcomes at all cost, for example because they arepoisonous, harmful, or otherwise dangerous. We testparticipants’ behavior in scenarios in which they haveto find the optimum of a function while at the sametime avoid outputs below a certain threshold. Intwo experiments, we find that Safe-Optimization, aGaussian Process-based exploration-exploitation algo-rithm, describes participants’ behavior well and thatparticipants seem to care first about whether a point issafe and then try to pick the optimal point from all suchsafe points. This means that their trade-off betweenexploration and exploitation indicates intelligent,approximate, and homeostasis-driven behavior