Search

We assess risks differently when they are explicitly described, compared to when we learn directly from experience, suggesting dissociable decision-making systems. Our needs, such as hunger, could globally affect our risk preferences, but do they affect described and learned risks equally? On one hand, explicit decision-making is often considered flexible and contextsensitive, and might therefore be modulated by metabolic needs. On the other hand, implicit preferences learned through reinforcement might be more strongly coupled to biological drives. To answer this, we asked participants to choose between two options with different risks, where the probabilities of monetary outcomes were either described or learned. In agreement with previous studies, rewarding contexts induced risk-aversion when risks were explicitly described, but risk-seeking when they were learned through experience. Crucially, hunger attenuated these contextual biases, but only for learned risks. The results suggest that our metabolic state determines risk-taking biases when we lack explicit descriptions.

Human decisions can be reflexive or planned, being governed respectively by model-free and model-based learning systems. These two systems might differ in their responsiveness to our needs. Hunger drives us to specifically seek food rewards, but here we ask whether it might have more general effects on these two decision systems. On one hand, the model-based system is often considered flexible and context-sensitive, and might therefore be modulated by metabolic needs. On the other hand, the model-free system’s primitive reinforcement mechanisms may have closer ties to biological drives. Here, we tested participants on a well-established two-stage sequential decision-making task that dissociates the contribution of model-based and model-free control. Hunger enhanced overall performance by increasing model-free control, without affecting model-based control. These results demonstrate a generalised effect of hunger on decision-making that enhances reliance on primitive reinforcement learning, which in some situations translates into adaptive benefits.Significance statementThe prevalence of obesity and eating disorder is steadily increasing. To counteract problems related to eating, people need to make rational decisions. However, appetite may switch us to a different decision mode, making it harder to achieve long-term goals. Here we show that planned and reinforcement-driven actions are differentially sensitive to hunger. Hunger specifically affected reinforcement-driven actions, and did not affect the planning of actions. Our data shows that people behave differently when they are hungry. We also provide a computational model of how the behavioural changes might arise.

Making accurate decisions often involves the integration of current and past evidence. Here we examine the neural correlates of conflict and evidence integration during sequential decision making. Patients implanted with deep-brain stimulation (DBS) electrodes and age-matched healthy controls performed an expanded judgement task, in which they were free to choose how many cues to sample. Behaviourally, we found that while patients sampled numerically more cues, they were less able to integrate evidence and showed suboptimal performance. Using recordings of Magnetoencephalography (MEG) and local field potentials (LFP, in patients) in the subthalamic nucleus (STN), we found that beta oscillations signalled conflict between cues within a sequence. Following cues that differed from previous cues, beta power in the STN and cortex first decreased and then increased. Importantly, the conflict signal in the STN outlasted the cortical one, carrying over to the next cue in the sequence. Furthermore, after a conflict, there was an increase in coherence between the dorsal premotor cortex and subthalamic nucleus in the beta band. These results extend our understanding of cortico-subcortical dynamics of conflict processing, and do so in a context where evidence must be accumulated in discrete steps, much like in real life. Thus, the present work leads to a more nuanced picture of conflict monitoring systems in the brain and potential changes due to disease.

Reward prediction errors (RPEs) and risk preferences have two things in common: both can shape decision making behavior, and both are commonly associated with dopamine. RPEs drive value learning and are thought to be represented in the phasic release of striatal dopamine. Risk preferences bias choices towards or away from uncertainty; they can be manipulated with drugs that target the dopaminergic system. The common neural substrate suggests that RPEs and risk preferences might be linked on the level of behavior as well, but this has never been tested. Here, we aim to close this gap. First, we apply a recent theory of learning in the basal ganglia to predict how exactly RPEs might influence risk preferences. We then test our behavioral predictions using a novel bandit task in which value and risk vary independently across options. Critically, conditions are included where options vary in risk but are matched for value. We find that subjects become more risk seeking if choices are preceded by positive RPEs, and more risk averse if choices are preceded by negative RPEs. These findings cannot be explained by other known effects, such as nonlinear utility curves or dynamic learning rates. Finally, we show that RPE-induced risk-seeking is indexed by pupil dilation: participants with stronger pupillary correlates of RPE also show more pronounced behavioral effects.Author’s summaryMany of our decisions are based on expectations. Sometimes, however, surprises happen: outcomes are not as expected. Such discrepancies between expectations and actual outcomes are called prediction errors. Our brain recognises and uses such prediction errors to modify our expectations and make them more realistic--a process known as reinforcement learning. In particular, neurons that release the neurotransmitter dopamine show activity patterns that strongly resemble prediction errors. Interestingly, the same neurotransmitter is also known to regulate risk preferences: dopamine levels control our willingness to take risks. We theorised that, since learning signals cause dopamine release, they might change risk preferences as well. In this study, we test this hypothesis. We find that participants are more likely to make a risky choice just after they experienced an outcome that was better than expected, which is precisely what out theory predicts. This suggests that dopamine signalling can be ambiguous--a learning signal can be mistaken for an impulse to take a risk.

Reward prediction errors (RPEs) and risk preferences have two things in common: both can shape decision making behavior, and both are commonly associated with dopamine. RPEs drive value learning and are thought to be represented in the phasic release of striatal dopamine. Risk preferences bias choices towards or away from uncertainty; they can be manipulated with drugs that target the dopaminergic system. Based on the common neural substrate, we hypothesize that RPEs and risk preferences are linked on the level of behavior as well. Here, we develop this hypothesis theoretically and test it empirically. First, we apply a recent theory of learning in the basal ganglia to predict how RPEs influence risk preferences. We find that positive RPEs should cause increased risk-seeking, while negative RPEs should cause risk-aversion. We then test our behavioral predictions using a novel bandit task in which value and risk vary independently across options. Critically, conditions are included where options vary in risk but are matched for value. We find that our prediction was correct: participants become more risk-seeking if choices are preceded by positive RPEs, and more risk-averse if choices are preceded by negative RPEs. These findings cannot be explained by other known effects, such as nonlinear utility curves or dynamic learning rates., Author summary Many of our decisions are based on expectations. Sometimes, however, surprises happen: outcomes are not as expected. Such discrepancies between expectations and actual outcomes are called prediction errors. Our brain recognizes and uses such prediction errors to modify our expectations and make them more realistic—a process known as reinforcement learning. In particular, neurons that release the neurotransmitter dopamine show activity patterns that strongly resemble prediction errors. Interestingly, the same neurotransmitter is also known to regulate risk preferences: dopamine levels control our willingness to take risks. We theorized that, since learning signals cause dopamine release, they might change risk preferences as well. In this study, we test this hypothesis. We find that participants are more likely to make a risky choice just after they experienced an outcome that was better than expected, which is precisely what our theory predicts. This suggests that dopamine signaling can be ambiguous—a learning signal can be mistaken for an impulse to take a risk.

The cortico–basal–ganglia circuit plays a critical role in decision making on the basis of probabilistic information. Computational models have suggested how this circuit could compute the probabilities of actions being appropriate according to Bayes' theorem. These models predict that the subthalamic nucleus (STN) provides feedback that normalizes the neural representation of probabilities, such that if the probability of one action increases, the probabilities of all other available actions decrease. Here we report the results of an experiment testing a prediction of this theory that disrupting information processing in the STN with deep brain stimulation should abolish the normalization of the neural representation of probabilities. In our experiment, we asked patients with Parkinson's disease to saccade to a target that could appear in one of two locations, and the probability of the target appearing in each location was periodically changed. When the stimulator was switched off, the target probability affected the reaction times (RT) of patients in a similar way to healthy participants. Specifically, the RTs were shorter for more probable targets and, importantly, they were longer for the unlikely targets. When the stimulator was switched on, the patients were still faster for more probable targets, but critically they did not increase RTs as the target was becoming less likely. This pattern of results is consistent with the prediction of the model that the patients on DBS no longer normalized their neural representation of prior probabilities. We discuss alternative explanations for the data in the context of other published results.

Memory consolidation underpins adaptive behavior and dopaminergic networks may be critical for prolonged, selective information storage. To understand the time course of the dopaminergic contribution to memory consolidation in humans, here we investigate the effect of dopaminergic medication on recall and recognition in the short and longer term in Parkinson disease (PD). Fifteen people with PD were each tested on or off dopaminergic medication during learning/early consolidation (Day 1) and/or late consolidation (Day 2). Fifteen age-matched healthy participants were tested only once. On Day 1 participants learned new information, and early episodic memory was tested after 30 min. Then on Day 2, recall and recognition were retested after a 24-hr delay. Participants on medication on Day 1 recalled less information at 30 min and 24 hr. In contrast, patients on medication on Day 2 (8–24 hr after learning) recalled more information at 24 hr than those off medication. Although recognition sensitivity was unaffected by medication, response bias was dependent on dopaminergic state: Medication during learning induced a more liberal bias 24 hr later, whereas patients off medication during learning were more conservative responders 24 hr later. We use computational modeling to propose possible mechanisms for this change in response bias. In summary, dopaminergic medication in PD patients during learning impairs early consolidation of episodic memory and makes delayed responses more liberal, but enhances late memory consolidation presumably through a dopamine-dependent consolidation pathway that may be active during sleep.

Several prominent neurocomputational models predict that an increase of choice alternatives is modulated by increased activity in the subthalamic nucleus (STN). In turn, increased STN activity allows prolonged accumulation of information. At the same time, areas in the medial frontal cortex such as the anterior cingulate cortex (ACC) and the pre-SMA are hypothesized to influence the information processing in the STN. This study set out to test concrete predictions of STN activity in multiple-alternative decision-making using a multimodal combination of 7 Tesla structural and functional Magnetic Resonance Imaging, and ancestral graph (AG) modeling. The results are in line with the predictions in that increased STN activity was found with an increasing amount of choice alternatives. In addition, our study shows that activity in the ACC is correlated with activity in the STN without directly modulating it. This result sheds new light on the information processing streams between medial frontal cortex and the basal ganglia.

It is widely held that dopamine signaling encodes predictions of future rewards and such predictions are regularly used to drive behavior, but the relationship between these two is poorly defined. Here, we demonstrate in rats that nucleus accumbens dopamine following a reward-predicting cue is attenuated unless movement is correctly initiated. These results demonstrate that dopamine release in this region is contingent upon correct action initiation and not just reward prediction.

The most widely used account of decision-making proposes that people choose between alternatives by accumulating evidence in favor of each alternative until this evidence reaches a decision boundary. It is frequently assumed that this decision boundary stays constant during a decision, depending on the evidence collected but not on time. Recent experimental and theoretical work has challenged this assumption, showing that constant decision boundaries are, in some circumstances, sub-optimal. We introduce a theoretical model that facilitates identification of the optimal decision boundaries under a wide range of conditions. Time-varying optimal decision boundaries for our model are a result only of uncertainty over the difficulty of each trial and do not require decision deadlines or costs associated with collecting evidence, as assumed by previous authors. Furthermore, the shape of optimal decision boundaries depends on the difficulties of different decisions. When some trials are very difficult, optimal boundaries decrease with time, but for tasks that only include a mixture of easy and medium difficulty trials, the optimal boundaries increase or stay constant. We also show how this simple model can be extended to more complex decision-making tasks such as when people have unequal priors or when they can choose to opt out of decisions. The theoretical model presented here provides an important framework to understand how, why, and whether decision boundaries should change over time in experiments on decision-making.

The motor symptoms of both Parkinson's disease and focal dystonia arise from dysfunction of the basal ganglia, and are improved by pallidotomy or deep brain stimulation of the Globus Pallidus interna (GPi). However, Parkinson's disease is associated with a greater degree of basal ganglia-dependent learning impairment than dystonia. We attempt to understand this observation in terms of a comparison of the electrophysiology of the output of the basal ganglia between the two conditions. We use the natural experiment offered by Deep Brain Stimulation to compare GPi local field potential responses in subjects with Parkinson's disease compared to subjects with dystonia performing a forced-choice decision-making task with sensory feedback. In dystonic subjects, we found that auditory feedback was associated with the presence of high gamma oscillations nestled on a negative deflection, morphologically similar to sharp wave ripple complexes described in human rhinal cortex. These were not present in Parkinson's disease subjects. The temporal properties of the high gamma burst were modified by incorrect trial performance compared to correct trial performance. Both groups exhibited a robust low frequency response to 'incorrect' trial performance in dominant GPi but not non-dominant GPi at theta frequency. Our results suggest that cellular processes associated with striatum-dependent memory function may be selectively impaired in Parkinson's disease even if dopaminergic drugs are administered, but that error detection mechanisms are preserved.

Much evidence indicates that discrimination of the familiarity of visual stimuli is dependent on the perirhinal cortex of the temporal lobe. A stimulus can become familiar to animals or humans either when a stimulus is seen once but is behaviourally significant, or when a stimulus is not significant but repeats many times. This paper shows that a previously developed network model of familiarity discrimination in the perirhinal cortex is also able to judge familiarity for these different types of stimuli. The network continues to achieve high capacity and discriminative accuracy. © 2002 Published by Elsevier Science B.V.

Recent experimental and theoretical work on reinforcement learning has shed light on the neural bases of learning from rewards and punishments. One fundamental problem in reinforcement learning is the credit assignment problem, or how to properly assign credit to actions that lead to reward or punishment following a delay. Temporal difference learning solves this problem, but its efficiency can be significantly improved by the addition of eligibility traces (ET). In essence, ETs function as decaying memories of previous choices that are used to scale synaptic weight changes. It has been shown in theoretical studies that ETs spanning a number of actions may improve the performance of reinforcement learning. However, it remains an open question whether including ETs that persist over sequences of actions allows reinforcement learning models to better fit empirical data regarding the behaviors of humans and other animals. Here, we report an experiment in which human subjects performed a sequential economic decision game in which the long-term optimal strategy differed from the strategy that leads to the greatest short-term return. We demonstrate that human subjects' performance in the task is significantly affected by the time between choices in a surprising and seemingly counterintuitive way. However, this behavior is naturally explained by a temporal difference learning model which includes ETs persisting across actions. Furthermore, we review recent findings that suggest that short-term synaptic plasticity in dopamine neurons may provide a realistic biophysical mechanism for producing ETs that persist on a timescale consistent with behavioral observations.

The signal averaging approach typically used in ERP research assumes that peaks in ERP waveforms reflect neural activity that is uncorrelated with activity in the ongoing EEG. However, this assumption has been challenged by research suggesting that ERP peaks reflect event-related synchronization of ongoing EEG oscillations. In this study, we investigated the validity of a set of methods that have been used to demonstrate that particular ERP peaks result from synchronized EEG oscillations. We simulated epochs of EEG data by superimposing phasic peaks on noise characterized by the power spectrum of the EEG. When applied to the simulated data, the methods in question produced results that have previously been interpreted as evidence of synchronized oscillations, even though no such synchrony was present. These findings suggest that proposed analysis methods may not effectively disambiguate competing views of ERP generation.

We present a method for estimating parameters of connectionist models that allows the model's output to fit as closely as possible to empirical data. The method minimizes a cost function that measures the difference between statistics computed from the model's output and statistics computed from the subjects' performance. An optimization algorithm finds the values of the parameters that minimize the value of this cost function. The cost function also indicates whether the model's statistics are significantly different from the data's. In some cases, the method can find the optimal parameters automatically. In others, the method may facilitate the manual search for optimal parameters. The method has been implemented in Matlab, is fully documented, and is available for free download from the Psychonomic Society Web archive at www.psychonomic.org/archive/.

Even simple behaviour requires us to make decisions based on combining multiple pieces of learned and new information. Making such decisions requires both learning the optimal response to each given stimulus as well as combining probabilistic information from multiple stimuli before selecting a response. Computational theories of decision making predict that learning individual stimulus–response associations and rapid combination of information from multiple stimuli are dependent on different components of basal ganglia circuitry. In particular, learning and retention of memory, required for optimal response choice, are significantly reliant on dopamine, whereas integrating information probabilistically is critically dependent upon functioning of the glutamatergic subthalamic nucleus (computing the ‘normalization term’ in Bayes’ theorem). Here, we test these theories by investigating 22 patients with Parkinson’s disease either treated with deep brain stimulation to the subthalamic nucleus and dopaminergic therapy or managed with dopaminergic therapy alone. We use computerized tasks that probe three cognitive functions—information acquisition (learning), memory over a delay and information integration when multiple pieces of sequentially presented information have to be combined. Patients performed the tasks ON or OFF deep brain stimulation and/or ON or OFF dopaminergic therapy. Consistent with the computational theories, we show that stopping dopaminergic therapy impairs memory for probabilistic information over a delay, whereas deep brain stimulation to the region of the subthalamic nucleus disrupts decision making when multiple pieces of acquired information must be combined. Furthermore, we found that when participants needed to update their decision on the basis of the last piece of information presented in the decision-making task, patients with deep brain stimulation of the subthalamic nucleus region did not slow down appropriately to revise their plan, a pattern of behaviour that mirrors the impulsivity described clinically in some patients with subthalamic nucleus deep brain stimulation. Thus, we demonstrate distinct mechanisms for two important facets of human decision making: first, a role for dopamine in memory consolidation, and second, the critical importance of the subthalamic nucleus in successful decision making when multiple pieces of information must be combined. * Abbreviations : DBS : deep brain stimulation UPDRS : Unified Parkinson’s Disease Rating Scale

Learning the reliability of different sources of rewards is critical for making optimal choices. However, despite the existence of detailed theory describing how the expected reward is learned in the basal ganglia, it is not known how reward uncertainty is estimated in these circuits. This paper presents a class of models that encode both the mean reward and the spread of the rewards, the former in the difference between the synaptic weights of D1 and D2 neurons, and the latter in their sum. In the models, the tendency to seek (or avoid) options with variable reward can be controlled by increasing (or decreasing) the tonic level of dopamine. The models are consistent with the physiology of and synaptic plasticity in the basal ganglia, they explain the effects of dopaminergic manipulations on choices involving risks, and they make multiple experimental predictions., Author Summary To maximize their chances for survival, animals need to base their decisions not only on the average consequences of chosen actions, but also on the variability of the rewards resulting from these actions. For example, when an animal’s food reserves are depleted, it should prefer to forage in an area where food is guaranteed over an area where the amount of food is higher on average but variable, thus avoiding the risk of starvation. To implement such policies, animals need to be able to learn about variability of rewards resulting from taking different actions. This paper proposes how such learning may be implemented in a circuit of subcortical nuclei called the basal ganglia. It also suggests how the information about reward uncertainty can be used during decision making, so that animals can make choices that not only maximize expected rewards but also minimize risks.

Coordinated activity spanning anatomically distributed neuronal networks underpins cognition and mediates limbic–cortical interactions during learning, memory, and decision-making. We used CP55940, a potent agonist of brain cannabinoid receptors known to disrupt coordinated activity in hippocampus, to investigate the roles of network oscillations during hippocampal and medial prefrontal cortical (mPFC) interactions in rats. During quiet wakefulness and rest, CP55940 dose-dependently reduced 0.1–30 Hz local field potential power in CA1 of the hippocampus while concurrently decreasing 30–100 Hz power in mPFC; these contrasting population-level effects were paralleled by differential effects on underlying single-unit activity in the two structures. During decision-making phases of a spatial working memory task, CP5540-induced deficits in hippocampal theta and prefrontal gamma oscillations were observed alongside disrupted theta-frequency coherence between the two structures. These changes in coordinated limbic–cortical network activities correlated with (1) reduced accuracy of task performance, (2) impaired phase-locking of prefrontal single-unit spiking to the local gamma and hippocampal theta rhythms, and (3) impaired task-dependent activity in a subset of mPFC units. In addition to highlighting the importance of CA1–mPFC network oscillations for cognition, these results implicate disrupted theta-frequency coordination of CA1–mPFC activity in the cognitive deficits caused by exogenous activation of brain cannabinoid receptors.

Reinforcement learning involves a tight cou- pling of reward-associated behavior and a type of learning that is very different from that promoted by education. However, the emerging understanding of its underlying processes may help derive principles for effective learning games that have, until now, been elusive. This article first reviews findings from cognitive neuroscience and psychology to provide insight into the motivating role of uncertain reward in games, including educational games. Then, a short experiment is reported to illustrate the potential of reward-based neurocomputational models of behavior in the understanding and development of effective learning games. In this study, a reward-based model of behavior is shown to predict recall of newly learned information during a simple learning game.

Learning from competitors poses a challenge for existing theories of reward-based learning, which assume that rewarded actions are more likely to be executed in the future. Such a learning mechanism would disadvantage a player in a competitive situation because, since the competitor's loss is the player's gain, reward might become associated with an action the player should themselves avoid. Using fMRI, we investigated the neural activity of humans competing with a computer in a foraging task. We observed neural activity that represented the variables required for learning from competitors: the actions of the competitor (in the player's motor and premotor cortex) and the reward prediction error arising from the competitor's feedback. In particular, regions positively correlated with the unexpected loss of the competitor (which was beneficial to the player) included the striatum and those regions previously implicated in response inhibition. Our results suggest that learning in such contexts may involve the competitor's unexpected losses activating regions of the player's brain that subserve response inhibition, as the player learns to avoid the actions that produced them.

In many situations, decision makers need to negotiate between the competing demands of response speed and response accuracy, a dilemma generally known as the speed-accuracy tradeoff (SAT). Despite the ubiquity of SAT, the question of how neural decision circuits implement SAT has received little attention up until a year ago. We review recent studies that show SAT is modulated in association and pre-motor areas rather than in sensory or primary motor areas. Furthermore, the studies suggest that emphasis on response speed increases the baseline firing rate of cortical integrator neurons. We also review current theories on how and where in the brain the SAT is controlled, and we end by proposing research directions that could distinguish between these theories.

Neurophysiological studies have identified a number of brain regions critically involved in solving the problem of action selection or decision making. In the case of highly practiced tasks, these regions include cortical areas hypothesized to integrate evidence supporting alternative actions and the basal ganglia, hypothesized to act as a central switch in gating behavioral requests. However, despite our relatively detailed knowledge of basal ganglia biology and its connectivity with the cortex and numerical simulation studies demonstrating selective function, no formal theoretical framework exists that supplies an algorithmic description of these circuits. This article shows how many aspects of the anatomy and physiology of the circuit involving the cortex and basal ganglia are exactly those required to implement the computation defined by an asymptotically optimal statistical test for decision making: the multihypothesis sequential probability ratio test (MSPRT). The resulting model of basal ganglia provides a new framework for understanding the computation in the basal ganglia during decision making in highly practiced tasks. The predictions of the theory concerning the properties of particular neuronal populations are validated in existing experimental data. Further, we show that this neurobiologically grounded implementation of MSPRT outperforms other candidates for neural decision making, that it is structurally and parametrically robust, and that it can accommodate cortical mechanisms for decision making in a way that complements those in basal ganglia.

In this article, the authors consider optimal decision making in two-alternative forced-choice (TAFC) tasks. They begin by analyzing 6 models of TAFC decision making and show that all but one can be reduced to the drift diffusion model, implementing the statistically optimal algorithm (most accurate for a given speed or fastest for a given accuracy). They prove further that there is always an optimal trade-off between speed and accuracy that maximizes various reward functions, including reward rate (percentage of correct responses per unit time), as well as several other objective functions, including ones weighted for accuracy. They use these findings to address empirical data and make novel predictions about performance under optimality.

Much evidence indicates that recognition memory involves two separable processes, recollection and familiarity discrimination, with familiarity discrimination being dependent on the perirhinal cortex of the temporal lobe. Here, we describe a new neural network model designed to mimic the response patterns of perirhinal neurons that signal information concerning the novelty or familiarity of stimuli. The model achieves very fast and accurate familiarity discrimination while employing biologically plausible parameters and Hebbian learning rules. The fact that the activity patterns of the model's simulated neurons are closely similar to those of neurons recorded from the primate perirhinal cortex indicates that this brain region could discriminate familiarity using principles akin to those of the model. If so, the capacity of the model establishes that the perirhinal cortex alone may discriminate the familiarity of many more stimuli than current neural network models indicate could be recalled (recollected) by all the remaining areas of the cerebral cortex. This efficiency and speed of detecting novelty provides an evolutionary advantage, thereby providing a reason for the existence of a familiarity discrimination network in addition to networks used for recollection.

Despite the complexity and variability of decision processes, motor responses are generally stereotypical and independent\ud of decision difficulty. How is this consistency achieved? Through an engineering analogy we consider how and why a\ud system should be designed to realise not only flexible decision-making, but also consistent decision implementation. We\ud specifically consider neurobiologically-plausible accumulator models of decision-making, in which decisions are made when\ud a decision threshold is reached. To trade-off between the speed and accuracy of the decision in these models, one can\ud either adjust the thresholds themselves or, equivalently, fix the thresholds and adjust baseline activation. Here we review\ud how this equivalence can be implemented in such models. We then argue that manipulating baseline activation is\ud preferable as it realises consistent decision implementation by ensuring consistency of motor inputs, summarise empirical\ud evidence in support of this hypothesis, and suggest that it could be a general principle of decision making and\ud implementation. Our goal is therefore to review how neurobiologically-plausible models of decision-making can manipulate\ud speed-accuracy trade-offs using different mechanisms, to consider which of these mechanisms has more desirable decision-\ud implementation properties, and then review the relevant neuroscientific data on which mechanism brains actually use.

The leaky competing accumulator (LCA) is a biologically inspired model of choice. It describes the processes of leaky accumulation and competition observed in neuronal populations during choice tasks and it accounts for reaction time distributions observed in psychophysical experiments. This paper discusses recent analyses and extensions of the LCA model. First, it reviews the dynamics and examines the conditions that make the model achieve optimal performance. Second, it shows that nonlinearities of the type present in biological neurons improve performance when the number of choice alternatives increases. Third, the model is extended to value-based choice, where it is shown that nonlinearities in the value function explain risk aversion in risky choice and preference reversals in choice between alternatives characterized across multiple dimensions.

This article seeks to integrate two sets of theories describing action selection in the basal ganglia: reinforcement learning theories describing learning which actions to select to maximize reward and decision-making theories proposing that the basal ganglia selects actions on the basis of sensory evidence accumulated in the cortex. In particular, we present a model that integrates the actor-critic model of reinforcement learning and a model assuming that the cortico-basal-ganglia circuit implements a statistically optimal decision-making procedure. The values of corico-striatal weights required for optimal decision making in our model differ from those provided by standard reinforcement learning models. Nevertheless, we show that an actor-critic model converges to the weights required for optimal decision making when biologically realistic limits on synaptic weights are introduced. We also describe the model's predictions concerning reaction times and neural responses during learning, and we discuss directions required for further integration of reinforcement learning and optimal decision-making theories.

Psychological experiments have shown that the capacity of the brain for discriminating visual stimuli as novel or familiar is almost limitless. Neurobiological studies have established that the perirhinal cortex is critically involved in both familiarity discrimination and feature extraction. However, opinion is divided as to whether these two processes are performed by the same neurons. Previously proposed models have been unable to simultaneously extract features and discriminate familiarity for large numbers of stimuli. We show that a well-known model of visual feature extraction, Infomax, can simultaneously perform familiarity discrimination and feature extraction efficiently. This model has a significantly larger capacity than previously proposed models combining these two processes, particularly when correlation exists between inputs, as is the case in the perirhinal cortex. Furthermore, we show that once the model fully extracts features, its ability to perform familiarity discrimination increases markedly.

When people make decisions they often face opposing demands for response speed and response accuracy, a process likely mediated by response thresholds. According to the striatal hypothesis , people decrease response thresholds by increasing activation from cortex to striatum, releasing the brain from inhibition. According to the STN hypothesis , people decrease response thresholds by decreasing activation from cortex to subthalamic nucleus (STN); a decrease in STN activity is likewise thought to release the brain from inhibition and result in responses that are fast but error-prone. To test these hypotheses—both of which may be true—we conducted two experiments on perceptual decision making in which we used cues to vary the demands for speed vs. accuracy. In both experiments, behavioral data and mathematical model analyses confirmed that instruction from the cue selectively affected the setting of response thresholds. In the first experiment we used ultra-high-resolution 7T structural MRI to locate the STN precisely. We then used 3T structural MRI and probabilistic tractography to quantify the connectivity between the relevant brain areas. The results showed that participants who flexibly change response thresholds (as quantified by the mathematical model) have strong structural connections between presupplementary motor area and striatum. This result was confirmed in an independent second experiment. In general, these findings show that individual differences in elementary cognitive tasks are partly driven by structural differences in brain connectivity. Specifically, these findings support a cortico-striatal control account of how the brain implements adaptive switches between cautious and risky behavior.

The advance of Parkinson's disease is associated with the existence of abnormal oscillations within the basal ganglia with frequencies in the beta band (13–30 Hz). While the origin of these oscillations remains unknown, there is some evidence suggesting that oscillations observed in the basal ganglia arise due to interactions of two nuclei: the subthalamic nucleus (STN) and the globus pallidus pars externa (GPe). To investigate this hypothesis, we develop a computational model of the STN–GPe network based upon anatomical and electrophysiological studies. Significantly, our study shows that for certain parameter regimes, the model intrinsically oscillates in the beta range. Through an analytical study of the model, we identify a simple set of necessary conditions on model parameters that guarantees the existence of beta oscillations. These conditions for generation of oscillations are described by a set of simple inequalities and can be summarized as follows: (1) The excitatory connections from STN to GPe and the inhibitory connections from GPe to STN need to be sufficiently strong. (2) The time required by neurons to react to their inputs needs to be short relative to synaptic transmission delays. (3) The excitatory input from the cortex to STN needs to be high relative to the inhibition from striatum to GPe. We confirmed the validity of these conditions via numerical simulation. These conditions describe changes in parameters that are consistent with those expected as a result of the development of Parkinson's disease, and predict manipulations that could inhibit the pathological oscillations.

In this paper we investigate trade-offs between speed and accuracy that are produced by humans when confronted with a sequence of choices between two alternatives. We assume that the choice process is described by the drift diffusion model, in which the speed–accuracy trade-off is primarily controlled by the value of the decision threshold. We test the hypothesis that participants choose the decision threshold that maximizes reward rate, defined as an average number of rewards per unit of time. In particular, we test four predictions derived on the basis of this hypothesis in two behavioural experiments. The data from all participants of our experiments provide support only for some of the predictions, and on average the participants are slower and more accurate than predicted by reward rate maximization. However, when we limit our analysis to subgroups of 30–50% of participants who earned the highest overall rewards, all the predictions are satisfied by the data. This suggests that a substantial subset of participants do select decision thresholds that maximize reward rate. We also discuss possible reasons why the remaining participants select thresholds higher than optimal, including the possibility that participants optimize a combination of reward rate and accuracy or that they compensate for the influence of timing uncertainty, or both.

Background The basal ganglia plays a key role in motor control, and its dysfunction is associated with various neurological disorders including Parkinson's disease. The symptoms of Parkinson's disease include bradykinesia, characterized by a general slowing of movement execution. The advance of bradykinesia is highly correlated with the presence of abnormal oscillations within basal ganglia in the beta band frequency [1]. Some evidence suggests the oscillations observed in the basal ganglia originate from the network comprised of two nuclei: subthalamic nucleus (STN) and globus pallidus pars externa (GPe) [2], while other authors attribute their source to the entire corticobasal-ganglia-thalamic circuit [3]. In this work, we attempt to address this question of the origin of beta band oscillations.

Empirical findings suggest that the mammalian brain has two decision-making systems that act at different speeds. We represent the faster system using standard signal detection theory. We represent the slower (but more accurate) cortical system as the integration of sensory evidence over time until a certain level of confidence is reached. We then consider how two such systems should be combined optimally for a range of information linkage mechanisms. We conclude with some performance predictions that will hold if our representation is realistic.

This chapter presents a model assuming that during decision making the cortico-basal-ganglia circuit computes probabilities that considered alternatives are correct, according to Bayes’ theorem. The model suggests how the equation of Bayes’ theorem is mapped onto the functional anatomy of a circuit involving the cortex, basal ganglia and thalamus. The chapter also describes the relationship of the model to other models of decision making and experimental data.

Recent physiological studies suggest that in motion discrimination tasks, neurons in the lateral intraparietal (LIP) area integrate sensory evidence during decision making process by carrying persistent response selective to the saccadic response. LIP neurons also discharge at high frequency shortly before the saccade onset. We propose that the later response is due to the activity form the bursting neurons in the Superior Colliculus (SC). To test the hypothesis we developed a decision making model with populations of neurons in LIP, Basal Ganglia (BG) and SC where BG and SC process the threshold detection and action generation. The model successfully describes the LIP activity from the experiment, and is also consistent with the behavioral measurements.

This article reviews recently proposed theories postulating that, during simple choices, the brain performs statistically optimal decision making. These theories are ecologically motivated by evolutionary pressures to optimize the speed and accuracy of decisions and to maximize the rate of receiving rewards for correct choices. This article suggests that the models of decision making that are proposed on different levels of abstraction can be linked by virtue of the same optimal computation. Also reviewed here are recent observations that many aspects of the circuit that involves the cortex and basal ganglia are the same as those that are required to perform statistically optimal choice. This review illustrates how optimal-decision theories elucidate current data and provide experimental predictions that concern both neurobiology and behaviour.

It has been proposed that the error-related negativity (ERN) is generated by phase resetting of theta-band EEG oscillations. The present research evaluates a set of analysis methods that have recently been used to provide evidence for this hypothesis. To evaluate these methods, we apply each of them to two simulated data sets: one set that includes theta phase resetting and a second that comprises phasic peaks embedded in EEG noise. The results indicate that the analysis methods do not effectively distinguish between the two simulated data sets. In particular, the simulated data set constructed from phasic peaks, though containing no synchronization of ongoing EEG activity, demonstrates properties previously interpreted as supporting the synchronized oscillation account of the ERN. These findings suggest that the proposed analysis methods cannot provide unambiguous evidence that the ERN is generated by phase resetting of ongoing oscillations.

This study compares the efficiency and plausibility of published computational models of familiarity discrimination in the perirhinal cortex. Substantial evidence indicates that the perirhinal cortex is involved in both the familiarity discrimination aspect of recognition memory and in perceptual functions involved with representations of complete stimuli (i.e., object identification). Published models of how the perirhinal cortex may perform familiarity discrimination can be divided into two groups. The first group assumes that a proportion of perirhinal neurons form a network specialised just for familiarity discrimination (these models may be based on Hebbian or anti-Hebbian synaptic plasticity). In contrast, the second group assumes that both familiarity discrimination and learning representations of complete stimuli are performed within a single combined network. This study establishes that when the responses of neurons that provide input to the familiarity discrimination network are correlated (as indicated by experimental data), specialised networks based on anti-Hebbian learning may recognise the previous occurrence of many more stimuli (i.e., have a capacity up to thousands of times larger) than specialised networks based on Hebbian learning. The currently published combined models do not learn an optimal stimulus representation (they do not fully extract statistically independent features), and hence their capacities are even lower than those of the specialised models based on Hebbian learning. Hence, the combined models published thus far are critically less efficient than the specialised models based on anti-Hebbian learning. This study also compares the consistency of the models with experimental observations concerning what is known of synaptic plasticity in the perirhinal cortex and the responses of its neurons. Many theoretically important parameters remain undetermined, and experiments are suggested to provide information critical for refining and distinguishing between the various models. However, the above theoretical arguments and currently published data favour the existence of a separate network specialised for familiarity discrimination.

Much evidence indicates that discrimination of the familiarity of visual stimuli is dependent on the perirhinal cortex of the temporal lobe. Within the monkey's perirhinal cortex, ∼25% of neurons respond strongly to the sight of novel objects but respond only weakly or briefly when these objects are seen again. These neurons can be divided into three populations based on their patterns of responsiveness. Specific temporal dependencies exist among the activities of the three populations of neurons, suggesting the existence of specific connections between them. This report concerns computer modelling that indicates how such connections may be used to increase reliability in the determination of whether or not a stimulus is being seen for the first time. © 2001 Elsevier Science B.V. All rights reserved.