Search

van Doorn et al. (2021) outlined various questions that arise when conducting Bayesian model comparison for mixed effects models. Seven response articles offered their own perspective on the preferred setup for mixed model comparison, on the most appropriate specification of prior distributions, and on the desirability of default recommendations. This article presents a round-table discussion that aims to clarify outstanding issues, explore common ground, and outline practical considerations for any researcher wishing to conduct a Bayesian mixed effects model comparison.

Interest in the processes that mediate between stimuli and responses is at the heart of most modern psychology and neuroscience. These processes cannot be directly measured but instead must be inferred from observed responses. Race models, through their ability to account for both response choices and response times, have been a key enabler of such inferences. Examples of such models appeared contemporaneously with the cognitive revolution, and since then have become increasingly prominent and elaborated, so that psychologists now have a powerful array of race models at their disposal. We showcase the state of the art for race models and describe why and how they are used.

The effects of distraction on responses manifest in three ways: prolonged reaction times, and increased error and response omission rates. However, the latter effect is often ignored or assumed to be due to a separate cognitive process. We investigated omissions occurring in two paradigms that manipulated distraction. One required simple stimulus detection of younger participants, the second required choice responses and was completed by both younger and older participants. We fit data from these paradigms with a model that identifies three causes of omissions: two are related to the process of accumulating the evidence on which a response is based: intrinsic omissions (due to between-trial variation in accumulation rates making it impossible to ever reach the evidence threshold) and design omissions (due to response windows that cause slow responses not to be recorded; a third, contaminant omissions, allows for a cause unrelated to the response process. In both data sets systematic differences in omission rates across conditions were accounted for by task-related omissions. Intrinsic omissions played a lesser role than design omissions, even though the presence of design omissions was not evident in descriptive analyses of the data. The model provided an accurate account of all aspects of the detection data and the choice-response data, but slightly underestimated overall omissions in the choice paradigm, particularly in older participants, suggesting that further investigation of contaminant omission effects is needed.

Stopping an already initiated action is crucial for human everyday behavior and empirical evidence points toward the prefrontal cortex playing a key role in response inhibition. Two regions that have been consistently implicated in response inhibition are the right inferior frontal gyrus (IFG) and the more superior region of the dorsolateral prefrontal cortex (DLPFC). The present study investigated the effect of offline 1 Hz transcranial magnetic stimulation (TMS) over the right IFG and DLPFC on performance in a gamified stop-signal task (SSG). We hypothesized that perturbing each area would decrease performance in the SSG, albeit with a quantitative difference in the performance decrease after stimulation. After offline TMS, functional short-term reorganization is possible, and the domain-general area (i.e., the right DLPFC) might be able to compensate for the perturbation of the domain-specific area (i.e., the right IFG). Results showed that 1 Hz offline TMS over the right DLPFC and the right IFG at 110% intensity of the resting motor threshold had no effect on performance in the SSG. In fact, evidence in favor of the null hypothesis was found. One theoretically intriguing interpretation of this result is that within-network compensation was triggered, canceling out the potential TMS effects as has been suggested in recent theorizing on TMS effects. This interpretation is particularly intriguing given that our task was closer to a real-life situation due to its increased complexity as compared to the standard stop-signal task version.

Response inhibition is a key attribute of human executive control. Standard stop-signal tasks require countermanding a single response; the speed at which that response can be inhibited indexes the efficacy of the inhibitory control networks. However, more complex stopping tasks, where one or more components of a multi-component action are cancelled (i.e., response-selective stopping) cannot be explained by the independent-race model appropriate for the simple task (Logan and Cowan, 1984). Healthy human participants (n=28; 10 male; 19-40 years) completed a response-selective stopping task where a `go' stimulus required simultaneous (bimanual) button presses in response to left and right pointing green arrows. On a subset of trials (30%) one, or both, arrows turned red (constituting the stop signal) requiring that only the button-press(es) associated with red arrows be cancelled. Electromyographic recordings from both index fingers (first dorsal interosseous) permitted voluntary motor responses that resulted in overt button presses, and activity that was cancelled prior to an overt response (i.e., partial, or covert, responses), to be assessed. We propose a simultaneously inhibit and start (SIS) model that extends the independent race model and provides a highly accurate account of response-selective stopping data. Together with fine-grained EMG analysis, our model-based analysis offers converging evidence that the selective-stop signal simultaneously triggers a process that stops the bimanual response and triggers a new unimanual response corresponding to the green arrow. Our results require a reconceptualization of response-selective stopping and offer a tractable framework for assessing such tasks in healthy and patient populations.

The “marginality principle” for linear regression models states that when a higher order term is included, its constituent terms must also be included. The target article relies on this principle for the fixed-effects part of linear mixed models of ANOVA designs and considers the implication that if extended to combined fixed-and-random-effects models, model selection tests specific to some fixed-effects ANOVA terms are not possible. We review the basis for this principle for fixed-effects models and delineate its limits. We then consider its extension to combined fixed-and-random-effects models. We conclude that we have been unable to find in the literature, including the target article, and have ourselves been unable to construct any satisfactory argument against the use of incomplete ANOVA models. The only basis we could find requires one to assume that it is not possible to test point-null hypotheses, something we disagree with, and which we believe is incompatible with the Bayesian model-selection methods that are the basis of the target article.

The stop-signal paradigm has become ubiquitous in investigations of inhibitory control. Tasks inspired by the paradigm, referred to as stop-signal tasks, require participants to make responses on go trials and to inhibit those responses when presented with a stop-signal on stop trials. Currently, the most popular version of the stop-signal task is the ‘choice-reaction’ variant, where participants make choice responses, but must inhibit those responses when presented with a stop-signal. An alternative to the choice-reaction variant of the stop-signal task is the ‘anticipated response inhibition’ task. In anticipated response inhibition tasks, participants are required to make a planned response that coincides with a predictably timed event (such as lifting a finger from a computer key to stop a filling bar at a predefined target). Anticipated response inhibition tasks have some advantages over the more traditional choice-reaction stop-signal tasks and are becoming increasingly popular. However, currently, there are no openly available versions of the anticipated response inhibition task, limiting potential uptake. Here, we present an open-source, free, and ready-to-use version of the anticipated response inhibition task, which we refer to as the OSARI (the Open-Source Anticipated Response Inhibition) task.

Inhibitory control is one of the most important control functions in the human brain. Much of our understanding of its neural basis comes from seminal work showing that lesions to the right inferior frontal cortex (rIFC) increase stop-signal reaction time (SSRT), a latent variable that expresses the speed of inhibitory control. However, recent work has identified substantial limitations of the SSRT method. Notably, SSRT is confounded by trigger failures: stop-signal trials in which inhibitory control was never initiated. Such trials inflate SSRT, but are typically indicative of attentional, rather than inhibitory deficits. Here, we used hierarchical Bayesian modeling to identify stop-signal trigger failures in human rIFC lesion patients, non-rIFC lesion patients, and healthy comparisons. Furthermore, we measured scalp-EEG to detect β-bursts, a neurophysiological index of inhibitory control. rIFC lesion patients showed a more than five-fold increase in trigger failure trials and did not exhibit the typical increase of stop-related frontal β- bursts. However, on trials in which such β-bursts did occur, rIFC patients showed the typical subsequent upregulation of β over sensorimotor areas, indicating that their ability to implement inhibitory control, once triggered, is intact. These findings suggest that the role of rIFC in inhibitory control has to be fundamentally reinterpreted.

In recent years, multiple initiatives have sought to improve the transparency and reproducibility of psychological research. One example is the Peer Reviewers’ Openness Initiative (PRO), which aims to promote the sharing of data and code. PRO signatories pledge to provide a full review only for manuscripts that publicly share data and code, or include a justification why sharing is not possible. The punitive element of this approach attracted criticism: PRO may be perceived as coercive and could lead to negative repercussions for its signatories. Therefore we conducted a survey to assess signatories’ experiences with PRO. Contrary to the criticism, the reported experiences were predominantly positive, and 92% (117/127) of the signatories who completed the survey indicated that they would sign the initiative again today. Only 19 out of 127 respondents (15%) experienced negative reactions related to their commitment to PRO. Almost 50 respondents suggested ways in which PRO could be improved. We conclude that, from the signatories’ perspective, the benefits of the PRO initiative outweigh its drawbacks.

Over the last decade, the Bayesian estimation of evidence-accumulation models has gained popularity, largely due to the advantages afforded by the Bayesian hierarchical framework. Despite recent advances in the Bayesian estimation of evidence-accumulation models, model comparison continues to rely on suboptimal procedures, such as posterior parameter inference and model selection criteria known to favor overly complex models. In this paper, we advocate model comparison for evidence-accumulation models based on the Bayes factor obtained via Warp-III bridge sampling. We demonstrate, using the linear ballistic accumulator (LBA), that Warp-III sampling provides a powerful and flexible approach that can be applied to both nested and non-nested model comparisons, even in complex and high-dimensional hierarchical instantiations of the LBA. We provide an easy-to-use software implementation of the Warp-III sampler and outline a series of recommendations aimed at facilitating the use of Warp-III sampling in practical applications.

Sir Ronald Fisher’s venerable experiment “The Lady Tasting Tea” is revisited from a Bayesian perspective. We demonstrate how a similar tasting experiment, conducted in a classroom setting, can familiarize students with several key concepts of Bayesian inference, such as the prior distribution, the posterior distribution, the Bayes factor, and sequential analysis.

Contains fulltext : 226717.pdf (Publisher’s version ) (Closed access) Despite an ongoing stream of lamentations, many empirical disciplines still treat the p value as the sole arbiter to separate the scientific wheat from the chaff. The continued reign of the p value is arguably due in part to a perceived lack of workable alternatives. In order to be workable, any alternative methodology must be (1) relevant: it has to address the practitioners' research question, which - for better or for worse- most often concerns the test of a hypothesis, and less often concerns the estimation of a parameter; (2) available: it must have a concrete implementation for practitioners' statistical workhorses such as the t test, regression, and ANOVA; and (3) easy to use: methods that demand practitioners switch to the theoreticians' programming tools will face an uphill struggle for adoption. The above desiderata are fulfilled by Harold Jeffreys's Bayes factor methodology as implemented in the open-source software JASP. We explain Jeffreys's methodology and showcase its practical relevance with two examples. 9 p.

Standard, well-established cognitive tasks that produce reliable effects in group comparisons also lead to unreliable measurement when assessing individual differences. This reliability paradox has been demonstrated in decision-conflict tasks such as the Simon, Flanker, and Stroop tasks, which measure various aspects of cognitive control. We aim to address this paradox by implementing carefully calibrated versions of the standard tests with an additional manipulation to encourage processing of conflicting information, as well as combinations of standard tasks. Over five experiments, we show that a Flanker task and a combined Simon and Stroop task with the additional manipulation produced reliable estimates of individual differences in under 100 trials per task, which improves on the reliability seen in benchmark Flanker, Simon, and Stroop data. We make these tasks freely available and discuss both theoretical and applied implications regarding how the cognitive testing of individual differences is carried out.

We propose a new cognitive-modelling methodology that disentangles the non-linear interactions among psychological mechanisms underpinning behaviour in a complex task. We apply the approach to the domain of cognitive aging and show that it affords simultaneous measures of the major mechanisms proposed to explain age-related deficits: capacity limits, processing speed, inhibition, and executive function. The measures’ validities derive from the precisely specified roles of the model parameters and the model’s comprehensive, detailed, and accurate account of the behaviour of our healthy older and younger participants. Because it derives constraint from all aspects of participant’s behaviour, the analysis is powerful enough to provide clear results from a relatively small sample of participants who provide high quality data. When controlling for all other factors, we identify processing speed as the primary cause of age-related deficits. Executive control was better for older than younger participants when task demands were lower, but as demands increased this advantage reduced or disappeared, consistent with age-related capacity limitations. We conclude that such detailed cognitive models have the potential to be widely applied to provide an integrated, quantitative, and conceptually rich account of the many interacting psychological mechanisms that support behaviour in complex tasks.

The effects of distraction on responses manifest in three ways: prolonged reaction times, and increased error and response omission rates. However, the latter effect is often ignored or assumed to be due to a separate cognitive process. We investigated omissions occurring in two paradigms that manipulated distraction. One required simple stimulus detection of younger participants, the second required choice responses and was completed by both younger and older participants. We fit data from these paradigms with a model that identifies three causes of omissions: two are related to the process of accumulating the evidence on which a response is based: intrinsic omissions (due to between-trial variation in accumulation rates making it impossible to ever reach the evidence threshold) and design omissions (due to response windows that cause slow responses not to be recorded; a third, contaminant omissions, allows for a cause unrelated to the response process. In both data sets systematic differences in omission rates across conditions were accounted for by task-related omissions. Intrinsic omissions played a lesser role than design omissions, even though the presence of design omissions was not evident in descriptive analyses of the data. The model provided an accurate account of all aspects of the detection data and the choice-response data, but slightly underestimated overall omissions in the choice paradigm, particularly in older participants, suggesting that further investigation of contaminant omission effects is needed.

The Adolescent Brain Cognitive Development (ABCD) Study is a longitudinal neuroimaging study of unprecedented scale that is in the process of following over 11,000 youth from middle childhood though age 20. However, a design feature of the study’s stop-signal task violates “context independence”, an assumption critical to current non-parametric methods for estimating stop-signal reaction time (SSRT), a key measure of inhibitory ability in the study. This has led some experts to call for the task to be changed and for previously collected data to be used with caution. We present a formal cognitive process model, the BEESTS-ABCD model, that provides a mechanistic explanation for the impact of this design feature, describes key behavioral trends in the ABCD data, and allows biases in SSRT estimates resulting from context independence violations to be quantified. We use the model to demonstrate that, although non-parametric SSRT estimates generally preserve the rank ordering of participants’ SSRT values, failing to account for context independence violations can lead to erroneous inferences in several realistic scenarios. Nonetheless, as the BEESTS-ABCD model can be used to accurately recover estimates of SSRT and other mechanistic parameters of interest from ABCD data, the impact of such violations can be effectively mitigated.

Cognitive models provide a substantively meaningful quantitative description of latent cognitive processes. The quantitative formulation of these models supports cumulative theory building and enables strong empirical tests. However, the non-linearity of these models and pervasive correlations among model parameters pose special challenges when applying cognitive models to data. Firstly, estimating cognitive models typically requires large hierarchical data sets that need to be accommodated by an appropriate statistical structure within the model. Secondly, statistical inference needs to appropriately account for model uncertainty to avoid overconfidence and biased parameter estimates. In the present work we show how these challenges can be addressed through a combination of Bayesian hierarchical modelling and Bayesian model averaging. To illustrate these techniques, we apply the popular diffusion decision model to data from a collaborative selective influence study.

Mean stop-signal reaction time (SSRT) is frequently employed as a measure of response inhibition in cognitive neuroscience research on attention deficit/hyperactivity disorder (ADHD). However, this measurement model is limited by two factors that may bias SSRT estimation in this population: (a) excessive skew in “go” RT distributions and (b) trigger failures, or instances in which individuals fail to trigger an inhibition process in response to the stop signal. We used a Bayesian parametric approach that allows unbiased estimation of the shape of entire SSRT distributions and the probability of trigger failures to clarify mechanisms of stop-signal task deficits in ADHD. Children with ADHD displayed greater positive skew than their peers in both go RT and SSRT distributions. However, they also displayed more frequent trigger failures, which appeared to drive ADHD-related stopping difficulties. Results suggest that performance on the stop-signal task among children with ADHD reflects impairments in early attentional processes, rather than inefficiency in the stop process.

Dissociations have served as a key source of evidence for theory development in experimental psychology. Claims about the existence of multiple distinct psychological processes or systems are often based on demonstrations that manipulations such as working memory load, mood or instructions have differential effects on task performance. For example, a manipulation may have a larger effect on performance in one task, and a smaller or no detectable effect in another, as identified by statistical models like analysis of variance. However, inferring distinct underlying processes based on such interaction effects can be misleading. Such an inference depends on the strong – and probably false – assumption that underlying psychological variables map linearly onto the observable dependent variables. Fortunately, state-trace analysis offers an alternative approach to test for multiple underlying variables, avoiding the linearity assumption. We apply state-trace analysis to databases of studies from reasoning and from category learning that have been cited as evidence for qualitatively distinct processes. We show that many of the dissociations thought to reflect the operation of distinct processes disappear against the stricter criteria of state-trace analysis. We argue that it is important for experiments to be designed with state-trace analysis in mind, and highlight the need for the development and more widespread use of similar techniques. This will lead to a more rigorous foundation for theoretical claims about distinct underlying psychological mechanisms.

Motorists often engage in secondary tasks unrelated to driving that increase cognitive workload, resulting in fatal crashes and injuries. An International Standards Organization method for measuring a driver's cognitive workload, the detection response task (DRT), correlates well with driving outcomes, but investigation of its putative theoretical basis in terms of finite attention capacity remains limited. We address this knowledge gap using evidence-accumulation modeling of simple and choice versions of the DRT in a driving scenario. Our experiments demonstrate how dual-task load affects the parameters of evidence-accumulation models. We found that the cognitive workload induced by a secondary task (counting backward by 3s) reduced the rate of evidence accumulation, consistent with rates being sensitive to limited-capacity attention. We also found a compensatory increase in the amount of evidence required for a response and a small speeding in the time for nondecision processes. The International Standards Organization version of the DRT was found to be most sensitive to cognitive workload. A Wald-distributed evidence-accumulation model augmented with a parameter measuring response omissions provided a parsimonious measure of the underlying causes of cognitive workload in this task. This work demonstrates that evidence-accumulation modeling can accurately represent data produced by cognitive workload measurements, reproduce the data through simulation, and provide supporting evidence for the cognitive processes underlying cognitive workload. Our results provide converging evidence that the DRT method is sensitive to dynamic fluctuations in limited-capacity attention. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Human operators often experience large fluctuations in cognitive workload over seconds timescales that can lead to sub-optimal performance, ranging from overload to neglect. Adaptive automation could potentially address this issue, but to do so it needs to be aware of real-time changes in operators’ spare cognitive capacity, so it can provide help in times of peak demand and take advantage of troughs to elicit operator engagement. However, it is unclear whether rapid changes in task demands are reflected in similarly rapid fluctuations in spare capacity, and if so what aspects of responses to those demands are predictive of the current level of spare capacity. We used the ISO standard detection response task (DRT) to measure cognitive workload approximately every 4 s in a demanding task requiring monitoring and refueling of a fleet of simulated unmanned aerial vehicles (UAVs). We showed that the DRT provided a valid measure that can detect differences in workload due to changes in the number of UAVs. We used cross-validation to assess whether measures related to task performance immediately preceding the DRT could predict detection performance as a proxy for cognitive workload. Although the simple occurrence of task events had weak predictive ability, composite measures that tapped operators’ situational awareness with respect to fuel levels were much more effective. We conclude that cognitive workload does vary rapidly as a function of recent task events, and that real-time predictive models of operators’ cognitive workload provide a potential avenue for automation to adapt without an ongoing need for intrusive workload measurements.

Human operators often experience large fluctuations in cognitive workload over seconds time scales that can lead to sub-optimal performance, ranging from overload to neglect. Adaptive automation could potentially address this issue, but to do so it needs to be aware of real-time changes in operators’ spare cognitive capacity, so it can provide help in times of peak demand and take advantage of troughs to elicit operator engagement. However, it is unclear whether rapid changes in task demands are reflected in similarly rapid fluctuations in spare capacity, and if so what aspects of responses to those demands are predictive the current level of spare capacity. We used the ISO standard Detection Response Task (DRT) to measure cognitive workload approximately every 4 seconds in a demanding task requiring monitoring and refuelling of a fleet of simulated unmanned aerial vehicles (UAVs). We showed that the DRT provided a valid measure that can detect differences in workload due to changes in the number of UAVs. We used cross-validation to assess whether measures related to task performance immediately preceding the DRT could predict detection performance as a proxy for cognitive workload. Although the simple occurrence of task events had weak predictive ability, composite measures that tapped operators’ situational awareness with respect to fuel levels were much more effective. We conclude that cognitive workload does vary rapidly as a function of recent task events, and that real-time predictive models of operators’ cognitive workload provide a potential avenue for automation to adapt without an ongoing need for intrusive workload measurements.

The ability to inhibit ongoing responses that suddenly become inappropriate is essential for safe and effective interaction with an ever-changing and unpredictable world. Response inhibition is quantified by the unobservablestop-signal reaction time (SSRT), the completion time of an inhibitory process triggered by a signal to stop responding. SSRTs can be inferred based on a model in which inhibitory and response processes race with each other to control behavior. Inhibition is usually studied in the context of choice responses, but there has been increasing interest in what is often a key component of skilled behavior, stopping a response that is timed to coincide with an anticipated event. We show that SSRT measurement via the standard race model fails for anticipated responses because the stop signal changes the perception of the passage of time, due to the long-known “filled-interval illusion”. We propose a computational model of anticipated response inhibitionthat takes account of this distortion of time perception and show that this model produces valid estimates of not only SSRT, but also another key process that determines inhibitory ability, lapses in attention. Our new modeland accompanying Bayesian estimation procedures provide a solid basis for the burgeoning study of timed-action control.

Roberts (2020, Learning & Behavior, 48[2], 191–192) discussed research claiming honeybees can do arithmetic. Some readers of this research might regard such claims as unlikely. The present authors used this example as a basis for a debate on the criterion that ought to be used for publication of results or conclusions that could be viewed as unlikely by a significant number of readers, editors, or reviewers.

Despite the increasing popularity of Bayesian inference in empirical research, few practical guidelines provide detailed recommendations for how to apply Bayesian procedures and interpret the results. Here we offer specific guidelines for four different stages of Bayesian statistical reasoning in a research setting: planning the analysis, executing the analysis, interpreting the results, and reporting the results. The guidelines for each stage are illustrated with a running example. Although the guidelines are geared towards analyses performed with the open-source statistical software JASP, most guidelines extend to Bayesian inference in general.

The stop-signal paradigm is a popular procedure to investigate response inhibition—the ability to stop ongoing responses. It consists of a choice response time (RT) task that is occasionally interrupted by a stop stimulus signaling participants to withhold their response. Performance in the stop-signal paradigm is often formalized as race between a set of go runners triggered by the choice stimulus and a stop runner triggered by the stop signal. We investigated whether evidence-accumulation processes, which have been widely used in choice RT analysis, can serve as the runners in the stop-signal race model and support the estimation of psychologically meaningful parameters. We examined two types of the evidence-accumulation architectures: the racing Wald model (Logan et al. 2014) and a novel proposal based on the lognormal race (Heathcote and Love 2012). Using a series of simulation studies and fits to empirical data, we found that these models are not measurement models in the sense that the data-generating parameters cannot be recovered in realistic experimental designs.

Contains fulltext : 226716.pdf (Publisher’s version ) (Closed access) Analysis of variance (ANOVA) is the standard procedure for statistical inference in factorial designs. Typically, ANOVAs are executed using frequentist statistics, where p-values determine statistical significance in an all-or-none fashion. In recent years, the Bayesian approach to statistics is increasingly viewed as a legitimate alternative to the p-value. However, the broad adoption of Bayesian statistics - and Bayesian ANOVA in particular - is frustrated by the fact that Bayesian concepts are rarely taught in applied statistics courses. Consequently, practitioners may be unsure how to conduct a Bayesian ANOVA and interpret the results. Here we provide a guide for executing and interpreting a Bayesian ANOVA with JASP, an open-source statistical software program with a graphical user interface. We explain the key concepts of the Bayesian ANOVA using two empirical examples. 24 p.

Psychological experiments often yield data that are hierarchically structured. A number of popular shortcut strategies in cognitive modeling do not properly accommodate this structure and can result in biased conclusions. To gauge the severity of these biases, we conducted a simulation study for a two-group experiment. We first considered a modeling strategy that ignores the hierarchical data structure. In line with theoretical results, our simulations showed that Bayesian and frequentist methods that rely on this strategy are biased towards the null hypothesis. Secondly, we considered a modeling strategy that takes a two-step approach by first obtaining participant-level estimates from a hierarchical cognitive model and subsequently using these estimates in a follow-up statistical test. Methods that rely on this strategy are biased towards the alternative hypothesis. Only hierarchical models of the multilevel data lead to correct conclusions. Our results are particularly relevant for the use of hierarchical Bayesian parameter estimates in cognitive modeling.

Parameter estimation in evidence-accumulation models of choice response times is demanding of both the data and the user. We outline how to fit evidence-accumulation models using the flexible, open-source, R-based Dynamic Models of Choice (DMC) software. DMC provides a hands-on introduction to the Bayesian implementation of two popular evidence-accumulation models: the diffusion decision model (DDM) and the linear ballistic accumulator (LBA). It enables individual and hierarchical estimation, as well as assessment of the quality of a model’s parameter estimates and descriptive accuracy. First, we introduce the basic concepts of Bayesian parameter estimation, guiding the reader through a simple DDM analysis. We then illustrate the challenges of fitting evidence-accumulation models using a set of LBA analyses. We emphasize best practices in modeling and discuss the importance of parameter- and model-recovery simulations, exploring the strengths and weaknesses of models in different experimental designs and parameter regions. We also demonstrate how DMC can be used to model complex cognitive processes, using as an example a race model of the stop-signal paradigm, which is used to measure inhibitory ability. We illustrate the flexibility of DMC by extending this model to account for mixtures of cognitive processes resulting from attention failures. We then guide the reader through the practical details of a Bayesian hierarchical analysis, from specifying priors to obtaining posterior distributions that encapsulate what has been learned from the data. Finally, we illustrate how the Bayesian approach leads to a quantitatively cumulative science, showing how to use posterior distributions to specify priors that can be used to inform the analysis of future experiments.

Bayesian parameter estimation and Bayesian hypothesis testing present attractive alternatives to classical inference using confidence intervals and p values. In part I of this series we outline ten prominent advantages of the Bayesian approach. Many of these advantages translate to concrete opportunities for pragmatic researchers. For instance, Bayesian hypothesis testing allows researchers to quantify evidence and monitor its progression as data come in, without needing to know the intention with which the data were collected. We end by countering several objections to Bayesian hypothesis testing. Part II of this series discusses JASP, a free and open source software program that makes it easy to conduct Bayesian estimation and testing for a range of popular statistical scenarios (Wagenmakers et al. this issue).

We outline a network method to synthesize a literature overview from search results obtained by multiple team members. Several network statistics are used to create a single representativeness ranking. We illustrate the method with the dispersed literature on a common misinterpretation of analysis of covariance (ANCOVA). The network method yields a top ten list of the most relevant articles that students and researchers can take as a point of departure for a more detailed study on this topic. The proposed methodology is implemented in Shiny, an open-source R package.

Human operators often experience large fluctuations in cognitive workload over seconds timescales that can lead to sub-optimal performance, ranging from overload to neglect. Adaptive automation could potentially address this issue, but to do so it needs to be aware of real-time changes in operators' spare cognitive capacity, so it can provide help in times of peak demand and take advantage of troughs to elicit operator engagement. However, it is unclear whether rapid changes in task demands are reflected in similarly rapid fluctuations in spare capacity, and if so what aspects of responses to those demands are predictive of the current level of spare capacity. We used the ISO standard detection response task (DRT) to measure cognitive workload approximately every 4 s in a demanding task requiring monitoring and refueling of a fleet of simulated unmanned aerial vehicles (UAVs). We showed that the DRT provided a valid measure that can detect differences in workload due to changes in the number of UAVs. We used cross-validation to assess whether measures related to task performance immediately preceding the DRT could predict detection performance as a proxy for cognitive workload. Although the simple occurrence of task events had weak predictive ability, composite measures that tapped operators' situational awareness with respect to fuel levels were much more effective. We conclude that cognitive workload does vary rapidly as a function of recent task events, and that real-time predictive models of operators' cognitive workload provide a potential avenue for automation to adapt without an ongoing need for intrusive workload measurements.

We demonstrate the use of three popular Bayesian software packages that enable researchers to estimate parameters in a broad class of models that are commonly used in psychological research. We focus on WinBUGS, JAGS, and Stan, and show how they can be interfaced from R and MATLAB. We illustrate the use of the packages through two fully worked examples; the examples involve a simple univariate linear regression and fitting a multinomial processing tree model to data from a classic false-memory experiment. We conclude with a comparison of the strengths and weaknesses of the packages. Our example code, data, and this text are available via https://osf.io/ucmaz/ .