Search

With the growing role of artificial intelligence (AI) in our lives, attention is increasingly turning to the way that humans and AI work together. A key aspect of human–AI collaboration is how people integrate judgements or recommendations from machine agents, when they differ from their own judgements. We investigated trust in human–machine teaming using a perceptual judgement task based on the judge–advisor system. Participants ( n = 89 ) estimated a perceptual quantity, then received a recommendation from a machine agent. The participants then made a second response which combined their first estimate and the machine's recommendation. The degree to which participants shifted their second response in the direction of the recommendations provided a measure of their trust in the machine agent. We analysed the role of advice distance in people's willingness to change their judgements. When a recommendation falls a long way from their initial judgement, do people come to doubt their own judgement, trusting the recommendation more, or do they doubt the machine agent, trusting the recommendation less? We found that although some participants exhibited these behaviours, the most common response was neither of these tendencies, and a simple model based on averaging accounted best for participants' trust behaviour. We discuss implications for theories of trust, and human–machine teaming. [ABSTRACT FROM AUTHOR]

Threatened species monitoring can produce enormous quantities of acoustic and visual recordings which must be searched for animal detections. Data coding is extremely time-consuming for humans and even though machine algorithms are emerging as useful tools to tackle this task, they too require large amounts of known detections for training. Citizen scientists are often recruited via crowd-sourcing to assist. However, the results of their coding can be difficult to interpret because citizen scientists lack comprehensive training and typically each codes only a small fraction of the full dataset. Competence may vary between citizen scientists, but without knowing the ground truth of the dataset, it is difficult to identify which citizen scientists are most competent. We used a quantitative cognitive model, cultural consensus theory, to analyze both empirical and simulated data from a crowdsourced analysis of audio recordings of Australian frogs. Several hundred citizen scientists were asked whether the calls of nine frog species were present on 1260 brief audio recordings, though most only coded a fraction of these recordings. Through modeling, characteristics of both the citizen scientist cohort and the recordings were estimated. We then compared the model's output to expert coding of the recordings and found agreement between the cohort's consensus and the expert evaluation. This finding adds to the evidence that crowdsourced analyses can be utilized to understand large-scale datasets, even when the ground truth of the dataset is unknown. The model-based analysis provides a promising tool to screen large datasets prior to investing expert time and resources. [ABSTRACT FROM AUTHOR]

Poorly differentiated chordoma (PDC) is a recently recognized subtype of chordoma characterized by expression of the embryonic transcription factor, brachyury, and loss of INI1. PDC primarily affects children and is associated with a poor prognosis and limited treatment options. Here we describe the molecular and immune tumour microenvironment profiles of two paediatric PDCs produced using whole-genome, transcriptome and whole-genome bisulfite sequencing (WGBS) and multiplex immunohistochemistry. Our analyses revealed the presence of tumour-associated immune cells, including CD8+ T cells, and expression of the immune checkpoint protein, PD-L1, in both patient samples. Molecular profiling provided the rationale for immune checkpoint inhibitor (ICI) therapy, which resulted in a clinical and radiographic response. A dominant T cell receptor (TCR) clone specific for a brachyury peptide–MHC complex was identified from bulk RNA sequencing, suggesting that targeting of the brachyury tumour antigen by tumour-associated T cells may underlie this clinical response to ICI. Correlative analysis with rhabdoid tumours, another INI1-deficient paediatric malignancy, suggests that a subset of tumours may share common immune phenotypes, indicating the potential for a therapeutically targetable subgroup of challenging paediatric cancers. [ABSTRACT FROM AUTHOR]

It is commonly assumed that a specific testing occasion (task, design, procedure, etc.) provides insights that generalize beyond that occasion. This assumption is infrequently carefully tested in data. We develop a statistically principled method to directly estimate the correlation between latent components of cognitive processing across tasks, contexts, and time. This method simultaneously estimates individual-participant parameters of a cognitive model at each testing occasion, group-level parameters representing across-participant parameter averages and variances, and across-task correlations. The approach provides a natural way to "borrow" strength across testing occasions, which can increase the precision of parameter estimates across all testing occasions. Two example applications demonstrate that the method is practical in standard designs. The examples, and a simulation study, also provide evidence about the reliability and validity of parameter estimates from the linear ballistic accumulator model. We conclude by highlighting the potential of the parameter-correlation method to provide an "assumption-light" tool for estimating the relatedness of cognitive processes across tasks, contexts, and time. [ABSTRACT FROM AUTHOR]

Purpose: To explore in a sample of adult cancer patients: (1) the relative influence of initiation source, information format and consultation format on preferred approach to life expectancy disclosure using a discrete choice experiment (DCE); and (2) whether patient age, cancer type and perceived prognosis were associated with preferences within the three attributes. Methods: A DCE survey of adult solid tumour and haematological cancer patients. Participants chose between three hypothetical scenarios about life expectancy disclosure consisting of three attributes: initiation source (i.e. doctor versus patient-initiated discussion), information content (i.e. estimate presented as best-worst-typical length of life case scenario versus median survival time) and consultation format (i.e. two 20-min versus one 40-min consultation). Respondents selected their most preferred scenario within each question. Results: Three hundred and two patients completed the DCE (78% consent rate). Initiation source was the most influential predictor of patient choice. More preferred a doctor deliver life expectancy information as soon as it is available rather than waiting for the patient to ask (59% vs 41% z = − 7.396, p < 0.01). More patients preferred the two 20-min rather than the one 40-min consultation format (55% vs 45%, z = 4.284, p < 0.01). Information content did not influence choice. Age, cancer type, and patient-perceived prognosis were not associated with preferences. Conclusion: Healthcare professionals should assess cancer patients' preferences for engaging in life expectancy discussions as soon as they have this information, and ensure patients have adequate time to consider the information they receive, seek additional information and involve others if they wish. [ABSTRACT FROM AUTHOR]

With the advancement of technologies like in-car navigation and smartphones, concerns around how cognitive functioning is influenced by "workload" are increasingly prevalent. Research shows that spreading effort across multiple tasks can impair cognitive abilities through an overuse of resources, and that similar overload effects arise in difficult single-task paradigms. We developed a novel lab-based extension of the Detection Response Task, which measures workload, and paired it with a Multiple Object Tracking Task to manipulate cognitive load. Load was manipulated either by changing within-task difficulty or by the addition of an extra task. Using quantitative cognitive modelling we showed that these manipulations cause similar cognitive impairments through diminished processing rates, but that the introduction of a second task tends to invoke more cautious response strategies that do not occur when only difficulty changes. We conclude that more prudence should be exercised when directly comparing multi-tasking and difficulty-based workload impairments, particularly when relying on measures of central tendency. [ABSTRACT FROM AUTHOR]

A standard assumption of most sequential sampling models is that decision-makers rely on a decision criterion that remains constant throughout the decision process. However, several authors have recently suggested that, in order to maximize reward rates in dynamic environments, decision-makers need to rely on a decision criterion that changes over the course of the decision process. We used dynamic programming and simulation methods to quantify the reward rates obtained by constant and dynamic decision criteria in different environments. We further investigated what influence a decision-maker's uncertainty about the stochastic structure of the environment has on reward rates. Our results show that in most dynamic environments, both types of decision criteria yield similar reward rates, across different levels of uncertainty. This suggests that a static decision criterion might provide a robust default setting. [ABSTRACT FROM AUTHOR]

Most data analyses rely on models. To complement statistical models, psychologists have developed cognitive models, which translate observed variables into psychologically interesting constructs. Response time models, in particular, assume that response time and accuracy are the observed expression of latent variables including 1) ease of processing, 2) response caution, 3) response bias, and 4) non-decision time. Inferences about these psychological factors, hinge upon the validity of the models' parameters. Here, we use a blinded, collaborative approach to assess the validity of such model-based inferences. Seventeen teams of researchers analyzed the same 14 data sets. In each of these two-condition data sets, we manipulated properties of participants' behavior in a two-alternative forced choice task. The contributing teams were blind to the manipulations, and had to infer what aspect of behavior was changed using their method of choice. The contributors chose to employ a variety of models, estimation methods, and inference procedures. Our results show that, although conclusions were similar across different methods, these "modeler's degrees of freedom" did affect their inferences. Interestingly, many of the simpler approaches yielded as robust and accurate inferences as the more complex methods. We recommend that, in general, cognitive models become a typical analysis tool for response time data. In particular, we argue that the simpler models and procedures are sufficient for standard experimental designs. We finish by outlining situations in which more complicated models and methods may be necessary, and discuss potential pitfalls when interpreting the output from response time models. [ABSTRACT FROM AUTHOR]

Decision-making involves a tradeoff between pressures for caution and urgency. Previous research has investigated how well humans optimize this tradeoff, and mostly concluded that people adopt a sub-optimal strategy that over-emphasizes caution. This emphasis reduces how many decisions can be made in a fixed time, which reduces the "reward rate". However, the strategy that is optimal depends critically on the timing properties of the experiment design: the slower the rate of decision opportunities, the more cautious the optimal strategy. Previous studies have almost uniformly adopted very fast designs, which favor very urgent decision-making. This raises the possibility that previous findings—that humans adopt strategies that are too cautious—could either be ascribed to human caution, or to the experiments' design. To test this, we used a slowed-down decision-making task in which the optimal strategy was quite cautious. With this task, and in contrast to previous findings, the average strategy adopted across participants was very close to optimal, with about equally many participants adopting too-cautious as too-urgent strategies. Our findings suggest that task design can play a role in inferences about optimality, and that previous conclusions regarding human sub-optimality are conditional on the task settings. This limits claims about human optimality that can be supported by the available evidence. [ABSTRACT FROM AUTHOR]

Objectives: To explore in a sample of medical oncology outpatients and their nominated support persons (SPs): (1) the relative influence of pain, consciousness and life extension on end-of-life choices using a discrete choice experiment (DCE); (2) the extent to which SPs can predict the choices of index patients and (3) whether having a previous end-of-life discussion was associated with dyad agreement.Methods: Adult medical oncology patients and their SPs were approached for consent to complete a survey containing a DCE. Participants chose between three unlabelled care scenarios characterised by three attributes: pain (mild, moderate or severe), consciousness (some, half or most of time) and extension of life (1, 2 or 3 weeks). Respondents selected (1) most-preferred and (2) least-preferred scenarios within each question. SPs answered the same questions but from patient's perspective.Results: A total of 110 patients and 64 SPs responded overall (42 matched patient-SP dyads). For patients, pain was the most influential predictor of most- and least-preferred scenarios (z = 12.5 and z = 12.9). For SPs, pain was the only significant predictor of most and least-preferred scenarios (z = 9.7 and z = 11.5). Dyad agreement was greater for choices about least- (69%) compared to most-preferred scenarios (55%). Agreement was slightly higher for dyads reporting a previous EOL discussion (68 versus 48%; p = 0.065).Conclusion: Patients and SPs place significant value on avoiding severe pain when making end-of-life choices, over and above level of consciousness or life extension. People's views about end-of-life scenarios they most as well as least prefer should be sought. [ABSTRACT FROM AUTHOR]

Evidence accumulation models of decision-making have led to advances in several different areas of psychology. These models provide a way to integrate response time and accuracy data, and to describe performance in terms of latent cognitive processes. Testing important psychological hypotheses using cognitive models requires a method to make inferences about different versions of the models which assume different parameters to cause observed effects. The task of model-based inference using noisy data is difficult, and has proven especially problematic with current model selection methods based on parameter estimation. We provide a method for computing Bayes factors through Monte-Carlo integration for the linear ballistic accumulator (LBA; Brown and Heathcote, 2008), a widely used evidence accumulation model. Bayes factors are used frequently for inference with simpler statistical models, and they do not require parameter estimation. In order to overcome the computational burden of estimating Bayes factors via brute force integration, we exploit general purpose graphical processing units; we provide free code for this. This approach allows estimation of Bayes factors via Monte-Carlo integration within a practical time frame. We demonstrate the method using both simulated and real data. We investigate the stability of the Monte-Carlo approximation, and the LBA’s inferential properties, in simulation studies. [ABSTRACT FROM AUTHOR]

Markov Chain Monte-Carlo (MCMC) is an increasingly popular method for obtaining information about distributions, especially for estimating posterior distributions in Bayesian inference. This article provides a very basic introduction to MCMC sampling. It describes what MCMC is, and what it can be used for, with simple illustrative examples. Highlighted are some of the benefits and limitations of MCMC sampling, as well as different approaches to circumventing the limitations most likely to trouble cognitive scientists. [ABSTRACT FROM AUTHOR]

Identification accuracy for sets of perceptually discriminable stimuli ordered on a single dimension (e.g., line length) is remarkably low, indicating a fundamental limit on information processing capacity. This surprising limit has naturally led to a focus on measuring and modeling choice probability in absolute identification research. We show that choice response time (RT) results can enrich our understanding of absolute identification by investigating dissociation between RT and accuracy as a function of stimulus spacing. The dissociation is predicted by the SAMBA model of absolute identification (Brown, Marley, Dockin, & Heathcote, ), but cannot easily be accommodated by other theories. We show that SAMBA provides an accurate, parameter free, account of the dissociation that emerges from the architecture of the model and the physical attributes of the stimuli, rather than through numerical adjustment. This violation of the pervasive monotonic relationship between RT and accuracy has implications for model development, which are discussed. [ABSTRACT FROM AUTHOR]

Psychological experiments often collect choice responses using buttonpresses. However, spoken responses are useful in many cases--for example, when working with special clinical populations, or when a paradigm demands vocalization, or when accurate response time measurements are desired. In these cases, spoken responses are typically collected using a voice key, which usually involves manual coding by experimenters in a tedious and error-prone manner. We describe ChoiceKey, an open-source speech recognition package for MATLAB. It can be optimized by training for small response sets and different speakers. We show ChoiceKey to be reliable with minimal training for most participants in experiments with two different responses. Problems presented by individual differences, and occasional atypical responses, are examined, and extensions to larger response sets are explored. The ChoiceKey source files and instructions may be downloaded as supplemental materials for this article from brm.psychonomic-journals.org/content/supplemental. [ABSTRACT FROM AUTHOR]