Search

In 2015, the Australian Government's Excellence in Research for Australia (ERA) assessment of research quality declined to rate 1.5 per cent of submissions from universities. The public debate focused on practices of gaming or "coding errors" within university submissions as the reason for this outcome. The issue was about the in/appropriate allocation of research activities to Fields of Research. This paper argues that such practices are only part of the explanation. With the support of statistical modelling, unrated outcomes are shown to have also arisen from particular evaluation practices within the discipline of Psychology and the associated Medical and Health Sciences Research Evaluation Committee. Given the high stakes nature of unrated outcomes and that the evaluation process breaches public administration principles by being not appealable nor appropriately transparent, the paper concludes with recommendations for the strengthening ERA policy and procedures to enhance trust in future ERA processes.

An important feature of human cognition is the ability to flexibly and efficiently adapt behavior in response to continuously changing contextual demands. We leverage a large-scale dataset from Lumosity, an online cognitive-training platform, to investigate how cognitive processes involved in cued switching between tasks are affected by level of task practice across the adult lifespan. We develop a computational account of task switching that specifies the temporal dynamics of activating task-relevant representations and inhibiting task-irrelevant representations and how they vary with extended task practice across a number of age groups. Practice modulates the level of activation of the task-relevant representation and improves the rate at which this information becomes available, but has little effect on the task-irrelevant representation. While long-term practice improves performance across all age groups, it has a greater effect on older adults. Indeed, extensive task practice can make older individuals functionally similar to less-practiced younger individuals, especially for cognitive measures that focus on the rate at which task-relevant information becomes available.

(Immunity 48, 812–830.e1–e14; April 17, 2018) In the originally published version of this article, the authors neglected to include Younes Mokrab and Aaron M. Newman as co-authors and misspelled the names of authors Charles S. Rabkin and Ilya Shmulevich. The author names have been corrected here and online. In addition, the concluding sentence of the subsection “Immune Signature Compilation” in the Method Details in the original published article was deemed unclear because it did not specify differences among the gene set scoring methods. The concluding sentences now reads “Gene sets from Bindea et al., Senbabaoglu et al., and the MSigDB C7 collection were scored using single-sample gene set enrichment (ssGSEA) analysis (Barbie et al., 2009), as implemented in the GSVA R package (Hänzelmann et al., 2013). All other signatures were scored using methods found in the associated citations.”

Most data analyses rely on models. To complement statistical models, psychologists have developed cognitive models, which translate observed variables into psychologically interesting constructs. Response time models, in particular, assume that response time and accuracy are the observed expression of latent variables including 1) ease of processing, 2) response caution, 3) response bias, and 4) non-decision time. Inferences about these psychological factors, hinge upon the validity of the models' parameters. Here, we use a blinded, collaborative approach to assess the validity of such model-based inferences. Seventeen teams of researchers analyzed the same 14 data sets. In each of these two-condition data sets, we manipulated properties of participants' behavior in a two-alternative forced choice task. The contributing teams were blind to the manipulations, and had to infer what aspect of behavior was changed using their method of choice. The contributors chose to employ a variety of models, estimation methods, and inference procedures. Our results show that, although conclusions were similar across different methods, these "modeler's degrees of freedom" did affect their inferences. Interestingly, many of the simpler approaches yielded as robust and accurate inferences as the more complex methods. We recommend that, in general, cognitive models become a typical analysis tool for response time data. In particular, we argue that the simpler models and procedures are sufficient for standard experimental designs. We finish by outlining situations in which more complicated models and methods may be necessary, and discuss potential pitfalls when interpreting the output from response time models.

Theories of perceptual decision making have been dominated by the idea that evidence accumulates in favor of different alternatives until some fixed threshold amount is reached, which triggers a decision. Recent theories have suggested that these thresholds may not be fixed during each decision but change as time passes. These collapsing thresholds can improve performance in particular decision environments, but reviews of data from typical decision-making paradigms have failed to support collapsing thresholds. We designed three experiments to test collapsing threshold assumptions in decision environments specifically tailored to make them optimal. An emphasis on decision speed encouraged the adoption of collapsing thresholds--most strongly through the use of response deadlines but also through instruction to a lesser extent--but setting an explicit goal of reward rate optimality through both instructions and task design did not. Our results suggest that collapsing thresholds models of decision-making are inconsistent with human behaviour even in some situations where there are normative motivations for these models.

We performed an extensive immunogenomic analysis of more than 10,000 tumors comprising 33 diverse cancer types by utilizing data compiled by TCGA. Across cancer types, we identified six immune subtypes-wound healing, IFN-γ dominant, inflammatory, lymphocyte depleted, immunologically quiet, and TGF-β dominant-characterized by differences in macrophage or lymphocyte signatures, Th1:Th2 cell ratio, extent of intratumoral heterogeneity, aneuploidy, extent of neoantigen load, overall cell proliferation, expression of immunomodulatory genes, and prognosis. Specific driver mutations correlated with lower (CTNNB1, NRAS, or IDH1) or higher (BRAF, TP53, or CASP8) leukocyte levels across all cancers. Multiple control modalities of the intracellular and extracellular networks (transcription, microRNAs, copy number, and epigenetic processes) were involved in tumor-immune cell interactions, both across and within immune subtypes. Our immunogenomics pipeline to characterize these heterogeneous tumors and the resulting data are intended to serve as a resource for future targeted studies to further advance the field.

How does the process of information transmission affect the cultural or linguistic products that emerge? This question is often studied experimentally and computationally via iterated learning, a procedure in which participants learn from previous participants in a chain. Iterated learning is a powerful tool because, when all participants share the same priors, the stationary distributions of the iterated learning chains reveal those priors. In many situations, however, it is unreasonable to assume that all participants share the same prior beliefs. We present four simulation studies and one experiment demonstrating that when the population of learners is heterogeneous, the behavior of an iterated learning chain can be unpredictable and is often systematically distorted by the learners with the most extreme biases. This results in group-level outcomes that reflect neither the behavior of any individuals within the population nor the overall population average. We discuss implications for the use of iterated learning as a methodological tool as well as for the processes that might have shaped cultural and linguistic evolution in the real world.

Understanding individual differences in cognitive performance is an important part of understanding how variations in underlying cognitive processes can result in variations in task performance. However, the exploration of individual differences in the components of the decision process--such as cognitive processing speed, response caution, and motor execution speed--in previous research has been limited. Here, we assess the heritability of the components of the decision process, with heritability having been a common aspect of individual differences research within other areas of cognition. Importantly, a limitation of previous work on cognitive heritability is the underlying assumption that variability in response times solely reflects variability in the speed of cognitive processing. This assumption has been problematic in other domains, due to the confounding effects of caution and motor execution speed on observed response times. We extend a cognitive model of decision-making to account for relatedness structure in a twin study paradigm. This approach can separately quantify different contributions to the heritability of response time. Using data from the Human Connectome Project, we find strong evidence for the heritability of response caution, and more ambiguous evidence for the heritability of cognitive processing speed and motor execution speed. Our study suggests that the assumption made in previous studies--that the heritability of cognitive ability is based on cognitive processing speed--may be incorrect. More generally, our methodology provides a useful avenue for future research in complex data that aims to analyze cognitive traits across different sources of related data, whether the relation is between people, tasks, experimental phases, or methods of measurement.

Context effects occur when a choice between 2 options is altered by adding a 3rd alternative. Three major context effects--similarity, compromise, and attraction--have wide-ranging implications across applied and theoretical domains, and have driven the development of new dynamic models of multiattribute and multialternative choice. We propose the multiattribute linear ballistic accumulator (MLBA), a new dynamic model that provides a quantitative account of all 3 context effects. Our account applies not only to traditional paradigms involving choices among hedonic stimuli, but also to recent demonstrations of context effects with nonhedonic stimuli. Because of its computational tractability, the MLBA model is more easily applied than previous dynamic models. We show that the model also accounts for a range of other phenomena in multiattribute, multialternative choice, including time pressure effects, and that it makes a new prediction about the relationship between deliberation time and the magnitude of the similarity effect, which we confirm experimentally.

BackgroundThe subgenual anterior cingulate cortex (sgACC) and its connected circuitry have been heavily implicated in emotional functioning in adolescent-onset major depressive disorder (MDD). While several recent studies have examined sgACC functional connectivity (FC) in depressed youth at rest, no studies to date have investigated sgACC FC in adolescent depression during negative emotional processing.MethodsNineteen medication-naïve adolescents with MDD and 19 matched healthy controls (HCL) performed an implicit fear facial affect recognition task during functional magnetic resonance imaging (fMRI). We defined seeds in bilateral sgACC and assessed FC using the psychophysiological interaction method. We also applied cognitive behavioral modeling to estimate group differences in perceptual sensitivity in this task. Finally, we correlated connectivity strength with clinical data and perceptual sensitivity.ResultsDepressed adolescents showed increased sgACC-amygdala FC and decreased sgACC-fusiform gyrus, sgACC-precuneus, sgACC-insula, and sgACC-middle frontal gyrus FC compared to HCL (p

Model comparison is the cornerstone of theoretical progress in psychological research. Common practice overwhelmingly relies on tools that evaluate competing models by balancing in-sample descriptive adequacy against model flexibility, with modern approaches advocating the use of marginal likelihood for hierarchical cognitive models. Cross-validation is another popular approach but its implementation remains out of reach for cognitive models evaluated in a Bayesian hierarchical framework, with the major hurdle being its prohibitive computational cost. To address this issue, we develop novel algorithms that make variational Bayes (VB) inference for hierarchical models feasible and computationally efficient for complex cognitive models of substantive theoretical interest. It is well known that VB produces good estimates of the first moments of the parameters, which gives good predictive densities estimates. We thus develop a novel VB algorithm with Bayesian prediction as a tool to perform model comparison by cross-validation, which we refer to as CVVB. In particular, CVVB can be used as a model screening device that quickly identifies bad models. We demonstrate the utility of CVVB by revisiting a classic question in decision making research: what latent components of processing drive the ubiquitous speed-accuracy tradeoff? We demonstrate that CVVB strongly agrees with model comparison via marginal likelihood, yet achieves the outcome in much less time. Our approach brings cross-validation within reach of theoretically important psychological models, making it feasible to compare much larger families of hierarchically specified cognitive models than has previously been possible. To enhance the applicability of the algorithm, we provide Matlab code together with a user manual so users can easily implement VB and/or CVVB for the models considered in this article and their variants. Progress in psychological science can be made by choosing between competing theories: Does sleep deprivation cause attentional lapses? Does alcohol impair the speed of information processing or reduce cautiousness, or both? When these theories are quantitative models that can be estimated from observed data, the problem is known as statistical model selection. Cross-validation is a desirable method for model selection as it directly addresses the question that scientists are often interested in: how well will this model predict new data? However, cross-validation is not used very widely in psychological science due to its computational cost. Modern psychological theories often include random effects, which are necessary to account for individual differences, but these can mean that Bayesian evaluation of the model can take days of computer time. This makes repeated evaluation for cross-validation impractical. Another practical limitation arises when comparing a very large number of competing hypotheses, and this can amplify issues of researcher bias: researchers are forced to focus on a limited subset of models, which they identify by idiosyncratic means. We propose crossvalidation with variational Bayes (CVVB) to tackle these issues. CVVB allows researchers to address larger sets of model-based hypotheses, even using modern multilevel theories. As a demonstration, we use CVVB to investigate an important question about strategies for trading speed versus caution in simple decision-making. [ABSTRACT FROM AUTHOR]

For decisions between many alternatives, the benchmark result is Hick's Law: that response time increases log-linearly with the number of choice alternatives. Even when Hick's Law is observed for response times, divergent results have been observed for error rates--sometimes error rates increase with the number of choice alternatives, and sometimes they are constant. We provide evidence from two experiments that error rates are mostly independent of the number of choice alternatives, unless context effects induce participants to trade speed for accuracy across conditions. Error rate data have previously been used to discriminate between competing theoretical accounts of Hick's Law, and our results question the validity of those conclusions. We show that a previously dismissed optimal observer model might provide a parsimonious account of both response time and error rate data. The model suggests that people approximate Bayesian inference in multi-alternative choice, except for some perceptual limitations. (Contains 7 figures and 1 note.)