Search

In 2015, the Australian Government's Excellence in Research for Australia (ERA) assessment of research quality declined to rate 1.5 per cent of submissions from universities. The public debate focused on practices of gaming or "coding errors" within university submissions as the reason for this outcome. The issue was about the in/appropriate allocation of research activities to Fields of Research. This paper argues that such practices are only part of the explanation. With the support of statistical modelling, unrated outcomes are shown to have also arisen from particular evaluation practices within the discipline of Psychology and the associated Medical and Health Sciences Research Evaluation Committee. Given the high stakes nature of unrated outcomes and that the evaluation process breaches public administration principles by being not appealable nor appropriately transparent, the paper concludes with recommendations for the strengthening ERA policy and procedures to enhance trust in future ERA processes.

For decisions between many alternatives, the benchmark result is Hick's Law: that response time increases log-linearly with the number of choice alternatives. Even when Hick's Law is observed for response times, divergent results have been observed for error rates--sometimes error rates increase with the number of choice alternatives, and sometimes they are constant. We provide evidence from two experiments that error rates are mostly independent of the number of choice alternatives, unless context effects induce participants to trade speed for accuracy across conditions. Error rate data have previously been used to discriminate between competing theoretical accounts of Hick's Law, and our results question the validity of those conclusions. We show that a previously dismissed optimal observer model might provide a parsimonious account of both response time and error rate data. The model suggests that people approximate Bayesian inference in multi-alternative choice, except for some perceptual limitations. (Contains 7 figures and 1 note.)

With the growth of client-centered and patient-as-consumer approaches to care, understanding the preferences of psychologists' patients has never been more important. Traditional methods for measuring preference, such as Likert-type rating scales, suffer from well-known limitations, including subjectivity and positive bias. Best-worst scaling (BWS) provides an opportunity to address some of these limitations. Despite the growing use of BWS to measure preference in other areas, BWS methods are not being used in the study of psychologists' patients. We demonstrate BWS data collection and analysis. With a sample of only 31 clients from 2 Australian psychology practices, we show the strength of preference for different aspects of psychologists' appointments can be measured accurately. Additionally, the inclusion of readily available timing data from responses improved measurement sensitivity and statistical power.