Search

We introduce the Individual Differences in Language Skills (IDLaS-NL) web platform, which enables users to run studies on individual differences in Dutch language skills via the Internet. IDLaS-NL consists of 35 behavioral tests, previously validated in participants aged between 18 and 30 years. The platform provides an intuitive graphical interface for users to select the tests they wish to include in their research, to divide these tests into different sessions and to determine their order. Moreover, for standardized administration the platform provides an application (an emulated browser) wherein the tests are run. Results can be retrieved by mouse click in the graphical interface and are provided as CSV file output via e-mail. Similarly, the graphical interface enables researchers to modify and delete their study configurations. IDLaS-NL is intended for researchers, clinicians, educators and in general anyone conducting fundamental research into language and general cognitive skills; it is not intended for diagnostic purposes. All platform services are free of charge. Here, we provide a description of its workings as well as instructions for using the platform. The IDLaS-NL platform can be accessed at www.mpi.nl/idlas-nl. [ABSTRACT FROM AUTHOR]

Anonymous web-based experiments are increasingly used in many domains of behavioral research. However, online studies of auditory perception, especially of psychoacoustic phenomena pertaining to low-level sensory processing, are challenging because of limited available control of the acoustics, and the inability to perform audiometry to confirm normal-hearing status of participants. Here, we outline our approach to mitigate these challenges and validate our procedures by comparing web-based measurements to lab-based data on a range of classic psychoacoustic tasks. Individual tasks were created using jsPsych, an open-source JavaScript front-end library. Dynamic sequences of psychoacoustic tasks were implemented using Django, an open-source library for web applications, and combined with consent pages, questionnaires, and debriefing pages. Subjects were recruited via Prolific, a subject recruitment platform for web-based studies. Guided by a meta-analysis of lab-based data, we developed and validated a screening procedure to select participants for (putative) normal-hearing status based on their responses in a suprathreshold task and a survey. Headphone use was standardized by supplementing procedures from prior literature with a binaural hearing task. Individuals meeting all criteria were re-invited to complete a range of classic psychoacoustic tasks. For the re-invited participants, absolute thresholds were in excellent agreement with lab-based data for fundamental frequency discrimination, gap detection, and sensitivity to interaural time delay and level difference. Furthermore, word identification scores, consonant confusion patterns, and co-modulation masking release effect also matched lab-based studies. Our results suggest that web-based psychoacoustics is a viable complement to lab-based research. Source code for our infrastructure is provided. [ABSTRACT FROM AUTHOR]

Many behavioural phenomena have been replicated using web-based experiments, but evaluation of the agreement between objective measures of web- and lab-based performance is required if scientists and clinicians are to reap the benefits of web-based testing. In this study, we investigated the reliability of a task which assesses early visual cortical function by evaluating the well-known 'oblique effect' (we are better at seeing horizontal and vertical edges than tilted ones) and the levels of agreement between remote, web-based measures and lab-based measures. Sixty-nine young participants (mean age, 21.8 years) performed temporal and spatial versions of a web-based, two-alternative forced choice (2AFC) orientation-identification task. In each case, orientation-identification thresholds (the minimum orientation difference at which a standard orientation could be reliably distinguished from a rotated comparison) were measured for cardinal (horizontal and vertical) and oblique orientations. Reliability was assessed in a subsample of 18 participants who performed the same tasks under laboratory conditions. Robust oblique effects were found, such that thresholds were substantially lower for cardinal orientations compared to obliques, for both web- and lab-based measures of the temporal and spatial 2AFC tasks. Crucially, web- and lab-based orientation-identification thresholds showed high levels of agreement, demonstrating the suitability of web-based testing for assessments of early visual cortical function. Future studies should assess the reliability of similar web-based tasks in clinical populations to evaluate their adoption into clinical settings, either to screen for visual anomalies or to assess changes in performance associated with progression of disease severity. [ABSTRACT FROM AUTHOR]

The goal of the current work was to develop and validate web-based measures for assessing English vocabulary knowledge. Two existing paper-and-pencil assessments, the Vocabulary Size Test (VST) and the Word Familiarity Test (WordFAM), were modified for web-based administration. In Experiment 1, participants (n = 100) completed the web-based VST. In Experiment 2, participants (n = 100) completed the web-based WordFAM. Results from these experiments confirmed that both tasks (1) could be completed online, (2) showed expected sensitivity to English frequency patterns, (3) exhibited high internal consistency, and (4) showed an expected range of item discrimination scores, with low frequency items exhibiting higher item discrimination scores compared to high frequency items. This work provides open-source English vocabulary knowledge assessments with normative data that researchers can use to foster high quality data collection in web-based environments. [ABSTRACT FROM AUTHOR]

Two measures for assessing English vocabulary knowledge, the Vocabulary Size Test (VST) and the Word Familiarity Test (WordFAM), were recently validated for web-based administration. An analysis of the psychometric properties of these assessments revealed high internal consistency, suggesting that stable assessment could be achieved with fewer test items. Because researchers may use these assessments in conjunction with other experimental tasks, the utility may be enhanced if they are shorter in duration. To this end, two "brief" versions of the VST and the WordFAM were developed and submitted to validation testing. Each version consisted of approximately half of the items from the full assessment, with novel items across each brief version. Participants (n = 85) completed one brief version of both the VST and the WordFAM at session one, followed by the other brief version of each assessment at session two. The results showed high test-retest reliability for both the VST (r = 0.68) and the WordFAM (r = 0.82). The assessments also showed moderate convergent validity (ranging from r = 0.38 to 0.59), indicative of assessment validity. This work provides open-source English vocabulary knowledge assessments with normative data that researchers can use to foster high quality data collection in web-based environments. [ABSTRACT FROM AUTHOR]

This paper describes the developmental phases of the digital TestDaF and shows how the new task types and the digital test administration strengthen the relationship to the context of higher education institutions in Germany and to the CEFR. The contribution deals extensively with the test construct, the test format, and the different types of tasks used in the reading, listening, writing, and speaking sections. Innovative approaches and methods of performance assessment and scoring are discussed as much as their practical implementation. Finally, the paper presents some perspectives of future studies on the digital TestDaF and elaborates on relations to contemporary language testing research. [ABSTRACT FROM AUTHOR]

Web-based experimental testing has seen exponential growth in psychology and cognitive neuroscience. However, paradigms involving affective auditory stimuli have yet to adapt to the online approach due to concerns about the lack of experimental control and other technical challenges. In this study, we assessed whether sounds commonly used to evoke affective responses in-lab can be used online. Using recent developments to increase sound presentation quality, we selected 15 commonly used sound stimuli and assessed their impact on valence and arousal states in a web-based experiment. Our results reveal good inter-rater and test-retest reliabilities, with results comparable to in-lab studies. Additionally, we compared a variety of previously used unpleasant stimuli, allowing us to identify the most aversive among these sounds. Our findings demonstrate that affective sounds can be reliably delivered through web-based platforms, which help facilitate the development of new auditory paradigms for affective online experiments. [ABSTRACT FROM AUTHOR]

Online experimental platforms can be used as an alternative to, or complement, lab-based research. However, when conducting auditory experiments via online methods, the researcher has limited control over the participants' listening environment. We offer a new method to probe one aspect of that environment, headphone use. Headphones not only provide better control of sound presentation but can also "shield" the listener from background noise. Here we present a rapid (< 3 min) headphone screening test based on Huggins Pitch (HP), a perceptual phenomenon that can only be detected when stimuli are presented dichotically. We validate this test using a cohort of "Trusted" online participants who completed the test using both headphones and loudspeakers. The same participants were also used to test an existing headphone test (AP test; Woods et al., 2017, Attention Perception Psychophysics). We demonstrate that compared to the AP test, the HP test has a higher selectivity for headphone users, rendering it as a compelling alternative to existing methods. Overall, the new HP test correctly detects 80% of headphone users and has a false-positive rate of 20%. Moreover, we demonstrate that combining the HP test with an additional test–either the AP test or an alternative based on a beat test (BT)–can lower the false-positive rate to ~ 7%. This should be useful in situations where headphone use is particularly critical (e.g., dichotic or spatial manipulations). Code for implementing the new tests is publicly available in JavaScript and through Gorilla (gorilla.sc). [ABSTRACT FROM AUTHOR]

Educational large-scale studies typically adopt highly standardized settings to collect cognitive data on large samples of respondents. Increasing costs alongside dwindling response rates in these studies necessitate exploring alternative assessment strategies such as unsupervised web-based testing. Before respective assessment modes can be implemented on a broad scale, their impact on cognitive measurements needs to be quantified. Therefore, an experimental study on N = 17,473 university students from the German National Educational Panel Study has been conducted. Respondents were randomly assigned to a supervised paper-based, a supervised computerized, and an unsupervised web-based mode to work on a test of scientific literacy. Mode-specific effects on selection bias, measurement bias, and predictive bias were examined. The results showed a higher response rate in web-based testing as compared to the supervised modes, without introducing a pronounced mode-specific selection bias. Analyses of differential test functioning showed systematically larger test scores in paper-based testing, particularly among low to medium ability respondents. Prediction bias for web-based testing was observed for one out of four criteria on study-related success factors. Overall, the results indicate that unsupervised web-based testing is not strictly equivalent to other assessment modes. However, the respective bias introduced by web-based testing was generally small. Thus, unsupervised web-based assessments seem to be a feasible option in cognitive large-scale studies in higher education. [ABSTRACT FROM AUTHOR]

Background: Digital cognitive assessment is becoming increasingly widespread in ageing research and care, especially since the COVID-19 pandemic. Remote online collection provides opportunities for ageing and dementia professionals to collect larger datasets, increase the diversity of research participants and patients and offer cost-effective screening and monitoring methods for clinical practice and trials. However, the reliability of self-administered at-home tests compared to their lab-based counterparts often goes unexamined, compromising the validity of adopting such measures., Objective: Our aim is to validate a self-administered web-based version of the visual short-term memory binding task (VSTMBT), a potential digital biomarker sensitive to Alzheimer's disease processes, suitable for use on personal devices., Methods: A final cross-sectional sample of 37 older-adult (51-77 years) participants without dementia completed our novel self-administered version of the VSTMBT, both at home on a personal device and in the lab, under researcher-controlled conditions., Results: ANOVA and Bayesian t -test found no significant differences between the task when it was remotely self-administered by participants at home compared to when it was taken under controlled lab conditions., Conclusions: These results indicate the VSTMBT can provide reliable data when self-administered at-home using an online version of the task and on a personal device. This finding has important implications for remote screening and monitoring practices of older adults, as well as supporting clinical practices serving diverse patient communities. Future work will assess remote administration in older adults with cognitive impairment and diverse socio-economic and ethno-cultural backgrounds as well as a bench-to-bedside application., Competing Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article., (© The Author(s) 2024.)

Online usability testing has long been of interest to usability engineers and designers. Previous studies found that online tests are as good as lab tests in measuring user performance, but not in finding usability problems. However, little is known about measuring the hedonic quality of user experience, such as joy, beauty, and attractiveness, in online tests. In this study, we conducted a systematic empirical comparison between lab and online testing using AttrakDiff, a validated semantic differential scale that measures pragmatic and hedonic quality of user experience. In our study, 32 participants were divided into three groups: lab testing with moderators, lab testing without moderators, and online testing. The participants performed tasks on a prototype of a cryptocurrency application and evaluated the prototype using the AttrakDiff questionnaire. The results showed a significant difference between lab and online tests in hedonic quality, but no difference was found in pragmatic quality. The difference was only between lab and online tests; the presence or absence of moderators in the lab tests did not produce any significant difference. We also found that the participants in online tests offered longer and more detailed free-comment feedback than the participants in lab tests. These results indicate a possible difference between lab and online tests concerning the measurement of the hedonic quality of user experience, as well as the prospect of using online tests to get rich user feedback. [ABSTRACT FROM AUTHOR]

This article presents a critical survey of the prevalent usage of the Montreal Battery of Evaluation of Amusia (MBEA; Peretz et al., 2003) to assess congenital amusia, a neuro-developmental disorder that has been claimed to be present in 4% of the population (Kalmus and Fry, 1980). It reviews and discusses the current usage of the MBEA in relation to cut-off scores, number of used subtests, manner of testing, and employed statistics, as these vary in the literature. Furthermore, data are presented from a large-scale experiment with 228 German undergraduate students who were assessed with the MBEA and a comprehensive questionnaire. This experiment tested the difference between scores that were obtained in a web-based study (at participants’ homes) and those obtained under laboratory conditions with a computerized version of the MBEA. In addition to traditional statistical procedures, the data were evaluated using Signal Detection Theory (SDT; Green and Swets, 1966), taking into consideration the individual’s ability to discriminate and their response bias. Results show that using SDT for scoring instead of proportion correct offers a bias-free and normally distributed measure of discrimination ability. In addition it is also demonstrated that a diagnosis based on an average score leads to cases of misdiagnosis. The prevalence of congenital amusia is shown to depend highly on the statistical criterion that is applied as cut-off score and on the number of subtests that is considered for the diagnosis. In addition, three different subtypes of amusics were found in our sample. Lastly, significant differences between the web-based and the laboratory group were found, giving rise to questions about the validity of web-based experimentation.

This study describes the adaptation process of a vocabulary knowledge test for British Sign Language (BSL) into American Sign Language (ASL) and presents results from the first round of pilot testing with 20 deaf native ASL signers. The web-based test assesses the strength of deaf children’s vocabulary knowledge by means of different mappings of phonological form and meaning of signs. The adaptation from BSL to ASL involved nine stages, which included forming a panel of deaf/hearing experts, developing a set of new items and revising/replacing items considered ineffective, and piloting the new version. Results provide new evidence in support of the use of this methodology for assessing signed languages, making a useful contribution toward the availability of tests to assess deaf children’s signed language skills. [ABSTRACT FROM AUTHOR]

As a consequence of the large-scale assessment studies (TIMMS; PISA) in compulsory schooling, attention is now being given to the modelling and measurement of competencies in initial vocational education and training. This new output-led perspective of teaching/training and learning/working processes demands new approaches to research. Using the concept of the curriculum–instruction–assessment triad this paper argues that competencies in the fields of business and commerce education can be determined and compared internationally via a large-scale assessment. Empirical results are presented that demonstrate the possibilities of running such an international study in VET. [ABSTRACT FROM PUBLISHER]

Web-based testing has become a ubiquitous self-assessment method for online learning. One useful feature that is missing from today's web-based testing systems is the reliable capability to fulfill different assessment requirements of students based on a large-scale question data set. A promising approach for supporting large-scale web-based testing is static test generation (STG), which generates a test paper automatically according to user specification based on multiple assessment criteria. And the generated test paper can then be attempted over the web by users for assessment purpose. Generating high-quality test papers under multiobjective constraints is a challenging task. It is a 0-1 integer linear programming (ILP) that is not only NP-hard but also need to be solved efficiently. Current popular optimization software and heuristic-based intelligent techniques are ineffective for STG, as they generally do not have guarantee for high-quality solutions of solving the large-scale 0-1 ILP of STG. To that end, we propose an efficient ILP approach for STG, called branch-and-cut for static test generation (BAC-STG). Our experimental study on various data sets and a user evaluation on generated test paper quality have shown that the BAC-STG approach is more effective and efficient than the current STG techniques. [ABSTRACT FROM PUBLISHER]

With the increasing sophistication and ubiquity of the Internet, behavioral research is on the cusp of a revolution that will do for population sampling what the computer did for stimulus control and measurement. It remains a common assumption, however, that data from self-selected Web samples must involve a trade-off between participant numbers and data quality. Concerns about data quality are heightened for performance-based cognitive and perceptual measures, particularly those that are timed or that involve complex stimuli. In experiments run with uncompensated, anonymous participants whose motivation for participation is unknown, reduced conscientiousness or lack of focus could produce results that would be difficult to interpret due to decreased overall performance, increased variability of performance, or increased measurement noise. Here, we addressed the question of data quality across a range of cognitive and perceptual tests. For three key performance metrics-mean performance, performance variance, and internal reliability-the results from self-selected Web samples did not differ systematically from those obtained from traditionally recruited and/or lab-tested samples. These findings demonstrate that collecting data from uncompensated, anonymous, unsupervised, self-selected participants need not reduce data quality, even for demanding cognitive and perceptual experiments. [ABSTRACT FROM AUTHOR]

Abstract: Computer- and web-based testing procedures are increasingly popular for the assessment of cognitive abilities and knowledge. This paper identified color red as a critical context factor that may influence the results. Two studies showed that color red may harm the performance in web-based tests of general knowledge. In Study 1 (N =131) a red (vs. green) progress bar impeded the performance in a knowledge test, but only for the male participants. In Study 2 (N =190) the color of the survey’s forward-button was manipulated (red vs. blue vs. mixed color) which led to a replication of the gender-dependent color effect. Evolutionary psychology and stereotype threat research explain why red impedes the activation of knowledge among men, but not among women. [ABSTRACT FROM AUTHOR]

We show that conjoint analysis, a popular multi-attribute preference assessment technique used in market research, is a well suited tool to evaluate a multitude of gamut mapping algorithms simultaneously. Our analysis is based on data from psychovisual tests assessed in a laboratory and in a web environment. Conjoint analysis allows us to quantify the contribution of every single parameter value to the perceived value of the algorithm; it also allows us to test the influence of additional parameters like gamut size or color shifts.We show that conjoint analysis can be individualized to images or observers if enough data is available. Especially promising in this respect is the combination of individual and population data. [ABSTRACT FROM AUTHOR]

Background: The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies. Objective: This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT). Methods: A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures. Results: The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P >.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P <.001), which could reflect test environment differences, including possible effects of mask-wearing on communication. Conclusions: These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality—particularly for studies involving high-risk or rare cohorts—and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance. J Med Internet Res 2022;24(1):e28368 doi:10.2196/28368 [ABSTRACT FROM AUTHOR]

One aim of testing is to identify weaknesses in students’ knowledge. Computerized tests are now one of the most important ways to judge learning, and, selecting tailored questions for each learner is a significant part of such tests. Therefore, one current trend is that computerized adaptive tests (CATs) not only assist teachers in estimating the learning performance of students, but also facilitate understanding of problems in their learning process. These tests, must effectively and efficiently select questions from a large-scale item bank, and to cope with this problem we propose a dynamic question generation system for web-based tests using the novel approach of particle swarm optimization (PSO). The dynamic question generation system is built to select tailored questions for each learner from the item bank to satisfy multiple assessment requirements. Furthermore, the proposed approach is able to efficiently generate near-optimal questions that satisfy multiple assessment criteria. With a series of experiments, we compare the efficiency and efficacy of the PSO approach with other approaches. The experimental results show that the PSO approach is suitable for the selection of near-optimal questions from large-scale item banks. [Copyright &y& Elsevier]

The article presents a study on the influence of multidisciplinary educational approach in implementing diabetic ketoacidosis (DKA) guidelines among internal medicine residents. The approach, which includes web-based educational needs assessment, individualized education and formal lectures and discussions, was found to be effective at improving knowledge and guideline compliance in diabetic ketoacidosis for the residents. However, there was limited changes in clinical outcomes that accompanied this improved knowledge and guideline compliance.

This paper presents a conceptual model for the Web-based virtual testing and learning (WVTL) in applied sciences, especially in the area of materials science and engineering. WVTL is not an existing field but rather an idea, which could be easily handled for educational purposes and staff training as well as for new materials, products and prototypes testing. The paper reports exclusively on the use and services of simulations on the Web and the potential use of WVTL as well as the trends and opportunities in the area. [ABSTRACT FROM AUTHOR]

Rapid growth in the use of internet-based instructional technology (IT) necessitates greater attention to the pedagogical implications of this transition. Though much of the literature on the uses of IT focuses on outcomes and student learning, we also need to ask whether and how different technologies force us to reconceive the boundaries for both student learning and academic honesty. One IT tool, the web quiz, demands this reconception by creating the possibility for an iterative game where typically only a single-shot game had existed in the past. This change in rule structure calls for the reassessment of definitions of learning and cheating that were developed in a previous era. I review the literature on IT, student learning, and academic honesty, as well as empirical classroom data to develop a typology of behavioral responses to the web quiz environment. Specifically, I identify five behavioral options for students presented with web quiz assignments, four of which would be considered “cheating” in the traditional quiz environment. Data from the classroom confirm the existence of two of these behaviors and offer circumstantial evidence for the other three. I also test the relationship between quiz type and retention of course material to see whether web quizzes or in-class quizzes better serve student learning. I conclude with recommendations for redefining academic honesty in light of IT's alterations to the learning environment. [ABSTRACT FROM AUTHOR]

Background: Web-based research allows cognitive psychologists to collect high-quality data from a diverse pool of participants with fewer resources. However, web-based testing presents unique challenges for researchers and clinicians working with aging populations. Older adults may be less familiar with computer usage than their younger peers, leading to differences in performance when completing web-based tasks in their home versus in the laboratory under the supervision of an experimenter. Objective: This study aimed to use a within-subjects design to compare the performance of healthy older adults on computerized cognitive tasks completed at home and in the laboratory. Familiarity and attitudes surrounding computer use were also examined. Methods: In total, 32 community-dwelling healthy adults aged above 65 years completed computerized versions of the word-color Stroop task, paired associates learning, and verbal and matrix reasoning in 2 testing environments: at home (unsupervised) and in the laboratory (supervised). The paper-and-pencil neuropsychological versions of these tasks were also administered, along with questionnaires examining computer attitudes and familiarity. The order of testing environments was counterbalanced across participants. Results: Analyses of variance conducted on scores from the computerized cognitive tasks revealed no significant effect of the testing environment and no correlation with computer familiarity or attitudes. These null effects were confirmed with follow-up Bayesian analyses. Moreover, performance on the computerized tasks correlated positively with performance on their paper-and-pencil equivalents. Conclusions: Our findings show comparable performance on computerized cognitive tasks in at-home and laboratory testing environments. These findings have implications for researchers and clinicians wishing to harness web-based testing to collect meaningful data from older adult populations. JMIR Aging 2021;4(1):e23384 doi:10.2196/23384 [ABSTRACT FROM AUTHOR]

Background: Web-based research allows cognitive psychologists to collect high-quality data from a diverse pool of participants with fewer resources. However, web-based testing presents unique challenges for researchers and clinicians working with aging populations. Older adults may be less familiar with computer usage than their younger peers, leading to differences in performance when completing web-based tasks in their home versus in the laboratory under the supervision of an experimenter., Objective: This study aimed to use a within-subjects design to compare the performance of healthy older adults on computerized cognitive tasks completed at home and in the laboratory. Familiarity and attitudes surrounding computer use were also examined., Methods: In total, 32 community-dwelling healthy adults aged above 65 years completed computerized versions of the word-color Stroop task, paired associates learning, and verbal and matrix reasoning in 2 testing environments: at home (unsupervised) and in the laboratory (supervised). The paper-and-pencil neuropsychological versions of these tasks were also administered, along with questionnaires examining computer attitudes and familiarity. The order of testing environments was counterbalanced across participants., Results: Analyses of variance conducted on scores from the computerized cognitive tasks revealed no significant effect of the testing environment and no correlation with computer familiarity or attitudes. These null effects were confirmed with follow-up Bayesian analyses. Moreover, performance on the computerized tasks correlated positively with performance on their paper-and-pencil equivalents., Conclusions: Our findings show comparable performance on computerized cognitive tasks in at-home and laboratory testing environments. These findings have implications for researchers and clinicians wishing to harness web-based testing to collect meaningful data from older adult populations., (©Andrée-Ann Cyr, Kristoffer Romero, Laura Galin-Corini. Originally published in JMIR Aging (http://aging.jmir.org), 01.02.2021.)

This article presents a critical survey of the prevalent usage of the Montreal Battery of Evaluation of Amusia (MBEA; Peretz et al., 2003) to assess congenital amusia, a neuro-developmental disorder that has been claimed to be present in 4% of the population (Kalmus and Fry, 1980). It reviews and discusses the current usage of the MBEA in relation to cut-off scores, number of used subtests, manner of testing, and employed statistics, as these vary in the literature. Furthermore, data are presented from a large-scale experiment with 228 German undergraduate students who were assessed with the MBEA and a comprehensive questionnaire. This experiment tested the difference between scores that were obtained in a web-based study (at participants' homes) and those obtained under laboratory conditions with a computerized version of the MBEA. In addition to traditional statistical procedures, the data were evaluated using Signal Detection Theory (SDT; Green and Swets, 1966), taking into consideration the individual's ability to discriminate and their response bias. Results show that using SDT for scoring instead of proportion correct offers a bias-free and normally distributed measure of discrimination ability. It is also demonstrated that a diagnosis based on an average score leads to cases of misdiagnosis. The prevalence of congenital amusia is shown to depend highly on the statistical criterion that is applied as cut-off score and on the number of subtests that is considered for the diagnosis. In addition, three different subtypes of amusics were found in our sample. Lastly, significant differences between the web-based and the laboratory group were found, giving rise to questions about the validity of web-based experimentation.