Search

The study analyzes and compares the validity of computerized adaptive testing, paper and pencil and computer-based forms of cognitive abilities tests. The research was conducted on a sample of 803 secondary school students (567 paper and pencil, 236 computer-based/computerized adaptive administration; 363 males, 440 females), their mean age was 16.8 years (SD = 1.33). The test set consisted of the Test of Intellect Potential and the Vienna Matrices Test. Overall results showed that the validity of CAT was reasonably comparable across administration modes. Consistent with previous research, CAT selecting only a small number of items gave results which, in terms of validity, were only marginally different from the results of traditional administration. CAT simulated administration of the TIP was roughly 55% and VMT 54% more economical than the traditional version. These results indicate that CAT is a useful way of improving methodology of psychological testing. [ABSTRACT FROM AUTHOR]

Objectives: Patient-reported outcomes (PROs; self-report assessments) are increasingly important in evaluating medical care and treatment efficacy. Electronic administration of PROs via computer is becoming widespread. This article reviews the literature addressing whether computer-administered tests are equivalent to their paper-and-pencil forms. Methods: Meta-analysis was used to synthesize 65 studies that directly assessed the equivalence of computer versus paper versions of PROs used in clinical trials. A total of 46 unique studies, evaluating 278 scales, provided sufficient detail to allow quantitative analysis. Results: Among 233 direct comparisons, the average mean difference between modes averaged 0.2% of the scale range (e.g., 0.02 points on a 10-point scale), and 93% were within ±5% of the scale range. Among 207 correlation coefficients between paper and computer instruments (typically intraclass correlation coefficients), the average weighted correlation was 0.90; 94% of correlations were at least 0.75. Because the cross-mode correlation (paper vs. computer) is also a test–retest correlation, with potential variation because of retest, we compared it to the within-mode (paper vs. paper) test–retest correlation. In four comparisons that evaluated both, the average cross-mode paper-to-computer correlation was almost identical to the within-mode correlation for readministration of a paper measure (0.88 vs. 0.91). Conclusions: Extensive evidence indicates that paper- and computer-administered PROs are equivalent. [ABSTRACT FROM AUTHOR]

This article explores the effect on assessment of ‘translating’ paper and pencil test items into their computer equivalents. Computer versions of a set of mathematics questions derived from the paper-based end of key stage 2 and 3 assessments in England were administered to age appropriate pupil samples, and the outcomes compared. Although in most cases the change to the different medium seems to make little difference, for some items the affordances of the computer profoundly affect how the question is attempted, and therefore what is being assessed when the item is used in a test. These differences are considered in terms of validity and legitimacy, that is whether the means used to answer a question in a particular medium are appropriate to the assessment intention. The conclusion is not only that translating paper and pencil items into the computer format sometimes undermines their validity as assessments, it is also that some paper and pencil items are less valid as assessments than their computer equivalents would be. [ABSTRACT FROM AUTHOR]

Current taxonomies of intelligence comprise two factors of mental speed, clerical speed (Gs), and elementary cognitive speed (Gt). Both originated from different research traditions and are conceptualized as dissociable constructs in current taxonomies. However, previous research suggests that tasks of one category can be transferred into the other category by simply changing the mode of administration, i.e., in form of a paper-and-pencil test or in from of a computer-based elementary cognitive task. However, cross-mode correlations for specific tasks are usually only moderate. In the present study, mental speed was assessed as a broad construct across different tasks and stimulus materials. This allowed modeling mental speed as a hierarchical construct for paper-and-pencil as well as for computer-based tests. Cross-mode correlations of the respective general factors were moderate (r = 0.64), while the cross-mode correlations of task-specific components depended on task type (r = 0.12 to r = 0.71). Only the g factors of mental speed, but not the task-specific components, were found to be related with working memory capacity as a marker of cognitive ability. The speed general factor modeled across computer-based tests was more highly correlated with working memory capacity (r = 0.66) than the general factor modeled across paper-and-pencil tests (r = 0.46). These findings corroborate a crucial role of the assessment method and imply that validity of speed tests is affected by the choice of the test format. [ABSTRACT FROM AUTHOR]