Search

To link to full-text access for this article, visit this link: http://dx.doi.org/10.1016/j.chb.2009.01.011 Byline: Jane Carstairs (a), Brett Myors (b) Keywords: Internet usage; Cognitive assessment; Achievement measures; Cheating; Educational measurement Abstract: A natural experiment is reported in which the performance of two cohorts of undergraduate students on identical tests of cognitive achievement is compared under differing conditions of proctoring and modes of administration, total N =302. Between-subjects and within-subjects analyses reveal clear effects of test score inflation in the unproctored conditions compared with formal, proctored, examination conditions. These results caution against the use of unproctored cognitive testing in high-stakes situations, including on-line assessment. Author Affiliation: (a) Psychology Department, University of Wolverhampton, Wulfruna Street, Wolverhampton, West Midlands WV1 1LY, United Kingdom (b) School of Psychology, Griffith University, Nathan, Queensland 4160, Australia

Researchers are often interested in testing the hypothesis that the effects of treatments, interventions, and so on are negligibly small rather than testing the hypothesis that treatments have no effect whatsoever. A number of procedures for conducting such tests have been suggested but have yet to be widely adopted. In this article, simple methods of testing such minimum-effect hypotheses are illustrated in a variety of applications of the general linear model. Tables and computational routines that can be used in conjunction with the familiar F test to evaluate the hypothesis that the effects of treatments or interventions exceed some minimum level are also provided.

Selection outcomes under banding are affected by characteristics of the selection system and the applicant pool. This study examined the effects of eight parameters on the proportions hired from higher- and lower-scoring groups: (a) selection ratio; (b) reliability; (c) fixed vs. sliding bands; (d) top-down vs. random within-band selection; (e) preferential vs. nonpreferential selection; (f) mean differences; (g) standard deviation differences; and (h) proportion of applicants from the lower-scoring group. Simulation results were analyzed in a fully-crossed eight-way ANOVA. Higher-order interactions among selection system and applicant pool characteristics had virtually no effect on selection outcomes; the proportion of the applicant pool from the lower-scoring group accounted for nearly half the variance in outcomes. Other important effects are, in order, the effects of standard deviation differences, mean differences, preferential hiring, and the selection ratio. Applicant pool characteristics have considerably more influence on selection outcomes than do selection system characteristics. (Reprinted by permission of the publisher.)

S.L. Martin and N.S. Raju (1992) proposed a procedure for setting the cutoff score that maximized the total utility of a selection program by including a continuous function of recruiting costs. In the present article, the authors comment on some limitations of Martin and Raju's approach and suggest a modified procedure that uses a step recruitment-cost function. The modified procedure has the flexibility of allowing variable combinations of recruiting sources in any order to maximize total dollar utility of the selection program. It is also suggested that researchers should work toward the goals of finding the optimal combination of recruiting sources and of developing a confidence interval around the final estimate of the number of applicants based on the estimated variability of labor supply. A working example of the modified procedure is discussed. (Reprinted by permission of the publisher.)