Search

The accuracy with which testees perceived the difficulty of ability-test items was investigated by administering two 41-item conventional tests on verbal ability. High- and low-ability groups responded to test items by choosing the correct alternative and then rating each item's difficulty relative to their levels of ability. Least-squares estimates of item difficulty correlated highly with proportion-correct and latent trait estimates of item difficulty based on a norming sample. Least-squares estimates of testee ability, which were based solely on the difficulty perceptions of the testees, correlated significantly with number-correct and maximum-likelihood ability scores based on the testees' responses to the items. These results show that item-difficulty often used in test construction, and that as testee ability level increased, the items were perceived as being relatively less difficult. The relationship between a testee's ability and his/her perception of an individual item's relative difficulty appeared to be weak. Of major importance was the finding that items which were appropriate in difficulty levels from a psychometric standpoint were perceived by the testees as being too difficult for their ability levels. (Author/DSE)

Volunteer college students were assigned to one of six computer administered vocabulary tests, one half with immediate knowledge of results (KR) after responding to each item, and the other half without knowledge of results. The six tests were designed to be at one of three levels of difficulty and consisted either of 50 preselected items (conventional testing) or tailored on the basis of previous candidate responses (stradaptive testing). Results indicated that the mean maximum-likelihood estimates of individuals' ability varied as a joint function of the KR condition and of test difficulty. Students in the KR condition scored higher than the other students on the most difficult test and lower on the least difficult test. Questionnaire results indicated that, although the students perceived the differences in test difficulty, there were not shown to be any effects on mean student anxiety or motivation scores attributable to test difficulty alone. Students in general reacted very favorably to receiving immediate knowledge of results and its provision increased the mean level of reported motivation. (Author/CTM)

In order to take advantage of advances in the field of mental measurement, the Armed Forces and the Department of Defense have supported the development of a computerized adaptive version of the Armed Services Vocational Aptitude Battery (ASVAB) for use in military personnel selection and classification. This report describes the development and calibration of item pools for each of nine ASVAB content areas. Domain specifications were developed for the content areas, and more than 3,600 items were written. The items were then pretested on samples of recruits. These data were used to select items for calibration in a sample of over 138,000 examinees tested in Military Entrance Processing Stations and their associated testing sites in May and June of 1983. The calibration data were then analyzed using both an equivalent-groups design and a joint-calibration design that used matched experimental and operational test data. Item response theory a, b, and c parameters based on the three-parameter logistic item response model were computed. The parameters resulting from the joint-calibration approach were recommended for operational use. (Author)