Search

CSAR (Characteristic Storage And Retrieval) is the name of an interactive item bank system for the storage, retrieval, manipulation, and summarization of information about multiple choice test items. The purpose of the system is to facilitate the test construction process. The system has two main components: one or more item banks and an interactive item selection program. Nonstatistical, traditional item analysis, and item response theory item information is stored in each bank. This information is then accessed interactively by test editors using a wide variety of different item selection criteria. The paper describes the CSAR system briefly and discusses the design decisions around which it was developed. Recommendations are offered to others who might be considering a similar venture. (Author)

The purpose of the research was to explore an alternative implementation strategy for Elementary Secondary Education Act Title I evaluation Model A; that is, the use of Rasch calibrated item banks rather than traditional norm-referenced tests. A methodology for obtaining normative information for such banks involved equating a normed test with selected subsets of bank items. Three conventional equating techniques and one Rasch model technique were used for the development of conversion tables. All four equating techniques yielded similar results, but the accuracy of the results was of concern. The largest discrepancies between atual and predicted performance were observed in the score range in which most Title I students would be expected to score. An application of the conversion tables developed through the Project to real Title I and non-Title I data was reported. (Author/GSK)

The purpose of the present study was to examine the accuracy of indirect trait estimates, i.e., estimates of some primary trait obtained from a second measure which have been equated to the first. The California Achievement Test in Reading was the primary measure and the Prescriptive Reading Inventory was the indirect measure. Four kinds of content-match between primary and indirect item sets were studied: match, similar, dissimilar, and partial. Within each content-match category, indirect trait estimates based on item sets containing 10, 20, and 30 items were compared to those obtained from the primary measure. A three-parameter logistic model was applied to an internal anchor test design to equate the measures. The results obtained were then evaluated for a cross-validation sample. The accuracy of indirect trait estimates was affected by both content-match and the number of items used to produce the estimate. As indirect item sets became more similar in content or as the number of items increased, the accuracy of the estimates increased. However, a considerable amount of equating bias was noted, even for the 30-item sets. (Author/CM)

Open testing and test disclosure legislation has fostered a renewed interest in the problems and issues associated with test equating. This paper focuses on a specific application of test equating, that of professional and occupational testing for certification and licensure. Test equating is the process of making test scores from different forms of the same test comparable. The role of test equating in the credentialing examination process, its usefulness for maintaining consistent standards, and monitoring performance trends across time are discussed. The issues which must be considered when implementing test equating in a credentialing examination program are described. Issues are considered as test development or technical concerns. This paper is designed to provide information for credentialing board members who hold the major responsibility for preparing, administering, and scoring credentialing examinations, as well as non-technical persons involved in the credentialing process. (Author/DWH)

This research, conducted for California State Department of Education, is one component of a feasibility study focusing on ways to make the testing process more efficient by application of multi-purpose tests (MPT). MPT are designed, administered and scored to serve more than one purpose and to consolidate and unify testing programs as a partial solution to test proliferation. A content review identifies common skills measured by the California Assessment Program (CAP) and the Stanford Achievement Test. Empirical information was collected about matched CAP and Stanford item performance from a mathematics field test. A core set of multi-purpose items was identified. A validation study of over 1,000 grade 6 students examined the accuracy of the estimates of norm-referenced test information from the multi-purpose item set identified. The general strategy was to estimate Stanford total test and subtest scores in mathematics by applying item response theory techniques to the identified multi-purpose item set. Estimated scores were then compared with total test and subtest scores actually earned by students. The correlations between pairs of estimated and actual scaled scores were large, and scaled root mean square differences were small. Results indicate: (1) some testing programs evaluate common skills, supporting the concept of a consolidated testing system; (2) there is a need for statewide content domains; and (3) using multi-purpose testing methodology is feasible. (PN)

Discrepancies among the standards produced by different criterion-referenced standard-setting techniques may be the result of a failure to adequately define the minimally competent candidate. Current research in this area is reviewed in terms of three categories: studies in which no formal assistance in conceptualization is given to judges, studies in which conceptualization is accomplished through discussion including the review of relevant materials, and studies in which empirical information about test items or test performance is used. Two standard-setting studies using a combination of these techniques are discussed in detail. (Author)

The accuracy of trait estimates obtained from three vertical equating methods was examined. The procedures studied included two anchor test designs and a single-group design. Data from two content areas and two grade combinations were studied. A three-parameter logistic model was used to perform the equatings. The results obtained were used to produce pairs of true score estimates for a cross-validation sample, one estimate for each of the equated tests. Comparisons among pairs of estimates indicated that the single-group method consistently produced the most accurate results, although the advantage was small. A differential effect due to content area was reported for the two anchor test methods. (Author)