Search

Based on initial Scholastic Aptitude Test (SAT) Verbal pretest data and hypotheses advanced in the research literature, 7 sentence completion and 16 analogy items with extreme levels of differential item functioning (DIF) were selected and then systematically revised and re-administered in an attempt to reduce or eliminate DIF. The apparent success of the effort makes similar attempts worth continuing. The particular terminology used in stems and keys, rather than the underlying skill being measured, seemed to be a recurring source of DIF in the SAT-Verbal items. Larger sample sizes, especially for minority focal groups, would help to stabilize the DIF categories used by Educational Testing Service (ETS) test developers. In addition, because the ETS delta metric is unbounded at the extremes, the use of both the Standardization (p-metric) and Mantel-Haenszel (delta-metric) methodologies is recommended for classifying the level of DIF for very easy and very difficult items. Further research is suggested to study the possible relationship between DIF and predictive validity. Nine tables and four figures present analysis results. An appendix summarizes DIF hypotheses. (Contains 26 references.) (Author/SLD)

Differential item functioning (DIF) assessment attempts to identify items or item types for which subpopulations of examinees exhibit performance differentials that are not consistent with the performance differentials typically seen for those subpopulations on collections of items that purport to measure a common construct. DIF assessment requires a rule for scoring items and a matching variable on which different subpopulations can be viewed as comparable for purposes of assessing their performance on items. Typically, DIF is operationally defined as a difference in item performance between subpopulations, e.g., Blacks and Whites, that exists after members of the different subpopulations have been matched on some total score. Constructed-response items move beyond traditional multiple-choice items, for which DIF methodology is well-defined, towards item types involving selection or identification, reordering or rearrangement, substitution or correction, completion, construction, and performance or presentation. This paper defines DIF, describes two standard procedures for measuring DIF and indicates how DIF might be assessed for certain constructed-response item types. The description of DIF assessment presented in this paper is applicable to computer-delivered constructed-response items as well as paper and pencil delivered items. (Contains 67 references and 5 tables.) (Author)

Alternative mathematical items administered as prototypes at the spring 1989 Field Trials of the Scholastic Aptitude Test (SAT) are evaluated for differential item functioning (DIF) and differential speededness. Results for Algebra Placement (AP) and Student Produced Response (SPR) items are presented and contrasted with results obtained on two current SAT-Math (SAT-M) items: Regular Math and Quantitative Comparison. Analyses on comparisons between female and comparable male examinees, and between Asian-American, Black, and Hispanic examinees in comparison to comparable White examinees indicate that both of these alternative items appear to have DIF. Additional DIF analyses comparing the use of an internal versus an external matching criteria for the SPR items show evidence of negative DIF with either criteria. Results using the Mantel Haenszel Delta DIF statistic are more extreme than DIF results using the standardization p metric DIF index. Differential speededness results indicate that the two Math prototypes have slightly higher levels of differential speededness than the SAT-M. The SPR items pose an interesting problem for DIF. The definition of an appropriate DIF matching criterion for constructed response items and metric differences between methods and their effect on difficult or easy items also need further exploration. One figure and 13 tables illustrate the analysis. (Contains 15 references.) (Author/SLD)

The purpose of this investigation was to identify item factors that may contribute to differential item functioning (DIF) for black examinees on Scholastic Aptitude Test (SAT) analogy items. Initially, items were classified according to several possible explanatory factors. Preliminary analyses identified several factors that seemed to affect DIF for blacks. Secondly, these factors were further investigated by classifying and analyzing analogy items from two additional SAT forms. Results indicate that black students need more time to complete the SAT verbal sections than do their white counterparts with comparable overall SAT verbal scores. Once DIF statistics were corrected for speededness, a smaller number of analogy items were identified as differentially more difficult. Some of the factors appear to be interdependent; these factors include item position within each analogy set, difficulty, subject content, and level of abstractness. The effects of homographs and semantic relationship types are also confounded with the previous factors. Generalizations to be drawn from these results are limited since analyses are based on regular administration items, which tend to have several factors operating in a single item and which contain infrequent occurrence of some factors. (TJH)

Standardization methodology was used to analyze omitted responses of Hispanic examinees on the Scholastic Aptitude Test. Study or focal groups were 2,956 Mexican-Americans, 3,230 Puerto Ricans, and 278,009 White test-takers. Results indicate that both Mexican-Americans and Puerto Rican students omitted fewer items than White students of comparable ability but that Mexican-Americans tended to omit even less. Examination of items with the least differential omit rate by both Hispanic groups indicate that items with true cognates or words with a common root in English and Spanish were omitted differentially less. Some dependency on ability level was observed for the omit patterns of Puerto Rican examinees. (Contains 2 tables, 6 figures, and 18 references.) (Author/SLD)

Two editions of the Verbal and Mathematical portions of the Scholastic Aptitude Test (SAT) were used to study differential speededness and differential omission and the relationships among differential item functioning (DIF), differential omission, and item difficulty for Asian Americans, Blacks, Hispanics, and females in a total sample exceeding 300,000 SAT examinees in 1984 and 1985. Consistent and replicable evidence of differential speededness was found for Blacks and Hispanics. Use of an unspeeded criterion for matching in place of the traditional total score, which contains speeded items, did not affect the DIF analyses of speeded items. A strong artifactual negative relationship between DIF and differential omission was found. The relationship between differential omission and difficulty was consistently positive on the Verbal sections for all comparison groups except the Asian-American group, for whom it was consistently negative. On the mathematical sections, this relationship was consistently found for the female/male comparison, for whom it was negative. Finally, the relationship between difficulty and DIF was negative but smaller than previously observed. (Contains 8 tables, 8 figures, and 31 references.) (Author/SLD)