A Comparison of a Statistical and Subjective Procedure to Ascertain Item Validity: One Step in the Test Validation Process

In an attempt to make commercial tests valid for users of both sexes and all race groups, many test publishers employ review procedures to remove potentially invalid (or biased) items from their tests. Statistical procedures to identify biased test items are often sophisticated and costly to test developers. Alternative approaches have employers review test items and select those that appear biased. To verify this methodology, a statistical and a subjective procedure were used to identify biased items in an elementary achievement test. The results show little agreement on items selected as biased. An implication to test developers is to adopt statistical support for a reviewer's selection of biased items.