Instructional Sensitivity Statistics Appropriate for Objectives-Based Test Items. CSE Report No. 91.

Two types of sensitivity indices were developed in this paper, one internal to the total test and the second external. To evaluate the success of these statistics the three criteria suggested for a satisfactory index of item quality were considered. The Internal Sensitivity Index appears to meet these demands. Certainly it is easily computed. In addition its moderately positive correlations with other traditional statistics confirms that the ISI provides unique information and yet is not inconsistent with these indices. However, when there are a large number of masters at the pretest an alternate form of the ISI is sometimes necessary to demonstrate item sensitivity. Finally, the theoretical construction of the ISI is both intuitively understandable and similar in form to other statistics. The External Sensitivity Index, on the other hand, does not fair as well as its internal counterpart. Although computationally simple it fails to demonstrate any consistent correlations with the traditional indices, suggesting a rather random statistic. Perhaps a single item is not sufficient to provide a stable, reliable measure of the effects of instruction. The ISI appears to provide a suitable measure of an item's ability to distinguish between those who have and have not benefited from instruction. Further, the most appropriate approach for evaluating item quality is an examination of the item in context with total test performance. (Author/BJG)