An Investigation on the Generalizability of Performance-Based Assessment in Mathematics.

This study examined the generalizability and dependability of a performance-based assessment in algebra. Four forms of a five-item test were constructed using different subsets of eight items based on attributes from task analysis. Subjects included 142 "algebra 2" students from 2 high schools in the Midwestern United States and 148 11th graders from 1 school in Japan. Students' responses were scored using a holistic scoring rubric from 0 to 4. Analyses of generalizability and dependability revealed that the four forms achieved moderate levels of generalizability, although they varied by school. Analyses of data from subsets of the items on the forms suggested that acceptable levels of generalizability could be achieved by using items if items were well chosen. The choice of subsets of the items did not affect the validity of the content coverage of the test. One U.S. school and one Japanese school showed more similarity than between two U.S. schools. Four appendixes contain a list of mathematics attributes, sample test items, a scoring guide, and a scoring example. (Contains 16 tables, 1 figure, and 18 references.) (Author/SLD)