Search

The author discusses valid and reliable ways to assign grades in an academic course in any discipline. "Validity" means the accuracy of a grade's reflection of student learning and achievement. "Reliability" concerns the degree of random error that might be present and affect validity. First the author defines a grade as a measure of achievement, then identifies problems with existing grading practices and outlines principles that lead to valid grading. Grading criteria, which should be carefully and thoughtfully selected, are reviewed and categorized as "highly recommended," "controversial," and "inappropriate." Whichever grading method instructors choose should accurately measure their students' achievement, because a grade represents a contract between instructor and student that accurately reflects a particular level of learning. Finally, the author discusses grade inflation.

Writing multiple-choice test items to measure student learning in higher education is a challenge. Based on extensive scholarly research and experience, the author describes various item formats, offers guidelines for creating these items, and provides many examples of both good and bad test items. He also suggests some shortcuts for developing test items. All of this advice is based on extensive scholarly research and experience. Creating valid multiple-choice items is a difficult task, but it contributes greatly to the teaching and learning process for undergraduate, graduate, and professional-school courses.

More attention is currently being paid to the distractors of a multiple-choice test item (Thissen, Steinberg, and Fitzpatrick, 1989). A systematic relationship exists between the keyed response and distractors in multiple-choice items (Levine and Drasgow, 1983). New scoring methods have been introduced, computer programs developed, and research conducted to estimate and evaluate the potential for useful information present in distractors. This study determines the efficacy of three linear polytomous scoring methods in the context of testing programs where pass/fail decisions are made. Recommendations are offered about which methods appear to be most effective and feasible. Additionally, future directions in research on polytomous scoring are suggested. Four tables and one figure are included. (Contains 17 references.) (Author)

The objectives of this study were to first determine whether or not the empirical item analysis of domain referenced tests (DR) was justified; and second, in the event that it was, which of a set of recommended procedures was most effective for determining item quality. The analysis that followed led to the conclusion that empirical procedures were highly desirable. When these empirical procedures were applied to test data, the results indicated that four different techniques provided almost identical information: Rasch statistics, instructional sensitivity indexes, traditional statistics, and Baysian indexes. Based on these results, it would seem that any one of these four would serve adequately. (Author/RC)

Two techniques for writing achievement test items to accompany instructional materials were contrasted: (1) generating items from statements of instructional objectives, and (2) generating items from rules for transforming instructional statements (adapted from Bormuth). Items of each type were written by two experienced item writers. Subjects were given tests employing these items before and after reading a programmed booklet. One item writer was found to produce consistently more difficult test items regardless of the technique used. This result supports the contention that objective-based item writing results in items of varying quality, but is in conflict with the hypothesis that the rule-generation technique eliminates "subjectivity" in item writing. The need for further investigation of fully-automated, linguistic-based rules for item writing is suggested. (Author)

This paper reports results of a large-scale, descriptive study of correlates of attitudes toward the mathematical, natural, and social sciences. Objectives of this study were to: (1) examine the levels and direction of attitudes toward the sciences as a function of grade level; (2) probe into possible causal determinants of these attitudes; and (3) examine correlates of attitudes toward the sciences as a function of grade level and specific subject matter. Subjects (N=5,804) in 277 classrooms in grades 4, 7, and 9 were administered the Inventory for Affective Aspects of Schooling (IAAS). First, product-moment correlations were computed, then ordinary least-squares regression analyses were done to determine the relative contributions of teams of variables for explaining criterion variance. Results are reported by grade level, by subject matter, and by the regression analyses. Conclusions are drawn concerning the role of the teacher in shaping attitude at the class level; and generalizations are made concerning a classroom in which there are positive collective attitudes about mathematics, science, or social studies. (CS)

The present study involved the testing of two common multiple-choice item writing rules. A recent review of research revealed that much of the advice given for writing multiple-choice test items is based on experience and wisdom rather than on empirical research. The rules assessed in this study include: (1) the phrasing of the stem in the form of a question versus a partial sentence; and (2) the use of the inclusive "none of the above" option instead of a specific content option. Limited empirical research suggests that using the partial sentence format and the inclusive "none of the above" option may lead to undesirable item and test characteristics, while textbook authors essentially are divided on their opinions about the validity of each rule. The items used in this study were from the instructor's manual for D. Myer's (1986) text entitled "Psychology." Items were randomly assigned to be rewritten to reflect the experimental conditions under investigation. Two instructors of an introductory psychology course selected 32 multiple-choice items for the study. The rewritten tests were administered to 228 students enrolled in two sections of an introductory psychology class. About half of the students in each section received Form A and the other half received Form B, resulting in 115 Form A and 113 Form B responses. The same manipulated items were combined with 18 different non-manipulated items in a third section of the class to comprise Forms C and D, whose administration resulted in 59 Form C and 59 Form D responses. Results offer no evidence to support the use of either type of stem and limited evidence to caution against use of the "none of the above" option. Two data tables and examples of the four item formats used are provided. (TJH)

This paper describes a model of the influence of personal and environmental factors on students' valuing of two deep-processing strategies for studying expository texts. In the model, task orientation (a form of intrinsic motivation in which learning and understanding are the major goals) interacts with perceptions of the teacher's goals to influence both subsequent task orientation and strategy-value beliefs. Questionnaire data from 281 high school science students, collected near the beginning and end of a school year, were used to test the model. The questionnaires included measures of two additional variables thought by other theorists to influence study strategy use: perceived ability and attitude toward science. Individual differences in the initial level of students' task orientation appear to exert a powerful influence, both on later motivation and on their belief in the value of deep-processing strategies. Perceptions that their teacher wanted them to think independently as well as thoroughly master the material appear to positively influence both students' task orientation and their strategy-value beliefs over the course of a school year. Neither perceived ability nor attitude toward science added significantly to the model's explanatory power. The theoretical and practical implications of these findings are discussed, with an emphasis on the potential for teachers to affect student goals and strategies for studying science. (Author)

The proposition that the optimal number of options in a multiple choice test item is three was examined. The concept of functional distractor, a plausible wrong answer that is negatively discriminating when total test performance is the criterion, is discussed. Three distinct groups of achievers (high, middle, and low) on a national standardized achievement test for physicians were identified. The number of functional distractors was identified for each sample condition and each item. Results suggest, as have theoretical analyses, that more functional distractors are desirable. However, with higher achieving students the number of functional distractors, and consequently the number of options, had no effect. The time and effort devoted to additional option item development is probably not worth the gains in item discrimination and reliability for high and middle achieving examinees. The three-option format is efficient to construct, and it results in better domain-referenced measures of achievement. (SLD)

An ideal process is outlined for test item development and the study of item responses to ensure that tests are sound. Qualitative and quantitative methods are used to assess the item-level validity evidence for high-stakes examinations. A checklist for assessment is provided. (SLD)

Articles in this special issue attempt to address the most significant aspect of certification and licensing testing programs: validity. Although construct validity has become the prevailing model, the theoretical formulation of the construct of competence remains a topic of considerable debate. (SLD)