Search

Multiple choice writing guidelines are decidedly split on the use of "none of the above" (NOTA), with some authors discouraging and others advocating its use. Moreover, empirical studies of NOTA have produced mixed results. Generally, these studies have utilized NOTA as either the correct response or a distractor and assessed its effect on difficulty and discrimination. In these studies, NOTA commonly yields increased difficulty when it is used as the correct response, and no change in discrimination regardless of usage. However, when NOTA is implemented as a distractor, rarely is consideration given to the distractor that could have been written in its place. Here, we systematically replaced each distractor in a series of questions with NOTA across different versions of an Introductory Psychology examination. This approach allowed us to quantify the quality of each distractor based on its relative discrimination index and assess the effect of NOTA relative to the quality of distractor it replaced. Moreover, our use of large Introductory Psychology examinations afforded highly stable difficulty and discrimination estimates. We found that NOTA increased question difficulty only when it was the correct response, with no effect on difficulty of replacing any distractor type with NOTA. Moreover, we found that NOTA decreased discrimination when it replaced the most effective distractors, with no effect on discrimination of replacing either the correct response or lowest quality distractor with NOTA. These results replicate the common finding that inclusion of NOTA as the correct response increases question difficulty by equally luring high-performing and low-performing students toward distractors. Moreover, we have shown that including NOTA as a distractor can reduce discrimination if used in lieu of a well written alternative, suggesting that multiple choice authors should avoid using NOTA on multiple choice tests.

This study examined whether audio feedback provided to undergraduates (N=51) about one paper would prove beneficial in terms of improving their grades on another, unrelated paper of the same type. We examined this issue both in terms of student "beliefs" about learning transfer, as well as their "actual ability" to transfer what had been learned on one assignment to another, subsequent assignment. Results indicated that students believed that they would be able to transfer what they had learned via audio feedback. Moreover, results also suggested that students actually did generalize the overarching comments about content and structure made in the audio files to a subsequent paper, the content of which differed substantially from the initial one. Both students and teaching assistants demonstrated very favourable responses to this type of feedback, suggesting that it was both clear and comprehensive.

Because multiple-choice testing is so widespread in higher education, we assessed the quality of items used on classroom tests by carrying out a statistical item analysis. We examined undergraduates' responses to 1198 multiple-choice items on sixteen classroom tests in various disciplines. The mean item discrimination coefficient was +0.25, with more than 30% of items having unsatisfactory coefficients less than +0.20. Of the 3819 distractors, 45% were flawed either because less than 5% of examinees selected them or because their selection was positively rather than negatively correlated with test scores. In three tests, more than 40% of the items had an unsatisfactory discrimination coefficient, and in six tests, more than half of the distractors were flawed. Discriminatory power suffered dramatically when the selection of one or more distractors was positively correlated with test scores, but it was only minimally affected by the presence of distractors that were selected by less than 5% of examinees. Our findings indicate that there is considerable room for improvement in the quality of many multiple-choice tests. We suggest that instructors consider improving the quality of their multiple-choice tests by conducting an item analysis and by modifying distractors that impair the discriminatory power of items. (Contains 5 figures and 4 tables.)

Multiple-choice questions are widely used in higher education and have some important advantages over constructed-response test questions. It seems, however, that many teachers underestimate the value of multiple-choice questions, believing them to be useful only for assessing how well students can memorize information, but not for assessing higher-order cognitive skills. Several strategies are presented for generating multiple-choice questions that can effectively assess students' ability to understand, apply, analyze, and evaluate information.

The authors examined how the grading scheme affects learning and students' reactions to the Immediate Feedback Assessment Technique (IFAT), an answer form providing immediate feedback on multiple-choice questions. Undergraduate students (N = 141) took a general-knowledge multiple-choice test of low, medium, or high difficulty. They used the IFAT with a number-correct (NC), partial-credit (PC), or correction-for-guessing (CG) grading scheme. After 1 week, they retook the test using a traditional response form with an NC scheme. When the authors used the NC or PC scheme for Test 1, the number of correct answers increased by more than 30% on Test 2. However, the increase was only about half as large with the CG scheme, suggesting that it interferes with the IFAT's learning benefits. Responses to questionnaire items provided no strong clues regarding the origin of this difference. Participants in all treatment conditions had positive attitudes toward the IFAT. (Contains 5 tables.)

The authors examined the relationship between the reactions of undergraduate students to using the Immediate Feedback Assessment Technique (IFAT), an answer form that provides immediate feedback on multiple-choice questions, for the first time on a major examination and their levels of test anxiety and trait anxiety. They also assessed whether students with higher levels of test anxiety and trait anxiety might be disadvantaged relative to other students by use of the IFAT in a testing situation. They found that preference of undergraduates (N =185) for the IFAT was not related to test anxiety, nor did evidence indicate that the IFAT put students with higher levels of test anxiety at a disadvantage with respect to test performance. Using the IFAT did not generally increase test-related anxiety, and for a majority of students, immediate feedback actually reduced it. Nineteen percent of students felt that immediate feedback interfered with their test performance but would nevertheless still prefer to use the IFAT in future tests. Potential concerns that test-anxious students may either dislike the IFAT or be disadvantaged by its use appear unwarranted and should not deter instructors from adopting the IFAT.

The Immediate Feedback Assessment Technique (IFAT) is a new multiple-choice response form that has advantages over more commonly used response techniques. The IFAT, which is commercially available at reasonable cost and can be used conveniently with large classes, has an answer-until-correct format that provides students with immediate, corrective, item-by-item feedback. Advantages of this learner-centered response form are that it: (a) actively promotes learning; (b) allows students' partial knowledge to be rewarded with partial credit; (c) is strongly preferred by students over other response techniques; and (d) lets instructors more easily maintain the security of multiple choice (MC) items so that they can be reused from one semester to the next. The IFAT's major shortcoming is that grading must be done manually because it does not yet have a compatible optical scanning device. Helpful suggestions are presented for instructors who may be considering using the IFAT for the first time. [ABSTRACT FROM AUTHOR]