Search

Multiple choice writing guidelines are decidedly split on the use of "none of the above" (NOTA), with some authors discouraging and others advocating its use. Moreover, empirical studies of NOTA have produced mixed results. Generally, these studies have utilized NOTA as either the correct response or a distractor and assessed its effect on difficulty and discrimination. In these studies, NOTA commonly yields increased difficulty when it is used as the correct response, and no change in discrimination regardless of usage. However, when NOTA is implemented as a distractor, rarely is consideration given to the distractor that could have been written in its place. Here, we systematically replaced each distractor in a series of questions with NOTA across different versions of an Introductory Psychology examination. This approach allowed us to quantify the quality of each distractor based on its relative discrimination index and assess the effect of NOTA relative to the quality of distractor it replaced. Moreover, our use of large Introductory Psychology examinations afforded highly stable difficulty and discrimination estimates. We found that NOTA increased question difficulty only when it was the correct response, with no effect on difficulty of replacing any distractor type with NOTA. Moreover, we found that NOTA decreased discrimination when it replaced the most effective distractors, with no effect on discrimination of replacing either the correct response or lowest quality distractor with NOTA. These results replicate the common finding that inclusion of NOTA as the correct response increases question difficulty by equally luring high-performing and low-performing students toward distractors. Moreover, we have shown that including NOTA as a distractor can reduce discrimination if used in lieu of a well written alternative, suggesting that multiple choice authors should avoid using NOTA on multiple choice tests.

This study examined whether audio feedback provided to undergraduates (N=51) about one paper would prove beneficial in terms of improving their grades on another, unrelated paper of the same type. We examined this issue both in terms of student "beliefs" about learning transfer, as well as their "actual ability" to transfer what had been learned on one assignment to another, subsequent assignment. Results indicated that students believed that they would be able to transfer what they had learned via audio feedback. Moreover, results also suggested that students actually did generalize the overarching comments about content and structure made in the audio files to a subsequent paper, the content of which differed substantially from the initial one. Both students and teaching assistants demonstrated very favourable responses to this type of feedback, suggesting that it was both clear and comprehensive.

Because multiple-choice testing is so widespread in higher education, we assessed the quality of items used on classroom tests by carrying out a statistical item analysis. We examined undergraduates' responses to 1198 multiple-choice items on sixteen classroom tests in various disciplines. The mean item discrimination coefficient was +0.25, with more than 30% of items having unsatisfactory coefficients less than +0.20. Of the 3819 distractors, 45% were flawed either because less than 5% of examinees selected them or because their selection was positively rather than negatively correlated with test scores. In three tests, more than 40% of the items had an unsatisfactory discrimination coefficient, and in six tests, more than half of the distractors were flawed. Discriminatory power suffered dramatically when the selection of one or more distractors was positively correlated with test scores, but it was only minimally affected by the presence of distractors that were selected by less than 5% of examinees. Our findings indicate that there is considerable room for improvement in the quality of many multiple-choice tests. We suggest that instructors consider improving the quality of their multiple-choice tests by conducting an item analysis and by modifying distractors that impair the discriminatory power of items. (Contains 5 figures and 4 tables.)

Multiple-choice questions are widely used in higher education and have some important advantages over constructed-response test questions. It seems, however, that many teachers underestimate the value of multiple-choice questions, believing them to be useful only for assessing how well students can memorize information, but not for assessing higher-order cognitive skills. Several strategies are presented for generating multiple-choice questions that can effectively assess students' ability to understand, apply, analyze, and evaluate information.

The authors assessed the effects of using "none of the above" as an option in a 40-item, general-knowledge multiple-choice test administered to undergraduate students. Examinees who selected "none of the above" were given an incentive to write the correct answer to the question posed. Using "none of the above" as the keyed option made items much more difficult (d = -1.11). Furthermore, 45% of the time that examinees correctly selected "none of the above," they wrote either a wrong answer (19%) or no answer (26%), and rescoring items to deny credit in these cases caused item discrimination to fall (d = -0.35). Thus, when "none of the above" is the keyed option, credit earned by examinees with knowledge deficiencies can make items appear to have more discriminatory power than is actually the case. The authors recommend that "none of the above" should not be used as an option in multiple-choice items.

The Monty Hall dilemma (MHD) is a remarkably difficult probability problem with a counterintuitive solution. Undergraduate students used an interactive digital learning object that provided a set-based, animated explanation of the solution to the MHD and let them play games designed to increase understanding of the solution. More than 60% of users could explain the solution to the classic three-door version of the MHD and generalized the solution to an extremely difficult six-door version of the game. Because the MHD has relevance to important aspects of science in general and psychology in particular, some instructors may find the learning object, which is freely available at www.brocku.ca/psychology/people/dibattista.htm, to be of substantial value in their teaching. (Contains 2 figures.)

The authors examined how the grading scheme affects learning and students' reactions to the Immediate Feedback Assessment Technique (IFAT), an answer form providing immediate feedback on multiple-choice questions. Undergraduate students (N = 141) took a general-knowledge multiple-choice test of low, medium, or high difficulty. They used the IFAT with a number-correct (NC), partial-credit (PC), or correction-for-guessing (CG) grading scheme. After 1 week, they retook the test using a traditional response form with an NC scheme. When the authors used the NC or PC scheme for Test 1, the number of correct answers increased by more than 30% on Test 2. However, the increase was only about half as large with the CG scheme, suggesting that it interferes with the IFAT's learning benefits. Responses to questionnaire items provided no strong clues regarding the origin of this difference. Participants in all treatment conditions had positive attitudes toward the IFAT. (Contains 5 tables.)

The authors examined the relationship between the reactions of undergraduate students to using the Immediate Feedback Assessment Technique (IFAT), an answer form that provides immediate feedback on multiple-choice questions, for the first time on a major examination and their levels of test anxiety and trait anxiety. They also assessed whether students with higher levels of test anxiety and trait anxiety might be disadvantaged relative to other students by use of the IFAT in a testing situation. They found that preference of undergraduates (N =185) for the IFAT was not related to test anxiety, nor did evidence indicate that the IFAT put students with higher levels of test anxiety at a disadvantage with respect to test performance. Using the IFAT did not generally increase test-related anxiety, and for a majority of students, immediate feedback actually reduced it. Nineteen percent of students felt that immediate feedback interfered with their test performance but would nevertheless still prefer to use the IFAT in future tests. Potential concerns that test-anxious students may either dislike the IFAT or be disadvantaged by its use appear unwarranted and should not deter instructors from adopting the IFAT.

Undergraduates completed a questionnaire after using the Immediate Feedback Assessment Technique (IFAT), a commercially available answer form for multiple-choice (MC) testing that can be used easily and conveniently with large classes. This simple new technique for MC testing provides immediate feedback for each item in an answer-until-correct format and permits the earning of partial credit when the student's initial response is incorrect. Reaction to the IFAT was extremely positive, with students saying it was easy to use and contributed to their learning, and they indicated a strong desire to use the IFAT for all MC tests. Liking for the IFAT was not related to either personal characteristics or test performance variables, indicating that it has broad appeal to students.

Although animals generally prefer to eat foods with familiar rather than unfamiliar flavors, adult golden hamsters (Mesocricetus auratus) were found to do the opposite. After having prolonged exposure to a food with a particular flavor, hamsters were allowed to select between the food with the familiar flavor and the same food with a novel flavor. Hamsters consistently ate more of the food with the novel flavor, and this preference was long-lasting and resistant to extinction. Furthermore, the novelty effect was robust, being manifested in both sexes and under a variety of experimental circumstances. In contrast, rats tested under identical conditions consistently preferred the food with the familiar flavor. The origins of the novelty effect in hamsters remain to be determined.

Golden hamsters (Mesocricetus auratus) were repeatedly exposed to conditioning diets differing in both caloric density (1.98 or 5.01 kcal/g) and flavor (anise or clove) to demonstrate flavor-calorie learning. During testing, hamsters were presented with 2 jars that both contained a test diet consisting of a mixture of equal amounts of the conditioning diets; the test diet in I cup contained the flavor previously associated with the high-calorie diet, and the test diet in the other cup contained the opposite flavor. Hamsters showed a strong preference for the test diet containing the flavor previously associated with the high-calorie diet during conditioning; however, the preference was not robust, disappearing after only 3 brief extinction trials. The learned flavor preference was specifically attributable to flavor-calorie learning and not to either flavor-flavor learning or the effects of mere exposure.

The available body of scientific evidence does not support the notion that sugar consumption has significant adverse effects on children's behaviour; however, responses to a mailed questionnaire indicated that more than 80% of 389 Canadian primary school teachers believed that sugar consumption contributes to increased activity of normal children and to the behavioural problems of hyperactive children. Moreover, in the previous three years, 55% of all respondents had counselled the parents of children whom they believed to be hyperactive to consider reducing their child's sugar consumption to control the child's activity, and parents frequently did so. These results indicate that teachers need to be provided with accurate and up-to-date information about the effects of sugar on children's behaviour and about effective treatments for hyperactivity. Possible origins of beliefs about the reputed adverse behavioural effects of sugar and about factors that may serve to perpetuate these beliefs are discussed.

Adult mammals generally demonstrate a lower preference for the disaccharide sugar lactose than for any other common sugar, and because adults typically have low levels of the intestinal enzyme lactase, lactose ingestion may cause gastrointestinal distress. The lactose intake of adult golden hamsters was examined in three experiments; the main findings were: (a) hamsters allowed to choose between tap water and lactose solutions (maximum concentration = 32% weight/volume) over a 20-day period showed a clear preference for lactose solutions and ingested substantial quantities of lactose (up to 3 g/100 g body weight/day) without noticeable adverse effects; (b) hamsters consuming a single diet with lactose added (maximum concentration = 50%) over a period of days ingested up to 3.42 g/100 g body weight/day of lactose without noticeable adverse effects; (c) both hamsters with prior exposure to lactose solutions and those without such exposure consumed similar amounts of 32% lactose solution over an 8-day period, suggesting that hamsters' lactose intake does not depend on the occurrence of adaptation. It is suggested that the fermentative capacity of the hamster's pregastric pouch may underlie this animal's unusual tolerance for lactose.

The Immediate Feedback Assessment Technique (IFAT) is a new multiple-choice response form that has advantages over more commonly used response techniques. The IFAT, which is commercially available at reasonable cost and can be used conveniently with large classes, has an answer-until-correct format that provides students with immediate, corrective, item-by-item feedback. Advantages of this learner-centered response form are that it: (a) actively promotes learning; (b) allows students' partial knowledge to be rewarded with partial credit; (c) is strongly preferred by students over other response techniques; and (d) lets instructors more easily maintain the security of multiple choice (MC) items so that they can be reused from one semester to the next. The IFAT's major shortcoming is that grading must be done manually because it does not yet have a compatible optical scanning device. Helpful suggestions are presented for instructors who may be considering using the IFAT for the first time. [ABSTRACT FROM AUTHOR]