Search

The purpose of this study was to compare student performance and question discrimination of multiple-choice questions (MCQs) that followed a standard format (SF) versus those that do not follow a SF, termed here as non-standard format (NSF). Medical physiology exam results of approximately 500 first-year medical students collected over a five-year period (2020–2024) were used. Classical test theory item analysis indices, e.g. discrimination (D), point-biserial correlation (rpbis), distractor analysis for non-functional distractors (NFDs), and difficulty (p) were determined and compared across MCQ format types. The results presented here are the mean ± standard error of the mean (SEM). The analysis showed that D (0.278 ± 0.008 vs 0.228 ± 0.006) and rpbis (0.291 ±.006 vs 0.273 ±.006) were significantly higher for NSF questions compared to SF questions, indicating NSF questions provided more discriminatory power. In addition, the percentage of NFDs was lower for the NSF items compared to the SF ones (58.3 ± 0.019% vs 70.2 ± 0.015%). Also, the NSF questions proved to be more difficult relative to the SF questions (p = 0.741 ± 0.007 for NSF; p = 0.809 ± 0.006 for SF). Thus, the NSF questions discriminated better, had fewer NFDs, and were more difficult than SF questions. These data suggest that using the selected non-standard item writing questions can enhance the ability to discriminate higher performers from lower performers on MCQs as well as provide more rigour for exams. [ABSTRACT FROM AUTHOR]

This study focuses on the reading comprehension problem with multiple-choice answers, using the BERT model to achieve the highest performance. The ultimate goal is to create a solution to help solve reading comprehension problems without any reasoning or knowledge, suitable for the level of students in grades six and seven. The model will solve factoid questions from a given text. Our research topic will use a deep learning model-based approach to create a model that automatically answers the English reading comprehension question. We obtain promising results to give an accuracy of 78 percent.

Recent advances in automated scoring technology have made it practical to replace multiple-choice questions (MCQs) with short-answer questions (SAQs) in large-scale, high-stakes assessments. However, most previous research comparing these formats has used small examinee samples testing under low-stakes conditions. Additionally, previous studies have not reported on the time required to respond to the two item types. This study compares the difficulty, discrimination, and time requirements for the two formats when examinees responded as part of a large-scale, high-stakes assessment. Seventy-one MCQs were converted to SAQs. These matched items were randomly assigned to examinees completing a high-stakes assessment of internal medicine. No examinee saw the same item in both formats. Items administered in the SAQ format were generally more difficult than items in the MCQ format. The discrimination index for SAQs was modestly higher than that for MCQs and response times were substantially higher for SAQs. These results support the interchangeability of MCQs and SAQs. When it is important that the examinee generate the response rather than selecting it, SAQs may be preferred. The results relating to difficulty and discrimination reported in this paper are consistent with those of previous studies. The results on the relative time requirements for the two formats suggest that with a fixed testing time fewer SAQs can be administered, this limitation more than makes up for the higher discrimination that has been reported for SAQs. We additionally examine the extent to which increased difficulty may directly impact the discrimination of SAQs. [ABSTRACT FROM AUTHOR]

Sex differences in mental rotation performance are one of the largest in cognitive psychology. Men outperform women by up to 1 SD in psychometric mental rotation tests, but it is often neglected that there are no or only small sex differences for chronometric tests. As both tests are supposed to measure the same ability, we suspect some features of the tests themselves to affect sex differences in performance. Following a theoretical review of the test features, we evaluate the effects of the number of possible answer alternatives, whether they are presented as pairwise mirrored, and their interaction on sex differences in mental rotation performance. In an online experiment, 838 German-speaking participants, 421 women, 417 men, M age = 42.58 (SD = 12.54) years, solved four blocks of mental rotation trials with two or eight alternatives, which were either pairwise mirrored or not. The results show that that the overall performance was lower for more alternatives and for mixed alternatives but not for their interaction. We could not determine explanations for sex differences as we did not observe meaningful sex differences at all. Possible reasons include the differences between men and women in age and education. This study suggests that the differences between tests affect performance. Sex differences, however, need more investigation, including possible effects and interactions of the test design, education, and age. [ABSTRACT FROM AUTHOR]

Although the assessment of therapists' competence is often conceptualized in a hierarchical model, the model's implications are underresearched. This study thus focuses on the association between theoretical knowledge and applied knowledge on one side, and communication and therapy skills on the other. Psychology students (N = 69) took part in a psychotherapy training study, including role-plays with standardized patients. The training interventions included reading treatment manuals and watching model videos. We measured theoretical knowledge using multiple-choice questions, applied knowledge using case vignettes, and psychotherapy skills in videotaped sessions with standardized patients using independent competence ratings. Theoretical knowledge correlated significantly with practical competences (r = .28-.36, p < .05) and predicted postintervention skills in standardized patients interactions (β = .21-.35, p < .05). Applied knowledge was significantly correlated with practical competences (r = .31-.54, p < .05), and predicted postintervention skills in one out of two treatment conditions (β = .47, p < .001). The results underline the importance of theoretical knowledge and applied knowledge in psychotherapy training in the early career of therapists. The discussion focuses on methodological aspects, and on implications for future studies and training. [ABSTRACT FROM AUTHOR]

Background: Multiple-choice examinations are frequently used in German dental schools. However, details regarding the used item types and applied scoring methods are lacking. Objective: This study aims to gain insight into the current use of multiple-choice items (ie, questions) in summative examinations in German undergraduate dental training programs. Methods: A paper-based 10-item questionnaire regarding the used assessment methods, multiple-choice item types, and applied scoring methods was designed. The pilot-tested questionnaire was mailed to the deans of studies and to the heads of the Department of Operative/Restorative Dentistry at all 30 dental schools in Germany in February 2023. Statistical analysis was performed using the Fisher exact test (P <.05). Results: The response rate amounted to 90% (27/30 dental schools). All respondent dental schools used multiple-choice examinations for summative assessments. Examinations were delivered electronically by 70% (19/27) of the dental schools. Almost all dental schools used single-choice Type A items (24/27, 89%), which accounted for the largest number of items in approximately half of the dental schools (13/27, 48%). Further item types (eg, conventional multiple-select items, Multiple-True-False, and Pick-N) were only used by fewer dental schools (≤67%, up to 18 out of 27 dental schools). For the multiple-select item types, the applied scoring methods varied considerably (ie, awarding [intermediate] partial credit and requirements for partial credit). Dental schools with the possibility of electronic examinations used multiple-select items slightly more often (14/19, 74% vs 4/8, 50%). However, this difference was statistically not significant (P =.38). Dental schools used items either individually or as key feature problems consisting of a clinical case scenario followed by a number of items focusing on critical treatment steps (15/27, 56%). Not a single school used alternative testing methods (eg, answer-until-correct). A formal item review process was established at about half of the dental schools (15/27, 56%). Conclusions: Summative assessment methods among German dental schools vary widely. Especially, a large variability regarding the use and scoring of multiple-select multiple-choice items was found. [ABSTRACT FROM AUTHOR]

Over four semesters of a large introductory statistics course the authors found students were engaging in contract cheating on Chegg.com during multiple choice examinations. In this paper we describe our methodology for identifying, addressing and eventually eliminating cheating. We successfully identified 23 out of 25 students using a combination of unique academic and digital fingerprints, and identified students who used virtual private networks (VPNs) to protect their online identity. There were two forms of cheating – posting questions and waiting for responses from tutors, and looking for questions that had already been solved. We found that 165 questions from these examinations were posted by 10 different students, but that the most common form of cheating was searching for answers that had already been posted. This paper discusses these patterns of Chegg usage, the consequences of not catching cheating early on, and how students reacted to being caught. Also provided are R and Python code that readers may use to identify cheating students in their own courses. [ABSTRACT FROM AUTHOR]

Even though the impact of the position of response options on answers to multiple-choice items has been investigated for decades, it remains debated. Research on this topic is inconclusive, perhaps because too few studies have obtained experimental data from large-sized samples in a real-world context and have manipulated the position of both correct response and distractors. Since multiple-choice tests' outcomes can be strikingly consequential and option position effects constitute a potential source of measurement error, these effects should be clarified. In this study, two experiments in which the position of correct response and distractors was carefully manipulated were performed within a Chilean national high-stakes standardized test, responded by 195,715 examinees. Results show small but clear and systematic effects of options position on examinees' responses in both experiments. They consistently indicate that a five-option item is slightly easier when the correct response is in A rather than E and when the most attractive distractor is after and far away from the correct response. They clarify and extend previous findings, showing that the appeal of all options is influenced by position. The existence and nature of a potential interference phenomenon between the options' processing are discussed, and implications for test development are considered. [ABSTRACT FROM AUTHOR]

Players in serious games may often need multiple trials for correctly completing a game task. Therefore, the number of attempts should be reflected in the score. This article presents three computational score models that take into account the number of attempts that a player makes to be successful. The models, which are extensions of test theory, quantify the random contributions to the scores that need to be removed. They also describe the influence of prior knowledge used for elimination of incorrect options, and take into account that the decision options in a node may not be equally plausible. In a series of simulation studies the score outcomes of the models are compared under various conditions. Results show that the number of trials cannot be ignored as they have a strong impact on the performance scores to be assigned. Neglecting the number of trials leads to inaccurate scores that significantly overrate the observed performances, occasionally up to 100% or even more. The effects depend on the number of decision options, the presence of obvious, correct or incorrect options given the player´s knowledge level and, to a lesser extent, different plausibility levels of the options to decide upon. The practical feasibility is high, because a simple score formula largely solves the problem.

Different versions of multiple-choice exams were administered to an undergraduate class in human physiology as part of normal testing in the classroom. The goal was to evaluate whether the number of options (possible answers) per question influenced the effectiveness of this assessment. Three exams (each with three versions) were given to each of two sections during an academic quarter. All versions were equally long, with 30 questions: 10 questions with 3 options, 10 questions with 4, and 10 questions with 5 (always one correct answer plus distractors). Each question appeared in all three versions of an exam, with a different number of options in each version (three, four, or five). Discrimination (point biserial and upper-lower discrimination indexes) and difficulty were evaluated for each question. There was a small increase in difficulty (a lower average score on a question) when more options were provided. The upper-lower discrimination index indicated a small improvement in assessment of student learning with more options, although the point biserial did not. The total length of a question (number of words) was associated with a small increase in discrimination and difficulty, independent of the number of options. Quantitative questions were more likely to show an increase in discrimination with more options than nonquantitative questions, but this effect was very small. Therefore, for these testing conditions, there appears to be little advantage in providing more than three options per multiple-choice question, and there are disadvantages, such as needing more time for an exam.

The purpose of this study is to introduce a method for converting scored 4-option multiple-choice (MC) items into scored 3-option MC items without re-pretesting the 3-option MC items. This study describes a six-step process for achieving this goal. Data from a professional credentialing exam was used in this study and the method was applied to 24 forms of the exam. The results found 100% accuracy in predicting the rounded passing score for all forms. [ABSTRACT FROM AUTHOR]

Only recently has eye-tracking been used to investigate test-taker reading behavior, and results have been primarily used to confirm a range of cognitive tasks elicited by test items. This study explores test taker reading behavior for its own sake by describing how ESL readers of different proficiency levels behaviorally view multiple-choice passages and test items at different difficulty levels. Data were gathered from 51 students at three proficiency levels attending a university-sponsored intensive English program (IEP). Participants read eight validated reading comprehension items at varying difficulty levels while their eye movements were recorded on the passage, multiple-choice stem, correct answer, and distractors. Reading behavior demonstrated that language proficiency had a limited effect while passage difficulty had a stronger effect on reading behavior: participants gave less visual attention to the reading passage and correct answers within easier items and when they had higher language proficiency. The interaction of proficiency and item difficult on reading behavior is important in understanding how learners experience tests. [ABSTRACT FROM AUTHOR]

Purpose: This report describes the essential steps in the development, implementation, evaluation and quality assurance of the written part of the Swiss Federal Licensing Examination for Human Medicine (FLE) and the insights gained since its introduction in 2011.Methods: Based on existing scientific evidence, international expertise, and experience gained from previous examinations, the FLE is developed by experts from all five medical faculties in Switzerland with the support of the Institute for Medical Education and is held simultaneously at five locations. The exam organisers document and review every examination held and continuously optimise the processes; they have summarised the results in this report.Results: The essential steps comprise the development, revision and translation of questions; construction of the exam and production of materials; candidate preparation; implementation and analysis. The quality assurance measures consist of guideline coherence in the development of the questions and implementation of the exam, revision processes, construction of the exam based on the national blueprint, multiphase review of the translations and exam material, and statistical analysis of the exam and the comments from candidates. The intensive collaboration, especially on the part of representatives from all the participating faculties and a central coordination unit, which provides methodological support throughout and oversees the analysis of the exam, has proven successful. Successfully completed examinations and reliable results in the eleven examinations so far implemented represent the outcomes of the quality assurance measures. Significant insights in recent years are the importance of appreciating the work of those involved and the central organisation of exam development, thus ensuring the long-term success of the process.Conclusion: Common guidelines and workshops, quality assurance measures accompanied by the continuous improvement of all processes, and appreciation of everyone involved, are essential to carrying out such an examination at a high-quality level in the long term.

Players in serious games may often need multiple trials for correctly completing a game task. Therefore, the number of attempts should be reflected in the score. This article presents three computational score models that take into account the number of attempts that a player makes to be successful. The models, which are extensions of test theory, quantify the random contributions to the scores that need to be removed. They also describe the influence of prior knowledge used for elimination of incorrect options, and take into account that the decision options in a node may not be equally plausible. In a series of simulation studies the score outcomes of the models are compared under various conditions. Results show that the number of trials cannot be ignored as they have a strong impact on the performance scores to be assigned. Neglecting the number of trials leads to inaccurate scores that significantly overrate the observed performances, occasionally up to 100% or even more. The effects depend on the number of decision options, the presence of obvious, correct or incorrect options given the player's knowledge level and, to a lesser extent, different plausibility levels of the options to decide upon. The practical feasibility is high, because a simple score formula largely solves the problem. [ABSTRACT FROM AUTHOR]

Copyright of GMS Journal for Medical Education is the property of German Medical Science Publishing House gGmbH and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

We match data on performance in a multiple‐choice examination with data on risk preferences from a classroom experiment. Students who are more loss averse leave more questions unanswered and perform worse in the exam when an incorrect answer is penalized compared with no answer. Thus, loss aversion parameters extracted from lottery choices in a controlled experiment have predictive power in a field environment of decision‐making under uncertainty. Furthermore, the degree of loss aversion appears to be persistent over time, as the experiment was conducted three months prior to the exam. Important differences across genders are partly explained by differences in loss aversion. [ABSTRACT FROM AUTHOR]

The study of construct validity on summative test items in multiple-choice Arabic aims to improve the test items' quality to make them valid in the construction aspect. The study was carried out on multiple-choice items based on the modified construction criteria instrument based on the Guttman scale. Items that match the indicator are marked with S and TS if they do not match. Based on a review of 40 items, it was found that the items were not suitable, namely: 25 items with unformulated stem clearly and firmly, two questions that pointed to the correct answer, and one question with pictures that confused. Meanwhile, other items are in accordance with all construct criteria indicators for multiple-choice questions. Based on the research results, the question maker must correct and pay attention to these aspects to produce a valid test kit.

Kahoot ist ein Beispiel für eine Multiple-Choice-Quiz-App. Im Rahmen von Bestrebungen, digitale Medien im Deutschunterricht einzusetzen, nehmen digitale Quizze oft eine zentrale Rolle ein, weil Schülerinnen und Schüler sie als spielerische Lernformen begreifen. Der Beitrag prüft verschiedene Szenarien, wie Quizze im Deutschunter-richt genutzt werden und formuliert eine Kritik an digitalen Quiz-Tools, die primär den Verzicht auf sprachliche Handlungen bemängelt.

Abstract This study aimed to examine the effect of test format on test performance by comparing Multiple Choice (MC) and Constructed Response (CR) vocabulary tests in an EFL setting. Also, this paper investigated the function of gender in MC and CR vocabulary measures. To this end, five 20-item stem-equivalent vocabulary tests (CR, and 3-, 4-, 5-, and 6-option MC) were administered to 243 (132 male and 111 female) pre-intermediate students. Results of the study revealed that MC tests were easier than CR. Results indicated a gender bias, in that, males scored better than females in all versions of MC tests while females outperformed males in CR. The findings implied that testers should consider the effect of test format while assessing vocabulary knowledge and use a combination of test formats (MC and CR) in vocabulary assessment to reduce gender bias and format effect.

Higher-order thinking skills (HOTs) are very crucial thinking skills needed by teachers to train students to develop 21st-century learning. This study aimed to develop Multiple Choice and Essay Questions to measure the HOTs of the prospective teachers of the elementary school education department. This study used a 4-D model by Thiagarajan which involved experts at natural science, evaluation studies, and primary school pedagogy in the content validation. We also involved 156 prospective teachers as the test subjects. The assessment of instrument quality by experts showed that the question quality was very good. This research succeeded in developing 10 multiple choice questions and 5 essays. The validity test by Rasch Model showed that there were 7 multiple choice questions classified as fit, and 3 questions were classified as a misfit, while the 2 essay questions are invalid and the other (3 questions) as valid. The reliability test with KR-20 on multiple-choice questions and Cronbach's alpha for the essay questions resulted reliable questions. The discrimination index showed discarded, sufficient, good, and very good. The item difficulty index showed that 3 questions are moderate (num 7, 1, 5) and 7 questions are difficult (num 4, 10, 6, 3, 2, 8, 9). The distractor efficiency showed that 59.2% of distractors worked, and 40.8% did not work. This instrument can be used to analyze prospective teachers’ HOTs. This data can be used as the reference for developing competency improvement programs for prospective teachers, for example through the HOTs-oriented learning models. [ABSTRACT FROM AUTHOR]

College students are always wondering if there is any relationship between the types of course materials introduced by teachers and the test formats they give. By knowing the existence or non-existence of such a relationship, they can be prepared better for a test and it may reduce their test anxiety. In this study, two universities that offer "English translation" programs at the B.A. level in Iran were randomly selected and examined. Here, the researcher looked at the kinds of materials for 47 courses in Mofid University (introduced by 15 teachers) and 42 courses in University of Kashan, then at the test formats used to assess the knowledge or skill of the students. After comparing the materials with the tests given, it was found that approximately 83 percent of tests in University of Kashan and 89 percent of them in Mofid University were of written form (essay type, multiple choice, etc.) and we came to the conclusion that there was no relationship between course materials and test formats for the students majoring in English translation. Rather, apart from teachers’ preferences, it was the nature of the courses that decided the type of questions and led to different types and formats of tests.

Copyright of Polizei & Wissenschaft is the property of Verlag fur Polizeiwissenschaft and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Background: The Corona pandemic has made it difficult to conduct face-to-face events, which is why two workshops planned for the development of multiple choice (MC) questions were conducted online. Whether the online format is suitable for MC question development has not yet been described to our knowledge. Questions: The study aimed to answer the following questions from the perspective of the participants: How are the two online workshops evaluated in terms of their implementation? Are these online workshops suitable for developing MC questions? Is the online or face-to-face format preferred? As a measure of efficiency, it was examined whether the expected question output (standard of comparable face-to-face workshops) was achieved in the online workshops.Methods: In May and June 2020, two online workshops with a total of 24 participants were conducted for Swiss professional societies with SWITCHinteract. The participants’ feedback was collected via an anonymous online survey with 21 questions.Results: 88% of the participants took part in the voluntary online survey. The participants were satisfied with the implementation and found the online format suitable. The majority of the participants did not show a preference for a certain format (online vs. face-to-face), although in case of a format preference the online format was indicated more often. The expected question output was exceeded in both workshops. Technical aspects were most frequently cited as requiring improvement. Conclusion: Based on the results, online workshops for MC question development can be considered as a resource-saving and efficient alternative to face-to-face workshops. Increased use and optimization of online tools could further facilitate implementation and influence the format preference.

The value of a third potential option or distractor can alter the way in which decisions are made between two other options. Two hypotheses have received empirical support: that a high value distractor improves the accuracy with which decisions between two other options are made and that it impairs accuracy. Recently, however, it has been argued that neither observation is replicable. Inspired by neuroimaging data showing that high value distractors have different impacts on prefrontal and parietal regions, we designed a dual route decision-making model that mimics the neural signals of these regions. Here we show in the dual route model and empirical data that both enhancement and impairment effects are robust phenomena but predominate in different parts of the decision space defined by the options’ and the distractor’s values. However, beyond these constraints, both effects co-exist under similar conditions. Moreover, both effects are robust and observable in six experiments.

BIOLOGY LEARNING BY DOING MULTIPLE CHOICE QUESTION FOR SENIOR HIGH SCHOOL STUDENTS. Multiple choice question learning and understanding can help learning process in the classroom. To maximize its use as learning tool, the students can have finish the task in 2 ways: first they will finish the Multiple-Choice Question (MCQ) task during the class hours and submit it when the class over. Or it can be homework to the students and hand it over at the next meeting session. The research showed that the instructor/teacher prefer on use multiple choice not only due to administration, time saving and simplicity but also objective and consistent. Furthermore, the student based on research result more like Multiple choice question assignments because they can eliminate wrong answers and these questions form more objective than anothers. Beside that with this method the student can participate actively in learning process.

Dialogue-based multiple-choice machine reading comprehension (MRC) is one of most difficult and novel tasks because it requires more advanced reading comprehension skills, such as speaker's intention analysis, non-extractive reasoning, commonsense knowledge. Previous models usually only compute attention scores from the fixed representation of entire dialogue, and also do not fully consider the contribution of dialogue, question, options, and their combinations respectively. In this paper, we introduce Utterance-focusing Multiway-matching Network (UMN), a simple but effective human mimicking model for dialogue-based multiple-choice MRC. First, two utterance-focusing mechanisms called ParaUF and AutoUF are proposed to extract the utterances that are most relevant to the question and option: ParaUF gets the bilinear weighted distance between each utterance of dialogue and question and option during training while AutoUF obtains the scores by the relevance, overlap and coverage (ROC) rules before training process. Second, we adopted the multiway-matching mechanism to capture the relationship among the question, option and selected utterances through calculating the attention weights between the quadruplet of four sequences: utterances, question, option and the concatenation of each two. We evaluate the proposed model on dialogue-based multiple-choice MRC tasks, DREAM, and outperformed recently published methods under the same pre-trained model. A series of detailed analysis is also conducted to interpret the differences of two utterance-focusing mechanisms and the effectiveness of the proposed multiway-matching mechanism. [ABSTRACT FROM AUTHOR]

We explored the relationship between self-regulated test duration and test performance in first-year optometry students using test administration software. The scores for two midterms (MT1 and MT2) and the final exam for a single course (Visual Sensation and Perception) and time from password submission to exam upload were obtained for 132 students in spring 2019. Statistical analysis suggests that the relationship between self-regulated test duration and test performance is inconsistent across the semester for this group. Exploration of self-regulated test duration and test performance for other courses and throughout the optometric program may provide additional insight. [ABSTRACT FROM AUTHOR]

Purpose: Testing is required in medical education. The large number of exams that students face requires effective learning strategies. Various methods of improving knowledge retention and recall have been discussed, two of the most widely evaluated of which are test-enhanced learning and pause procedures. This study investigated the effect of voluntary multiple-choice questions on students' performance. Methods: In a prospective study from April 2013 to March 2015, 721 students were randomly assigned to receive supplementary online material only (control group) or additional multiple-choice questions (investigative group) accompanying lectures. Their performance in the final exam was evaluated. Results: A total of 675 students were ultimately included, with 299 randomly assigned to the investigative group and 376 to the control group. Students in the investigative group scored significantly better in relation to grades and points (2.11 vs. 2.49; 33 vs 31.31; p < 0.05). The effect declined over time. Conclusion: This is the first study of the use of voluntary multiple-choice questions to improve medical students' performance. The results support test-enhanced learning and the feasibility of implementing multiple-choice questions in lectures. [ABSTRACT FROM AUTHOR]

Copyright of Australasian Marketing Journal is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This study aimed to examine the effect of test format on test performance by comparing Multiple Choice (MC) and Constructed Response (CR) vocabulary tests in an EFL setting. Also, this paper investigated the function of gender in MC and CR vocabulary measures. To this end, five 20-item stem-equivalent vocabulary tests (CR, and 3-, 4-, 5-, and 6-option MC) were administered to 243 (132 male and 111 female) pre-intermediate students. Results of the study revealed that MC tests were easier than CR. Results indicated a gender bias, in that, males scored better than females in all versions of MC tests while females outperformed males in CR. The findings implied that testers should consider the effect of test format while assessing vocabulary knowledge and use a combination of test formats (MC and CR) in vocabulary assessment to reduce gender bias and format effect. [ABSTRACT FROM AUTHOR]

Reasoning is regarded to be an essential facet of fundamental cognitive abilities. As examinee characteristics may affect performance in Reasoning tests, concern about maintaining fairness is expressed. The purpose of the current study was to examine effects of response format on psychometric properties and fairness of a matrices test according to examinee's sex, risk propensity, and test anxiety. A total of 433 German-speaking pupils (aged 14–20) were randomly assigned to either a multiple choice or a free response version of the same 25-item test. Data analysis yielded Rasch-homogeneous 23-item versions, with higher reliability, but lower criterion validity for the free response test. No interactions between response format and sex, test anxiety, or risk propensity were revealed, but a significant main effect of sex: men out-performed women in reasoning irrespective of response format. Results are discussed with reference to attributes of the test situation and sample characteristics.

Taking a test improves memory for that tested information, a finding referred to as the testing effect. Multiple-choice tests tend to produce smaller testing effects than do cued-recall tests, and this result is largely attributed to the different processing that the two formats are assumed to induce. Specifically, it is generally assumed that the multiple-choice format bypasses the need to retrieve information. Research suggests, however, that multiple-choice questions can be constructed to induce retrieval of information pertaining to the incorrect alternatives. In the present research, we investigated the processes that individuals use to answer multiple-choice questions and how those processes relate to later memory, particularly for information pertaining to the incorrect alternatives. Most critically, we found that participants sometimes spontaneously recall information pertaining to incorrect alternatives, and these spontaneous retrievals are associated with retention of those alternatives as correct answers to related questions later. Although multiple-choice questions can be constructed so as to bypass retrieval, they can also be constructed to induce retrieval, and when they are, learning benefits are likely to occur. The present work has practical implications for how instructors can create multiple-choice questions to induce processes that facilitate learning. [ABSTRACT FROM AUTHOR]

Abstrakt I högre utbildning har användandet av flervalsuppgifter för examination ökat. Detta medför problem när man använder samma betygsgränser som för textbaserade uppgifter, då gissning blir lönsamt. Här undersöks effekterna av gissning i fallet med flervalsuppgifter och jämförelser mellan olika poängsättningsalgoritmer. Olika algoritmer jämförs i fallet med ett standardtest med 100 frågor, där kunskapsnivån ger en poäng och resten av frågorna besvaras slumpmässigt för att imitera gissning. Resultatet är att gissning ger en icke-försumbar påverkan på sannolikheten att få ett betyg som inte svarar mot kunskapsnivån i fallet med en dikotom poängsättning.

The article is the commentary to the report of O.A. Reshetnikova, the director of the Federal Institute of Educational Measuring Materials, named "Assessment Materials for State Final Attestation. Specifics and ways of development" represented on scientific-research seminar "Measuring in social sciences" 20.05.2016. The article also includes suggestions for increasing efficiency of measuring and evaluation procedures taking into account their application.