Search

Evaluation is one of the most difficult areas of educational pursuits. The most common type of evaluation and assessment is through a written examination. The majority of the time, questions is drafted right before the examination without going through a quality control process, which might cause confusion or incorrect interpretation of the questions by the students. Students' self-confidence and self-esteem grow as a result of their increased understanding of the question paper's structure, and they then affirm their ability to complete the task. The design of the question paper is prepared, with consideration given to the objectives, the various forms and types of questions, and the various content areas, a projected time, Level of difficulty anticipated. In order to enhance the standard of student evaluation and assessment, this article focuses on offering instructions and a scientific technique for creating effective question papers. [ABSTRACT FROM AUTHOR]

Computer-based curriculum-based measurement (CBM) is a relatively common practice, but surprisingly few studies have examined the reliability of computer-based CBM. This study sought to examine the reliability of CBM administered via paper/pencil versus the computer. Twenty-one of 25 students in two third-grade classes (N = 21) participated in two generalizability studies. The primary facet of interest, format of assessment, was examined for two measures, a fact operation (fact family measure) and a more complex operation (multidigit addition). Researchers administered four alternate forms of each measure under both computer and paper/pencil conditions. Results indicated ideal reliability of measurement within all conditions and results were consistent across assessment formats for the fact operation. However, assessment format explained a large proportion of variance for the more complex skill (multidigit addition). Results indicate a need to consider the reliability of transitioning assessment from paper/pencil to computer-based response on a measure-by-measure basis. [ABSTRACT FROM AUTHOR]

Examination process is an important activity for educational institutions to assess student performance. However, preparing the exam questions is very challenging, tedious and time consuming for the instructors. Generating an effective question paper is a task of great importance for any educational institute. The traditional method, where lecturers manually prepare the question paper, is very tedious and challenging. The aim is to automate the entire process of question paper generation and correction using the PC. Hence, with the help of this project we present the solution in form of "Teacher's Assistant" (TAS). This system includes several modules like user administration, subject selection, difficulty level specification, question entry, paper generation, answer checker and paper management. Thus the system optimizes the human effort involved and also reduces the overall time consumed in the process of generating question papers [ABSTRACT FROM AUTHOR]

Copyright of Chinese University Education Journal is the property of Chinese University of Hong Kong and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Introduction: The aim of this systematic review was to compare the effectiveness of electronic recording with paper-based recording of clinical skills assessments for entry-level health professional students. Methods: A comprehensive database search was undertaken using AMED, CINAHL, CENTRAL, EMBASE, Medline, PsycINFO, Scopus, and Web of Science on 12–16 June 2017, and updated 9 April 2018. Studies investigating electronic and paper-based recordings of clinical skills assessments of students in entry-level health professional programs were eligible for inclusion. Two independent researchers completed screening of studies for inclusion, quality assessments, and data extraction, with discrepancies resolved by consensus. Quality assessment was performed using the Critical Appraisal Skills Program Diagnostic checklist. Results: From 2264 studies identified, five observational cohort studies were included. Published between 2006 and 2016, included studies investigated electronic and paper examinations of clinical skills assessments of students from medical, dentistry, and physical therapy programs. Electronic assessments were reported to be more time efficient than paper-based assessments with an added advantage of no missing data. Quality and quantity of quantitative and qualitative student feedback increased with electronic assessment compared to paper-based assessments. Conclusions: Electronic assessments were used successfully across a range of health professional programs, take significantly less time to complete and provide higher quality feedback to students. Future studies are needed with more robust psychometric testing and cost-effective analysis to inform the increasing uptake of electronic assessment tools in health professional training. [ABSTRACT FROM AUTHOR]

In some previous research studies, quite successful results have been obtained in using online tools to increase students' academic achievement. In many fields; the benefits of online formative quizzes have been widely recognized by educators. However, there is little research determining whether weekly formative online exams affect students' academic achievement, especially in engineering courses. In this quasi‐experimental study, it was investigated whether the weekly quiz application method had an effect on students' academic achievement in an engineering course. One hundred and ninety sophomores participated in the study. Although the experimental group (n = 81) used the quizzes prepared on the Course Portal online to learn the Sensors and Transducers Unit, the control group (n = 109) solved the quizzes in the textbook on a paper‐pencil basis. After the online quizzes were solved, the experimental group was given which questions they answered incorrectly and the related video link of these incorrectly answered questions, and the relevant section of the book was given by the Course Portal as feedback. The pretest and posttest data of the students were obtained, and analysis of covariance was utilized as the analysis method to reveal the answer to the study question. The statistical analysis of the students' average achievement scores in the posttest (midterm exam) showed that the experimental group's average achievement score was significantly better than the control group. Results show that online weekly exams have significant effects on students' academic achievement. Weekly online quizzes are recommended to engineering lecturers as they offer students to improve their students' academic success. [ABSTRACT FROM AUTHOR]

Comparability of paper and online versions of a statewide assessment was evaluated for the purposes of test score reporting, to appropriately adjust equated score conversion tables for students testing online as warranted. In particular, comparability of online and paper tests was designed with a propensity score matching method. The propensity score was used to match the online test sample with the paper–pencil test sample. The results of this study suggest that neither a separate assessment nor a conversion table would be required in a state’s assessment. The online test appears to provide a credible and comparable option to the paper and pencil modality. [ABSTRACT FROM PUBLISHER]

This study investigated the performance of four widely used data-collection designs in detecting test-mode effects (i.e., computer-based versus paper-based testing). The experimental conditions included four data-collection designs, two test-administration modes, and the availability of an anchor assessment. The test-level and item-level results were analyzed with inferential statistics and multidimensional scaling, respectively. The test-level results supported the superiority of the single-group counterbalanced design and the random-groups design over the single-group design without counterbalancing and the anchor test design in the recovery of the actual test-mode effects. Analysis at the item level revealed the presence of a two-dimensional solution, with the data-collection design contributing significant variability over and beyond the test mode. [ABSTRACT FROM AUTHOR]

Nowadays the size of university classes along with the growing number of students to be tested for final examination raise unprecedented challenges relating to the management, monitoring and evaluation of learning. Technology may provide some solutions that deserve to be investigated. This paper explores the potentialities and limitations of Computer‐Based Testing (CBT), specifically BYOD e‐test, compared to traditional Paper‐Based Testing (PBT) to verify whether, and to what extent, an electronic mode of assessment can become a suitable alternative to PBT. It is based on a study carried out at the University of Florence during 2016/17. 606 students participated in the study, of whom 443 opted for CBT using their own devices, while 163 preferred PBT. Participants who experienced CBT also answered a survey on perceptions, self‐efficacy and satisfaction. The results show that students' performances were better with CBT, and that a positive relationship exists between the perceived level of self‐efficacy and the propensity to adopt digital tests. In addition, students greatly appreciated the electronic system, especially for the possibility of immediate feedback. Some critical issues emerged relating to on‐screen reading, which suggests the need for careful design of testing tools. [ABSTRACT FROM AUTHOR]

There are sound educational and examining reasons for the use of coursework assessment and practical assessment of student work by teachers in schools for purposes of reporting examination grades. Coursework and practical work test a range of different curriculum goals to final papers and increase the validity and reliability of the result. However, the use of coursework and practical work in tests and examinations has been a matter of constant political as well as educational debate in England over the last 30 years. The paper reviews these debates and developments and argues that as accountability pressures increase, the evidence base for published results is becoming narrower and less valid as the system moves back to wholly end-of-course testing. [ABSTRACT FROM AUTHOR]

In recent years, computer-based testing (CBT) has grown in popularity, is increasingly being implemented across the United States, and will likely become the primary mode for delivering tests in the future. Although CBT offers many advantages over traditional paper-and-pencil testing, assessment experts, researchers, practitioners, and users have expressed concern about the comparability of scores between the two test administration modes. To help provide an answer to this issue, a meta-analysis was conducted to synthesize the administration mode effects of CBTs and paper-and-pencil tests on K-12 student reading assessments. Findings indicate that the administration mode had no statistically significant effect on K-12 student reading achievement scores. Four moderator variables—study design, sample size, computer delivery algorithm, and computer practice—made statistically significant contributions to predicting effect size. Three moderator variables—grade level, type of test, and computer delivery method—did not affect the differences in reading scores between test modes. [ABSTRACT FROM AUTHOR]

This study investigated the equivalence of scores from computerized and paper-and-pencil formats of a series of K-3 reading screening tests. Concerns about score equivalence on the computerized formats were warranted because of the use of reading passages, computer unfamiliarity of primary school students, and teacher versus computer administration of the test. The results indicated that the computerized format produced lower scores than the paper-and-pencil format across all four grades. These difficulty differences could be related to the differences in family income but further research is needed into the causal factors. However, confirmatory factor analysis supported the construct and measurement equivalence of the scores from the two formats. [ABSTRACT FROM AUTHOR]

This study investigated the effect of several different modes of test administration on scores and completion times. In Experiment 1, paper-based assessment was compared to computer-based assessment. Undergraduates completed the computer-based assessment faster than the paper-based assessment, with no difference in scores. Experiment 2 assessed three different computer interfaces that provided students various levels of flexibility to change and review answers. No difference in scores was observed among the three modes, but students completed the least-flexible mode faster than the other two modes. It appears that less flexible test modes are faster and do not result in poorer performance than more flexible modes. [ABSTRACT FROM AUTHOR]

This paper provides a critical commentary on the Green Paper, Assessment for Success in Primary Schools, released on 7 May 1998. It reveals that the arguments advanced in support of compulsory national testing were neither new nor innovative. In fact, New Zealand had experimented with national primary school tests from 1878 to 1937 and had finally abandoned them because there was absolutely no evidence that they had improved the quality of teaching and learning in the nation's classrooms. We conclude that the Green Paper and the subsequent Information for Better Learning policy statement highlighted the then National government's neo-liberal obsession with using national tests not only to measure school 'outputs' and teacher 'effectiveness' but also to exercise greater political surveillance and control over teachers' work in the nation's primary schools. The Labour government's recently announced voluntary primary school literacy and numeracy assessment initiative avoids the pitfalls associated with compulsory national testing. [ABSTRACT FROM AUTHOR]

The purpose of this study was to present a multifaceted approach to investigating the equivalence of the computer-based and the paper-and-pencil tests and exemplify it using the Stanford Diagnostic Reading Tests. The multifaceted approach includes construct equivalence evaluation, test-level comparison and item-level comparisons, especially differential functioning of items across the two modes. The results show that less than 1% of items in grades 3 and 4 were flagged with DIF, though about 5% of the items in grade 2 were flagged with DIF. The CBT and the PPT for the same grade test shared similar test characteristics. The mode did not affect the raw score to scale score conversion relationship significantly. The underlying constructs between the two modes of the same grade test satisfied the three levels of construct equivalence, i.e., congeneric, τ-equivalent and parallel equivalent. [ABSTRACT FROM AUTHOR]

The terminal examination of post-primary education in Ireland, the Leaving Certificate, is often criticised for the reliance on memory recall over higher order thinking skills in the assessment process. In order to examine the evidence base for these critiques, this article presents an empirical investigation of the intellectual skills and knowledge domains implicit in the tasks in the written examination papers of 23 subjects in the Leaving Certificate in Ireland from 2005 to 2010. Data were collected from two sources: examination papers and student interviews. In an in-depth document analysis of the examination papers, 14,910 occurrences of command verbs were coded for the intellectual skill and knowledge domains required by the assessment task. As the same verb can require different intellectual skills in different subjects and in different tasks, each occurrence of every verb was assigned a specific value depending on its context. The article presents the frequencies and distributions of intellectual skills and knowledge domains within and across subjects. In light of key points in the literature search, the findings indicate concern regarding the level of challenge and stimulation for the development of students of the Leaving Certificate. [ABSTRACT FROM AUTHOR]

Taking a test online rather than on paper is becoming increasingly common. However, there has been little research directly addressing the testing mode (taking a test on paper or online) in chemistry courses, particularly when students take multiple practice tests before an exam. Two studies were conducted to investigate student performance on two proctored general chemistry practice tests as a function of the testing mode. Data were collected in 2013 (Study 1) and again in 2015 (Study 2). The participants were 422 undergraduate students (Study 1 N = 207 and Study 2 N = 215) from a first-semester general chemistry course at a midwestern university. In each study students took two practice tests. Each test included 17 algorithmic, 5 conceptual, and 2 definition questions and was administered on computer or paper. The mode combination of Test 1-Test 2 identified the four conditions: Computer-Computer, Computer-Paper, Paper-Computer, and Paper-Paper. The results show minor differences between online and paper modes. In particular, no significant difference was found between Computer-Paper and Paper-Paper conditions. This pattern suggests that online testing is a promising alternative to the traditional paper-and-pencil mode most often used in chemistry. [ABSTRACT FROM AUTHOR]

Construct: In this study, we examine the differences in test performance between the paper-based and the computer-based version of the Berlin formative Progress Test. In this context it is the first study that allows controlling for students’ prior performance.Background: Computer-based tests make possible a more efficient examination procedure for test administration and review. Although university staff will benefit largely from computer-based tests, the question arises if computer-based tests influence students’ test performance.Approach: A total of 266 German students from the 9th and 10th semester of medicine (comparable with the 4th-year North American medical school schedule) participated in the study (paper = 132, computer = 134). The allocation of the test format was conducted as a randomized matched-pair design in which students were first sorted according to their prior test results. The organizational procedure, the examination conditions, the room, and seating arrangements, as well as the order of questions and answers, were identical in both groups.Results: The sociodemographic variables and pretest scores of both groups were comparable. The test results from the paper and computer versions did not differ. The groups remained within the allotted time, but students using the computer version (particularly the high performers) needed significantly less time to complete the test. In addition, we found significant differences in guessing behavior. Low performers using the computer version guess significantly more than low-performing students in the paper-pencil version.Conclusions: Participants in computer-based tests are not at a disadvantage in terms of their test results. The computer-based test required less processing time. The reason for the longer processing time when using the paper-pencil version might be due to the time needed to write the answer down, controlling for transferring the answer correctly. It is still not known why students using the computer version (particularly low-performing students) guess at a higher rate. Further studies are necessary to understand this finding. [ABSTRACT FROM AUTHOR]

This study attempts to determine if the conclusions of Fordham and Hayes (2009) can be confirmed to include exam performance (they study only quiz performance) by examining the impact of paper color on student final exam performance. One thousand three hundred seventy-two final exams were evaluated covering ten academic quarters, from the Winter 2007 through the Winter 2011. Over this time period, eight different paper colors were used, including seven pastel colors and white. The results show that paper color has no impact on student exam performance and that paper color and student exam performance are independent factors. If we assume that students are motivated to take a final exam and less so to take a quiz, then student motivation (as per Soldat et al. 1997) may be a plausible explanation for the differing results between this study and Fordham and Hayes (2009). The results of this study suggest there may not be a need to restrict the use of pastel paper color on exams, although we describe some evidence and opinions that suggest certain paper colors, including white, may impact those with certain disabilities (i.e., dyslexia, Attention Deficit Disorder [ADD], and Attention Deficit Hyperactivity Disorder [ADHD]). Further, we determine that the widespread use of colored paper may be somewhat of a U.S. phenomenon. [ABSTRACT FROM AUTHOR]

Aim & Objectives: Developing a tactful paper writing skill, through delivery and depiction of the necessary expressions required for in standard or superior essay writing. Understanding relevance and tact of theoretical expression in exam paper writing Learning Indices of standard or quality theory/essay answer (SAQ/LAQ). Applying knowledge and skill gained through these theory writing exercises and assignments to achieve high or better scores in examinations Methods and Materials:The study subjects were divided into two groups- Group A (17 students) and Group B students (10students). The students were selected from II M.B.B.S 4th term. Students of Group A were sensitized on how to write a theory paper and went through 4 phases namely pre-sensitization test, sensitization (imparting them with skills of good theory paper writing through home assignments and deliberations/ guidance), post-sensitization test and Evaluation. Students of Group A (17 students) undertook theory tests (twice, i.e. before and after sensitization) and Students of Group B (10 students) who were not sensitized and took the theory test with post sensitized Group A students (random 10 students). Both groups were given general pathology as the test syllabus, taught to both groups in didactic lectures during the last 6 months. The results of pre and Post-sensitization tests from both groups were analyzed. Intra group comparisons (pre sensitized Group A with Post sensitized Group A) and inter group comparisons (Non-sensitized group B with Sensitized Group A) were made. Results: Significant results were found between results of pre and Post-sensitization tests in Group A (intra group analysis) and inter group (Group A and B) Post-sensitization tests, as there was remarkable improvement in student theory paper writing skills post sensitizing the students of Group A. Conclusion: Medical students should be mandatorily guided and exposed to the nuances and tact of writing the theory paper for their examinations, as it definitely gives them better understanding of presentations ultimately improving their score in the theory exams. [ABSTRACT FROM AUTHOR]

The aim of this paper is to disclose the critical domain of academic writing and its pivotal role in scholarly communication and professional achievement. The study seeks to evaluate the efficacy of a structured academic writing approach using a mixed-methods design, incorporating pre-test and post-test assessments alongside qualitative insights derived from the participant feedback. The participants underwent an initial pre-test covering the structural aspects of academic writing, followed by engagement in a structured undergraduate course emphasising specific nuances of academic papers and mastery of cohesive elements for academic and professional purposes. The qualitative aspect provided a nuanced understanding of the learners’ experiences and perceptions. The structured undergraduate course entailed practical activities and tests covering fundamental aspects of academic writing. These encompass understanding the intended audience and purpose, recognising conventions and standards, grasping the subtleties of formal tone and language precision, refining skills in evidence-based argumentation, analysing the organisational structure of academic texts, evaluating the effectiveness of thesis statements, and incorporating tasks related to professional writing. The research underscores the practical application of acquired academic writing skills, aligning them with professional purposes. Overall, the study aims to contribute valuable insights into the effectiveness of a structured academic writing approach in preparing individuals for success in both academic and professional contexts. [ABSTRACT FROM AUTHOR]

Copyright of Brazilian Journal of Occupational Therapy / Cadernos Brasileiros de Terapia Ocupacional is the property of Cadernos de Terapia Ocupacional da UFSCar and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This study investigates the effect of mode of administration of the Raven Standard Progressive Matrices test on distribution, accuracy, and meaning of raw scores. A random sample of high school students take counterbalanced paper-and-pencil and computer-based administrations of the test and answer a questionnaire surveying preferences for computer-delivered test administrations. Administration mode effect is studied with repeated measures multivariate analysis of variance, internal consistency reliability estimates, and confirmatory factor analysis approaches. Results show a lack of test mode effect on distribution, accuracy, and meaning of raw scores. Participants indicate their preferences for the computer-delivered administration of the test. The article discusses findings in light of previous studies of the Raven Standard Progressive Matrices test. [ABSTRACT FROM AUTHOR]

It is widely recognised that high-stakes assessment can significantly influence what is taught in the classroom. Many argue that high-stakes assessment results in a narrowed curriculum where students learn by rote rather than developing higher cognitive skills. This paper describes a study investigating the various cognitive objectives present from Bloom’s Taxonomy Educational Objectives on the Leaving Certificate biology examination. The study analysed examination papers from the past and current biology syllabuses. Analysis was also carried out to determine the marks being awarded to the different cognitive objectives. The findings show that the examination predominately includes questions that do not promote higher levels of thinking. The majority of the marks on the paper were allocated to the lower objectives of the taxonomy, suggesting students can rely on rote learning to succeed when undertaking the biology examination. This study strongly highlights how high-stake examinations have a narrow scope in terms of student achievement and shows how current biology examination procedures promote low-level learning. This low level of thinking promotes rote learning and regurgitation of facts, requiring little to no understanding of the topics. To prepare students for the working world, there needs to be a shift from only terminal exams to a mixed approach. [ABSTRACT FROM PUBLISHER]

The present study focuses on the implementation of a self - assessment procedure in a Physics class, extended during a seven weeks period in a European secondary level school. The researchers used three modes of assessment based on paper and pencil, computer-web and mobile devices respectively. The aim of the study is to investigate the effect of each mode of assessment on students' motivation and achievement. Analysis of pre- and post-motivation tests revealed a more positive motivational orientation of students towards computers and mobile devices as assessment delivery mediums. Also, student evaluation implemented after the phase of the experimental procedure showed a significant increase in learning achievement for low-achieving students who participated in the mobile-based and computer-based assessment. The positive effects of computers and mobile devices on students' learning motivation suggest that they can be used as a promising alternative to paper and pencil assessment procedures. [ABSTRACT FROM AUTHOR]

This article describes three experiments comparing paper-and-pencil tests (PPTs) to computer-based tests (CBTs) in terms of test method preferences and student performance. In Experiment 1, students took tests using three methods: PPT in class, CBT in class, and CBT at the time and place of their choosing. Results indicate that test method did not affect performance and students liked the idea of additional CBTs. In Experiment 2, students chose the test time and administration method for four exams. CBT was the preferred method and CBT performance did not differ from a PPT comparison group. In Experiment 3, students chose the test method, but not the time, for three exams. PPT was the preferred method. Possible explanations for these findings are discussed. [ABSTRACT FROM PUBLISHER]

The relationship of expected and actual difficulty of items on six mathematics question papers designed for 16-year olds in England was investigated through paired comparison using experts and testing with students. A variant of the Rasch model was applied to the comparison data to establish a scale of expected difficulty. In testing, the papers were taken by 2933 students using an equivalent-groups design, allowing the actual difficulty of the items to be placed on the same measurement scale. It was found that the expected difficulty derived using the comparative judgement approach and the actual difficulty derived from the test data was reasonably strongly correlated. This suggests that comparative judgement may be an effective way to investigate the comparability of difficulty of examinations. The approach could potentially be used as a proxy for pretesting high-stakes tests in situations where pretesting is not feasible due to reasons of security or other risks. [ABSTRACT FROM AUTHOR]

The use of computer-based tests (CBTs) has spread rapidly in recent years, as such tests offer real-time scoring and immediate feedback, facilitate the use of individualised testing methods, improve test administration and reduce test expenses. Thus, most previous studies have tended to focus on the technical advantages of CBTs and on implementation issues. However, objections to the use of CBTs have begun to surface, and the primary concern is whether the scores of CBTs and those of paper-based tests (PBTs) are equivalent. The aim of this article is to compare the scores of Korean students on computer-based and paper-based versions of the same test. We focus on the differences between the scores of male and female participants and between scores on tests examining different subject matter. Surprisingly, even though the Korean students who participated in this study had more exposure to advanced information technologies such as computers, the Internet and multimedia than did students in other countries, they did not achieve higher CBT scores than PBT scores. This finding shows that familiarity with information technology and adaptation to CBTs are distinct. We also identified a fundamental reason for low CBT scores. [ABSTRACT FROM PUBLISHER]

Discusses the difference between electronic and paper-based educational assessment. Questions raised in relation to technology-induced assessment; Comparison of student performance; Solutions to potential problems in testing methods.

This study investigated the equivalence of scores from computerized and paper-and-pencil versions of a reading placement test. Concerns about score equivalence on the computerized versions were warranted because of the speeded nature of the paper-and-pencil version and differences in text delivery and response modes. The results indicated that both forms of the computerized versions produced higher vocabulary scores than the paper-and-pencil format and one form also had higher comprehension and total scores on the computerized version. These difficulty differences, especially for the vocabulary scores, appeared related to the differences in response speed associated with use of a mouse to record responses in contrast to a pencil and answer sheet. Scale scores for the computerized versions had similar predictive power for course placement as paper-and-pencil scores. However, because these results were based on students from only seven institutions, additional studies are needed to investigate the comparability of these versions. [ABSTRACT FROM AUTHOR]

As part of GCSE (General Certificate of Secondary Education) reforms in England, requirements for assessing application in science increased. Setting examination questions in context facilitates testing application as students need to apply what they know and understand to a particular situation. This research explored the nature of the contexts used in reformed GCSE combined science examinations and compared contexts used in a specification which specifically emphasises contextualised learning to those in other specifications. Eight combined science specimen examination papers were selected, including four from GCSE Twenty First Century Science (21C). A qualitative coding frame was used to code each contextualised item. Various strategies for testing in context were present. Contextual features that might risk introducing construct-irrelevant variance were infrequent but may suggest areas for attention in setter training. 21C papers included a higher proportion of items with detailed contexts and a higher proportion of items set in science-related adult/professional settings. [ABSTRACT FROM AUTHOR]

Errors in examination papers and other assessment instruments can compromise fairness. For example, a history question containing an incorrect historical date could be impossible for students to answer. Incorrect instructions at the start of an examination could lead students to answer the wrong number of questions. As there is little research on this issue within the educational assessment community, we reviewed the literature on minimizing errors in other industries and domains, including aviation, energy, and medicine. We identified generalizable principles and applied them to our context of educational assessment. We argue that since assessment instrument construction is a complex system comprising numerous interacting components, a holistic approach to system improvement is required. Assessment instrument errors stem primarily from human failure. When human failure occurs, it is not good enough to suggest that 'to err is simply human'. Instead it is necessary to look deeper, evaluating the latent working conditions that underpin the efficacy of procedures, making the human failure more or less likely. Drawing from the aviation industry's ergonomic SHELLO model, we articulate and explore three of the most critical working conditions that relate to our context: (i) time pressure, (ii) workload and stress, and (iii) wider organizational culture, including good error data collection. We conclude with recommendations for best practice in minimizing errors in assessment instruments. A 'good' error culture should be promoted, which avoids blaming individuals. Errors should be acknowledged readily by all, and system owners should take a scientific approach to understanding and learning from them. [ABSTRACT FROM AUTHOR]

Many English Examination Boards will make some use of the differentiated paper method of assessment in the new scheme of 16+ examining, but it is not clear whether problems of comparability have been adequately solved. This article presents an analysis based on well-defined assumptions incorporated into a measurement model and utilising regression. Marks on the paper not taken by a group are estimated from their joint marks on the common paper and the paper specific to their group. The marks are then weighted using the measurement model and combined to produce a rank order over both the groups. In an appendix, the weighting method used is related to the more usual method of weighting adopted when combining scores. [ABSTRACT FROM AUTHOR]

A call for proposals for papers, training sessions, and coordinated sessions for the 2012 Annual Meeting of the National Council on Measurement in Education (NCME), with the theme "Supporting Student Success in a Global Economy: Methods, Practices, and Policies for Assessment and Learning," is presented.

Abstract: People often attribute their reluctance to study texts on screen to technology-related factors rooted in hardware or software. However, previous studies have pointed to screen inferiority in the metacognitive regulation of learning. The study examined the effects of time pressure on learning texts on screen relative to paper among undergraduates who report only moderate paper preference. In Experiment 1, test scores on screen were lower than on paper under time pressure, with no difference under free regulation. In Experiment 2 the time condition was manipulated within participants to include time pressure, free regulation, and an interrupted condition where study was unexpectedly stopped after the time allotted under time pressure. No media effects were found under the interrupted study condition, although technology-related barriers should have taken their effect also in this condition. Paper learners who preferred this learning medium improved their scores when the time constraints were known in advance. No such adaptation was found on screen regardless of the medium preference. Beyond that, paper learning was more efficient and self-assessments of knowledge were better calibrated under most conditions. The results reinforce the inferiority of self-regulation of learning on screen and argue against technology-related factors as the main reason for this. [Copyright &y& Elsevier]

Literature in the United States provides many examples of no difference in student achievement when measured against the mode of test administration i.e., paper–pencil and online versions of the test. However, most of these researches centre on ‘regular’ students who do not require differential teaching methods or different evaluation processes and techniques. This research provides evidence that students who have learning disabilities, like their counterparts in the regular educational programme, do not lag behind in computer adaptation and use. The study, using differential item functioning analysis with an ‘external’ variable and an analysis of covariance, shows that items and tests can be created to have no practical differences in the mode of administration for this special group of students, and as such, is in keeping with the trend for using online testing with its many advantages (cost savings, flexibility in administration, etc.) in lieu of the paper and pencil version of the test. [ABSTRACT FROM PUBLISHER]

In educational and psycho-diagnostic assessment procedures, computerised assessment is gradually replacing paper-based instruments. In computerised instruments, measuring reasoning skills, such as the ability to solve analogies, few differences are found with their paper-based counterparts; however young children may form an exception. Often in these comparisons of assessment mode, only accuracy is considered, whereas examining solution strategies and measuring learning over time are of particular interest to educational psychologists. The current study investigates whether the learning of analogical reasoning and progression of strategy change follow the same patterns in a computerised setting. The equivalence of paper-based and computerised versions of figural analogy puzzles were examined with regard to both accuracy and strategy change in 69 5-year old children (M=5.4 years). All children were asked to solve figural analogy matrices by constructing their answers from puzzle pieces. In the paper condition (N=33), the puzzle pieces could be grabbed and turned over by hand and then placed in the empty quadrant to solve the analogy. In the computer condition (N=36), the children used a computer mouse to carry out these manoeuvres. The assessment of children's solution strategy progression took place during weekly sessions over four consecutive weeks. Results indicate that the computerised figural analogy task is comparable to the original paper-based version with regard to both difficulty level and progression of strategy change. However, the computerised version took significantly less time to administer and analyse. Another advantage is that testing with computers allows for the registration of supplementary information that could prove useful for more individualised intervention. We conclude that computerised assessment is a sound method for educational psychologists to evaluate 5-year-olds' analogical reasoning ability and development. [ABSTRACT FROM AUTHOR]

Background: Medical workplace learning consists largely of individual activities, since workplace settings do not lend themselves readily to group learning. An electronic Learning Management with System Computer-Supported Collaborative Learning (CSCL) could enable learners at different workplace locations to discuss personal clinical experiences at a distance to enhance learning. Aim: To explore whether CSCL-enabled structured asynchronous discussions on an authentic task has additional value for learning in the medical workplace. Methods: Between January 2008 and June 2010, we conducted an exploratory evaluation study among senior medical students that were engaged in clinical electives. Students wrote a Critical Appraisal of a Topic paper about a clinical problem they had encountered and discussed it in discipline homogeneous subgroups on an asynchronous forum in a CSCL environment. A mixed method design was used to explore students'' perceptions of the CSCL arrangement with respect to their preparation and participation, the design and knowledge gains. We analysed the messages recorded during the discussions to investigate which types of interactions occurred. Results: Students perceived knowledge improvement of their papers. The discussions were mostly task-focused. The students considered an instruction session and a manual necessary to prepare for CSCL. A high amount of sent messages and a high activity in discussion seem to influence scores on perceptions: ''participation'' and ''knowledge gain'' positively. Conclusion: CSCL appears to offer a suitable environment for peers to provide formative feedback on a Critical Appraisal of a Topic paper during workplace learning. The CSCL environment enabled students to collaborate in asynchronous discussions, which positively influenced their learning. [ABSTRACT FROM AUTHOR]

The aim of this study is to investigate whether personalization of mathematical word problems affects student achievement; and if it does, to find out whether the outcomes vary depending on computer and class environment in the teaching of mathematics. The sample of the research, conducted in the 2006–2007 fall semester, consists of 90 seventh grade in four sub-groups. During the application in the classroom, paper-based personalized and paper-based non-personalized problems were given. As to the computer environment, web-based personalized and web-based non-personalized problems were presented. After the application, an achievement test was performed as an achievement post-test parallel to the pre-test. The results show no significant difference between the scores of the groups who used personalized materials and the groups who used non-personalized materials. The results also indicate that no significant difference exists between those who studied in the computer-based environment and those in the paper-based environment. [Copyright &y& Elsevier]

This paper describes a study comparing paper-based marking (PBM) and onscreen marking (OSM) in Hong Kong utilising English language essay scripts drawn from the live 2007 Hong Kong Certificate of Education Examination (HKCEE) Year 11 English Language Writing Paper. In the study, 30 raters from the 2007 HKCEE Writing Paper marked on paper 100 scripts that they had marked on screen 9 months previously. Subsequent to the rating, they completed a questionnaire, from which it was concluded that, technologically, raters had no problems with OSM. Statistical results indicated no bias favouring either form of rating. Attitudinal differences surfaced, however, between new raters who had solely rated on screen as against veteran raters, whose previous rating experiences had only been with paper-based rating. The paper concludes with the prediction that as OSM becomes more prevalent, it will be accepted as the norm, with prejudice about fairness or convenience gradually fading. [ABSTRACT FROM AUTHOR]

In this commentary, we summarize some of the main themes of the NRC report and note ways in which the papers by Mislevy and Haertel, Gorin, and Abedi and Gándara address the Panel's recommendations. We then briefly review and offer reflections on each paper. We see much to applaud here and also in the broader effort to build bridges between the cognitive and measurement sciences. However, much work remains, not only in building bridges, be also in educating the next generation of psychometricians about cognition and the next generation of teachers about psychometrics. Until that day, even the best work here will have little impact on the business of large-scale assessment. [ABSTRACT FROM AUTHOR]

Computer-based assessment (CBA) is yet to have a significant impact on high-stakes educational assessment, but the equivalence between CBA and paper-and-pencil (P&P) test scores will become a central concern in education as CBA increases. It is argued that as CBA and P&P tests provide test takers with qualitatively different experiences, the impact of individual differences on the testing experience, and so statistical equivalence of scores, needs to be considered. As studies of score equivalence have largely ignored individual differences such as computer experience, computer anxiety and computer attitudes, the purpose of this paper is to highlight the potential effects of these. It is concluded that each of these areas is of significance to the study of equivalence and that the often inconsistent findings result from the rapid changes in exposure to technology. [Copyright &y& Elsevier]

There is a lack of empirical research in the use of computers in assessment. A study was conducted to compare student performance in computerized and paper-and-pen multiple-choice tests and assess the extent and effect of 'computer anxiety'. The LXR.TEST 5.1 software was used for the computerized test. A validated rating scale was used to assess computer anxiety. Most students had little computer experience other than in word processing. Only about a third of the students had moderate to mild computer anxiety. Neither computer experience nor anxiety correlated significantly with performance in the computer test. Whereas students performed significantly better in the paper test than in the computer test, there was no statistical difference in ranking in the two assessments formats. This suggests that computer-based assessment could be used with confidence for the purposes of ranking students, but care should be taken in using it for grading. [ABSTRACT FROM AUTHOR]

As access and reliance on technology continue to increase, so does the use of computerized testing for admissions, licensure/certification, and accountability exams. Nonetheless, full computer-based test (CBT) implementation can be difficult due to limited resources. As a result, some testing programs offer both CBT and paper-based test (PBT) administration formats. In such situations, evidence that scores obtained from different formats are comparable must be gathered. In this study, we illustrate how contemporary statistical methods can be used to provide evidence regarding the comparability of CBT and PBT scores at the total test score and item levels. Specifically, we looked at the invariance of test structure and item functioning across test administration mode across subgroups of students defined by SES and sex. Multiple replications of both confirmatory factor analysis and Rasch differential item functioning analyses were used to assess invariance at the factorial and item levels. Results revealed a unidimensional construct with moderate statistical support for strong factorial-level invariance across SES subgroups, and moderate support of invariance across sex. Issues involved in applying these analyses to future evaluations of the comparability of scores from different versions of a test are discussed. [ABSTRACT FROM AUTHOR]

The purposes of this study were to assess the comparability of scores obtained from computer and paper-and-pencil versions of the Iowa Tests of Educational Development and to evaluate examinees' attitudes about multiple aspects of test administration in the two modes. Findings supported the comparability of scores across administration modes with regard to scaling (means and standard deviations), internal consistency, and criterion- and construct-related validity. Overall, examinees preferred taking the computerized tests and valued many operational features of those tests. The least favorable attitudes were reported for the literary skills tests and their scrollable reading passages in particular. [ABSTRACT FROM AUTHOR]

The author investigated the impact of assessment methods on student performance on accounting tests. Specifically, the author used analysis of variance to determine whether the use of computer-based tests instead of paper-based tests affects students' traditional test scores in accounting examinations. The author included 2 independent variables, student gender and student class, as covariates. The findings indicate that there was no significant difference in the values of the students' performance according to the 2 methods of assessment. The findings also revealed that neither student gender nor class was correlated to the test scores in either form of testing. [ABSTRACT FROM AUTHOR]