Search

This research explores how paper and online assessments (OA) can affect EFL learners' personal development variables such as self-esteem, mindfulness, demotivation, and language learning development. Sixty intermediate English as a foreign language (EFL) students participated in the current research and were randomly grouped into the experimental group (EG) and the control group (CG). Before the intervention, the data collection tools assessing language learners' self-esteem, mindfulness, demotivation, and language learning development were administered as pre-tests to all the participants. Subsequently, the EG was taught all lessons of the coursebook, Top Notch 2, by utilizing OA while the CG was taught the identical lessons using a conventional (pen and paper) assessment. In the post-treatment phase, the two groups completed the post-tests on self-esteem mindfulness, demotivation, and language learning progress. The study demonstrated that the EG group outperformed the CG group on the four post-tests. The findings indicate that the intervention of OA significantly enhanced language learning outcomes and psychological well-being regarding the measured personal development variables. The enhanced efficacy in the EG group can be ascribed to the utilization of OA during the treatment phase, which has a beneficial effect on academic and personal development. This study also posits that integrating OA augments psychological engagement and language development among EFL learners. It highlights the implications of the results associating teaching EFL via OA with the observed personal development aspects and attributes them to effects of positive psychology. [ABSTRACT FROM AUTHOR]

Inquiry-based learning (IBL) is a type of problem-based learning. While IBL aims at reflecting the work of practicing researchers, only some students will pursue an academic career. We therefore designed a course that provides opportunities for IBL by applying participant observation to internship work experience inside and outside academia. Using this internship course as an example, we investigated, how master's students in health science executed an IBL assignment regarding the application of participant observation and presentation of findings. In addition, the understanding of occupational fields provided through the IBL assignment was examined. To determine whether learning objectives were met, a document analysis of IBL assignments comprising presentation slides and field protocols was performed. Within content analysis, a category grid was used examining (a) suitability of chosen research objectives, (b) execution of field protocols, (c) sufficiency of reporting and (d) extent of reflection. 49 IBL assignments from the years 2020–2022 were included. Sides of IBL observations were: (a) organizations providing health care (n = 28); (b) administrations of health care (n = 8); and (c) research institutes (n = 13). Within students' field protocols, the level of detail of descriptions differed. 30 assignments included reflection on the methods used and research experience. In about a third of IBL assignments, indication of observation type, theoretical background and data analysis was missing. Using participant observation of internship work-experience for IBL can serve as a teaching tool for students to develop methodological skills. For future courses, we developed a checklist to strengthen reporting, reflection and use of theory. As internships are often integrated in degree programs in health sciences similar courses could be implemented in different programs, given qualified methodological guidance. [ABSTRACT FROM AUTHOR]

Student Evaluation of Teaching (SET) instruments can be administered online or with paper and pencil. These procedures involve different interactions between respondents and instruments which could cause errors when administrators and researchers want to compare or relate online results with paper-based results. This study aimed to analyze if the Romanian version of the Student Evaluations of Educational Quality instrument presents similar psychometric characteristics regardless if it is completed online or by paper and pencil. Data was collected from two groups of students (N = 809). One group completed the SEEQ on paper-pencil support (312 responders, 38.6%). The other group completed the scale for the same teachers and courses, but online and one year later. Psychometrical checks and measurement invariance analysis suggested that the Romanian version of SEEQ has passed the internal validity and reliability test and presented configural, metric, and scalar invariance across the results obtained from both administration procedures. • Nowadays, SET instruments are administered online or by paper-pencil procedures. • The procedures involve different interactions between respondents and instruments. • Checking the measure invariance is necessary when using both SET procedures. • The SEEQ has good invariance indices across data collected with both procedures. • The SEEQ has a good fit for the Romanian population. [ABSTRACT FROM AUTHOR]

In this article, I will outline an assessment model that allows instructors to continuing assigning term papers and argumentative papers without compromising the authenticity of student assessment. This path forward relies upon a pseudo flipped classroom model in which students will complete a scaffolded term paper through a series of in-class assessments that build upon previously completed components. The final steps of completing this assignment will require producing a draft and final version of a traditional term paper outside of the classroom but only after the instructor ensures they have critically thought through the preceding components independently. The intention is to maintain the nuance, critical thinking, engagement with primary and secondary sources, and depth of content comprehension that a thoughtful paper requires, while also ensuring that students are completing the work themselves and not merely outsourcing bits and pieces of it to LLMs. [ABSTRACT FROM AUTHOR]

In this paper, we present a system for automatic evaluation of the quality of a question paper. Question paper plays a major role in educational assessment. The quality of a question paper is crucial to fulfilling the purpose of the assessment. In many education sectors, question papers are prepared manually. A prior analysis of a question paper might help in finding the errors in the question paper, and better achieving the goals of the assessment. In this experiment, we focus on higher education in the technical domain. First, we conducted a student survey to identify the key factors that affect the quality of a question paper. The top factors we identified are question relevance, question difficulty, and time requirement. We explored the strategies to handle these factors and implemented them. We employ various concepts and techniques for the implementation. The system finally assigns a numerical quality score against these factors. The system is evaluated using a set of question papers collected from various sources. The experimental results show that the proposed system is quite promising. [ABSTRACT FROM AUTHOR]

Comparative studies on paper and pencil– and computer-based tests principally focus on statistical analysis of students' performances. In educational assessment, comparing students' performance (in terms of right or wrong results) does not imply a comparison of problem-solving processes followed by students. In this paper, we present a theoretical tool for task analysis that allows us to highlight how students' problem-solving processes could change in switching from paper to computer format and how these changes could be affected by the use of one environment rather than another. In particular, the aim of our study lies in identifying a set of indexes to highlight possible consequences that specific changes in task formulation have, in terms of task comparability. Therefore, we propose an example of the use of the tool for comparing paper-based and computer-based tasks. [ABSTRACT FROM AUTHOR]

This article discusses the importance of integrating educational theory and research into the production and evaluation of educational media. It highlights the lack of grounding in educational theory and research in many educational media products, such as apps. However, it also acknowledges examples of educational media, like Sesame Street, that are built upon a strong foundation of theory and research. The article introduces a special section of papers that provide insight into the integration of theory and practice in the creation and evaluation of an educational television series, using Sesame Street's curriculum focus on "playful problem solving" as an example. These papers trace the development of the curriculum from its theoretical underpinnings to its implementation and impact on the target audience. The article concludes by emphasizing the educational benefits of Sesame Street and Playful Learning, and encourages educators, producers, and researchers to share their insights and experiences to further merge theory and practice in educational media. [Extracted from the article]

Nowadays the size of university classes along with the growing number of students to be tested for final examination raise unprecedented challenges relating to the management, monitoring and evaluation of learning. Technology may provide some solutions that deserve to be investigated. This paper explores the potentialities and limitations of Computer‐Based Testing (CBT), specifically BYOD e‐test, compared to traditional Paper‐Based Testing (PBT) to verify whether, and to what extent, an electronic mode of assessment can become a suitable alternative to PBT. It is based on a study carried out at the University of Florence during 2016/17. 606 students participated in the study, of whom 443 opted for CBT using their own devices, while 163 preferred PBT. Participants who experienced CBT also answered a survey on perceptions, self‐efficacy and satisfaction. The results show that students' performances were better with CBT, and that a positive relationship exists between the perceived level of self‐efficacy and the propensity to adopt digital tests. In addition, students greatly appreciated the electronic system, especially for the possibility of immediate feedback. Some critical issues emerged relating to on‐screen reading, which suggests the need for careful design of testing tools. [ABSTRACT FROM AUTHOR]

The one-minute paper (OMP) is an easy-to-implement classroom assessment tool that is particularly well-suited to the familiar one-shot library instruction session. This article explores the ways that the one-minute paper can benefit academic librarians. Examples from the literature on teaching & learning and the author's own experience are used to illustrate potential benefits, including individualized outreach to students and program planning. [ABSTRACT FROM AUTHOR]

There are sound educational and examining reasons for the use of coursework assessment and practical assessment of student work by teachers in schools for purposes of reporting examination grades. Coursework and practical work test a range of different curriculum goals to final papers and increase the validity and reliability of the result. However, the use of coursework and practical work in tests and examinations has been a matter of constant political as well as educational debate in England over the last 30 years. The paper reviews these debates and developments and argues that as accountability pressures increase, the evidence base for published results is becoming narrower and less valid as the system moves back to wholly end-of-course testing. [ABSTRACT FROM AUTHOR]

Copyright of Journal for Educational Research Online / Journal für Bildungsforschung Online is the property of Waxmann Verlag GmbH and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The use of generative artificial intelligence (GenAI) in academia is a subjective and hotly debated topic. Currently, there are no agreed guidelines towards the usage of GenAI systems in higher education (HE) and, thus, it is still unclear how to make effective use of the technology for teaching and learning practice. This paper provides an overview of the current state of research on GenAI for teaching and learning in HE. To this end, this study conducted a systematic review of relevant studies indexed by Scopus, using the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines. The search criteria revealed a total of 625 research papers, of which 355 met the final inclusion criteria. The findings from the review showed the current state and the future trends in documents, citations, document sources/authors, keywords, and co-authorship. The research gaps identified suggest that while some authors have looked at understanding the detection of AI-generated text, it may be beneficial to understand how GenAI can be incorporated into supporting the educational curriculum for assessments, teaching, and learning delivery. Furthermore, there is a need for additional interdisciplinary, multidimensional studies in HE through collaboration. This will strengthen the awareness and understanding of students, tutors, and other stakeholders, which will be instrumental in formulating guidelines, frameworks, and policies for GenAI usage. [ABSTRACT FROM AUTHOR]

This study investigated the effect of several different modes of test administration on scores and completion times. In Experiment 1, paper-based assessment was compared to computer-based assessment. Undergraduates completed the computer-based assessment faster than the paper-based assessment, with no difference in scores. Experiment 2 assessed three different computer interfaces that provided students various levels of flexibility to change and review answers. No difference in scores was observed among the three modes, but students completed the least-flexible mode faster than the other two modes. It appears that less flexible test modes are faster and do not result in poorer performance than more flexible modes. [ABSTRACT FROM AUTHOR]

The purpose of this study was to analyse the students’ evaluations of the course and instructor for all statistics courses offered during fall semester 2009 at a large university in the southern United States. Data were collected and analysed for course evaluations administered both online and on paper to students in both undergraduate and graduate courses. Unlike most previous studies on this subject, class section rather than student was treated as the unit of analysis. It was of specific interest to verify prior research findings that evaluation surveys administered online would not result in lower course and instructor ratings and lower response rates. The results showed that there is not sufficient evidence within the collected data to conclude that either course and instructor ratings or response rates are lower for evaluations administered online (online evaluations) than they are for evaluations administered on paper (paper evaluations). Of secondary interest was whether class ratings would be associated with students’ attendance and a comparison of variability among answers for undergraduate vs. graduate students. It was observed that class and teacher ratings were not related to students’ attendance and individual students did not tend to give the same answer for every question on their survey. [ABSTRACT FROM AUTHOR]

This article explores the effect on assessment of ‘translating’ paper and pencil test items into their computer equivalents. Computer versions of a set of mathematics questions derived from the paper-based end of key stage 2 and 3 assessments in England were administered to age appropriate pupil samples, and the outcomes compared. Although in most cases the change to the different medium seems to make little difference, for some items the affordances of the computer profoundly affect how the question is attempted, and therefore what is being assessed when the item is used in a test. These differences are considered in terms of validity and legitimacy, that is whether the means used to answer a question in a particular medium are appropriate to the assessment intention. The conclusion is not only that translating paper and pencil items into the computer format sometimes undermines their validity as assessments, it is also that some paper and pencil items are less valid as assessments than their computer equivalents would be. [ABSTRACT FROM AUTHOR]

A range of research studies have been conducted on either quizes or minute papers as formative assessment tools. A comparison of both tools was done by Kwan through his survey of students learning about the principles of economics. Kwan's study compared the application of a one-minute paper and quiz in the form of extended true-false, multiple-choice, and short answers. However, this article experimentally compared a matching quiz and half-minute paper as formative assessment tools in an Educational Research Design class, a content-based subject taught to university students. The results of the descriptive statistics showed that the matching quiz was more effective than the half-minute paper as a formative assessment tool. Inferential statistics using the independent sample t-test, however, revealed that the difference between the experimental and control groups' posttest mean scores was not statistically significant. This indicated that the matching quiz and half-minute paper could possibly be used interchangeably; however, the teacher should be aware that the matching quiz is more practical but cannot develop students' higher-order thinking skills, while the half-minute paper encourages active learning but takes more time to apply. [ABSTRACT FROM AUTHOR]

As part of GCSE (General Certificate of Secondary Education) reforms in England, requirements for assessing application in science increased. Setting examination questions in context facilitates testing application as students need to apply what they know and understand to a particular situation. This research explored the nature of the contexts used in reformed GCSE combined science examinations and compared contexts used in a specification which specifically emphasises contextualised learning to those in other specifications. Eight combined science specimen examination papers were selected, including four from GCSE Twenty First Century Science (21C). A qualitative coding frame was used to code each contextualised item. Various strategies for testing in context were present. Contextual features that might risk introducing construct-irrelevant variance were infrequent but may suggest areas for attention in setter training. 21C papers included a higher proportion of items with detailed contexts and a higher proportion of items set in science-related adult/professional settings. [ABSTRACT FROM AUTHOR]

Errors in examination papers and other assessment instruments can compromise fairness. For example, a history question containing an incorrect historical date could be impossible for students to answer. Incorrect instructions at the start of an examination could lead students to answer the wrong number of questions. As there is little research on this issue within the educational assessment community, we reviewed the literature on minimizing errors in other industries and domains, including aviation, energy, and medicine. We identified generalizable principles and applied them to our context of educational assessment. We argue that since assessment instrument construction is a complex system comprising numerous interacting components, a holistic approach to system improvement is required. Assessment instrument errors stem primarily from human failure. When human failure occurs, it is not good enough to suggest that 'to err is simply human'. Instead it is necessary to look deeper, evaluating the latent working conditions that underpin the efficacy of procedures, making the human failure more or less likely. Drawing from the aviation industry's ergonomic SHELLO model, we articulate and explore three of the most critical working conditions that relate to our context: (i) time pressure, (ii) workload and stress, and (iii) wider organizational culture, including good error data collection. We conclude with recommendations for best practice in minimizing errors in assessment instruments. A 'good' error culture should be promoted, which avoids blaming individuals. Errors should be acknowledged readily by all, and system owners should take a scientific approach to understanding and learning from them. [ABSTRACT FROM AUTHOR]

Many English Examination Boards will make some use of the differentiated paper method of assessment in the new scheme of 16+ examining, but it is not clear whether problems of comparability have been adequately solved. This article presents an analysis based on well-defined assumptions incorporated into a measurement model and utilising regression. Marks on the paper not taken by a group are estimated from their joint marks on the common paper and the paper specific to their group. The marks are then weighted using the measurement model and combined to produce a rank order over both the groups. In an appendix, the weighting method used is related to the more usual method of weighting adopted when combining scores. [ABSTRACT FROM AUTHOR]

This document is an editorial from the journal Frontiers in Education, published in 2024. It discusses the field of assessment in education, highlighting new insights, developments, challenges, and future perspectives. The editorial emphasizes the importance of measurement, assessment, and testing in decision-making, but acknowledges the need for new approaches due to the increasing complexity of the world. It includes a summary of research topics covered in the journal, such as teacher assessment competence, scale development and validation, the relationship between test scores and degree completion, and the use of AI in assessment and formative practices. The editorial also discusses innovations in assessment protocols, such as dynamic assessment and synchronous self-assessment strategies. It concludes by commending the papers as a valuable contribution to the field. It is important for library patrons to critically evaluate the information provided in the article, as the publisher does not guarantee or endorse any product or claim mentioned. [Extracted from the article]

This paper describes a study comparing paper-based marking (PBM) and onscreen marking (OSM) in Hong Kong utilising English language essay scripts drawn from the live 2007 Hong Kong Certificate of Education Examination (HKCEE) Year 11 English Language Writing Paper. In the study, 30 raters from the 2007 HKCEE Writing Paper marked on paper 100 scripts that they had marked on screen 9 months previously. Subsequent to the rating, they completed a questionnaire, from which it was concluded that, technologically, raters had no problems with OSM. Statistical results indicated no bias favouring either form of rating. Attitudinal differences surfaced, however, between new raters who had solely rated on screen as against veteran raters, whose previous rating experiences had only been with paper-based rating. The paper concludes with the prediction that as OSM becomes more prevalent, it will be accepted as the norm, with prejudice about fairness or convenience gradually fading. [ABSTRACT FROM AUTHOR]

In this commentary, we summarize some of the main themes of the NRC report and note ways in which the papers by Mislevy and Haertel, Gorin, and Abedi and Gándara address the Panel's recommendations. We then briefly review and offer reflections on each paper. We see much to applaud here and also in the broader effort to build bridges between the cognitive and measurement sciences. However, much work remains, not only in building bridges, be also in educating the next generation of psychometricians about cognition and the next generation of teachers about psychometrics. Until that day, even the best work here will have little impact on the business of large-scale assessment. [ABSTRACT FROM AUTHOR]

Copyright of Pesquisa em Educação Ambiental is the property of Pesquisa em Educacao Ambiental and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The September issue of Pedagogy in Health Promotion focuses on the future of teaching and learning in an evolving digital world. The papers in this issue address the challenges posed by advances in artificial intelligence and online communication. One set of papers discusses the need for educators to equip students with skills to navigate the widespread misinformation and disinformation in public health. Another set of papers explores the use of artificial intelligence in teaching and learning, highlighting its benefits and ethical considerations. The issue also emphasizes the importance of providing faculty with training in online pedagogy. Overall, the journal aims to encourage further exploration of how pedagogy in health promotion is adapting to the digital world. [Extracted from the article]

The author investigated the impact of assessment methods on student performance on accounting tests. Specifically, the author used analysis of variance to determine whether the use of computer-based tests instead of paper-based tests affects students' traditional test scores in accounting examinations. The author included 2 independent variables, student gender and student class, as covariates. The findings indicate that there was no significant difference in the values of the students' performance according to the 2 methods of assessment. The findings also revealed that neither student gender nor class was correlated to the test scores in either form of testing. [ABSTRACT FROM AUTHOR]

This study investigated factors related to score differences on computerized and paper-and-pencil versions of a series of primary K–3 reading tests. Factors studied included item and student characteristics. The results suggest that the score differences were more related to student than item characteristics. These student characteristics include response style variables, especially omitting, and socioeconomic status as measured by free lunch eligibility. In addition, response style and socioeconomic status appear to be relatively independent factors in the score differences. Variables studied but not found to be related to the format score differences included association of items with a reading passage, item difficulty, and teacher versus computer administration of items. However, because this study is the 1st to study the factors behind these score differences below Grade 3, and because a number of states are increasing computer testing at the primary grades, additional studies are needed to verify the importance of these 2 factors. [ABSTRACT FROM AUTHOR]

Context: A growing number of health professions schools have implemented programmes for community-based education (CBE) for their students. There are indications, however, that particularly in developing countries, CBE programmes are not always optimally implemented or sustained. Objective: To test the suitability of an established method for curriculum evaluation, combined with a set of generic objectives for CBE programmes, for evaluation of CBE programmes. Methods: As a case study, Coles and Grant's model for curriculum evaluation was applied to the CBE programme of the Medical Faculty of Diponegoro University (MFDU) in Semarang, Indonesia. Document analysis yielded information on the programme on paper; participatory observation and staff interviews on the programme in action. In addition, MFDU's CBE programme was evaluated against a set of generic objectives for CBE programmes recently designed by us. Results: MFDU has created great opportunities for its CBE programme in which, however, also significant weaknesses were revealed. (1) In the community, much time was spent on formal teaching; (2) Students' work in the community was not jointly identified with community members regarding the community's felt health needs; (3) There was rarely continuity, and evaluation or follow-up of the students' work in the community; and (4) No systematic programme evaluations are carried out. Discussion: This evaluation study showed shortcomings in the implementation of MFDU's CBE programme. The major weaknesses identified point at an underutilization of the opportunities and potential jeopardization of the facilities in the community. On the other hand, more time is needed in the CBE programme to establish the health needs to be addressed jointly with the community and to assess the impact of activities undertaken. A thorough review of the CBE programme, perhaps taking the outcomes of this study into account, could turn MFDU's CBE programme into a fine example for other medical schools in Indonesia and beyond. Conclusion: Coles and Grant's method for curriculum evaluation proved suitable for evaluation of a CBE programme in a developing country. After additional comparison with a reference list of objectives for CBE programmes, reasoned suggestions for programme can be made. [ABSTRACT FROM AUTHOR]

Nowadays, the younger university students are increasingly using digital solutions, therefore teachers should turn to these solutions as well. One of the tools for this is to use an online platform instead of paper-based solutions for short end-of-class assessments. There are several such available software on the market, and this paper presents the use and results of Kahoot through two case studies. The details of the case studies also show the impact in logistics education. [ABSTRACT FROM AUTHOR]

The theoretical background of the conservation cultural heritage has been based on ethical principles phrased in the past rather than on findings of materials science, as illustrated quite impressively in 2005 by the well-known conservator and art historian Salvador Muñoz Viñas. In spite of the fact that this imbalance causes numerous problems for use and evaluation of conservational interventions into the material substance of historic objects, history’s long due transition towards principles of material science is very hesitant. Unfortunately, conservation-restoration, which is eventually in charge of maintaining the material integrity of cultural goods, does not have a satisfying number of suitable instructional materials on a sufficiently scientific basis to live up to the material diversity and complexity of historic objects. This is particularly deplorable with specific respect to the German-speaking area, where the most recent publications in the art date back to about 30 years ago and no longer meet the requirements prevailing today. Nevertheless, or maybe precisely because the German-speaking conservation scene is still forced to follow “ancient” teaching tools, innovative changes in conservator training and a refurbishment of the current mentality continue to be difficult and slow. This paper will analyze historical instructional materials in the field of paper conservation and set out a variety of additional factors that influence the development of the field. [ABSTRACT FROM AUTHOR]

The article presents a study which identifies that general practice registrars regard the current Membership of the Royal College of General Practitioners (MRCGP) written paper with short answer and Modified Easy Question (MEQ) formats as a valid assessment of approach to practice. The study found that almost 90 percent of the respondents thought that the questions were relevant to every general practice. The study showed that the educational impact of preparing for the paper was high.

The written paper (Paper 1) in the modular MRCGP is due to end in October 2008 as the new MRCGP assessment becomes the compulsory qualification for entry into general practice. As part of quality assurance in the old examination, candidate feedback was sought on the written module immediately after sitting the Spring 2006 paper. This showed high levels of satisfaction with the paper and provides evidence of the validity of the questions set. These results reinforce some of the findings of an original study exploring the views of a cohort of candidates from the Northern Deanery. Further, it supports the idea that the MRCGP written paper has driven educational goals, particularly in the area of critical appraisal skills. It also provides some assurance that the paper did not discriminate against minority groups taking the paper. [ABSTRACT FROM AUTHOR]

Computer assisted assessment is becoming more and more common through further and higher education. There is some debate about how easy it will be to migrate current assessment practice to a computer enhanced format and how items which are currently re-used for formative purposes may be adapted to be presented online. This paper proposes an evaluatory framework to assess and enhance the practicability of large-scale CAA migration for existing items and assessments. The framework can also be used as a tool for exposing compromises between delivery mechanism and validity--exposing the limits of validity of modified paper based assessments and highlighting the crucial areas for transformative assessments. [ABSTRACT FROM AUTHOR]

Few historical studies of government’s interest in student achievement exist and, of those that do, most concern themselves with relatively short periods of time, a decade or two in general. This discussion takes a longer view of measurement practices in one jurisdiction, British Columbia. Based on archival records, it examines testing and assessment developments in Canada’s westernmost province from the establishment of public education in 1872, to 1999, when narrow test‐based approaches to measuring achievement gave way to broader assessment practices based on formative, anecdotal, and portfolio reporting strategies. The study illustrates that government’s interest in measuring student achievement – and thereby demonstrating public accountability for educational expenditures – is anything but new and may be traced in an unbroken line of concern that stretches back to the Victorian Era and to the earliest days of public schooling in the province. [ABSTRACT FROM AUTHOR]