Search

The public release of generative artificial intelligence (GAI) tools (e.g., ChatGPT) has had a disruptive effect on the assessment practices of higher education institutions (HEIs) worldwide. Concerns have largely been associated with academic integrity, cheating and plagiarism. HEIs have had to develop guidelines in response to GAI. As many of these guidelines were developed in haste and could affect a large number of instructors and students, there is a need to examine their content, coverage and suitability. This review examines the extent to which the world's 50 top-ranking HEIs have developed or modified their assessment guidelines to address GAI use and, where guidelines exist, the primary content and advice given to guide instructors in their GAI assessment design and practices. The findings show that just under half of the institutions have developed publicly available guidelines. The guidelines cover three main areas: academic integrity, advice on assessment design and communicating with students. Amongst the suggestions for teachers on assessment design, two appear particularly pertinent in helping develop effective assessment tasks and developing learners’ AI literacy: first, running assessment tasks through GAI to check the extent to which the tool can accomplish the task and, second, having students use GAI as part of the assessment process. Overall, the review suggests that HEIs have come to accept the use of GAI and drafted assessment guidelines to advise instructors on its use. In the article, we argue that it may be beneficial to embrace GAI as a part of the assessment process since this is the reality of today's educational and job landscape. This will require instructors to develop a new competence - generative artificial intelligence assessment literacy - which is conceptualised in this article.

Uncertainties within most teaching and learning contexts make it difficult for assessment designers to predict the challenge experienced by students during an assignment. For this reason, ‘post-hoc’ adaptation of mark schemes sometimes take place after an assessment has been completed. This is often to compensate for inaccurate assumptions embedded within the assessment itself. Such adjustments occur within many spheres of education and our goal here is to open conversations around the use of these strategies. The overall aim of this paper is to explore and investigate some of these practices and propose a set of guidelines for educators in designing flexible mark schemes and achieving adaptable, yet consistent, marking processes. [ABSTRACT FROM AUTHOR]

Summary: The National Osteoporosis Guideline Group (NOGG) has developed intervention thresholds based on FRAX® to characterise patients at high and very high risk of fracture. Introduction: Guidelines for the assessment of fracture risk have begun to categorise patients eligible for treatment into high and very high risk of fracture to inform choice of therapeutic approach. The aim of the present study was to develop intervention thresholds based on the hybrid assessment model of NOGG. Methods: We examined the impact of intervention thresholds in a simulated cross-sectional cohort of women age 50 years or more from the UK with the distribution of baseline characteristics based on that in the FRAX cohorts. The prevalence of very high risk using the hybrid model was compared with age-dependent thresholds used by the International Osteoporosis Foundation and the European Society for Clinical and Economic Aspects of Osteoporosis and Osteoarthritis (IOF/ESCEO). The appropriateness of thresholds was tested based on the populations treated with anabolic agents. Results: With an upper intervention threshold using the IOF/ESCEO criteria, 56% of women age 50 years or more would be characterised at very high risk. This compares with 36% using the IOF/ESCEO criteria and an age-specific intervention threshold over all ages. With an upper intervention threshold of 1.6 times the pre-existing intervention threshold, 10% of women age 50 years or more would be characterised at very high risk. The data from phase 3 studies indicate that most trial participants exposed to romosozumab or teriparatide would fall into the very high-risk category. Conclusions: Proposals for FRAX-based criteria for very high risk for the NOGG hybrid model categorise a small proportion of women age 50 years or more (10%) in this highest risk stratum. The level of risk identified was comparable to that of women enrolled in trials of anabolic agents. [ABSTRACT FROM AUTHOR]

Digital archaeology in the Netherlands is connected with investments in a new large-scale and innovative Dutch research infrastructure for the next 10 years. Priorities are set to combine forces at a national level in order to contribute to the international position of the Netherlands as a 'knowledge country'. Researchers need to coordinate and collaborate even more to set up a FAIR enabling data infrastructure with limited resources. Regarding the archaeological discipline, the use of formal quality standards and legislation that certifies archaeological organisations to carry out archaeological work improves national collaboration and stimulates the digital workflow. DANS is the dedicated national repository for archaeology in the Netherlands and is launching the Data Station Archaeology, a repository meeting the latest technological standards. Finding and sharing data of high quality facilitates knowledge of archaeological discoveries; a flourishing open access trend in Dutch archaeology stimulates a strong growth in the use of data. To keep up with innovative developments, a growing community of archaeologists and other specialists are working together in international projects to secure the future of European Archaeology.

Background: The outcome of the undergraduate medical programme is to produce clinically competent health care providers relevant for the South African context. Educational institutions find it hard to ensure the quality of assessments where competency must be assessed. This study aimed to compile an assessment framework that can be used to benchmark current assessment practices in the clinical phase of the undergraduate medical programme where competency must be certified. Methods: In this observational, descriptive study, qualitative data were gathered using the steps described by the World Health Organization for rapid reviews. Literature was searched, screened and selected before data were analysed and a framework was constructed. Results: Twenty-five official documents were included in the study. The framework addressed the three components of quality assessment, namely, accreditation, assessment and quality assurance. Assessors should attend to the principles of assessment, namely, validity, reliability, fairness, feasibility, educational effect and acceptability, but realise that no assessment meets all these criteria. The first step to ensure quality assessment is to identify a clear outcome. Assessment should be planned and aligned with this outcome. Conclusion: It is clear that clinical assessment is multidimensional and that no assessment is perfect. Programme accreditation, assessment practices and psychometrics can assist to improve the quality of assessment but cannot judge clinical competence. Using experienced assessors with a variety of assessment methods on a continuous basis is the proposed way to assess clinical competence. An assessment framework can assist to improve assessment, but it cannot guarantee quality assessment.

The outcome of the undergraduate medical programme is to produce clinically competent health care providers relevant for the South African context. Educational institutions find it hard to ensure the quality of assessments where competency must be assessed. This study aimed to compile an assessment framework that can be used to benchmark current assessment practices in the clinical phase of the undergraduate medical programme where competency must be certified. In this observational, descriptive study, qualitative data were gathered using the steps described by the World Health Organization for rapid reviews. Literature was searched, screened and selected before data were analysed and a framework was constructed. Twenty-five official documents were included in the study. The framework addressed the three components of quality assessment, namely, accreditation, assessment and quality assurance. Assessors should attend to the principles of assessment, namely, validity, reliability, fairness, feasibility, educational effect and acceptability, but realise that no assessment meets all these criteria. The first step to ensure quality assessment is to identify a clear outcome. Assessment should be planned and aligned with this outcome. It is clear that clinical assessment is multidimensional and that no assessment is perfect. Programme accreditation, assessment practices and psychometrics can assist to improve the quality of assessment but cannot judge clinical competence. Using experienced assessors with a variety of assessment methods on a continuous basis is the proposed way to assess clinical competence. An assessment framework can assist to improve assessment, but it cannot guarantee quality assessment. [ABSTRACT FROM AUTHOR]

Purpose – The purpose of this paper is to develop an understanding of academics’ approaches to managing team assessment at an Australian University with a view to informing policy development and assessment design. Design/methodology/approach – The research was conducted using a single exploratory case study approach focussing on the team assessment approach of academics teaching in two colleges at an Australian University. A desktop audit of publicly available assessment policy from 38 Australian universities was conducted alongside a review of relevant college subject guides. The findings of this review framed a subsequent focus group and online survey of academic staff. Findings – Results suggest that staff have adopted highly diverse and idiosyncratic approaches to team assessment and have mixed views about varied approaches to managing and assessing teamwork. Findings identify a need for explicit guidance and professional development on designing, managing and grading team assessments. Institutional limits and criteria should be introduced to ensure a whole-of-course approach to developing teamwork skills and ensure students are not burdened with an excessive number of team assessment tasks in a degree. Research limitations/implications – The paper reports results from an exploratory case study at a single Australian University. As such, the results are not generalizable. Practical implications – The findings could inform guidelines, policies or support resources for designing team assessment tasks. Originality/value – The research explores a challenging area for academics: team assessment, which the research indicates is not currently adequately managed through university policy and procedure. The findings highlight options for universities to consider when developing policies and procedures to manage team assessment. The study also provides recommendations for academics to consider when developing and managing team assessment. [ABSTRACT FROM AUTHOR]

Scottish Intercollegiate Guidelines Network (SIGN) recently issued guidance on the management of osteoporosis and the prevention of fragility fractures. The aim of this paper was to critically review the guidance. The SIGN guidance utilises risk factors for fracture as an initial step for assessment, but recommends treatment only in individuals with a T-score of -2.5. There are many problems with the sole use of BMD as the sole gateway to treatment. Moreover, the assessment tools to determine risk (FRAX or QFracture) are not designed to detect osteoporosis but rather fracture risk. Whereas SIGN assumes that FRAX overestimates fracture probability, there are compelling reasons to believe that the disparity is related to the inadequate calibration of QFracture. The disparities make the use of a single threshold for BMD testing problematic. The SIGN guidance for men at high risk of fracture provides a set of confused and inconsistent recommendations that are in direct conflict with regulatory authorizations and is likely to increase further the large treatment gap in men. For women, the number of women eligible for treatment (i.e. with osteoporosis) is 81,700 with the use of FRAX but only 12,300 with QFracture representing 8.2 and 1.2 % of the total population at risk, respectively. We conclude that serious problems with the SIGN guidance preclude its implementation. [ABSTRACT FROM AUTHOR]

Résumé Introduction Les recherches qualitatives dans les sciences de la santé sont de plus en plus nombreuses, cependant elles restent difficiles à publier et font souvent l’objet d’une moindre reconnaissance de la part des commanditaires et/ou des experts des revues. Devant cet état de fait, les chercheurs anglo-saxons ont développé un ensemble de grilles et critères qui devraient permettre d’établir les « standards » de qualité d’une recherche qualitative. Objectifs Les auteurs de cette étude effectuent une analyse comparative de huit grilles et des critères qui les composent, afin d’en évaluer la comparabilité. Méthode Une recension des grilles d’évaluation existantes dans les sciences de la santé, via les moteurs de recherche Web of Science, Science Direct, PsychoInfo, Medline, etc, permet de présenter un bref état des lieux. Une deuxième partie présente huit grilles, dont la traduction française est inédite, issues de la psychologie/psychiatrie et de la médecine générale. Enfin, les auteurs procèdent à une analyse descriptive et thématique du contenu et de la structure de ces grilles. Résultats La mise en perspective des grilles montre les difficultés à trouver des accords dans les définitions des critères entre les divers auteurs. Ces difficultés mettent en évidence l’écart entre ces grilles tant au point de vue de l’épistémologie qui les sous-tend, qu’au point de vue des critères retenus (nombre, type, forme, contenus). Discussion Les auteurs montrent que les différences entre elles correspondent à la diversité des paradigmes auxquels se réfèrent les créateurs des grilles, sans jamais le mentionner explicitement dans leurs textes de référence. Conclusion Les auteurs concluent que l’augmentation des publications de type qualitatif n’a pas permis de dépasser les difficultés à établir des critères de qualité définis et partagés, et que cette grande hétérogénéité des concepts soulève des problèmes majeurs, non seulement méthodologiques, mais surtout épistémologiques et théoriques. Introduction There is an increasing number of qualitative research in the health sciences. However, such research encounters difficulty in being published and is less recognised by funding agencies and/or reviewers of journals. Given this situation, researchers have developed a set of grids and criteria intended to establish the “standards” of the quality of qualitative research. Objectives The authors of this study performed a comparative analysis of eight grids and the constituent criteria, in order to assess their comparability. Method A review of existing evaluation grids in the health sciences was conducted via the search engines: Web of Science, Science Direct, PsychoInfo, Medline, etc. This established a brief overview of the state of the research. Following this, eight grids from psychology/psychiatry and general medicine, which have been translated into French for the first time, are presented. Finally, the authors provide a descriptive and thematic analysis of the content and structure of these grids. Results The comparison between the grids demonstrates the difficulties in finding an agreement in the definitions of criteria between the various authors. These difficulties evidence the differences between the grids from the point of view of the underlying epistemology and the point of view of the criteria used (number, type, form, content). Discussion The authors show that the differences between the grids reflect the diversity of paradigms implicitly referred to by the authors of the grids, although never explicitly mentioned in their texts. Conclusion The authors conclude that the increase in publications of qualitative research has failed to overcome the difficulties in establishing criteria, both defined and shared, and that the great heterogeneity of concepts raises major problems, not only methodological, but also epistemological and theoretical. [ABSTRACT FROM AUTHOR]

This work presents the design and construction of the guidelines for usability assessment in virtual learning environments, based on some criteria identification, which allow to quantify the satisfaction rate of users of this type of tools. For this purpose, the following stages were developed: identification of usability attributes, virtual learning environment characterization, guidelines' criteria and metrics structuring and finally, validation of the proposed guidelines. To do so, two virtual learning environments were assessed: one of free use and the other one of proprietary use. It was shown that the proposed guidelines area tool that can be applied by different types of users to verify the usability degree of such applications. [ABSTRACT FROM AUTHOR]

[english] In 2008 the German Association for Medical Education (GMA) published guidelines to support medical departments in the implementation of assessments. Furthermore, these guidelines were designed to ensure an adequate quality of assessment. The responsible persons of five Bavarian faculties were asked about the organization of assessment at their institution with regard to the new German medical licensing regulations of 2002. Specifically, they were asked about the extent to which their assessment procedures comply with the guidelines of the GMA.Among the results considered important, it was reported that all of the faculties have a person responsible for assessment in their Dean’s offices. Furthermore all of the faculties have implemented or are currently implementing a catalog of learning goals. There is still potential for development in the fields of item review prior to testing and subsequent item analysis.[german] Die Gesellschaft für Medizinische Ausbildung (GMA) hat im Jahr 2008 Leitlinien zur Durchführung von universitären Prüfungen veröffentlicht, die zur Qualitätssicherung im Sinne reliabler und qualitativ hochwertiger Prüfungen beitragen sollen. Die Prüfungsverantwortlichen aller fünf bayrischen Medizinfakultäten wurden im Rahmen einer Bestandsaufnahme zur Organisation ihrer fakultären Prüfungen nach § 27 der ÄAppO befragt, inwieweit die Prüfungsdurchführung mit den Leitlinien der GMA konform ist.Als wichtige Ergebnisse sind festzuhalten, dass an allen Fakultäten ein Gesamtprüfungsverantwortlicher des Dekanats existiert. Weiterhin existieren an den meisten Fakultäten bereits Lernzielkataloge oder solche sind im Aufbau. Entwicklungspotenzial bieten noch die Durchführung von Reviews und einer umfassenden Itemanalyse.

[english] With the introduction of the new medical licensing regulation [1] in 2002, a reform process was initiated nationwide that aimed to redesign medical degree programs at universities to make them more practice-oriented and to establish quality assurance in teaching. Five centers of excellence were set up at the medical schools in Baden-Württemberg that merged to form the Network for Excellence in Medical Education.The Network for Excellence in Medical Education carried out a survey among the five medical schools in Baden-Württemberg on the implementation of the requirements set down in the GMA guidelines for university-specific, high-quality assessments during the clinical phase of medical studies. The network’s next steps for improving the quality of assessments at medical schools were prioritized in order to determine the urgency of support to be given to Baden-Württemberg’s faculties of medicine in implementing the GMA requirements.It became apparent that essential parts of the requirements formulated in the guidelines had already been fulfilled or at least could be fulfilled in the near future. It was observed that establishing structured review processes during assessment preparation and in particular the follow-up presented considerable difficulties. A focus group analysis identified the following objectives for the network: supporting medical schools in their review processes; compiling a comprehensive assessment program and assessment reports with statistical analysis. Training the people responsible for assessment is another priority.The results largely correspond to those from a similar survey carried out among medical schools in Bavaria and show that organizational structures, such as those that have been established in the centers of excellence in education and assessment in Baden-Württemberg and Bavaria, can offer essential support in the quality assurance of assessment.[german] Mit der Einführung der novellierten Ärztlichen Approbationsordnung im Jahr 2002 wurde bundesweit ein Reformprozess in Gang gesetzt, mit dem Ziel, das Medizinstudium an den Universitäten praxisorientierter zu gestalten und eine Qualitätssicherung im Bereich der Lehre zu etablieren. Um hierbei die medizinischen Fakultäten Baden-Württembergs zu unterstützen, wurden fünf Kompetenzzentren an den medizinischen Fakultäten des Landes gegründet, die sich zum Kompetenznetz Lehre Baden-Württemberg zusammengeschlossen haben.Das Kompetenznetz Lehre hat an den fünf medizinischen Fakultäten Baden-Württembergs eine Befragung zum Umsetzungsstatus der in der GMA-Leitlinie für fakultätsinterne Leistungsnachweise dargelegten Anforderungen an qualitativ hochwertige Prüfungen im klinischen Abschnitt des Medizinstudiums durchgeführt. Um den vordringlichen Bedarf für die Unterstützung der medizinischen Fakultäten des Landes Baden-Württemberg bei der Umsetzung der Leitlinien durch das Kompetenznetz Lehre zu ermitteln, wurde eine Priorisierung der nächsten Aufgaben bei der Verbesserung der Qualität der universitären medizinischen Prüfungen vorgenommen.Es zeigte sich, dass wesentliche Teile des in den Leitlinien formulierten Anforderungskatalogs bereits erfüllt wurden oder wenigstens mittelfristig an den Fakultäten umsetzbar sind. Festgestellt wurde aber, dass die Etablierung strukturierter Reviewprozesse bei der Prüfungsvorbereitung und insbesondere der Nachbereitung noch auf deutliche Schwierigkeiten stößt. Im Rahmen einer Fokusgruppenanalyse wurden als vorrangige Arbeitsziele für das Kompetenznetz Lehre die Unterstützung der medizinischen Fakultäten für den Begutachtungsprozess der Prüfungsaufgaben („Review“), die Erstellung eines Gesamtprüfungsprogramms und von Prüfungsberichten mit statistischer Auswertung definiert. Einen weiteren Schwerpunkt bildet die Qualifizierung von Prüfungsverantwortlichen zum Thema Prüfungen.Die Ergebnisse decken sich im Wesentlichen mit einer ähnlichen Befragung der medizinischen Fakultäten in Bayern und zeigen, dass organisatorische Strukturen, wie sie mit den Kompetenzzentren für Lehre und Prüfungen in Baden-Württemberg und Bayern geschaffen wurden, eine wesentliche Hilfestellung bei der Qualitätssicherung von Prüfungen bieten können.

[english] With the implementation of a new licensing regulation for doctors in 2002 the responsibility for assessment was assigned from the German national examination board to the individual faculties, especially so for clinical subjects.39 graded reports have to be issued between the first national board examination (after 2 years of preclinical studies) and the start of the internship (6th year). This new task provides special demands for assessment organization at faculty level. A number of prerequisites have to be met to guarantee adequate quality in assessment. To support faculties in meeting these requirements, guidelines were issued in cooperation with GMA assessment committee and the Baden-Württemberg competence centre for assessment in medical education. Developed on broad consensus these guidelines define quality requirements based on international standards. Their effect on quality improvement in German medical faculty’s examinations in clinical subjects is to be researched concomitantly.[german] Die neue ärztliche Approbationsordnung (ÄAppO) aus dem Jahre 2002 verlagert die Prüfungsverantwortung insbesondere für den klinischen Studienabschnitt auf die Fakultäten:39 benotete Leistungsnachweise sind zwischen der ersten Staatsprüfung und praktischem Jahr zu erstellen. Diese neue Aufgabe stellt besondere Anforderungen an das fakultäre Prüfungswesen. Um eine angemessene Qualität der fakultären Prüfungen sicher zu stellen, sind eine Reihe von Voraussetzungen zu erfüllen. Die vorliegenden Leitlinien des GMA-Ausschusses Prüfungen und des Kompetenzzentrums Prüfungen des Landes Baden-Württemberg wurden erstellt, um die Fakultäten in dieser Aufgabe zu unterstützen. Die vorliegenden Leitlinien definieren die Qualitätsanforderungen unter Berücksichtigung internationaler Standards und wurden in breitem Konsens entwickelt. Ihre Wirkung im Hinblick auf die Qualitätsverbesserung der fakultären Prüfungen im klinischen Studienabschnitt in Deutschland soll durch Begleitforschungsprojekte zukünftig dokumentiert werden.

New assessment guidelines for osteoporosis in Japan include the use of the WHO risk assessment tool (FRAX) that computes the 10-year probability of fracture. The aim of this study was to determine the distribution of fracture probabilities and to assess the impact of probability-based intervention thresholds in women from Japan aged 50 years and older. Age-specific simulation cohorts were constructed from the prevalences of clinical risk factors and femoral neck bone mineral density to determine the distribution of fracture probabilities as assessed by FRAX. These data were used to estimate the number and proportion of women at or above a 10-year fracture probability of 5, 10, 15, 20, 25, and 30 %. In addition, case scenarios that applied a FRAX probability threshold of 15 % were compared with current guidance. In the absence of additional criteria for treatment, a 15 % fracture probability threshold would identify approximately 32 % of women over the age of 50 years (9.3 million women) as eligible for treatment. Because of expected changes in population demography, the 15 % fracture probability threshold would capture approximately 38 % of women over the age of 50 years (12.7 million women), mainly those aged 80 years or older. The introduction of a FRAX threshold of 15 % would permit treatment in women with clinical risk factors that would otherwise fall below previously established intervention thresholds. The incorporation of FRAX into assessment guidelines is likely to redirect treatments for osteoporosis from younger women at low risk to elderly women at high fracture risk. [ABSTRACT FROM AUTHOR]

Placement providers can be successfully involved in assessing students. This is demonstrated in the Monash University Faculty of Information Technology Cooperative Education Program. Students are thoroughly prepared for their placements and industry supervisors receive written assessment guidelines. Not one student has been awarded a pass grade by the programme director, when the host supervisor thought the student would not make it in the profession and should not pass the placement. [ABSTRACT FROM AUTHOR]

Research in the cognition and learning sciences has demonstrated that the human brain contains basic structures whose functions are to perform a variety of specific spatial reasoning tasks and that children are capable of learning basic spatial concepts at an early age. There has been a call from within geography to recognize research on spatial cognition in a meaningful way in primary school curriculum. This article utilizes the spatial thinking taxonomy proposed by Gersmehl and Gersmehl (2006) to examine to the extent to which spatial thinking concepts are being practiced in U.S. schools. The National Geography Standards and forty-nine state social studies or geography standards are examined. Using standards as a measure of geography content, it is concluded that while some of spatial thinking concepts appear often in curriculum, others are largely absent. Designing geography standards that address the findings of spatial cognition research may serve as a means of improving geography instruction. [ABSTRACT FROM AUTHOR]

El objetivo de este estudio es desarrollar y validar una pauta para la evaluación del proceso de enseñanza clínica en radiología; Medicina Universidad Católica, Radiología 32 ítems (MEDUC-RX32). Se utilizó metodología mixta para el desarrollo del instrumento: dos entrevistas grupales (residentes y docentes) y una individual a un jefe de programa de radiología. Utilizando técnica Delfi modificada para lograr acuerdo, un panel de validez nacional evaluó la importancia de cada ítem. El panel Delfi refinó el instrumento de 88 a 32 ítems luego de dos rondas. La pauta final fue piloteada en 55 residentes del programa. El instrumento definitivo presenta alta confiabilidad (coeficiente alfa de Cronbach de 0,957). El promedio de las evaluaciones del desempeño de los docentes fue de 6,23±0,8 (escala Likert 0 a 7) constituyéndose en una pauta válida y confiable para la evaluación de docentes de programas de la especialidad de radiología en países de habla hispana.