Search

Background: The analytical rubric serves as a permanent reference for guidelines on clinical performance for undergraduate dental students. This study aims to assess the rubric system used to evaluate clinical class II composite restorations performed by undergraduate dental students and to explore the impact of gender on overall student performance across two academic years. Additionally, we investigated the relationship between cumulative grade point averages (CGPAs) and students' clinical performance., Methods: An analytical rubric for the assessment of clinical class II composite restoration in the academic years of 2022/2023 and 2023/2024 was used by two evaluators. These two evaluators were trained to use the rubric before doing the evaluations. The scores were based on a 4-point scale for the evaluation of five major parameters for pre-operative procedures (10 points), cavity preparation (20 points), restoration procedures (20 points), and time management (4 points). At the same time, chairside oral exam parameter was 15 points based on a 5-point scale. Descriptive statistics were calculated for the different analytical rubric parameters, and the independent t-test was used to compare the scores between the student groups and the evaluators. Other tests, such as the Kappa test and Pearson's correlation coefficient, were used to measure the association among CGPA, evaluators, and gender participants., Results: The overall score out of 69 slightly increased for females/males (61.28/59.42) and (61.18/59.49) in the 2022/2023 and 2023/2024 academic years, respectively, but the differences were not statistically significant. In the 2022/2023 academic year, female students scored significantly higher than male students in pre-operative procedures, as evaluated by both evaluators (p = 0.001), and in time management, as assessed by both evaluators (p = 0.031). The Kappa test demonstrated a moderate to substantial level of agreement between the two evaluators in both academic years. Strong and significant correlations were noted between students' CGPA and some tested parameters (p = 0.000)., Conclusion: The overall performance was very good and high among both genders, but it was marginally higher among females than among males. This study found some differences in performance between male and female students and variability in the evaluations by the two raters ranging from moderate to substantial agreement and similar performances for students with different CGPA., (© 2024. The Author(s).)

Purpose: Objective structured clinical examinations (OSCE) are a valuable assessment within healthcare education, as they provide the opportunity for students to demonstrate clinical competency, but can be resource intensive to provide faculty graders. The purpose of this study was to determine how overall OSCE scores compared between faculty, peer, and self-evaluations within a Doctor of Pharmacy (PharmD) curriculum., Methods: This study was conducted during the required nonprescription therapeutics course. Seventy-seven first-year PharmD students were included in the study, with 6 faculty members grading 10-15 students each. Students were evaluated by 3 graders: self, peer, and faculty. All evaluators utilized the same rubric. The primary endpoint of the study was to compare the overall scores between groups. Secondary endpoints included interrater reliability and quantification of feedback type based on the evaluator group., Results: The maximum possible score for the OSCE was 50 points; the mean scores for self, peer, and faculty evaluations were 43.3, 43.5, and 41.7 points, respectively. No statistically significant difference was found between the self and peer raters. However, statistical significance was found in the comparison of self versus faculty (p = 0.005) and in peer versus faculty (p < 0.001). When these scores were correlated to a letter grade (A, B, C or less), higher grades had greater similarity among raters compared to lower scores. Despite differences in scoring, the interrater reliability, or W score, on overall letter grade was 0.79, which is considered strong agreement., Conclusions: This study successfully demonstrated how peer and self-evaluation of an OSCE provides a comparable alternative to traditional faculty grading, especially in higher performing students. However, due to differences in overall grades, this strategy should be reserved for low-stakes assessments and basic skill evaluations., Competing Interests: Declaration of competing interest We declare no conflicts of interest or financial interests that the authors or members of their immediate families have in any product or service discussed in the manuscript, including grants (pending or received), employment, gifts, stock holdings or options, honoraria, consultancies, expert testimony, patents, and royalties., (Copyright © 2024. Published by Elsevier Inc.)

Constructed-response questions (CRQs) require effective marking schemes to ensure that the intended learning objectives and/or professional competencies are appropriately addressed, and valid inferences regarding examinee competence are drawn from such assessments. While the educational literature on writing rubrics has proliferated in recent years, this is largely targeted at classroom use and formative purposes. There is comparatively little guidance on how to develop appropriate marking schemes for summative assessment contexts. The different purposes mean that different principles and practices apply to mark schemes for examinations. In this article, we draw on the educational literature as well as our own practical experience of working with medical and health professional educators on their questions and marking schemes to offer 12 key principles or tips for designing and implementing effective marking schemes.

Background: Rubrics are assessment tools customarily designed to judge a student's skills in diagnosis, clinical assessment, presentations. as well as academic performance. The aim of the present study was to assess the reliability of rubrics in clinical oral presentations by students., Materials and Methods: This was a prospective control study that enrolled 300 undergraduate students, students in their internships, and post-graduate students in various clinical streams. Ethical committee approval was obtained before finalizing the study from appropriate regulatory and institutional bodies. A pre-validated questionnaire was designed and distributed to all the participating students. All students were assessed using the traditional examination method as well as by rubric assessment. The responses obtained were compared and statistically analyzed using one-way ANOVA and the Chi-square statistical tool., Results: Rubric assessment was found to have high statistical significance ( P < 0.0001) when compared with the traditional method of examination., Conclusion: In the present study, rubrics were found to serve as an effective method for judging student skills following a clinical oral presentation. However, due to contradictory scientific evidence, there is a requirement for a large sample size to be included for rubric or traditional assessment method studies., Competing Interests: There are no conflicts of interest., (Copyright: © 2024 Journal of Education and Health Promotion.)

Introduction: Peer assessment can enhance understanding of the simulation-based learning (SBL) process and promote feedback, though research on its rubrics remains limited. This study assesses the validity and reliability of a peer assessment rubric and determines the appropriate number of items and raters needed for a reliable assessment in the advanced cardiac life support (ACLS) context., Methods: Ninety-five third-year medical students participated in the ACLS course and were assessed by two teachers (190 ratings) and three peers (285 ratings). Students rotated roles and were assessed once as a team leader on a ten-item rubric in three domains: electrocardiogram and ACLS skills, management and mechanisms, and affective domains. Messick's validity framework guided the collection of validity evidence., Results: Five sources of validity evidence were collected: (1) content: expert reviews and alpha, beta, and pilot tests for iterative content validation; (2) response process: achieved acceptable peer interrater reliability (intraclass correlation = 0.78, p = 0.001) and a Cronbach's alpha of 0.83; (3) internal structure: demonstrated reliability through generalizability theory, where one peer rater with ten items achieved sufficient reliability (Phi-coefficient = 0.76), and two raters enhanced reliability (Phi-coefficient = 0.85); construct validity was supported by confirmatory factor analysis. (4) Relations to other variables: Peer and teacher ratings were similar. However, peers rated higher in scenario management; further generalizability theory analysis indicated comparable reliability with the same number of teachers. (5) Consequences: Over 80% of students positively perceived peer assessment on a 5-point Likert scale survey., Conclusion: This study confirms the validity and reliability of ACLS SBL rubrics while utilizing peers as raters. Rubrics can exhibit clear performance criteria, ensure uniform grading, provide targeted feedback, and promote peer assessment skills., (© 2024. The Author(s).)

Purpose The portfolio can be used as a crucial tool for self-reflection, which allows us not only to showcase achievements but also course correct on our personal and professional journey. However, there is a significant lack of awareness among medical professionals about portfolios. Arranging a workshop to impart this knowledge could be a potential mitigation approach. This study aims to assess the impact of workshops on portfolios on students' and faculty's knowledge. In addition, the study also analyzes the effect of using rubrics on reflective writing skills. Method A portfolio workshop was organized for the medical faculty and students in the Bundelkhand Government Medical College, Sagar, M.P. The Kirkpatrick model of training evaluation along with a rubric for the evaluation of reflective writing skills were used to measure the effectiveness of the workshop. Pre and post-tests for the workshop, pre and post-reflective writing skills, and workshop feedback were collected using questionnaires. The Shapiro-Wilk test and the Wilcoxon signed rank test were applied to the data collected. Results Out of 89 registrations for the workshop, only 81 people consented to the workshop and participated in the study. The total number of faculty was only 17 and the rest were students from all the phases. Both the Shapiro-Wilk test and the Wilcoxon signed rank test showed a significantly small p-value, stating that there was a significant positive impact on the knowledge, perception, and effectiveness of the workshop. Conclusion This study clearly outlines the positive impact of conducting a workshop on portfolios. A significant increase in participants' knowledge of portfolios is identified. Similarly, employing rubrics has a significant increase in the quality of reflective writing skills., Competing Interests: The authors have declared that no competing interests exist., (Copyright © 2024, Singh et al.)

An effective evaluation of a bachelor's thesis (BT) needs the use of valid tools such as rubrics. There are few studies providing a validation of rubrics for these theses and even fewer in the academic field of food science and technology; hence the aim of this study was to validate a rubric for the assessment of relevant competencies in the BT. Ninety-seven students presented their thesis and 56 teachers participated as members of the committees. The degree of relevance, suitability, and clarity of the rubrics were assessed as well as the relationship between student perception and academic performance and the congruence between the teacher's and students' evaluations. The content validity index and intraclass correlation coefficient (ICC) were calculated. Students had a moderate level of knowledge about rubrics, but they mostly agreed with the premise that the rubrics are an accurate tool to assess the quality of BTs. Teacher's and student's marks mostly aligned. No significant relationship was found between the positive perception of rubric's validity and the final grades. Regarding clarity, suitability, and relevance as perceived by teachers, the used rubrics were highly valued. The ICC of criteria indicated that the rubrics were valid in the studied terms. Hence, the validation of a food science and technology degree BT assessment system based on a rubric has been achieved. Supporting Information PRACTICAL APPLICATION: The conclusions drawn from this research could enable teachers at other universities to use this methodology for validating their rubrics for bachelor's theses. Furthermore, they could use this validated rubric to evaluate projects presented at their respective institutions., (© 2024 The Authors. Journal of Food Science published by Wiley Periodicals LLC on behalf of Institute of Food Technologists.)

Background: The development of procedural skills is essential in health sciences education. Rubrics can be useful for learning and assessing these skills. To this end, a set of rubrics were developed in case of neurophysiotherapy maneuvers for undergraduates. Although students found the rubrics to be valid and useful in previous courses, the analysis of the practical exam results showed the need to change them in order to improve their validity and reliability, especially when used for summative purposes. After reviewing the rubrics, this paper analyzes their validity and reliability for promoting the learning of neurophysiotherapy maneuvers and assessing the acquisition of the procedural skills they involve., Methods: In this cross-sectional and psychometric study, six experts and 142 undergraduate students of a neurophysiotherapy subject from a Spanish university participated. The rubrics' validity (content and structural) and reliability (inter-rater and internal consistency) were analyzed. The students' scores in the subject practical exam derived from the application of the rubrics, as well as the rubrics' criteria difficulty and discrimination indices were also determined., Results: The rubrics´ content validity was found to be adequate (Content Validity Index > 0.90). These showed a unidimensional structure, and an acceptable internal consistency (α = 0.71) and inter-rater reliability (Fleiss' ƙ=0.44, ICC = 0.94). The scores of the subject practical exam practically covered the entire range of possible theoretical scores, showing all the criterion medium-low to medium difficulty indices - except for the one related to the physical therapist position-. All the criterion exhibited adequate discrimination indices (rpbis > 0.39), as did the rubric as a whole (Ferguson's δ = 0.86). Students highlighted the rubrics´ usefulness for learning the maneuvers, as well as their validity and reliability for formative and summative assessment., Conclusions: The changed rubrics constitute a valid and reliable instrument for evaluating the execution quality of neurophysiotherapy maneuvers from a summative evaluation viewpoint. This study facilitates the development of rubrics aimed at promoting different practical skills in health-science education., (© 2024. The Author(s).)

Background: Health education games make health-related tasks enjoyable and interactive, thereby encouraging user participation. Entrepreneurs and health educators can leverage online crowdfunding platforms, such as Kickstarter, to transform their innovative ideas into funded projects., Objective: This research focuses on health education game initiatives on Kickstarter. Through an online user survey, it aims to understand user perceptions and evaluate the significance of 8 distinct components that may influence the success of such crowdfunding initiatives., Methods: A total of 75 participants evaluated games using 8 dimensions: game rules, learning objectives, narrative, content organization, motivation, interactivity, skill building, and assessment and feedback. The survey data were analyzed using descriptive statistical analysis, exploratory factor analysis, the Wilcoxon-Mann-Whitney test, and multivariate analysis., Results: Exploratory data analysis showed that, among the 8 dimensions, skill building, content organization, and interactivity were the top-ranking dimensions most closely associated with crowdfunding health education game. The 8 dimensions can be grouped into 3 categories from the exploratory factor analysis: game content-, instruction-, and game design-related components. Further statistical analysis confirmed the correlation between these dimensions with the successful crowdfunding of health education games., Conclusions: This empirical analysis identified critical factors for game proposal design that can increase the likelihood of securing crowdfunding support., (© Hong Huang, Han Yu, Wanwan Li. Originally published in JMIR Serious Games (https://games.jmir.org).)

Scholarly writing is an important skill in all fields of study. Despite a strong focus on writing in many courses, faculty and students have disparate expectations related to scholarly writing. Herein, a classroom exercise is presented in which students were asked to develop a rubric that would be used to evaluate their summative writing assessment. Students were provided with a list of elements that commonly represent good scholarly writing, asked to define what effectively demonstrating these elements looks like, and asked to assign the weight that would be given to each element. The weights given to each element by students were compared to a faculty-generated, departmental writing rubric. Students assigned significantly higher weights to ideas, and significantly lower weights to sentence fluency. Overall, students favored content over writing mechanics. A random selection of student papers was scored using both the departmental rubric and the student rubric, with about a half-letter grade difference between the two groups, though the difference was not statistically significant. The outcomes suggest this exercise may be valuable in offering insight into student perceptions of scholarly writing and in furthering student engagement in the writing process., (© 2023 The Authors. Pharmacology Research & Perspectives published by British Pharmacological Society and American Society for Pharmacology and Experimental Therapeutics and John Wiley & Sons Ltd.)

This article details how and why we have developed a flexible and responsive process-based rubric exemplar for teaching, learning, and assessing critical and creative thinking. We hope to contribute to global discussions of and efforts toward instrumentalising the challenge of assessing, but not standardising, creativity in compulsory education. Here, we respond to the key ideas of the four interrelated elements in the critical and creative thinking general capability in the Australian Curriculum learning continuum: inquiring; generating ideas, possibilities, actions; reflecting on thinking processes; and analysing, synthesising and evaluating reasoning and procedures. The rubrics, radical because they privilege process over outcome, have been designed to be used alongside the current NAPLAN tests in Years 5, 7 and 9 to build an Australian-based national creativity measure. We do so to argue the need for local and global measures of creativity in education as the first round of testing and results of the PISA Assessment of Creative Thinking approach and to contribute to the recognition of creative thinking (and doing) as a core twenty-first century literacy alongside literacy and numeracy., (© The Author(s) 2022.)

Research for development (R4D) funding is increasingly expected to demonstrate value for money (VfM). However, the dominance of positivist approaches to evaluating VfM, such as cost-benefit analysis, do not fully account for the complexity of R4D funds and risk undermining efforts to contribute to transformational development. This paper posits an alternative approach to evaluating VfM, using the UK's Global Challenges Research Fund and the Newton Fund as case studies. Based on a constructivist approach to valuing outcomes, this approach applies a collaboratively developed rubric-based peer review to a sample of projects. This is more appropriate for the complexity of R4D interventions, particularly when considering uncertain and emergent outcomes over a long timeframe. This approach could be adapted to other complex interventions, demonstrating that our options are not merely "CBA or the highway" and there are indeed alternative routes to evaluating VfM., Supplementary Information: The online version contains supplementary material available at 10.1057/s41287-022-00565-7., Competing Interests: Conflict of interestI am reporting that I was previously an employee of UK Research and Innovation (UKRI), sponsored by BEIS; the work reported in this article was conducted while I was an employee of UKRI. I am no longer a UKRI employee and do not have a financial interest in publishing this paper., (© European Association of Development Research and Training Institutes (EADI) 2022, Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.)

Introduction: In the health care area, tuition is an essential part to provide the instrument that proves the graduates have acquired the necessary skills in their specialties., Objective: Evaluation of the improvement in quality of resources for residents after standardized digital training program with rubrics., Methods: Prospective observational study of first year medical residents in seven medical specialties in four different training centers. Five dimensions were considered to scale the quality of medical resident research: Validation of rubric in investigation methodology topics for each block in b-learning mode; initial and ending evaluation; colloquium investigation rubric; results of final investigation; satisfaction survey of 360 degrees. The instruments were validated using the delphi method with a minimum agreement of 0.8. We considered global values greater than 80 points as good quality., Results: 85 medical residents participated and obtained a final average of 80.62 (±9.59), and the satisfaction of the course was qualified as excellent/good in 82.5%. A positive relation was observed between the scope of the evaluation and the level of satisfaction. Mean quality score for the course was good. There is no relationship between the research experience of the students and the final average r = 0.123 ( p = 0.291)., Conclusions: The implementation of research seminars in b-learning mode results in improving the education program for health residents after a training program with a rubric system and their acquiring research skills, and, as a consequence, the final product also showed better quality, even when the student did not have any experience in a scientific publication.

Background/purpose: Effective evaluation of students' self-assessment ability is crucial. This study was to develop a standardized 3D printed teaching model accompanied by structured scoring rubrics for preclinical endodontic training, to appraise students' self-assessment ability, and to evaluate their perceptions of the training system., Methods: The 3D printing model was designed to house a standardized central incisor. Forty-four undergraduate dental students were enrolled and their endodontic performance was self-assessed and also rated by a teaching assistant and a tutor using 3 structured rubrics including access cavity, mechanical preparation, and obturation assessments. In total, 21 rubric evaluation items of preclinical tasks were assessed, and the assessment results from the student, teaching assistant, and tutor were compared. Furthermore, questionnaires were used to evaluate students' experiences with the new system., Results: The tutor, teaching assistant, and students all had similar scorings and high intraclass correlation coefficients (ICC) for mechanical preparation and obturation rubrics, while the students had underestimated scores on the access cavity rubric with a rather low ICC of 0.387, which also lead to the underestimated overall scoring (P < 0.05). Among the 21 evaluation criteria, 9 items were rated statistically different (P < 0.05), which denoted the major deficiencies of students' self-assessments. More than 80% of students rated satisfied for most of the questions regarding the new training system, except the tactile sensation, hardness, and radiopacity., Conclusion: The proposed standardized 3D printed model and structured scoring rubrics is feasible for preclinical endodontic training, and standardized evaluation of students' self-assessment ability., Competing Interests: Declaration of competing interest The authors have no conflicts of interest relevant to this article., (Copyright © 2022 Formosan Medical Association. Published by Elsevier B.V. All rights reserved.)

Prior to the COVID-19 pandemic, most of us had little to no experience with online teaching. With the realization that we would need to teach our students remotely during the pandemic, we needed to develop our online teaching skills and apply this knowledge to create effective online classes for our students. Tulane University's Center for Engaged Learning and Teaching (CELT, http://celt.tulane.edu) and its Innovative Learning Center (ILC, http://it.tulane.edu/innovative-learning-center) partnered to teach an Online Teaching Training (OTT) course for all instructors. This five-week session covered many of the topics needed to teach an engaging and interactive online course. The training included not only the theory but also practical applications of many resources available to online course instructors. This training prepared me to teach a fully online course for the first time in Summer, 2020 and again the following summer. Student evaluations compared between the Emergency Remote Learning done in quick response to the COVID-19 pandemic (Spring 2020) and the online course after completion of the OTT (Summer 2020) indicated that student experiences were much more positive when taught by a trained instructor. Many of the resources/techniques for online courses can be incorporated into in-person or hybrid classes, and vice versa. Further, sharing ideas among colleagues, especially as new resources become available, is critical for the success of all instructors. In this article I share my lessons learned, insights, and thoughts for moving forward as we approach a new era of neuroscience instruction., (Copyright © 2022 Faculty for Undergraduate Neuroscience.)

Background: Doctor of Nursing Practice (DNP) students lack sufficient opportunities to practice writing. Students and faculty require clear expectations and consistent feedback to improve skills., Objective: This study evaluated a rubric-driven scientific writing development program., Design: A mixed methods design was used., Setting: The study was conducted in a post-Master's DNP Program., Participants: The sample included DNP students and faculty., Methods: The intervention was delivered to 10 students and writing proficiency was assessed over five semesters. Overall doctoral project quality and rigor were assessed at the end of the program and compared to a similar group of students (n = 20). Seven faculty and eight students participated in qualitative interviews., Results: Performance improved from Semesters 1 to 5; and though quality and rigor did not differ, the intervention group's final papers were more efficiently written with approximately 17 fewer pages and an average review time of eight fewer minutes than the comparison group. Participants identified the rubric, feedback, and scaffolding as helpful program components., Conclusions: Scientific writing development is essential to DNP education. The intervention improved skill performance and writing efficiency., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

Introduction: Reflection and self-assessment are critical skills for healthcare providers. Identification of gaps in knowledge, skills and attitudes, along with the ability to critically think and problem solve to fill gaps, is the ultimate outcome for lifelong learning. The aims of this study were to (a) refine an instrument used for measuring reflective ability, and conduct comprehensive reliability testing, and (b) describe a process for rater calibration., Materials and Methods: Students develop e-portfolios over a four-year span with assignments that require reflection and self-assessment. The final piece of the portfolio includes a global reflection written the last semester of the programme. Three faculty raters independently evaluated 106 dental students' global reflections using the revised grading rubric. An intraclass correlation coefficient measured the level of agreement between the three raters., Results: Analysis of the 318 faculty ratings (106/rater) resulted in an intraclass correlation of .708. Based on a 5-point grading scale (0 = does not respond to the assignment to 5 = reconstructing), the ratings of the 106 global reflections ranged from 1.3 to 5.0 (M = 3.1, SD =0.66)., Discussion: This study provides confidence in the reliability of a grading rubric designed to assess reflective ability, along with suggestions for calibration. An overall mean of 3.1 (Level 3 = relating-includes evidence of lessons learned) illustrates the complexity of teaching reflection and self-assessment., Conclusion: Use of a reliable grading rubric for assessing reflective writing could assist schools interested in incorporating reflection and self-assessment into the curriculum, ultimately supporting lifelong and enhanced health care., (© 2020 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.)

One of the main challenges faced by physical therapy (PT) students is to learn the practical skills involved in neurological physical therapy (PT). To help them to acquire these skills, a set of rubrics were designed for formative purposes. This paper presents the process followed in the creation of these rubrics and their application in the classroom, noting that students perceived them as valid, reliable, and highly useful for learning. The perception of the validity and usefulness of the rubrics has different closely related dimensions, showing homogeneous values across the students´ sociodemographic and educational variables, with the exception of dedication to studying, which showed a significant relationship with schoolwork engagement and course satisfaction. The adequacy of the hypothesized structural model of the relationships among the variables was confirmed. Direct effects of the perception of the rubrics' validity and engagement on course satisfaction were found, as well as direct effects of the assessment of the usefulness of the rubrics on schoolwork engagement and indirect effects on course satisfaction through this latter variable. The results are discussed taking into account the conclusions of previous research and different instructional implications.

The core concept of genetic information flow was identified in recent calls to improve undergraduate biology education. Previous work shows that students have difficulty differentiating between the three processes of the Central Dogma (CD; replication, transcription, and translation). We built upon this work by developing and applying an analytic coding rubric to 1050 student written responses to a three-question item about the CD. Each response was previously coded only for correctness using a holistic rubric. Our rubric captures subtleties of student conceptual understanding of each process that previous work has not yet captured at a large scale. Regardless of holistic correctness scores, student responses included five or six distinct ideas. By analyzing common co-occurring rubric categories in student responses, we found a common pair representing two normative ideas about the molecules produced by each CD process. By applying analytic coding to student responses preinstruction and postinstruction, we found student thinking about the processes involved was most prone to change. The combined strengths of analytic and holistic rubrics allow us to reveal mixed ideas about the CD processes and provide a detailed picture of which conceptual ideas students draw upon when explaining each CD process., (© 2020 The Authors. Biochemistry and Molecular Biology Education published by Wiley Periodicals LLC on behalf of International Union of Biochemistry and Molecular Biology.)

When determining the score given to candidates in multiple mini-interview (MMI) stations, raters have to translate a narrative judgment to an ordinal rating scale. When adding individual scores to calculate final ranking, it is generally presumed that the values of possible scores on the evaluation grid are separated by constant intervals, following a linear function, although this assumption is seldom validated with raters themselves. Inaccurate interval values could lead to systemic bias that could potentially distort candidates' final cumulative scores. The aim of this study was to establish rating scale values based on rater's intent, to validate these with an independent quantitative method, to explore their impact on final score, and to appraise their meaning according to experienced MMI interviewers. A 4-round consensus-group exercise was independently conducted with 42 MMI interviewers who were asked to determine relative values for the 6-point rating scale (from A to F) used in the Canadian integrated French MMI (IFMMI). In parallel, relative values were also calculated for each option of the scale by comparing the average scores concurrently given to the same individual in other stations every time that option was selected during three consecutive IFMMI years. Data from the same three cohorts was used to simulate the impact of using new score values on final rankings. Comments from the consensus group exercise were reviewed independently by two authors to explore raters' rationale for choosing specific values. Relative to the maximum (A = 100%) and minimum (F = 0%), experienced raters concluded to values of 86.7% (95% CI 86.3-87.1), 69.5% (68.9-70.1), 51.2% (50.6-51.8), and 29.3% (28.1-30.5), for scores of B, C, D and E respectively. The concurrent score approach was based on 43,412 IFMMI stations performed by 4345 medical school applicants. It provided quasi-identical values of 87.1% (82.4-91.5), 70.4% (66.1-74.7), 51.2% (47.1-55.3) and 31.8% (27.9-35.7), respectively. Qualitative analysis explained that while high scores are usually based on minor details of relatively low importance, low scores are usually attributed for more serious offenses and were assumed by the raters to carry more weight in the final score. Individual drop or increase in final MMI ranking with the use of new scale values ranged from - 21 to + 5 percentiles, with the average candidate changing by ± 1.4 percentiles. Consulting with experienced interviewers is a simple and effective approach to establish rating scale values that truly reflects raters' intent in MMI, thus improving the accuracy of the instrument and contributing to the general fairness of the process.

Background: The undergraduate thesis evaluation process is one of the most prevalent weak points in universities; the literature indicates the need to orient undergraduate thesis evaluators through specific training initiatives., Objectives: The aim of this study was to analyze the discrepancies between the grades awarded by undergraduate thesis supervisors and by the panel of faculty examiners, and the effect of training targeted to faculty members., Design and Methods: A cross-sectional study was performed during the 2015-2016 academic year in a Spanish Nursing School., Settings and Participants: We analyzed the mean grades from undergraduate thesis evaluation rubrics and the effects of a professional training course on the evaluation process., Results: A total of 920 rubrics were analyzed. There were significant differences between the 460 grades awarded by the panel (7.63/10.0, standard deviation [SD] 1.57) and the 460 awarded by undergraduate thesis supervisors (8.62/10.0, SD 1.51). The results of the multivariate analysis of variance revealed significant differences according to participation in the training course, with a moderate effect size (ɳ 2 p = 0.063)., Conclusions: A training initiative targeted to faculty involved in developing and evaluating undergraduate thesis reduces subjectivity and enhances fairness and quality in student evaluations., (Copyright © 2020 Elsevier Ltd. All rights reserved.)

In neuroscience and other scientific disciplines, instructors increasingly appreciate the value of writing. Teaching students to write well helps them succeed in school, not only because they perform better on assessments but also because well-structured writing assignments improve learning. Moreover, the ability to write well is an essential professional skill, because good clear writing in conjunction with good clear thinking results in increased success in fellowship applications, grant proposals, and publications. However, teaching writing in neuroscience classrooms is challenging for several reasons. Students may not initially recognize the importance of writing, teachers may lack training in the pedagogy of writing instruction, and both teachers and students must commit substantial time and effort to writing if progress is to be made. Here, we detail effective strategies for teaching writing to undergraduates, including scaffolding of teaching assignments, both within a class and across a curriculum; use of different types of writing assignments; early integration of writing into courses; peer review and revision of assignments; mentoring by student tutors; and use of defined rubrics. We also discuss how these strategies can be utilized effectively in the context of multicultural classrooms and labs., (Copyright © 2020 Elsevier B.V. All rights reserved.)

With the introduction of competency-based undergraduate curriculum in India, a paradigm shift in the assessment methods and tools will be the need of the hour. Competencies are complex combinations of various attributes, many of which being not assessable by objective methods. Assessment of affective and communication domains has always been neglected for want of objective methods. Areas like professionalism, ethics, altruism, and communication-so vital for being an Indian Medical Graduate, can be assessed longitudinally applying subjective means only. Though subjectivity has often been questioned as being biased, it has been proven time and again that a subjective assessment in expert hands gives comparable results as that of any objective assessment. By insisting on objectivity, we may compromise the validity of the assessment and deprive the students of enriched subjective feedback and judgement also. This review highlights the importance of subjective assessment in competency-based assessment and ways and means of improving the rigor of subjective assessment, with particular emphasis on the development and use of rubrics., Competing Interests: None

Background: Authentic assessment and effective feedback are among various strategies that promote learning in the assessment process. These strategies are commonly used during clinical placements. However, they are rarely implemented in the didactic portion of physiotherapy education despite the benefits this type of assessment may bring to achieving students' learning and outcome., Methods: This mixed method study investigated how an authentic continuous assessment coupled with rubrics facilitated physiotherapy students' learning process in a real-life complex skill of exercise prescription and instruction. The study also explored the relationship between different activities in the Learning Management System (LMS) and learning outcomes. Qualitative data was collected using a focus group and an analysis of discussion forum posts. Quantitative data included various information from a questionnaire, the LMS and assessment score., Results: Thematic analyses from the focus group and discussion forum posts suggest that students used a cyclical self-regulated learning process as a result of authentic task design and rubrics for feedback facilitation. Interestingly, the discussion forum access was found to be moderately and significantly correlated with assessment score by Spearman's rank correlation (ρ = 0.59, p < 0.01), while the students did not find the discussion forum useful., Conclusions: Overall results suggest the promotion of self-regulated learning in this authentic continuous assessment. The roles and goals of each authentic task within the assessment should be made explicit in order to raise cognitive awareness of benefits.

Objective: To explore the literature relating to the use of rubrics in Higher Education., Design: A systematic search using three databases was undertaken, the question used to guide the search strategy was: What are the benefits and challenges of using rubrics as part of the assessment process in Higher Education?, Data Sources: Three electronic databases were searched: British Education Index, Education Resources Information Centre and the Cumulative Index to Nursing and Allied Health Literature., Review Methods: The review utilised an integrative approach to the retrieval and appraisal of the research. As the papers retrieved used different methodologies to explore the use of rubrics they were analysed using either thematic analysis or narrative synthesis., Results: Fifteen papers were identified that met the inclusion and exclusion criteria for the review, these spanned a range of disciplines including education, medicine and design. Four main themes related to the use of rubrics were identified: the reliability and validity of the rubric, student performance, students' perceptions of the rubric and the implementation of the rubric., Conclusions: Student self-assessment, self-regulation and understanding of assessment criteria were all found to be enhanced by the use of rubrics. However students also reported that rubrics could be restrictive and student stress related to assessments could be increased. Student involvement in the design and implementation of a rubric was identified as being critical to their success. Rubrics were judged favourably by the studies reviewed in this paper, however they were found to be most effective when used as part of an overall assessment strategy that was co-created with students., (Crown Copyright © 2018. Published by Elsevier Ltd. All rights reserved.)

A rubric is a tool that can support evaluators in a core function of their practice-the process of combining evidence with values to determine merit, worth, or significance-however, little guidance specific to evaluation exists. This study examined, through semi-structured interviews, how a rare group of nine rubric-using seasoned evaluators from across the globe use and learned to use rubrics in their program evaluation practice. Key findings revealed rubrics were a critical component to the practice of these evaluators to make determinations, but also as frameworks to sharpen an evaluation's focus. Additionally, findings support the notion that there is a paucity of formal channels for learning about rubrics and indicate these early adopters are instead, honing their skills through informal channels such as trial and error and by tapping into a community of practice. Future directions for training and research should include expanding understanding, application, and acceptance of use., (Copyright © 2018. Published by Elsevier Ltd.)

Objective: This study aimed to investigate the use of an analytic rubric system for the evaluation of all-ceramic crown preparation on the right maxillary central incisor performed by the dental students., Materials and Methods: Seventy-two-third-year students and 8 faculty members from prosthodontics participated in this double-blind study. The students prepared an ivorine tooth # 11 for all-ceramic crown. The students were given clear instructions regarding the all-ceramic crown preparation and informed about the criteria for the assessment of the preparation. An analytic rubric based on 10-point scale for assessment of various preparation parameters was used by the 8 examiners. Descriptive statistics, ANOVA and post hoc Tukey tests were used for statistical analysis., Results: One-way analysis of variance indicated significance amongst the examiners for all the parameters except for time management. The overall mean scoring by examiners was 7.60 ± 1.18, with highest and lowest mean scores for Examiner 1 (8.02 ± 1.06) and Examiner 4 (6.82 ± 1.50), respectively. The highest number of interexaminer variation (difference) in scoring was found for two plane reduction, and the least difference amongst the examiners was observed for finishing of margins and walls of the preparation. Examiner 4 had the highest number of significant difference with the rest of the examiners. The students scored least marks in axial reduction (56.33%) and preservation of adjacent teeth (66.9%)., Conclusions: Criteria-based assessment using analytic rubric for crown preparations in pre-clinical fixed prosthodontics is an effective tool for finding the errors/weak areas of dental students., (© 2018 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.)

Background: Writing a Bachelor thesis is the last step in obtaining a university degree. The thesis may be job- or research-orientated, but it must demonstrate certain degree-level competences. Rubrics are a useful way of unifying the assessment criteria., Objectives: To design a system of rubrics for assessing the competences associated with the Bachelor thesis of a nursing degree, to examine the system's reliability and validity and to analyse results in relation to the final thesis mark., Design: Cross-sectional and psychometric study conducted between 2012 and 2014., Settings: Nursing degree at a Spanish university., Participants: Twelve tutors who designed the system of rubrics. Students (n = 76) who wrote their Bachelor thesis during the 2013-2014 academic year., Methods: After deciding which aspects would be assessed, who would assess them and when, the tutors developed seven rubrics (drafting process, assessment of the written thesis by the supervisor and by a panel, student self-assessment, peer assessment, tutor evaluation of the peer assessment and panel assessment of the viva). We analysed the reliability (inter-rater and internal consistency) and validity (convergent and discriminant) of the rubrics, and also the relationship between the competences assessed and the final thesis mark., Results: All the rubrics had internal consistency coefficients >0.80. The rubric for oral communication skills (viva) yielded inter-rater reliability of 0.95. Factor analysis indicated a unidimensional structure for all but one of the rubrics, the exception being the rubric for peer assessment, which had a two-factor structure. The main competences associated with a good quality Bachelor thesis were written communication skills and the ability to work independently., Conclusion: The assessment system based on seven rubrics is shown to be valid and reliable. Writing a Bachelor thesis requires a range of degree-level competences and it offers nursing students the opportunity to develop their evidence-based practice skills., (Copyright © 2018 Elsevier Ltd. All rights reserved.)

The purpose of this study was to evaluate interrater agreement between faculty and virtual assessments of preparations for complete coverage restorations in preclinical fixed prosthodontics. Teeth prepared during preclinical fixed prosthodontics practical exams at the University at Buffalo School of Dental Medicine were used in this study. Teeth were prepared for fabrication of complete cast, metal ceramic, and all ceramic crowns. The specimens were digitized using an intraoral scanner. Then, they were virtually superimposed on the corresponding standard preparations using Compare software. The software was used to quantify comparison percentages, average finish line widths, and average axial wall heights. Two calibrated faculty members assessed preparations for occlusal/incisal reduction, finish line location, axial wall height, and finish line width using traditional assessment forms. Cohen's kappa coefficient was used to measure interrater agreement between faculty and virtual assessments. Kappa interrater agreement scores ranged between 0.83 and 0.88 for virtually assessed comparison percentages and sums of faculty-assessed occlusal/incisal reduction and finish line location. Kappa interrater agreement score ranges were 0.64-0.94 and 0.74-0.89 for comparisons of virtual and faculty assessments for axial wall height and finish line width, respectively. Virtual assessments are similar to faculty assessments for occlusal/incisal reduction, finish line location, axial wall height, and finish line width in fixed prosthodontics and can be used as equivalent evaluations of student performance for these criteria.

Rubrics are used by evaluators who seek to move evaluations from being mere descriptions of an evaluand (i.e., the programme, project or policy to be evaluated) to determining the quality and success of the evaluand. However, a problem for evaluators interested in using rubrics is the literature relating to rubric development is scattered and mostly located in the education field with a particular focus on teaching and learning. In this short article we review and synthesise key points from the literature about rubrics to identify best practice. In addition we draw on our rubric teaching experience and our work with a range of stakeholders on a range of evaluation projects to develop evaluation criteria and rubrics. Our intention is to make this information readily available and to provide guidance to evaluators who wish to use rubrics to make value judgements as to the quality and success of evaluands., (Copyright © 2017 Elsevier Ltd. All rights reserved.)

Honors projects that supplement standard coursework are a widely used practice in undergraduate curricula. These projects can take many forms, ranging from laboratory research projects to performing service learning to literature analyses. Here we discuss an honors project focused on interviewing neuroscientists to learn about individual scientific practice and career paths, and synthesizing the resulting information into a personal reflection essay. We detail step-by-step instructions for performing this type of project, including how to develop interview questions, a sample project timeline, deliverables, learning objectives and outcomes, and address potential pitfalls. We provide sample interview questions, an interview solicitation email, and in the supplemental materials an example student reflection essay, assessment rubrics, and the transcription of a student-conducted interview of Drs. John Godwin and Santosh Mishra of North Carolina State University. This type of project is a promising method to enable student-researcher communication, and potentially useful to a broad spectrum of both honors and non-honors neuroscience coursework.

Background and Purpose: To evaluate an electronic counseling rubric to facilitate timely student feedback and explore differences in student performance, student anxiety, and self-perceived preparedness in a high stakes practical exam when using a paper rubric versus an electronic rubric., Educational Activity and Setting: Two cohorts of students in the third professional year were evaluated using the same rubric criteria: cohort 1 (n = 97) used traditional paper rubrics and cohort 2 (n = 104) used electronic rubrics. Cohorts were surveyed to measure anxiety and perceived preparedness in patient counseling skills one week prior to a practical exam, and cohort responses were compared. Student practical exam performance was also compared between the two cohorts., Findings: Results showed no significant relationship between electronic rubric use and student anxiety (p = 0.07) or student exam performance [average score 53.42 points (SD 3.65) and 53.93 points (SD 3.78) in Cohort 1 and Cohort 2, respectively]. Perceived exam preparedness was higher among students using electronic rubrics, with timing of feedback being the mediating process in increasing preparedness (p < 0.01)., Discussion and Summary: Electronic rubrics resulted in more timely feedback on patient counseling skills, and students felt more prepared for their practical exam. This did not result in a significant difference in practical exam performance between the two cohorts. Additional methods to incorporate electronic rubrics into the course will be explored., (Copyright © 2017 Elsevier Inc. All rights reserved.)

Aims: The aim of this study was to evaluate the reliability of using rubrics in dental education, specifically for undergraduate students' assessment in orthodontic oral presentation., Methods: A rubric-based case presentation assessment form was introduced to three contributing instructors. In each instructor's group, the course director, along with the assigned instructor, assessed 8 randomly selected fourth year male dental students utilizing the same assessment form (total of 24 students). The two final scorings made by the assigned instructor and the course director were then gathered for each student. The data of this prospective comparative study then was analyzed using paired t -test to look for any significant differences in the scoring of the course director and each instructor in each group., Results: No significant statistical differences were detected in grading variables between the instructors and the course director. Furthermore, the data showed no significant correlations between the students' final course grade, and their case presentation grades scored by instructors'/course director., Conclusion: Despite the elaborate nature of the routine orthodontic case presentation, the use of rubrics was found to be a promising reliable assessment element.

Background: Writing a bachelor thesis (BT) is the last step to obtain a nursing degree. In order to perform an effective assessment of a nursing BT, certain reliable and valid tools are required., Objectives: To develop and validate a 3-rubric system (drafting process, dissertation, and viva) to assess final year nursing students' BT., Design: A multi-disciplinary study of content validity and psychometric properties. The study was carried out between December 2014 and July 2015., Settings: Nursing Degree at Universitat Jaume I. Spain., Participants: Eleven experts (9 nursing professors and 2 education professors from 6 different universities) took part in the development and content validity stages. Fifty-two theses presented during the 2014-2015 academic year were included by consecutive sampling of cases in order to study the psychometric properties., Methods: First, a group of experts was created to validate the content of the assessment system based on three rubrics (drafting process, dissertation, and viva). Subsequently, a reliability and validity study of the rubrics was carried out on the 52 theses presented during the 2014-2015 academic year., Results: The BT drafting process rubric has 8 criteria (S-CVI=0.93; α=0.837; ICC=0.614), the dissertation rubric has 7 criteria (S-CVI=0.9; α=0.893; ICC=0.74), and the viva rubric has 4 criteria (S-CVI=0.86; α=8.16; ICC=0.895)., Conclusion: A nursing BT assessment system based on three rubrics (drafting process, dissertation, and viva) has been validated. This system may be transferred to other nursing degrees or degrees from other academic areas. It is necessary to continue with the validation process taking into account factors that may affect the results obtained., (Copyright © 2015 Elsevier Ltd. All rights reserved.)

The use of simulation technology has introduced a challenge for simulation nurse educators: evaluation of student performance. The subjectivity of student performance evaluation has been in need of improvement. It is imperative to provide clear and consistent information to the learner of expectations for their performance. Educators use objectives to define for the learner what the primary focus will be in the learning activities. Creation of rubrics to replace checklists to evaluate learner performance is a team task. Improved rubrics assist instructors in providing valuable, immediate, and postactivity feedback and consistency among instructors, and improved inter-rater reliability., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Objective: To determine the amount and type of feedback needed to improve pharmacy students' problem-solving skills using team-based learning (TBL) and a problem-solving rubric., Methods: A problem-solving rubric was developed to assess the ability of pharmacy students' to prioritize, organize, and defend the best and alternative options on TBL cases The study involved 3 groups of pharmacy students: second-year students in a cardiology class who received no feedback (control group), third-year students in an endocrinology class who received written feedback only, and third-year students in an endocrinology class who received written and verbal feedback. Students worked in groups on all TBL cases except the first and last one (beginning and end of course), which students completed independently as it served as a pretest and posttest., Results: Significant improvements were seen in the ability of the third-year students who received verbal and written feedback to prioritize the information presented in the case and in their total score on the problem-solving rubric., Conclusion: Providing pharmacy students with written and verbal explanations may help them improve their problem-solving skills overall. During verbal feedback, faculty members can provide more examples of how to improve and can field questions if needed.