Search

Background The outcome of assessments is determined by the standard-setting method used. Standard setting is the process of deciding what is good enough. A cutoff score of 50% was commonly used in dental schools in Malaysia. This study aims to compare the conventional, norm-referenced, and modified-Angoff standard-setting methods. Methods The norm-referenced method of standard setting was applied to the real scores of 40 final-year dental students on a multiple-choice question (MCQ), a short answer question (SAQ), and an objective structured clinical examination (OSCE). A panel of 10 judges set the standard using the modified-Angoff method for the same paper in one sitting. One judge set the passing score of 10 OSCE questions after 2 weeks. A comparison of the grades and pass/fail rates derived from the absolute standard, norm-referenced, and modified-Angoff methods was made. The intra-rater and inter-rater reliabilities of the modified-Angoff method were assessed. Results The passing rate for the absolute standard was 100% (40/40), for the norm-referenced method it was 62.5% (25/40), and for the modified-Angoff method it was 80% (32/40). The modified-Angoff method had good inter-rater reliability of 0.876 and excellent test-retest reliability of 0.941. Conclusion There were significant differences in the outcomes of these three standard-setting methods, as shown by the difference in the proportion of candidates who passed and failed the assessment. The modified-Angoff method was found to have good reliability for use with a professional qualifying dental examination.

The aim of this study was to assess the association among admissions variables, dental school performance, and licensing exam performance for six cohorts of graduates of one dental school. Data from all dental school graduates of Schulich School of Medicine & Dentistry, Western University, Canada, from 2009 to 2014 who had matching National Dental Examining Board of Canada (NDEB) data (N=298) were analyzed. In the results, significant differences between cohorts were found on both the NDEB objective structured clinical examination (OSCE) and written scores. Approximately 18% of the variation in OSCE scores was attributable to cohort differences and 82% to student differences. Approximately 10% of the variation in written scores was attributable to cohort differences and 90% to student differences. Several multilevel models were conducted. The final predictive model for NDEB OSCE scores consisted of age, Canadian Dental Aptitude Test (DAT) reading comprehension scores, year 2 average, and year 4 average. For predicting NDEB written exam scores, the final model consisted of DAT chemistry and year 1, 2, and 4 averages. The findings of this study showed that academic performance on admissions variables and in training predicted performance on dental licensing exams, whereas variables that captured noncognitive or interpersonal skills, such as interview scores, were not predictive. This difference may be due to construct mismatch, such that the outcome variables had no theoretical association with the predictors. Additional outcome measures (including noncognitive) are needed that have greater ecological validity in predicting potential for competence in practice.

Teaching radiographic interpretation skills to predoctoral dental students is challenging. The results of an objective structured clinical examination (OSCE) at one U.S. dental school failed to yield expected outcomes for students' radiographic interpretation skills. The purpose of this article is to describe the development of the radiology OSCE and subsequent curricular revisions based on the outcomes. Examples of curricular changes are a series of self-assessed radiographic interpretation cases using the university learning management system. Each case contains a set of four intraoral radiographs and a series of questions. In year 4, an OSCE and final radiographic interpretation competency exam are administered; students are required to pass both exams. OSCE outcomes over a period of six years were used to evaluate the effectiveness of the curricular revisions. A questionnaire developed to capture student perceptions of the curricular changes was administered. The pilot OSCE of 2.9% first attempt pass rates initiated curricular revision. The strongest results to date occurred in 2018 with a 73.3% first attempt pass rate. Results from the questionnaire showed that students reported greater confidence in their interpretation skills when it came to recognizing radiographic errors, dental caries, and periodontal disease/pathologies (65%, 64%, and 57%, respectively). The use of the OSCE for programmatic assessment revealed the need for curricular revision in radiology. Students' participation in the newly revised radiology curriculum resulted in improved student performance and outcomes. This article emphasizes the importance of global and programmatic assessment for assessing student competence along with analysis of how assessment and accompanying data can inform curricular decisions.

Although reviewing dental students' clinical competency assessments is an important aspect of instruction, finding time to give individual feedback to each student poses a challenge for faculty members, and some students may prefer to receive feedback from a peer. The aim of this study was to explore dental students' perceived value of feedback on their performance in a simulated patient care activity from either a faculty member or a peer. Participants were third- and fourth-year dental students who had completed two years of interprofessional instruction and a videotaped objective structured clinical examination (OSCE) with standardized patients. Participants in two cohorts were randomly assigned to a faculty or peer feedback group. Cohort one (2015-16) consisted of 66 students: 21 in faculty-led groups, and 45 in peer-led groups. Cohort two (2017) consisted of 60 students: 17 in faculty-led groups, and 43 in peer-led groups. In both types of pairings, the protocol consisted of jointly observing a video recording of student performance in the simulated patient encounter and discussing questions about the student's performance in non-technical competencies such as communication, patient safety, scope of practice, and conflict resolution. For cohort two, prior to the feedback sessions, students in the peer feedback groups received a 60-minute training on providing constructive feedback. All 126 students in the two cohorts completed an evaluation questionnaire after the experience. The results showed that students in both types of feedback sessions perceived value in the feedback and believed it enhanced their skills. However, students rated faculty feedback significantly higher (p

The aim of this study was to establish and implement an objective structured clinical examination (OSCE) for stage assessment in standardized training for oral residents, evaluating its validity and suitability. An OSCE was established for stage assessment of 158 residents in a program in Jiangsu Province, China. Its validity and suitability were evaluated using indices such as reliability and discrimination coefficients of assessment results. The established OSCE had eight stations: interpretation of auxiliary examination results, basic knowledge, doctor-patient communication and clinical examination, clinical case analysis, medical record and prescription writing, public skills, first aid skills, and specialist's operation skills. The mean overall score and consistency coefficient (Cronbach's alpha) for the test subjects were 83.64±3.69 points and 0.732, respectively. The developed OSCE was reasonably established, with credible assessment results, and indices such as mean and discrimination coefficient of test scores from each station were appropriate. Therefore, the proposed protocol was found to be reliable and suitable.

The aim of this pilot study was to measure and compare dental hygiene students' level of text anxiety and performance when being tested on an objective structured clinical examination (OSCE) using camera supervision versus direct examiner observation. This randomized, controlled trial, conducted in October 2016, assessed students' anxiety levels and performance using the Zung Self-Rating Anxiety Scale and performance on an OSCE. All 32 first-year students in one U.S. dental hygiene program were invited to participate, and all 32 participated for a 100% response rate. Participants were randomly assigned to the direct observation (N=16) and camera-supervised (N=16) groups. The comparison of pre-OSCE anxiety levels and performance scores evaluated with direct observation (36.4 and 20.4) versus camera-supervised (37.5 and 20.9) found no statistically significant difference between the two groups (p=0.496, p=0.403). The difference in post-OSCE anxiety levels between the direct observation and camera-supervised groups (33.8 and 37.8) approached significance (p=0.051). This study found that student anxiety levels were similar between the two groups except for the post-OSCE direct observation anxiety levels, which were lower. No significant difference was found between the two groups' performance scores. Due to similar performance scores and anxiety levels for the direct examiner and camera-supervised methods, the camera-supervised OSCE warrants further exploration as an alternative approach to direct observation for evaluation of students' performance.

The aim of this study was to gain insight into the nature of the role played by self- and peer assessment in the development of dental students' reflective practice skills and the value gained through structured encounters with standardized patients. Four standardized patient encounters in an objective structured clinical exam (OSCE) format served as a learning experience for students to demonstrate decision making and communication skills in complex scenarios regarding issues of ethics. Self- and peer assessment and peer-to-peer discourse were used to enhance student reflection. A sample of 16 peer pairs was randomly selected from the population of 108 fourth-year students who participated in the 2014 Clinical Dental Ethics OSCE. Data were collected from self- and peer assessment forms. Five overall performance themes (personal affect, verbal communication, professional demeanor, relationship-building, and patient management) and three student learning themes (application and knowledge, ways to change, and impressed with peer/increased confidence in self) were identified. The results showed that peer assessment ratings were consistently higher than those in the self-assessments, but overall the students deemed both their peers' and their own decision making and communication skills to be quite good. These students rated their experience with the OSCE and self- and peer assessments as positive, appreciating the importance of reflection and learning from their peers. These results provide support for the continued formative use of standardized patient OSCEs and self- and peer assessment to help students develop skills in decision making, communication, professionalism, and reflection.

The aim of this study was to evaluate whether the objective structured clinical examination (OSCE) and case presentation (CP) as forms of active assessment were effective measures of overall didactic knowledge and clinical performance in a predoctoral dental curriculum. This evaluation was conducted by statistical analysis of quality points (QP) awarded for didactic and clinical performance, CP grades, and OSCE scores for 185 students at Harvard School of Dental Medicine who graduated during the period 2010-14. As part of the requirements for graduation, each student takes three OSCEs and presents two patient cases. Data for the study were obtained from the Office of the Registrar. The results showed no direct correlation between QP and CP grades and no correlation between CP grades and OSCE scores. However, there was a correlation between OSCE scores and QP. Students with honors-level scores on any of the three OSCEs received significantly more QP than students who did not receive honors. In addition, students with passing scores on OSCEs 2 and 3 received significantly more QP than students with failing or marginal OSCE scores. Innovative formats of active assessment such as OSCEs and CPs can promote a student-centered learning environment. These data indicated that, within this study population, there was a positive association between OSCE scores and clinical and didactic performance, supporting the value of OSCEs as a means of assessment.

The purpose of this study was to evaluate the difference in grading of objective structured clinical examinations (OSCEs) at Harvard School of Dental Medicine (HSDM) by full-time faculty examiners, part-time faculty examiners, and postgraduate resident examiners. The OSCE is an evaluation of clinical competence and is used as a multidisciplinary examination at HSDM. Two examiners are selected for each of ten disciplines. Evaluators meet to review the case before the OSCE is given, and faculty examiners are given the opportunity to write exam questions based on the students' expected level of knowledge and ability. All examiners also meet on the day of the OSCE to review the case and discuss relevant issues. Students are randomly assigned to examiners and meet with one examiner at a time in each discipline during the examination. Analysis of OSCE scores on four exams given to HSDM students between 2012 and 2013 suggests that part-time faculty members tended to score students significantly higher than full-time faculty members or postgraduate residents. This may be a result of reduced contact time between students and the part-time faculty although it may also point to a need for more efforts in calibration of the part-time faculty members who take part in the OSCE.

The aim of this study was to evaluate the use of the dental school admissions interview score as a noncognitive indicator of performance in predoctoral dental education, with specific attention to whether a correlation existed between the admissions interview scores and performance on the objective structured clinical examination (OSCE). The study population consisted of all 175 students in the Harvard School of Dental Medicine (HSDM) DMD Classes of 2012 through 2016. Data on students' gender and age on entering dental school were self-reported using their applications for admission to the HSDM DMD program. Data on students' OSCE scores for three examination sessions were collected from the Office of Dental Education. The results showed that the students' interview scores did not significantly correlate with OSCE performance on any of the three exams. Performance on the first and second OSCEs did, however, correlate with performance on the third OSCE (p

The aim of this study was to examine student perceptions of the educational value of a comprehensive, multidisciplinary objective structured clinical examination (OSCE). The examination is administered after the second year of the curriculum, prior to the beginning of clinical training at the Columbia University College of Dental Medicine. The quantitative component of the study consisted of a survey administered to students who had taken the OSCE (N=78). Due to the ordinal level of data produced by the Likert-scale survey, statistical analysis was performed through calculation of the median and interquartile range (IQR). Overall, student perceptions of the educational value of the OSCE, as measured using a five-point scale (1=strongly disagree to 5=strongly agree) were positive, demonstrating students’ agreement that the exam required the ability to think critically and problem-solve (median=4, IQR=1), assessed clinically relevant skills (median=5, IQR=1), and was a learning experience (median=5, IQR=1). A statistically significant chi-square value (p

In this study, a comparison was made between the 1999–2009 graduates of the two-year structured program for foreigntrained dentists at the Schulich School of Dentistry, University of Western Ontario, Canada, and the school’s four-year program for domestic students. Both the students’ final-year course grades and their results on the National Dental Examining Board (NDEB) written examination and objective structured clinical examination (OSCE) were compared. Differences of ≥10 percent in mean final-year grades and ≥5 percent for each component of the NDEB examination were considered academically meaningful for the purpose of this analysis. Although there were no academically meaningful and statistically significant differences in the mean overall final-year course grades between the two-year and four-year classes, there were differences in individual courses in prosthodontics, oral medicine, and practice administration. There were no differences (≥10 percent) in mean course grades since 2007. An academically meaningful difference in performance was observed on the NDEB written examination in three of the eleven years. There were no meaningful differences (≥5 percent) between the two groups in any year for the NDEB OSCE. The relevance and implications of these findings to both the two-year structured program and the newly introduced alternative pathway to certification are discussed.

This study examined the reliability and predictive validity of a comprehensive, multidisciplinary objective structured clinical examination (OSCE) used to assess the transition from preclinical to clinical education in a dental school. The examination is administered prior to clinical training at the Columbia University College of Dental Medicine to assess student preparedness for patient care activity. The study population consisted of seventy-five students in the Class of 2010 and seventy students in the Class of 2011. The total reliability of the exam was assessed by Cronbach's coefficient alpha. Using polynomial regression, the predictive validity of the examination was evaluated by correlating student scores on the OSCE with clinical performance as measured by successfully completed clinical procedures during the first year of clinical training. Findings indicate a highly reliable exam (α=0.86 and α=0.80) with a moderately high correlation predicting future clinical performance (r=0.614, p

Educators in all academic disciplines have been encouraged to utilize assessment strategies to evaluate students' critical thinking. The purpose of this study was to assess the viability of the modified objective structured clinical examination (m-OSCE) to evaluate critical thinking in dental hygiene education. This evaluation utilized a convenience sample of senior dental hygiene students. Students participated in the m-OSCE in which portions of a patient case were revealed at four stations. The exam consisted of multiple-choice questions intended to measure students' ability to utilize critical thinking skills. Additionally, there was one fill-in-the-blank question and a treatment plan that was completed at the fifth station. The results of this study revealed that the m-OSCE did not reliably measure dental hygiene students' critical thinking. Statistical analysis found no satisfactory reliability within the multiple-choice questions and moderately reliable results within the treatment planning portion of the examination. In addition, the item analysis found gaps in students' abilities to transfer clinical evidence/data to basic biomedical knowledge as demonstrated through the multiple-choice questioning results. This outcome warrants further investigation of the utility of the m-OSCE, with a focus on modifications to the evaluation questions, grading rubric, and patient case.

The objective structured clinical examination (OSCE) is being widely used for assessment of skills in dental education around the world. In India, OSCE awareness is rising, and a few exploratory attempts have been made in its implementation. This article describes use of a five-station mini-OSCE for formative assessment of dental diagnostic and radiographic skills in an undergraduate curriculum. Besides gaining experience in OSCEs, the purpose of this project was to study their validity, objectivity, feasibility, acceptability to students and faculty, and impact on student performance. The mini-OSCE was found to be a fairly valid and reliable tool for formative assessment, though it required more planning, preparation, and resources than other means of assessment. A specially developed orientation module improved its feasibility. The nineteen students perceived it to be a meaningful examination and a fair method due to uniformity of tasks and time allocation; they found the scoring to be transparent and objective. The specific and immediate feedback received was appreciated by both students and faculty members.

Harvard School of Dental Medicine (HSDM) uses a hybrid problem-based approach to teaching in the predoctoral program. The objective structured clinical examination (OSCE) is a formative examination designed to assess the performance of students in the problem-based learning (PBL) curriculum. At HSDM three comprehensive examinations with OSCE components are administered during the third and fourth years of clinical training. The National Board Dental Examination (NBDE) Part II is taken in the final year of the predoctoral program. This study examines the association between the NBDE Part II and the comprehensive exams held at HSDM. Predoctoral students from the HSDM classes of 2005 and 2006 were included in this study. The outcome variable of interest was the scores obtained by students in the NBDE Part II, and the main independent variable of interest was the performance of students in the comprehensive exams (honors, pass, make-up exam to pass). The Mann-Whitney U-test was used to examine the association between the grades obtained in the each of the three comprehensive exams and the NBDE Part II scores. Multivariable linear regression analysis was also used to examine the association between the NBDE Part II scores and the comprehensive exam grades. The effect of potential confounding factors including age, sex, and race/ethnicity was adjusted. The results suggest that students who performed well in the comprehensive exams performed better on the NBDE Part II, even after adjusting for confounding factors. Future studies will examine the long-term impact of PBL on postdoctoral plans and career choices.

In order to facilitate effective tobacco cessation services within dental school clinics, it is necessary to understand the perceived barriers encountered by dental students while providing these services. The aim of this study was to identify which factors fourth-year dental students perceive to be associated with barriers to providing tobacco intervention services. A written survey was developed and completed by incoming fourth-year dental students (a convenience sample of seventy students) at the University of Iowa College of Dentistry in 2008. The survey assessed the perceived barriers to providing tobacco intervention services and related factors. Descriptive, bivariate, and linear regression analyses were conducted. The response rate was 97 percent. The most frequently reported barriers were patients' resistance to tobacco intervention services (96 percent), inadequate time available for tobacco intervention services (96 percent), and forgetting to give tobacco intervention advice (91 percent). The following variables were significantly (p

Tracking student performance in preclinical and clinical courses can be helpful in developing and refining a curriculum. Our objective was to correlate student performance on three fixed prosthodontic examinations taken by eighty junior dental students. Examinations included a knowledge-based objective structured clinical examination (OSCE), a manual skills exercise completed on a typodont (Typodont), and a competency casting exam (Casting CE) on a patient. Multiple regression analysis indicated that the OSCE and Typodont exam scores, as independent variables, were not statistically significant predictors (P=0.07; P=0.87, respectively) of Casting CE exam performance, which was the dependent variable. Correlations were weak for the OSCE (r=0.21) and nearly nonexistent for the Typodont exam(r=0.03) when compared to the Casting CE. Our results indicate a weak correlation between an OSCE-based knowledge exam measuring students' knowledge of critical errors in preparations and castings and a competency exam involving the preparation of a full veneer crown. Results also indicate virtually no correlation between a typodont preparation examination designed to provide a measure of students' clinical skill and a clinical competency exam involving the preparation of a full crown.

The purpose of this study was to assess the concurrent validity of the National Dental Examining Board of Canada (NDEB) Written Examination and Objective Structured Clinical Examination (OSCE) by correlating students' scores with their performance in the final year of the D.D.S./D.M.D. program. The subjects of this study were the 2,317 students at nine Canadian dental schools who completed both NDEB examinations between 1995 and 2000. The findings indicate positive correlations (r = 0.43 and r = 0.46, p < .001, for the written and OSCE examinations respectively) between students' examination scores and final year results. Year-to-year and school-to-school variations were minimal. These findings supported the concurrent validity of both NDEB examinations.

The Objective Structured Clinical Examination (OSCE) is becoming more widely used for performance assessment in dentistry. The department of pediatric dentistry at Baylor College of Dentistry (BCD) began incorporating the OSCE into its curriculum in 1995. This article describes the evolution of the department's use of the OSCE and its impact on teaching and the curriculum. The discussion focuses on logistics and station design, curricular content and order, student anxiety, writing and scoring exams, and curriculum assessment. BCD has found that using an OSCE-based testing format is time-consuming and labor-intensive, but provides unprecedented feedback about students' understanding and pinpoints areas of confusion. The demands of an OSCE-based testing format reveal that students can master, to the level of competency, only a finite amount of information in a given time period. The timed, interactive aspects of the OSCE create high levels of student anxiety that must be addressed. Writing and scoring OSCE items are different from traditional test items. The OSCE is a valuable mechanism to assess the students' progress toward competency. This review of the process of incorporating OSCEs into a curriculum is the foundation for future assessment of the OSCE and its use for curricular improvement.

Careful measurements of knowledge, attitude, and psychomotor and communication skills are necessary components of testing in a competency-based approach to education in dentistry. In an effort to address these requirements, Baylor College of Dentistry (BCD), Dallas, Texas, has applied Purposeful Assessment Techniques (PAT) to the Objective Structured Clinical Examination (OSCE) currently in use. PAT are those techniques that allow one to work toward development of linear measurement scales that are "person-free" and "item-free." Person-free measurement means that useful data are produced regardless of the group being measured and item-free measurement means that it does not matter which mix of items is completed over the course of an assessment. The Rasch probabilistic model and a guiding definition of Objective Measurement were used in an effort to implement PAT for the BCD OSCE. A Rasch analysis of a BCD-administered OSCE produced an item map that demonstrated the range of difficulty of items by student performance. This item map can be used to determine which items can be repeated on subsequent tests to allow for linear measurement of students9 progression through the curriculum. The movement toward PAT described in this paper demonstrates how careful and evolving measurement in dental education can be of great benefit to faculty, staff, students, and the public.

In 1994, following a request from the ten Provincial Licensing Authorities, the National Dental Examining Board of Canada (NDEB) implemented significant changes to the certification process for dentists seeking a license to practice in Canada. Prior to 1994, graduates of accredited Canadian dental programs were certified without further examination while graduates of United States and other international programs (non-Canadian, non-U.S.) were required to complete successfully a written and three-phase clinical certification examination. Changes implemented in 1994 required graduates of accredited Canadian programs to take both a Written and Objective Structured Clinical Examination (OSCE) Examination. The analysis of the results of the Written Examination for all candidates over the 1994-1996 period supports the following conclusions. There was no meaningful difference in performance of graduates across the ten Canadian dental programs; there was a small difference between the performance of graduates of Canadian and U.S. programs; and Canadian and U.S. graduates performed significantly better than graduates of other international programs. This level of candidate performance and changes to the respective accreditation processes supported the formal agreement providing reciprocal recognition of dental accreditation in Canada and the United States. As of January 1, 1997, graduates of dental programs in Canada and the United States are required to take the same certification examination while international graduates are required to complete a different certification process. These changes to the certification process were ratified by all ten Provincial Licensing Authorities, therefore maintaining a system of national portability for dental licensure in Canada that does not require preclinical or clinical board examinations for graduates of accredited North American dental programs.

The objective of this study was to compare the impact on knowledge and counseling skills of face-to-face and Internet-based oral health training programs on medical students. Participants consisted of 148 (82 percent) of the 180 invited students attending their fifth academic year at the Faculty of Medicine, University of Sao Paulo, Brasil, in 2007. The interventions took place during a three-month training period in the clinical Center for Health Promotion, which comprised part of a clerkship in Internal Medicine. The students were divided into four groups: 1) Control Group (Control), with basic intervention; 2) Brochure Group (Br), with basic intervention plus complete brochure with oral health themes; 3) Cybertutor Group (Cy), with basic intervention plus access to an Internet-based training program about oral health themes; and 4) Cybertutor + Contact Group (Cy+C), the same as Cy plus brief proactive contact with a tutor. The impact of these interventions on student knowledge was measured with pre- and post assessments, and student skills in asking and counseling about oral health were assessed with an objective structured clinical examination (OSCE). Multivariate logistic regression models were applied to identify the odds ratios of scoring above Control's medians on the final assessment and the OSCE. In the results, Cy+C performed significantly better than Control on both the final assessment (OR 9.4; 95% CI 2.7-32.8) and the OSCE (OR 5.6; 95% CI 1.9-16.3) and outperformed all the other groups. The Cy+C group showed the most significant increase in knowledge and the best skills in asking and counseling about oral health.

The Common Achievement Test (CAT) in Japan, which will be implemented in 2005, involves a medical interview that is the core task to be completed by students during an Objective Structured Clinical Examination (OSCE). Standardized/Simulated Patient instructors (SPs), posing as patients in medical interviews, are trained in standard fashion in terms of expression of symptoms as well as the emotional affect of actual patients. Institution of appropriate training programs for SP instructors in the CAT is also necessary. We trained seven individuals to function as standardized patients (in-school SPs) during a three-day SP training program described in this article. Following completion of the OSCE, we conducted a comparison study among evaluations completed by the evaluators and two types of SP instructors. We observed high correlation, according to Spearman significance testing, between scores of evaluators and those of both newly trained in-school SPs and veteran SPs who had more than five years of experience. Correlation coefficients between the veteran SPs (r=0.77) and the in-school SPs (r=0.73) were nearly identical. These results suggest that our training program for SP instructors is an effective protocol, particularly with respect to reliability and efficiency.