Search

Objectives: To develop a valid and reliable tool for an objective structured clinical examination (OSCE) on hoarseness. To pilot-test the feasibility by assessing residents' clinical skills in various core competencies while assessing hoarseness on a standardized patient (SP). Study Design: Educational tool development. Methods: The OSCE checklists were developed using modified Delphi technique after obtaining feedback from faculty involved in providing care to hoarseness patients. SP-based and rest stations were created to assess clinical skills. Twelve Otolaryngology- Head and Neck Surgery residents participated in the study. Video recordings of residents' performance and their written documentation were rated by faculty members. Results: The OSCE that we developed is a valid method of assessing residents' clinical skills for evaluating hoarseness. Senior residents performed better in all of the tasks such as obtaining history and performing a physical exam on an SP, ability to perform flexible laryngoscopy on a mannequin, and interpretation of radiologic findings. Internal consistency assessed by Cronbach's alpha as measure of interitem reliability was 0.92 for laryngoscopic station and 0.95 for radiology station. Conclusions: This OSCE can be effectively used for the objective assessment of clinical competency in hoarseness. Our pilot study evaluated multiple competencies on a single occasion, including medical knowledge, patient care, professionalism, and communication and interpersonal skills. Clinical competence in history taking, physical examination, flexible fiberoptic laryngoscopy, and ability to interpret radiologic findings improved with increasing year of training. This OSCE provides targeted assessment of practicebased learning and feedback for improvement of clinical performance. [ABSTRACT FROM AUTHOR]

Objectives/Hypothesis: The purpose of our study was to identify the number of attempts required to attain competency in performing flexible laryngoscopy. Study Design: Cross-sectional prospective study. Methods: Fifteen medical students were recruited to perform flexible laryngoscopy on a mannequin. Each participant was given unlimited time and attempts to perform the procedure until considered competent by the evaluator for two consecutive attempts. Three evaluators used a flexible laryngoscopy checklist to score performance on each step of the procedure. Time required to perform the procedure was recorded, as well as number of times the scope hit the mucosa. The criteria for attaining competence were achieving a minimum score of 3 out of 5 on all the items of the checklist and being deemed competent by the evaluator. Results: A total of 105 flexible laryngoscopies were performed by 15 medical students. A mean of six attempts (range, 2-17) were necessary for a medical student to become competent in performing flexible laryngoscopy. An 80% probability of becoming competent was achieved with the 14th attempt. An inverse relationship was noted between the number of times the scope hit the mucosa and the probability of being competent. The time taken to perform the procedure decreased with increasing number of attempts. Conclusions: Our results suggest that it takes six attempts on average for a novice to become competent in performing flexible laryngoscopy. This finding has implications for residency programs because it indicates the learning curve can be overcome in the laboratory rather than with patients. Laryngoscope, 2010 [ABSTRACT FROM AUTHOR]

Objectives/Hypothesis: To establish milestones toward achievement of surgical competency by using an objective assessment tool designed to measure the development of mastoidectomy skills in the operating room (OR). Study Design: Prospective longitudinal validation study. Methods: Fifty-six evaluations were conducted in the OR on nine otolaryngology residents from PGY (postgraduate year) 2 to PGY 5 over a period of 3 years. Technical performance was measured over time using a task-based checklist developed for assessment of mastoidectomy skills. Results: Three sets of technical milestones represented achievement of competency for progressively more complicated procedural steps: the first set was achieved after a mean of 6 ± 4.3 cases, the second set after 9 ± 6.7 cases (range of mean = 8–10 cases), and the third set after 13 ± 6.4 cases (range of mean = 12–14 cases). Conclusions: The acquisition of mastoidectomy skills can be integrated into surgical teaching in the OR, and this approach yields information that can aid individual skill development and program improvement. The identification of milestones in particular can help establish training benchmarks toward achievement of competency and in identifying trainees in need of remediation. [ABSTRACT FROM AUTHOR]

Objectives/Hypothesis: To identify the barriers faced by otolaryngology program directors as they implement competency-based education and assessment and to identify preferred approaches to meet these challenges as suggested by program directors. Study Design: A national survey of otolaryngology–head and neck surgery program directors. Methods: We developed a 20-item questionnaire that was distributed to 102 otolaryngology program directors through SurveyMonkey. Nonrespondents were reminded by follow-up email and phone calls. Results were analyzed by descriptive statistical analysis. Results: A total of 88 (86%) program directors responded to the survey. There was a marked discrepancy between the income received and time spent performing the duties of the program director. Program director workload was recognized as the most important barrier to the implementation of competency-based education. Creating a practical clearinghouse of existing and emerging assessment tools was given the highest rating among the approaches to meet the challenges faced by program directors. Conclusions: Program directors in otolaryngology do not have sufficient financial support, protected time, and personnel to fulfill their administrative and educational responsibilities. They should be provided with additional institutional assistance to help them achieve the goals of the Accreditation Council for Graduate Medical Education outcome project. [ABSTRACT FROM AUTHOR]

Background: Otolaryngology residency programs are required by the Accreditation Council for Graduate Medical Education to evaluate residents' operative competency. Many such tools based on the model of objective structured assessment of technical skills (OSATSs) have been developed in other surgical specialties, but no such instruments exist for otolaryngologic procedures except for tonsillectomy. Endoscopic sinus surgery (ESS) is among the most common rhinologic procedure and lends itself to objective evaluation of operative competency. The purpose of this study was to develop and test a new tool for ESS, focusing on feasibility, content and construct validity, and interrater agreement that can be used for such assessment in the operating room and the cadaver dissection course. Methods: Faculty input via the modified Delphi technique helped develop the content of a new OSATS-based instrument. The instrument underwent serial improvements based on 3 years of endoscopic sinus surgery (ESS) cadaver courses. All evaluations were used to calculate construct validity while paired observations were used to determine interrater agreement. Regional and national faculty input was incorporated for increasing generalizability. Internal consistency was calculated using Cronbach's alpha. Results: A total of 51 assessments were completed for 28 residents who were evaluated by 15 faculty members as they performed ESS on cadavers over a period of 3 years. A high degree of internal consistency (0.99) and feasibility was noted for the instrument, which took 7 minutes to complete. The interrater agreement improved with focused faculty development for the 3rd year of the course. Conclusion: Our results and experience suggest that a feasible, reliable, and valid instrument for objective evaluation of operative competency can be developed for ESS. Further experience at other otolaryngology programs and efforts focused on faculty development will be needed to enhance faculty buy-in. The instrument can be used for formative and summative feedback as well as for identifying residents needing remediation. [ABSTRACT FROM AUTHOR]

Objectives/Hypothesis: 1) Assess current status of operative competency assessment and feedback among US Otolaryngology residency programs. 2) Evaluate correlations between assessment or feedback tools and remediation. Study Design: Survey of Program Directors. Methods: The survey was sent to 100 Program Directors inquiring about program size, mode of feedback, assessment tools, surgical portfolios, remediation, and number of residents requiring remediation in past 5 years. Associations between assessment questions and remediation (yes or no) were statistically assessed with the χ2 goodness-of-fit test. For more complicated analyses, multiple logistic regression models were created to assess predictors of the binary outcome of radiation. Results: Among 72 respondents, subjective evaluations are used by 95.8% (69 of 72). Formative feedback after cases is used by 38.9% (28 of 72). Summative feedback at end of the rotation is done by 57.0% (41 of 72). Objective Structured Assessments of Technical Skills is used by 15.3% (11 of 72) of programs, most are large on the basis of number of residents or faculty or both. Operative portfolios are kept by 48.6% (35 of 72) of programs. The percentage of programs with mechanisms for remediation is 41.7% (30 of 72). Similarly, of the programs who responded to the remediation question, the percentage with at least one resident requiring remediation is 37.5%. Programs who report at least one resident remediation are statistically more likely to have a remediation mechanism in place (75% vs. 27.5%; P <.001). There is also a statistically significant association with formative feedback and having resident remediation (58.3% vs. 30%; P = .025). Conclusions: Standardized surgical curricula and assessment tools are needed. Programs with more intensive evaluation or remediation mechanisms are statistically more likely to report residents requiring remediation. This may reflect their ability to better identify the struggling resident. Alternatively, having a resident requiring remediation may be the catalyst that initiates the need for more intensive evaluation tools and remediation mechanisms. The Accreditation Council for Graduate Medical Education's and society will demand that we produce competent surgeons. Therefore, all Otolaryngology training programs should be developing and implementing formal surgical assessment tools, methods to identify residents requiring remediation, and remediation mechanisms. [ABSTRACT FROM AUTHOR]