Search

Background: First-year medical students typically have limited exposure to patients in diverse care settings, such as rehabilitation facilities and nursing homes., Purpose: It is unknown whether students bring predetermined attitudes toward these patients, or whether attitudes are influenced by early exposure. We studied this in a new course that provides opportunities for students to interact with patients of various ages and disabilities., Methods: We conducted surveys of 1st-year medical students at the University of Rochester in the year prior to the new course and during its initial year. We used factor analysis to derive underlying dimensions of students' responses. We also investigated the impact that the course had on their perceptions., Results: In both years, we found that students conceptualize patient care along 2 affective dimensions (comfort and pleasure) and 2 attitudinal dimensions (bias and pessimism), rather than by type of disability., Conclusions: This 10-week course improved their affect toward these patient groups but had little effect on their general attitudes toward the value of caring for them.

Background: Grading committees give excessive weight to standardized-examination scores., Purpose: Understanding that biases are often ingrained in grading processes, we sought to assess the influence of a structured grading policy in limiting this effect., Methods: All 7 clerkship grading committees derived students' clinical scores while blinded to examination scores. Scores were combined to yield a final rank order, which was used to derive grade cutoffs. Logit regression was performed to assess the contribution of clinical and examination scores to final grades. Results were compared to a similar analysis where committees were not blinded to examination scores., Results: In contrast to prior findings, grading committees consistently assigned greater weight to clinical-performance scores in assigning final grades when blinded to examination scores., Conclusions: Grading committees may be unaware of the extent to which they discount clinical assessments when they are at odds with the results of standardized examinations. This can be addressed with a procedure that blinds grading committees to examination scores.

Background and Objectives: Currently available tools to measure teamwork, an essential component of primary care, are generally very resource intensive and thus cannot be administered frequently. To explore the possibility of developing a brief teamwork-assessment instrument, we first administered 29 questions about teamwork from the Practice Environment Checklist (PEC) to all members of six clinical teams in a residency outpatient practice (n=56). We found that the scale assessed a single dimension of teamwork and that a five item survey has acceptable reliability (Cronbach alpha=0.89). In a subsequent validation study among an expanded sample of clinic staff (n=89), we found that the five-item questionnaire could be completed in less than 3 minutes. It continued to have an acceptable internal consistency (Cronbach alpha=0.82) and that all five items had sizeable item-total correlations. The resulting short form of the PEC may be useful for frequent assessment of team function.

Context: Subjective rating scales for communication skills may yield more personally meaningful responses than more standardised rating schemes. It is unclear, however, whether such evaluations may be overly biased by respondents' rating styles, which may lead to unreliable measurement of examinees' communication skills., Methods: Our study involved 212 students from the classes of 2005 and 2006 at the University of Rochester School of Medicine and Dentistry. All students were rated by actors depicting standardised patients (SPs) on the same seven cases using the 19-item Rochester Communication Rating Scale (RCRS). Different students were assigned to different actors playing the same SP. We assessed the extent to which actors' personal rating styles influenced the scores they assigned to students. Main outcome measures were: between-actor variability in responses; the degree to which actors' response styles contribute to overall scores, and improvements in reliability achieved by standardising actors' ratings., Results: There were statistically significant differences between actors in their mean assigned scores. Scores aggregated over 18 separate SP cases have an expected generalisability coefficient of 0.79. If raw RCRS scores are used, a total of 27 replications of the RCRS are required to achieve a Cronbach's alpha of 0.8; standardisation reduces this number to 18., Conclusions: Although actors are variable in their use of a standardised subjective scale of communication, such differences contribute to an acceptably small proportion of the total variance if scores are combined across a large number of cases. Reliability can be markedly improved by standardising scores across raters.

Background: It is known that male and female medical students have different experiences in their clinical training., Aims: To assess whether male and female medical students change in their self-rated work habits and interpersonal habits during the first year of clinical training., Method: Longitudinal study of self- and peer-assessment among 224 medical students in 3 consecutive classes at a private US medical school. Students rated themselves on global work habits (WH) and interpersonal attributes (IA). Students also rated and were rated by 6-12 peers on the same scale., Results: In the second year of medical school, there were no differences between men and women in quartiles of self-assessed WH or IA. At the end of the third year, however, women were more likely to be in the lower quartiles of self-assessed WH (X(2) = 6.77; p = 0.03), as well as the highest quartiles of self-assessed IA (X(2) = 11.36; p = 0.003). In both years, women rated their own WH skills significantly lower than they rated their peers, while men rated themselves similarly to peers. There were no sex differences in self-assessed IA., Conclusions: Although second-year male and female medical students appear similar to one another in terms of self-assessed WH and IA, by the end of the third year women rate themselves relatively lower in WH, while men rate themselves relatively lower in IA.

Background: It is not known how well dean's letter rankings predict later performance in residency., Purpose: To assess the accuracy of dean's letter rankings to predict clinical performance in internship., Method: Participants were medical students who graduated from the University of Rochester School of Medicine and Dentistry in the classes of 2003 and 2004. In their Dean's Letter, each student was ranked as either "Outstanding" (upper quartile), "Excellent" (second quartile), "Very good" (lower 2 quartiles), or "Good" (lowest few percentile). We compared these dean's letter rankings against results of questionnaires sent to program directors 9 months after graduation., Results: Response rate to the questionnaire was 58.9% (109 of 185 eligible graduates). There were no differences in response rate across the four dean's letter ranking categories. Program directors rated students in the top two categories of dean's letter rankings significantly higher than those in the very good group. Students in all three groups were rated significantly higher than those in the good group, F (3, 105) = 13.37, p < .001. Students in the very good group were most variable in their ratings by program directors, with many receiving similarly high ratings as students in the upper 2 groups. There were no differences by gender or specialty., Conclusion: Dean's letter rankings are a significant predictor of later performance in internship among graduates of our medical school. Students in the bottom half of the class are most likely either to underperform or overperform in internship.

Background: It is not known to what extent the dean's letter (medical student performance evaluation [MSPE]) reflects peer-assessed work habits (WH) skills and/or interpersonal attributes (IA) of students., Objective: To compare peer ratings of WH and IA of second- and third-year medical students with later MSPE rankings and ratings by internship program directors., Design and Participants: Participants were 281 medical students from the classes of 2004, 2005, and 2006 at a private medical school in the northeastern United States, who had participated in peer assessment exercises in the second and third years of medical school. For students from the class of 2004, we also compared peer assessment data against later evaluations obtained from internship program directors., Results: Peer-assessed WH were predictive of later MSPE groups in both the second (F = 44.90, P < .001) and third years (F = 29.54, P < .001) of medical school. Interpersonal attributes were not related to MSPE rankings in either year. MSPE rankings for a majority of students were predictable from peer-assessed WH scores. Internship directors' ratings were significantly related to second- and third-year peer-assessed WH scores (r = .32 [P = .15] and r = .43 [P = .004]), respectively, but not to peer-assessed IA., Conclusions: Peer assessment of WH, as early as the second year of medical school, can predict later MSPE rankings and internship performance. Although peer-assessed IA can be measured reliably, they are unrelated to either outcome.

Context: Although peer-assessment appears promising as a method to assess interpersonal skills among medical students, results may be biased by method of peer selection, particularly if different kinds of classmates are assigned systematically by different methods. It is also unclear whether students with lower interpersonal skills may be more negative towards their classmates than students with higher levels of interpersonal skills and, if so, how much bias this may introduce into the results of peer assessment. It is also unclear whether low-rated students are more likely to ask to rate one another., Design, Setting, and Participants: Participants were 296 medical students from the classes of 2004, 2005 and 2006 at the University of Rochester School of Medicine and Dentistry. All participants were evaluated by between 6 and 12 classmates near the end of both their second and third years. In some classes peer-raters were assigned from class lists, while in other classes students were asked to recommend classmates to rate and be rated by., Main Outcome Measures: Mean numerical ratings of interpersonal-attributes scores received from peers, compared with mean scores simultaneously assigned to peers. We also examined students' selection of peer-raters, as well as scores received by their peer-raters., Results: For all 3 classes in both second and third years, students in the lowest quartile of received scores were significantly more likely to simultaneously assign lower scores to their peers. Among third-year students, such low-scoring students were significantly more likely to ask to rate and be rated by other low-scoring students. Among third-year students, low-rated students were more likely to rate and be rated by other low-rated students, irrespective of how rater assignments are made. Most students who were originally in the lowest quartile of received scores remained so after removing the scores assigned by low-rated raters. Among pre-clinical second-year students, however, low-rated students were no more likely than high-rated students to be rated by other low-rated students., Conclusion: Students with low levels of peer-assessed interpersonal attributes are more negative in their judgements of classmates. Third-year medical students with low levels of interpersonal attributes are more likely to be rated by other students with low levels of interpersonal attributes, irrespective of method of rater assignment. This effect was not seen in the second year of medical school. Biases in selection of raters among third-year students, however, do not appear to affect the results of peer assessment. Thus, medical schools that choose to institute peer assessment can select the method of peer-rater assignment based on practicality and logistical considerations. Students' affiliations with one another in the third year appear to be related to peer-assessed interpersonal attributes.

Peer assessment has been increasingly recommended as a way to evaluate the professional competencies of medical trainees. Prior studies have only assessed single groups measured at a single timepoint. Thus, neither the longitudinal stability of such ratings nor differences between groups using the same peer-assessment instrument have been reported previously. Participants were all members of 2 consecutive classes of medical students (n = 77 and n = 85) at the University of Rochester School of Medicine and Dentistry who completed Years 2 and 3 of medical school consecutively. All participants were evaluated by 6-12 classmates near the end of both Years 2 and 3. Main outcome measures were mean numerical ratings on peer-assessed scales of professional work habits (WH) and interpersonal attributes (IA). Both scales had high internal consistencies in both years (Cronbach's alpha 0.84-0.94). The IA and WH scales were moderately correlated with one another (r = 0.36 in Year 2, r = 0.28 in Year 3). Year 2 scores were predictive of Year 3 scores for both scales (WH: r = 0.64; IA; r = 0.62). Generalisability and decision analyses revealed that 1 class was consistently more discriminating with the WH scale, while the other was more discriminating with of the IA scale. Depending on the class, year and scale, the number of raters needed to achieve a reasonable reliability ranged between 7 and 28. Although Year 3 peer ratings were consistently higher than Year 2 peer ratings for both WH and IA, individual scores were highly correlated across the 2 years, despite the fact that different individuals were chosen as peer raters. Abilities appear to be stable between Years 2 and 3 of medical school. Groups may differ in their ability to discriminate different kinds of skills. Generalisability analysis can be used to discover these patterns within groups.