Search

Although problem-based learning (PBL) has been widely used in medical schools, few studies have attended to the assessment of PBL processes using validated instruments. This study examined reliability and validity for an instrument assessing PBL performance in four domains: Problem Solving, Use of Information, Group Process, and Professionalism. Two cohorts of medical students (N = 310) participated in the study, with 2 years of PBL evaluation data extracted from archive rated by a total of 158 faculty raters. Analyses based on generalizability theory were conducted for reliability examination. Validity was examined through following the Standards for Educational and Psychological Testing to evaluate content validity, response processes, construct validity, predictive validity, and the relationship to the variable of training. For construct validity, correlations of PBL scores with six other outcome measures were examined, including Medical College Admission Test, United States Medical Licensing Examination (USMLE) Step 1, National Board of Medical Examiners (NBME) Comprehensive Basic Science Examination, NBME Comprehensive Clinical Science Examination, Clinical Performance Examination, and USMLE Step 2 Clinical Knowledge. Predictive validity was examined by using PBL scores to predict five medical school outcomes. The highest percentage of PBL total score variance was associated with students (60%), indicating students in the study differed in their PBL performance. The generalizability and dependability coefficients were moderately high (Ep[superscript 2] = 0.68, ? = 0.60), showing the instrument is reliable for ranking students and identifying competent PBL performers. The patterns of correlations between PBL domain scores and the outcome measures partially support construct validity. PBL performance ratings as a whole significantly (p < 0.01) predicted all the major medical school achievements. The second year PBL scores were significantly higher than those of the first year, indicating a training effect. Psychometric findings provided support for reliability and many aspects of validity of PBL performance assessment using the instrument.

To determine the direction and extent to which medical student scores (as observed by small-group tutors) on four problem-based-learning-related domains change over nine consecutive blocks during a two-year period (Domains: Problem Solving/Use of Information/Group Process/Professionalism). Latent growth curve modeling is used to analyze performance trajectories in each domain of two cohorts of 1st and 2nd year students (n = 296). Slopes of the growth trajectories show similar linear increments in the first three domains. Further analysis revealed relative strong individual variability in initial scores but not in their later increments. Professionalism, on the other hand, shows low variability and has very small, insignificant slope increments. In this study, we showed that the learning domains (Problem Solving, Use of Information, and Group Process) observed during PBL tutorials are not only related to each other but also develop cumulatively over time. Professionalism, in contrast to the other domains studied, is less affected by the curriculum suggesting that this represents a stable characteristic. The observation that the PBL tutorial has an equal benefit to all students is noteworthy and needs further investigation.

Clinical rotations play an important role in the medical curriculum and are considered crucial for student learning. However, competencies that should be learned can differ from those that are assessed. In order to explore which competencies are considered important for daily performance of student on the wards and to what extent clinical teachers consider the same competencies important for clerkship grading, a survey that consisted of 21 different student characteristics was administered to clinical teachers. Two independent factor analyses using structural equation modeling were conducted to abstract underlying latent relationships among the different student characteristics and to define a clinical competence profile for daily performance of students on the wards and clerkship grading. Differences between the degree of importance for student daily ward performance and clerkship grading are considered and discussed. The results of the survey indicate that the degree of importance of competencies are rated different for daily performance of students on the wards and clerkship grades. Competencies related to the diagnostic process are more important for clerkship grading, whereas interpersonal skills, professional qualities, and motivation are more important for daily ward performance. It is concluded that the components of clinical competence considered important for adequate performance are not necessarily in alignment with what is required for grading. Future research should focus on an explanation why clinical educators think differently about the importance of competencies for student examination in contrast to what is required for adequate daily performance on the wards.

Background: Before the 1970s, research into the development of clinical competence was mainly focused on general problem-solving abilities. The scope of research changed when Elstein and colleagues discovered that individual ability to solve clinical problems varies considerably across cases. It was concluded that problem solving abilities are highly dependent on domain-specific knowledge rather than on general problem solving skills. Elstein called this phenomenon "case specificity." Purpose: The finding of content specificity will be contrasted with the existence of a general clinical problem solving ability, and the relationship between preclinical knowledge and a problem solving ability will be investigated. Methods: A correlation matrix was calculated with clerkship final scores from 10 disciplines to examine the magnitude of the interrelations. A confirmatory factor analysis was applied to the corresponding covariances using structural equation modeling to investigate whether scores on finals shared any common variance across clerkships. Finally, two additional models were tested to examine the nature of the relationship between preclinical knowledge and problem solving. Results: Low to moderate correlations across clerkship disciplines were found, supporting the original findings of content specificity. Further investigation showed that in addition to specific knowledge, a general, content-independent ability is needed to perform on these examinations. Conclusions: Clinical competence, as measured in this study, is based on a "combination" of specific preclinical knowledge and a problem-solving ability. Case specificity fits perfectly well in this interactional perspective on clinical problem solving but does not explain it. The phenomenon "case specificity" is therefore not solely a result of content knowledge, but of level of experience and level of case difficulty.

Formative assessments are systematically designed instructional interventions to assess and provide feedback on students' strengths and weaknesses in the course of teaching and learning. Despite their known benefits to student attitudes and learning, medical school curricula have been slow to integrate such assessments into the curriculum. This study investigates how performance on two different modes of formative assessment relate to each other and to performance on summative assessments in an integrated, medical-school environment. Two types of formative assessment were administered to 146 first-year medical students each week over 8 weeks: a timed, closed-book component to assess factual recall and image recognition, and an un-timed, open-book component to assess higher order reasoning including the ability to identify and access appropriate resources and to integrate and apply knowledge. Analogous summative assessments were administered in the ninth week. Models relating formative and summative assessment performance were tested using Structural Equation Modeling. Two latent variables underlying achievement on formative and summative assessments could be identified; a "formative-assessment factor" and a "summative-assessment factor," with the former predicting the latter. A latent variable underlying achievement on open-book formative assessments was highly predictive of achievement on both open- and closed-book summative assessments, whereas a latent variable underlying closed-book assessments only predicted performance on the closed-book summative assessment. Formative assessments can be used as effective predictive tools of summative performance in medical school. Open-book, un-timed assessments of higher order processes appeared to be better predictors of overall summative performance than closed-book, timed assessments of factual recall and image recognition.

A study was conducted to establish the relationship between student body racial and ethnic composition with diversity-related outcomes in medical schools in the US. Results confirmed the hypothesis of a relationship between the two.

Background: The logistical necessity of students taking required clinical clerkship rotations in non-uniform sequences may have significant consequences on performance. Aims: To investigate (1) the impact of previous clinical clerkship experience and the national licensing examination (USMLE Step 1) rankings on end-of-clerkship overall skills assessments and (2) the effect of clinical clerkship order on end-of-year objective structured clinical examination (OSCE) performance. Method: The core clinical phase of UCLA consists at six clinical clerkships during a 48-week continuum. The clerkships are structured in two different tracks with different rotation order. Clinical performance scores were analyzed for 124 year-3 students in the two different tracks. Results: Performance, as reviewed on a standardized clinical clerkship evaluation form by faculty and residents, improved significantly across the clinical year, regardless of track suggesting an accumulated advantage of previous experiences. There were no significant differences between tracks on the OSCE. Conclusions: Overall clinical skills are more directly impacted by the time-of-year and not order whereas knowledge of specific specialties may be impacted by the order rotations that are taken. While this may not be obvious to students, perhaps we should share these results for their use in tracking their personal growth in clinical skills. [ABSTRACT FROM AUTHOR]

Background: National statistics reveal that efforts to reduce medical student mistreatment have been largely ineffective. Some hypothesize that as supervisors gain skills in professionalism, medical students become more sensitive. Aims: The purpose of this study was to determine if medical student perceptions of mistreatment are correlated with mistreatment sensitivity. Method: At the end of their third year, 175 medical students completed an Abuse Sensitivity Questionnaire, focused on student assessment of hypothetical scenarios which might be perceived as abusive, and the annual Well-Being Survey, which includes measurement of incident rates of mistreatment. It was hypothesized that those students who identified the scenarios as abusive would also be more likely to perceive that they had been mistreated. Results: Student perceptions of mistreatment were not statistically correlated with individual's responses to the scenarios or to a statistically derived abuse sensitivity variable. There were no differences in abuse sensitivity by student age or ethnicity. Women were more likely than men to consider it 'harsh' to be called incompetent during rounds ( p < 0.0005). Conclusion: This study provides preliminary evidence that challenges the hypothesis that medical students who perceive mistreatment by their superiors are simply more sensitive. [ABSTRACT FROM AUTHOR]

As students at the David Geffen School of Medicine at UCLA, the student authors were given the opportunity to develop their own creative projects which would be used to teach future medical students. They chose their own topics, planned and researched their projects, and then implemented the projects in interactive digital Adobe Flash files. In the first project they created interactive case-based radiology teaching files. In the second project they integrated photographic images into the existing illustrative anatomy files. Students in subsequent years have learned from these files on computers both at home and in the school's anatomy lab. The experience of creating the files served as an opportunity for hands-on learning for the student authors, both of the material and of the practice of teaching. In this paper they describe why they undertook these projects, what exactly they did, and the impact their creation had on them. The projects demonstrate that student-driven educational materials are both possible and beneficial. Furthermore, their experience has allowed them to conclude that faculty at other medical schools should consider providing students with opportunities to develop their own creative projects that contribute to the curriculum. [ABSTRACT FROM AUTHOR]

A series of three annual surveys of David Geffen School of Medicine (DGSOM) at UCLA students and UCR/UCLA Thomas Haider Program in Biomedical Sciences students were administered from 2010 to 2012 to ascertain student perceptions of which anatomy pedagogy--prosection or dissection--was most valuable to them during the first year of preclinical medical education and for the entire medical school experience in general. Students were asked, "What value does gross anatomy education have in preclinical medical education?" We further asked the students who participated in both prosection and dissection pedagogies, "Would you have preferred an anatomy curriculum like the Summer Anatomy Dissection during your first year in medical school instead of prosection?" All students who responded to the survey viewed anatomy as a highly valued part of the medical curriculum, specifically referring to four major themes: Anatomy is (1) the basis for medical understanding, (2) part of the overall medical school experience, (3) a bridge to understanding pathology and physiology, and (4) the foundation for clinical skills. Students who participated in both prosection and dissection pedagogies surprisingly and overwhelmingly advocated for a prosection curriculum for the first year of medical school, not a dissection curriculum. Time efficiency was the dominant theme in survey responses from students who learned anatomy through prosection and then dissection. Students, regardless of whether interested in surgery/radiology or not, appreciated both pedagogies but commented that prosection was sufficient for learning basic anatomy, while dissection was a necessary experience in preparation for the anatomical medical specialties. This suggests that anatomy instruction should be integrated into the clinical years of medical education. [ABSTRACT FROM AUTHOR]

We describe student beliefs of how anatomy education influenced their preparation for standardized clinical assessments and clinical skills. We conducted three annual surveys of students of the David Geffen School of Medicine (DGSOM) at the University of California Los Angeles (UCLA) and students of the University of California, Riverside (UCR)/UCLA Thomas Haider Program in Biomedical Sciences from 2010 to 2012. Students were asked, "What specific knowledge or skills did you learn from your gross anatomy experience that helped you prepare for USMLE board exams, third-year clerkships, and physical examination skills?" All students who responded to the survey viewed anatomy as a highly valued part of the medical curriculum. Almost all students felt that anatomy knowledge in general was useful for their success with United States Medical Licensing Examination (USMLE) exams, how they perceived their physical exam skills, and how they perceived their preparation for third- or fourth-year clerkships. On the other hand, when asked about how the anatomy curriculum helped prepare students for fourth-year clerkships, there was a downward trend over a three-year period with each subsequent class. Although anatomy is a highly valued part of the medical school experience, students value integration of the anatomical and clinical sciences, as evidenced by a perceived diminishing value of anatomy pedagogy taught outside of clinical context with subsequent classes over the course of three years. [ABSTRACT FROM AUTHOR]

Context: The finding of case or content specificity in medical problem solving moved the focus of research away from generalisable skills towards the importance of content knowledge. However, controversy about the content dependency of clinical performance and the generalisability of skills remains., Objectives: This study aimed to explore the relative impact of both perspectives (case specificity and generalisable skills) on different components (history taking, physical examination, communication) of clinical performance within and across cases., Methods: Data from a clinical performance examination (CPX) taken by 350 Year 3 students were used in a correlated traits-correlated methods (CTCM) approach using confirmatory factor analysis, whereby 'traits' refers to generalisable skills and 'methods' to individual cases. The baseline CTCM model was analysed and compared with four nested models using structural equation modelling techniques. The CPX consisted of three skills components and five cases., Results: Comparison of the four different models with the least-restricted baseline CTCM model revealed that a model with uncorrelated generalisable skills factors and correlated case-specific knowledge factors represented the data best. The generalisable processes found in history taking, physical examination and communication were responsible for half the explained variance, in comparison with the variance related to case specificity. Conclusions Pure knowledge-based and pure skill-based perspectives on clinical performance both seem too one-dimensional and new evidence supports the idea that a substantial amount of variance contributes to both aspects of performance. It could be concluded that generalisable skills and specialised knowledge go hand in hand: both are essential aspects of clinical performance.

Today's medical student curriculum is a lock-step experience that provides a broad survey of medicine with little opportunity to pursue fully integrated, in-depth learning. To teach students about the human dimensions of health care, many schools simply have added courses that survey general areas such as ethics, values, and patient-doctor relationships. However, a superficial, broad-brush approach does not offer students sufficient opportunity to engage with these topics in substantive and meaningful ways. The authors propose that a theme-based, individualized, in-depth learning experience (in which students pursue a focused project comprehensively and in detail)--one that is an integral part of the curriculum--helps students learn to blend values and ethics with medicine in a way that cannot occur during rapid-paced topical survey courses. Furthermore, it is in the depths of a learning experience that one comes face to face with the realities of uncertainty: the realization that unanswerable questions outnumber answerable ones; the awareness of the difficulty in accumulating sufficient evidence to answer a question that is, in fact, answerable; the recognition that many patients' problems transcend available evidence and must be addressed by the art of medicine; the realization that a patient can have a condition that one cannot diagnose and that may even get better for reasons that one cannot understand. The authors describe three initiatives at the University of Pittsburgh School of Medicine, two of which have been offered for more than 10 years, that illustrate the value of in-depth learning experiences. These in-depth experiences blend situated learning, reflective exercises, faculty mentoring, critical reading of literature, and constructive feedback in a prescribed but individualized curriculum.