Search

Although problem-based learning (PBL) has been widely used in medical schools, few studies have attended to the assessment of PBL processes using validated instruments. This study examined reliability and validity for an instrument assessing PBL performance in four domains: Problem Solving, Use of Information, Group Process, and Professionalism. Two cohorts of medical students (N = 310) participated in the study, with 2 years of PBL evaluation data extracted from archive rated by a total of 158 faculty raters. Analyses based on generalizability theory were conducted for reliability examination. Validity was examined through following the Standards for Educational and Psychological Testing to evaluate content validity, response processes, construct validity, predictive validity, and the relationship to the variable of training. For construct validity, correlations of PBL scores with six other outcome measures were examined, including Medical College Admission Test, United States Medical Licensing Examination (USMLE) Step 1, National Board of Medical Examiners (NBME) Comprehensive Basic Science Examination, NBME Comprehensive Clinical Science Examination, Clinical Performance Examination, and USMLE Step 2 Clinical Knowledge. Predictive validity was examined by using PBL scores to predict five medical school outcomes. The highest percentage of PBL total score variance was associated with students (60%), indicating students in the study differed in their PBL performance. The generalizability and dependability coefficients were moderately high (Ep[superscript 2] = 0.68, ? = 0.60), showing the instrument is reliable for ranking students and identifying competent PBL performers. The patterns of correlations between PBL domain scores and the outcome measures partially support construct validity. PBL performance ratings as a whole significantly (p < 0.01) predicted all the major medical school achievements. The second year PBL scores were significantly higher than those of the first year, indicating a training effect. Psychometric findings provided support for reliability and many aspects of validity of PBL performance assessment using the instrument.

To determine the direction and extent to which medical student scores (as observed by small-group tutors) on four problem-based-learning-related domains change over nine consecutive blocks during a two-year period (Domains: Problem Solving/Use of Information/Group Process/Professionalism). Latent growth curve modeling is used to analyze performance trajectories in each domain of two cohorts of 1st and 2nd year students (n = 296). Slopes of the growth trajectories show similar linear increments in the first three domains. Further analysis revealed relative strong individual variability in initial scores but not in their later increments. Professionalism, on the other hand, shows low variability and has very small, insignificant slope increments. In this study, we showed that the learning domains (Problem Solving, Use of Information, and Group Process) observed during PBL tutorials are not only related to each other but also develop cumulatively over time. Professionalism, in contrast to the other domains studied, is less affected by the curriculum suggesting that this represents a stable characteristic. The observation that the PBL tutorial has an equal benefit to all students is noteworthy and needs further investigation.

Background: Before the 1970s, research into the development of clinical competence was mainly focused on general problem-solving abilities. The scope of research changed when Elstein and colleagues discovered that individual ability to solve clinical problems varies considerably across cases. It was concluded that problem solving abilities are highly dependent on domain-specific knowledge rather than on general problem solving skills. Elstein called this phenomenon "case specificity." Purpose: The finding of content specificity will be contrasted with the existence of a general clinical problem solving ability, and the relationship between preclinical knowledge and a problem solving ability will be investigated. Methods: A correlation matrix was calculated with clerkship final scores from 10 disciplines to examine the magnitude of the interrelations. A confirmatory factor analysis was applied to the corresponding covariances using structural equation modeling to investigate whether scores on finals shared any common variance across clerkships. Finally, two additional models were tested to examine the nature of the relationship between preclinical knowledge and problem solving. Results: Low to moderate correlations across clerkship disciplines were found, supporting the original findings of content specificity. Further investigation showed that in addition to specific knowledge, a general, content-independent ability is needed to perform on these examinations. Conclusions: Clinical competence, as measured in this study, is based on a "combination" of specific preclinical knowledge and a problem-solving ability. Case specificity fits perfectly well in this interactional perspective on clinical problem solving but does not explain it. The phenomenon "case specificity" is therefore not solely a result of content knowledge, but of level of experience and level of case difficulty.

Formative assessments are systematically designed instructional interventions to assess and provide feedback on students' strengths and weaknesses in the course of teaching and learning. Despite their known benefits to student attitudes and learning, medical school curricula have been slow to integrate such assessments into the curriculum. This study investigates how performance on two different modes of formative assessment relate to each other and to performance on summative assessments in an integrated, medical-school environment. Two types of formative assessment were administered to 146 first-year medical students each week over 8 weeks: a timed, closed-book component to assess factual recall and image recognition, and an un-timed, open-book component to assess higher order reasoning including the ability to identify and access appropriate resources and to integrate and apply knowledge. Analogous summative assessments were administered in the ninth week. Models relating formative and summative assessment performance were tested using Structural Equation Modeling. Two latent variables underlying achievement on formative and summative assessments could be identified; a "formative-assessment factor" and a "summative-assessment factor," with the former predicting the latter. A latent variable underlying achievement on open-book formative assessments was highly predictive of achievement on both open- and closed-book summative assessments, whereas a latent variable underlying closed-book assessments only predicted performance on the closed-book summative assessment. Formative assessments can be used as effective predictive tools of summative performance in medical school. Open-book, un-timed assessments of higher order processes appeared to be better predictors of overall summative performance than closed-book, timed assessments of factual recall and image recognition.

Although problem-based learning (PBL) has been widely used in medical schools, few studies have attended to the assessment of PBL processes using validated instruments. This study examined reliability and validity for an instrument assessing PBL performance in four domains: Problem Solving, Use of Information, Group Process, and Professionalism. Two cohorts of medical students (N = 310) participated in the study, with 2 years of PBL evaluation data extracted from archive rated by a total of 158 faculty raters. Analyses based on generalizability theory were conducted for reliability examination. Validity was examined through following the Standards for Educational and Psychological Testing to evaluate content validity, response processes, construct validity, predictive validity, and the relationship to the variable of training. For construct validity, correlations of PBL scores with six other outcome measures were examined, including Medical College Admission Test, United States Medical Licensing Examination (USMLE) Step 1, National Board of Medical Examiners (NBME) Comprehensive Basic Science Examination, NBME Comprehensive Clinical Science Examination, Clinical Performance Examination, and USMLE Step 2 Clinical Knowledge. Predictive validity was examined by using PBL scores to predict five medical school outcomes. The highest percentage of PBL total score variance was associated with students (60 %), indicating students in the study differed in their PBL performance. The generalizability and dependability coefficients were moderately high (Ep(2) = .68, ϕ = .60), showing the instrument is reliable for ranking students and identifying competent PBL performers. The patterns of correlations between PBL domain scores and the outcome measures partially support construct validity. PBL performance ratings as a whole significantly (p < .01) predicted all the major medical school achievements. The second year PBL scores were significantly higher than those of the first year, indicating a training effect. Psychometric findings provided support for reliability and many aspects of validity of PBL performance assessment using the instrument. [ABSTRACT FROM AUTHOR]

As students at the David Geffen School of Medicine at UCLA, the student authors were given the opportunity to develop their own creative projects which would be used to teach future medical students. They chose their own topics, planned and researched their projects, and then implemented the projects in interactive digital Adobe Flash files. In the first project they created interactive case-based radiology teaching files. In the second project they integrated photographic images into the existing illustrative anatomy files. Students in subsequent years have learned from these files on computers both at home and in the school's anatomy lab. The experience of creating the files served as an opportunity for hands-on learning for the student authors, both of the material and of the practice of teaching. In this paper they describe why they undertook these projects, what exactly they did, and the impact their creation had on them. The projects demonstrate that student-driven educational materials are both possible and beneficial. Furthermore, their experience has allowed them to conclude that faculty at other medical schools should consider providing students with opportunities to develop their own creative projects that contribute to the curriculum. [ABSTRACT FROM AUTHOR]

The article discusses the ethnic and racial diversity of student bodies in medical schools, examining whether such diversity prepares graduating physicians to treat a diverse society. Topics include attitudes concerning access to care, an intent to practice medicine in areas underserved by doctors, and discriminatory admission practices by medical schools. Also discussed are the attitudes of white medical students in the highest diversity quintile, racial disparities in health care quality, and the educational benefits of a racially diverse medical school.

Formative assessments are systematically designed instructional interventions to assess and provide feedback on students’ strengths and weaknesses in the course of teaching and learning. Despite their known benefits to student attitudes and learning, medical school curricula have been slow to integrate such assessments into the curriculum. This study investigates how performance on two different modes of formative assessment relate to each other and to performance on summative assessments in an integrated, medical-school environment. Two types of formative assessment were administered to 146 first-year medical students each week over 8 weeks: a timed, closed-book component to assess factual recall and image recognition, and an un-timed, open-book component to assess higher order reasoning including the ability to identify and access appropriate resources and to integrate and apply knowledge. Analogous summative assessments were administered in the ninth week. Models relating formative and summative assessment performance were tested using Structural Equation Modeling. Two latent variables underlying achievement on formative and summative assessments could be identified; a “formative-assessment factor” and a “summative-assessment factor,” with the former predicting the latter. A latent variable underlying achievement on open-book formative assessments was highly predictive of achievement on both open- and closed-book summative assessments, whereas a latent variable underlying closed-book assessments only predicted performance on the closed-book summative assessment. Formative assessments can be used as effective predictive tools of summative performance in medical school. Open-book, un-timed assessments of higher order processes appeared to be better predictors of overall summative performance than closed-book, timed assessments of factual recall and image recognition. [ABSTRACT FROM AUTHOR]

Background This study was directed at illuminating a well known phenomenon in the medical expertise literature, the ‘intermediate effect’ in clinical case recall. This robust phenomenon consists of the finding that medical students of intermediate levels of expertise outperform both experts and novices in clinical case recall after diagnosing cases. It deals in particular with the findings of OME researchers who have reported a monotonically increasing recall with level of expertise. Purpose To address possible causes for this anomaly in medical expertise and to experimentally demonstrate how data elaboration can cause expertise effects in clinical case recall. Method Expert nephrologists, intermediate level students and novices were presented with 6 medical cases under 3 different conditions: laboratory data cases without special instructions, laboratory data cases with instructions to elaborate, and cases with laboratory data and a relevant clinical context. Results Only when participants were required to elaborate on each of the information units presented to them did case recall show an expertise effect. If laboratory data are framed within the context of a patient's history and physical examination data, the ‘intermediate effect’ appears. Conclusions The instructions used in the elaboration condition seem to have induced a deeper, more detailed, analysis of the patient case. It is therefore interesting to note that these instructions only affected the recall of the experts and had no effect on the novices' or intermediates' recall. We might conclude from this that expertise effects in clinical case recall are only produced when the normal processing of patient information is disrupted. [ABSTRACT FROM AUTHOR]

A series of three annual surveys of David Geffen School of Medicine (DGSOM) at UCLA students and UCR/UCLA Thomas Haider Program in Biomedical Sciences students were administered from 2010 to 2012 to ascertain student perceptions of which anatomy pedagogy--prosection or dissection--was most valuable to them during the first year of preclinical medical education and for the entire medical school experience in general. Students were asked, "What value does gross anatomy education have in preclinical medical education?" We further asked the students who participated in both prosection and dissection pedagogies, "Would you have preferred an anatomy curriculum like the Summer Anatomy Dissection during your first year in medical school instead of prosection?" All students who responded to the survey viewed anatomy as a highly valued part of the medical curriculum, specifically referring to four major themes: Anatomy is (1) the basis for medical understanding, (2) part of the overall medical school experience, (3) a bridge to understanding pathology and physiology, and (4) the foundation for clinical skills. Students who participated in both prosection and dissection pedagogies surprisingly and overwhelmingly advocated for a prosection curriculum for the first year of medical school, not a dissection curriculum. Time efficiency was the dominant theme in survey responses from students who learned anatomy through prosection and then dissection. Students, regardless of whether interested in surgery/radiology or not, appreciated both pedagogies but commented that prosection was sufficient for learning basic anatomy, while dissection was a necessary experience in preparation for the anatomical medical specialties. This suggests that anatomy instruction should be integrated into the clinical years of medical education. [ABSTRACT FROM AUTHOR]

We describe student beliefs of how anatomy education influenced their preparation for standardized clinical assessments and clinical skills. We conducted three annual surveys of students of the David Geffen School of Medicine (DGSOM) at the University of California Los Angeles (UCLA) and students of the University of California, Riverside (UCR)/UCLA Thomas Haider Program in Biomedical Sciences from 2010 to 2012. Students were asked, "What specific knowledge or skills did you learn from your gross anatomy experience that helped you prepare for USMLE board exams, third-year clerkships, and physical examination skills?" All students who responded to the survey viewed anatomy as a highly valued part of the medical curriculum. Almost all students felt that anatomy knowledge in general was useful for their success with United States Medical Licensing Examination (USMLE) exams, how they perceived their physical exam skills, and how they perceived their preparation for third- or fourth-year clerkships. On the other hand, when asked about how the anatomy curriculum helped prepare students for fourth-year clerkships, there was a downward trend over a three-year period with each subsequent class. Although anatomy is a highly valued part of the medical school experience, students value integration of the anatomical and clinical sciences, as evidenced by a perceived diminishing value of anatomy pedagogy taught outside of clinical context with subsequent classes over the course of three years. [ABSTRACT FROM AUTHOR]