Search

SUMMARY Although the assessment of medical students’ clinical skills through the use of an Objective Structured Clinical Examination (OSCE) is becoming more widespread among American medical schools, there is still a signi® cant number of medical schools that do not utilize this type of examination. The reasons for this are many, including considerations about expense, commitment of faculty time and the logistics of setting up and administering an OSCE program. Although studies have been published on various aspects of the OSCE, there have been very few overviews on how to develop this type of examination from initial needs assessment to implementation. This paper describes the process of developing an OSCE at a American medical school of intermediate size. The development process is divided into seven activity areas: initial preparation, case writing, developing a standardized patient program, procedures for conducting the exam, faculty participation, measurement and evaluation, and ® nancial support. Results of innovations that were tried during the pilot phase of development are discussed.

The Maternal-Infant Clinical Nurse Specialist Performance Evaluation Tool was used to evaluate the clinical performance of two classes of graduate students (n = 7, n = 4) enrolled in maternal-infant clinical practicums. When used by raters very familiar with the tool, i.e., those participating in its development, it yielded valid (.90) and reliable (.93) ratings of performance. When used by raters with limited familiarity with the tool, both validity and reliability declined but were still quite acceptable (.68 and .81, respectively). Better training for such raters should produce even more useful results. These results warrant continued cautious, monitored use of the tool in maternal-infant nursing. The generic content of the tool offers the possibility of its generalization to other specialties.

The proposed theory provides a basis for both measuring and correcting rater stringency error in some grossly incomplete rating data matrices. The theoretical model fit (R = .92, .85, .82; joint p less than .000001) ratings made by faculty and resident physicians (n = 47, 31, and 29) of student clinical performance in each of three junior year medical student cohorts (n = 29, 30, and 35) better than alternative models. In these data the percentage of variation attributable to stringency and ability was about 35 and 40, respectively. Three-month test-retest reliability for rater stringencies was .16 less than r less than .29 (joint p less than .04). Cross-validation supported the proposed model (r = .61) over the conventional alternative (r = .41; z = 2.62, p less than .004). Both reliability and convergent validity of the ability construct were .20 greater for one corrected rating than for one observed (uncorrected) rating.

Application of a rating theory model to reviews of abstracts submitted to an international research conference resulted in good fit with the data and detection of significant interreviewer differences in reviewer stringency. Computation of calibrated ratings (i.e., those from which reviewer differences had been removed) permitted selection of abstracts that reflected more accurately the true quality of each abstract without regard to who happened to have reviewed it and thus enhanced both the validity and reliability of the peer review process.

The considerable amount of time that medical students spend in clinical encounters with patients implies an expectation of many educators and clinicians, based on little reported research, that cognitive knowledge will increase as a result. In this study, 119 junior medical students were assigned to various paediatric subspecialty clinics. Using a correlated t-test, students' performances on those written examination items that corresponded to subspecialty clinics attended were compared with their performances on items for clinics not attended. Attendance in the clinics was not found to enhance mean test performances significantly (t = 0.8; df = 113; P less than 0.21). However, significantly greater variation in test performance was observed for clinics attended than those not attended (t = 581.3; df = 112; P less than 0.001). Discussion includes possible reasons for these results, other benefits the students may have received, and implications for clinic teaching.

A quasi-experimental study was conducted to examine the effects of information about an impending threatening event (barium enema) on subjects' expectations and the intensity of their emotional response to the event. Twenty subjects were assigned to one of two information conditions: sensation of procedure. Prior to receiving any information, all subjects completed a preinformation sensation inventory and the trait portion of the State-Trait Anxiety Inventory (STAI). A taped message of either sensation or procedural information was heard, followed by completion of the post-information sensation inventory. After the barium enema, all subjects completed the post-procedure sensation inventory and the state portion of the STAI. In neither the expected sensations (expectations) reported prior to hearing information or the responses provided after the procedure (actual experiences) were there significant differences between the two groups. However, after hearing the information, significant differences in reported expected sensations were found. Subjects receiving sensation information reported significantly less anxiety and expectations more congruent with their actual experiences than did subjects receiving procedural information.