Search

Objective: To compare postpartum hemorrhage (PPH) patient outcomes before and after an in situ instructional design based PPH simulation attended by obstetrics and gynecology (OBGYN) residents.Methods: This uncontrolled before-and-after study was conducted in Recife, Brazil including all 1388 women delivering from June to August 2012 and all 1357 delivering from June to August 2013. The 36 OBGYN residents were divided into13 teams of two or three participants and were trained through ID based PPH simulation training with the following eight steps: (1) prior knowledge activation, (2) video demonstration, (3) dual-coding PPH protocol discussion-an image association during the training, (4) training scenario # 1, (5) debriefing, (6) training scenario # 2 with immediate feedback, (7) training scenario # 3, and (8) debriefing with self-assessment. The training scenarios had an increasing level of complexity. The main goal of the training was the adequate management of PPH and situational awareness improvement-the ability to anticipate, recognize, and intercept unfolding error chains. The primary patient outcomes rates used for the before and after comparison were therapeutic uterotonics use within 24 h of birth and blood transfusion. Secondary outcomes were therapeutic oxytocin mean dosage IU within 24 h of birth, postpartum Hb < 6 g/dL, among others. Chi-square test was used for categorical variables comparison and independent t-test for continuous variables.Results: PPH rates were 100 (7.2% of 2012 deliveries) and 80 cases (5.9% of 2013 deliveries), respectively. Comparison of primary post- and pre-simulation outcomes revealed no significant differences. However, in the comparison for therapeutic oxytocin mean dosage IU within 24 h of birth, there was an increase found after the simulation (15.98 ± 7.4 versus 25.1 ± 12.3; p < .001). For all other outcome measures, there were no statistical differences.Conclusions:In situ ID based PPH simulation leads to an increase in the mean dosage of oxytocin after training, in selected cases. This may indicate better situational awareness when managing women with PPH. [ABSTRACT FROM AUTHOR]

Introduction: A new form of internationalization has been trending upward in the medical education realm: crossborder medical curriculum partnerships established to deliver the same, or adapted, curriculum to groups of geographically separated students. This study aims to investigate crossborder medical curriculum partnerships by exploring the experiences of teachers at the recipient institution who have a key role in delivering the program. Methods: From four pioneering recipient medical schools, 24 teachers participated in a Q-sort study. Each participant rank-ordered 42 statements about teaching in a crossborder medical curriculum on a scale from −5 (indicating strong disagreement) to +5 (indicating strong agreement). The authors conducted a "by-person" factor analysis to uncover distinct patterns in the ranking of statements, using the statistical results and participants' comments about their Q sorts to interpret these patterns and translate them into distinct viewpoints. Results: Three viewpoints emerged, reflecting: (1) a feeling of connectedness with the partner institution, trust in the quality of the curriculum, and appreciation of interinstitutional relationships; (2) the partnership's attractiveness because of the career opportunities it offers; and (3) concerns over the quality of graduates because of doubts about the appropriateness of the didactic model and insufficient attention to local healthcare needs, and over the practical feasibility of such partnerships. Conclusions: The three viewpoints identified revealed a pallet of views on how host teachers might experience their work. It shows the heterogeneous features of this group and seems to counterbalance reports that they are feeling "deprived" from their role as teacher. Two viewpoints featured an appreciation of interinstitutional relationships and of the partnership, especially when perceiving a degree of autonomy. Partners can capitalize on all different viewpoints by deploying procedure and policies to raise the quality of education delivery. [ABSTRACT FROM AUTHOR]

Background:It remains unclear which item format would best suit the assessment of clinical reasoning: context-rich single best answer questions (crSBAs) or key-feature problems (KFPs). This study compared KFPs and crSBAs with respect to students’ acceptance, their educational impact, and psychometric characteristics when used in a summative end-of-clinical-clerkship pediatric exam. Methods:Fifth-year medical students (n = 377) took a computer-based exam that included 6–9 KFPs and 9–20 crSBAs which assessed their clinical reasoning skills, in addition to an objective structured clinical exam (OSCE) that assessed their clinical skills. Each KFP consisted of a case vignette and three key features using a “long-menu” question format. We explored students’ perceptions of the KFPs and crSBAs in eight focus groups and analyzed statistical data of 11 exams. Results:Compared to crSBAs, KFPs were perceived as more realistic and difficult, providing a greater stimulus for the intense study of clinical reasoning, and were generally well accepted. The statistical analysis revealed no difference in difficulty, but KFPs resulted more reliable and efficient than crSBAs. The correlation between the two formats was high, while KFPs correlated more closely with the OSCE score. Conclusions:KFPs with long-menu exams seem to bring about a positive educational effect without psychometric drawbacks. [ABSTRACT FROM PUBLISHER]

Background:Progress tests (PT) are used to assess students on topics from all medical disciplines. Progress testing is usually one of the assessment methods of the cognitive domain. There is limited knowledge on how positioning of the PT in a program of assessment (PoA) influences students’ PT scores, use of PT feedback and perceived learning value. Methods:We compared PT total scores and use of a PT test feedback (ProF) system in two medical courses, where the PT is either used as a summative assessment or embedded in a comprehensive PoA and used formatively. In addition, an interview study was used to explore the students’ perception on use of PT feedback and perceived learning value. Results:PT total scores were higher, with considerable effect sizes (ESs) and students made more use of ProF when the PT was embedded in a comprehensive PoA. Analysis of feedback in the portfolio stimulated students to look for patterns in PT results, link the PT to other assessment results, follow-up on learning objectives, and integrate the PT in their learning for the entire PoA. Conclusions:Embedding the PT in an assessment program designed according to the principles of programmatic assessment positively affects PT total scores, use of PT feedback, and perceived learning value. [ABSTRACT FROM PUBLISHER]

Background:Portfolios are used as tools to coach and assess students in the workplace. This study sought to evaluate the content validity of portfolios as reflected in their capacity to adequately assess achieved competences of medical students during clerkships. Methods: We reviewed 120 workplace portfolios at three medical universities (Belgium and the Netherlands). To validate their content, we developed a Validity Inventory for Portfolio Assessment (VIPA) based on the CanMEDS roles. Two raters evaluated each portfolio and indicated for each VIPA item whether the portfolio provided sufficient information to enable satisfactory assessment of the item. We ran a descriptive analysis on the validation data and computed Cohen’s Kappa to investigate interrater agreement. Results:The portfolios adequately covered the items pertaining to the communicator (90%) and professional (87%) roles. Coverage of the medical expert, collaborator, scholar and manager roles ranged between 75% and 85%. The health advocate role, covering 59%, was clearly less well represented. This role also exhibited little interrater agreement (Kappa < 0.4). Conclusions:This study lends further credence to the evidence that portfolios can indeed adequately assess the different CanMEDS roles during clerkships, the health advocate role, which was less well represented in the portfolio content, excepted. [ABSTRACT FROM PUBLISHER]

Background: Teachers’ conceptions of learning and teaching are important for faculty development to result in enduring changes in teaching practices. Until now, studies on these conceptions have mostly focused on traditional, lecture-based curricula rather than on small-group student-centred educational formats, which are gaining ground worldwide. Aim: To explore which factors predict teachers’ conceptions in student-centred curricula. Methods: In two Dutch medical schools with 10 and 40 years of student-centred education, teachers were asked to fill out the Conceptions of Learning and Teaching (COLT) Questionnaire to assess their ‘teacher-centredness’, ‘appreciation of active learning’ and ‘orientation to professional practice’. Next, we quantitatively assessed the relations of teachers’ conceptions with their personal and occupational characteristics and institute. Results: Overall response was 49.4% (N = 319/646). Institute was the main predictor for variance in all three scales, and discipline, gender and teaching experience significantly explained variance in two of the scales. More than 80% of the variance was not explained by these factors. Conclusion: Longer exposure to a student-centred curriculum was associated with fewer teacher-centred conceptions, greater ‘appreciation of active learning’ and stronger ‘orientation towards professional practice’. In line with studies on lecture-based curricula, discipline, gender and teaching experience also appeared important for teachers’ conceptions in student-centred curricula. More research is necessary to better understand the influence of institute on the three teachers’ conceptions scales. [ABSTRACT FROM PUBLISHER]

Morning reports offer opportunities for intensive work-based learning. In this controlled study, we measured learning processes and outcomes with the report of paediatric emergency room patients. Twelve specialists and 12 residents were randomised into four groups and discussed the same two paediatric cases. The groups differed in their presentation modality (verbal only vs. verbal + text) and the use of buzz groups (with vs. without). The verbal interactions were analysed for clinical reasoning processes. Perceptions of learning and judgment of learning were reported in a questionnaire. Diagnostic accuracy was assessed by a 20-item multiple-choice test. Combined bimodal presentation and buzz groups increased the odds ratio of clinical reasoning to occur in the discussion of cases by a factor of 1.90 (p = 0.013), indicating superior reasoning for buzz groups working with bimodal materials. For specialists, a positive effect of bimodal presentation was found on perceptions of learning (p < 0.05), and for residents, a positive effect of buzz groups was found on judgment of learning (p < 0.005). A positive effect of bimodal presentation on diagnostic accuracy was noted in the specialists (p < 0.05). Combined bimodal presentation and buzz group discussion of emergency cases improves clinicians’ clinical reasoning and learning. [ABSTRACT FROM PUBLISHER]

Background: Virtual patients (VPs) are increasingly used to train clinical reasoning. So far, no validated evaluation instruments for VP design are available. Aims: We examined the validity of an instrument for assessing the perception of VP design by learners. Methods: Three sources of validity evidence were examined: (i) Content was examined based on theory of clinical reasoning and an international VP expert team. (ii) The response process was explored in think-aloud pilot studies with medical students and in content analyses of free text questions accompanying each item of the instrument. (iii) Internal structure was assessed by exploratory factor analysis (EFA) and inter-rater reliability by generalizability analysis. Results: Content analysis was reasonably supported by the theoretical foundation and the VP expert team. The think-aloud studies and analysis of free text comments supported the validity of the instrument. In the EFA, using 2547 student evaluations of a total of 78 VPs, a three-factor model showed a reasonable fit with the data. At least 200 student responses are needed to obtain a reliable evaluation of a VP on all three factors. Conclusion: The instrument has the potential to provide valid information about VP design, provided that many responses per VP are available. [ABSTRACT FROM AUTHOR]

Background: The psychological construct of emotional intelligence (EI), its theoretical models, measurement instruments and applications have been the subject of several research studies in health professions education. Aim: The objective of the current study was to investigate the factorial validity and reliability of a bilingual version of the Schutte Self Report Emotional Intelligence Scale (SSREIS) in an undergraduate Arab medical student population. Methods: The study was conducted during April-May 2012. A cross-sectional survey design was employed. A sample ( n = 467) was obtained from undergraduate medical students belonging to the male and female medical college of King Saud University, Riyadh, Saudi Arabia. Exploratory and confirmatory factor analysis was performed using SPSS 16.0 and AMOS 4.0 statistical software to determine the factor structure. Reliability was determined using Cronbach's alpha statistics. Results: The results obtained using an undergraduate Arab medical student sample supported a multidimensional; three factor structure of the SSREIS. The three factors are Optimism, Awareness-of-Emotions and Use-of-Emotions. The reliability (Cronbach's alpha) for the three subscales was 0.76, 0.72 and 0.55, respectively. Conclusion: Emotional intelligence is a multifactorial construct (three factors). The bilingual version of the SSREIS is a valid and reliable measure of trait emotional intelligence in an undergraduate Arab medical student population. [ABSTRACT FROM AUTHOR]

Background: Emotional Intelligence (EI) is the ability to deal with your own and others emotions. Medical students are inducted into medical schools on the basis of their academic achievement. Professionally, however, their success rate is variable and may depend on their interpersonal relationships. EI is thought to be significant in achieving good interpersonal relationships and success in life and career. Therefore, it is important to measure EI and understand its correlates in an undergraduate medical student population. Aim: The objective of study was to investigate the relationship between the EI of medical students and their academic achievement (based on cumulative grade point average [CGPA]), age, gender and year of study. Methods: A cross-sectional survey design was used. The SSREIS and demographic survey were administered in the three medical schools in Saudi Arabia from April to May 2012. Results: The response rate was 30%. For the Optimism subscale, the mean score was M = 3.79, SD ± 0.54 ( α = 0.82), for Awareness-of-emotion subscale M = 3.94, SD ± 0.57 ( α = 0.72) and for Use-of-emotion subscale M = 3.92, SD ± 0.54 ( α = 0.63). Multiple regression showed a significant positive correlation between CGPA and the EI of medical students ( r = 0.246, p = 0.000) on the Optimism subscale. No correlation was seen between CGPA and Awareness of Emotions and Use of Emotions subscales. No relationship was seen for the other independent variables. Conclusion: The current study demonstrates that CGPA is the only significant predictor, indicating that Optimism tends to be higher for students with a higher CPGA. None of the other independent variables (age, year of study, gender) showed a significant relationship. [ABSTRACT FROM AUTHOR]

Background: Earlier studies suggested national culture to be a potential barrier to curriculum reform in medical schools. In particular, Hofstede's cultural dimension 'uncertainty avoidance' had a significant negative relationship with the implementation rate of integrated curricula. Aims: However, some schools succeeded to adopt curriculum changes despite their country's strong uncertainty avoidance. This raised the question: 'How did those schools overcome the barrier of uncertainty avoidance?' Method: Austria offered the combination of a high uncertainty avoidance score and integrated curricula in all its medical schools. Twenty-seven key change agents in four medical universities were interviewed and transcripts analysed using thematic cross-case analysis. Results: Initially, strict national laws and limited autonomy of schools inhibited innovation and fostered an 'excuse culture': ' It's not our fault. It is the ministry's'. A new law increasing university autonomy stimulated reforms. However, just this law would have been insufficient as many faculty still sought to avoid change. A strong need for change, supportive and continuous leadership, and visionary change agents were also deemed essential. Conclusions: In societies with strong uncertainty avoidance strict legislation may enforce resistance to curriculum change. In those countries opposition by faculty can be overcome if national legislation encourages change, provided additional internal factors support the change process. [ABSTRACT FROM AUTHOR]

Background: The Dutch postgraduate medical training has been revised to focus on seven competencies. The role as manager is one of these competencies. Recent studies show that this competency receives little attention during the residency training. In an earlier study, we discovered that residents perceived their competency as managers to be moderate. Aim: In this study, we investigated how medical specialists perceived the managerial competencies of medical residents and their need for management education. Methods: In September 2010, a 46-item questionnaire was designed which examined medical specialists' perceptions of the competency and needs of residents in the field of medical management. Two hundred ninety-eight specialists were invited via email to participate. Results: Hundred twenty-nine specialists (43.3%) responded to our survey. They rated the residents' competencies in contract negotiating skills, knowledge of the healthcare system, and specialist department poorly. They felt that residents were competent in updating their medical knowledge. Ninety-four percent reported a need for training in management among residents. Preferred topics were time management and healthcare organization. The preferred training method was a workshop given during residency by an extramural expert. Conclusion: Dutch medical specialists perceive the management competencies of residents in some areas to be inadequate. They feel that training in medical management during residency is necessary. [ABSTRACT FROM AUTHOR]

Aim: There is growing pressure from the government and the public to define proficiency standards for surgical skills. Aim of this study was to estimate the reliability of the Program for Laparoscopic Urological Skills (PLUS) assessment and to set a certification standard for second-year urological residents. Methods: Fifty participants were assessed on performance time and performance quality to investigate the reliability of the PLUS assessment. Generalisability coefficient of 0.8, on a scale of 0 to 1.0, was considered to indicate good reliability for assessment purposes. Pass/fail standards were based on laparoscopic experience: Novices, intermediates, and experts (>100 procedures). The pass/fail standards were investigated for the PLUS performances of 33 second-year urological residents. Results: Fifteen novices, twenty-three intermediates and twelve experts were included. An inter-trial reliability of >0.80 was reached with two trials for each task. Inter-rater reliability of the quality measurements was 0.79 for two judges. Pass/fail scores were determined for the novice/intermediate boundary and the intermediate/expert boundary. Pass rates for second-year residents were 63.64% and 9.09%, respectively. Conclusion: The PLUS assessment is reliable for setting a certification standard for second-year urological residents that serves as a starting point for residents to proceed to the next level of laparoscopic competency. [ABSTRACT FROM AUTHOR]

There has been increasing use and significance of progress testing in medical education. It is used in many ways and with several formats to reflect the variety of curricula and assessment purposes. These developments have occurred alongside a recognised sensitivity for error variance inherent in multiple choice tests from which challenges to its validity and reliability have arisen. This Guide presents a generic, systemic framework to help identify and explore improvements in the quality and defensibility of progress test data. The framework draws on the combined experience of the Dutch consortium, an individual medical school in the United Kingdom, and the bulk of the progress test literature to date. It embeds progress testing as a quality-controlled assessment tool for improving learning, teaching and the demonstration of educational standards. The paper describes strengths, highlights constraints and explores issues for improvement. These may assist in the establishment of potential or new progress testing in medical education programmes. They can also guide the evaluation and improvement of existing programmes. [ABSTRACT FROM AUTHOR]

Background It is often assumed that the way teachers approach their teaching is determined by the way they think about learning. This study explores how teachers of an undergraduate medical programme (UMP) think about learning, how they approach teaching and whether their conceptions of learning relate to their teaching approaches. Methods Quantitative data of academic teachers involved in the undergraduate programme in medicine were collected and analysed. We used a questionnaire designed to measure teachers'' conceptions of their own learning (COL) and of student learning as well as teachers'' approaches to teaching (AT). Results Teachers of the medical undergraduate programme hold a variety of COL, of how students learn and their AT. No significant correlations were found between teachers'' conceptions of learning and their AT. Conclusions Although UMP teachers'' ideas on learning and teaching are very diverse, some of their conceptions are interrelated. Teachers'' ideas on their own learning is sometimes -- but not always -- related to how they think about student learning. But most importantly, the way UMP teachers think about learning is not automatically converted into the way they approach teaching. [ABSTRACT FROM AUTHOR]

The potential richness of the feedback for learners and teachers is one of the educational advantages of progress tests (PTs). Every test administration yields information on a student's knowledge level in each sub-domain of the test (cross-sectional information), and it adds a next point to the corresponding knowledge growth curve (longitudinal information). Traditional paper-based feedback has severe limitations and requires considerable effort from the learners to give meaning to the data. We reasoned that the PT data should be flexibly accessible in all pathways and with any available comparison data, according to the personal interest of the learner. For that purpose, a web-based tool (Progress test Feedback, the ProF system) was developed. This article presents the principles and features of the generated feedback and shows how it can be used. In addition to enhancement of the feedback, the ProF database of longitudinal PT-data also provides new opportunities for research on knowledge growth, and these are currently being explored. [ABSTRACT FROM AUTHOR]

In this paper, staff members' perceptions about the organisational culture are measured. The questions addressed are: what are their opinions about the current and preferred organisational culture? Are there differences between the current and preferred situation? Do the perceptions differ per department? The Organisational Culture Assessment Instrument was sent to staff involved in 18 different departments within Dutch universities of applied science. The results demonstrated that both a flexibility- and control-oriented culture were moderately experienced in practice. In addition, significant differences were found between the current and preferred situation as well as a significant variation over departments for the current situation. Staff members in general, prefer a flexibility-oriented culture to a control-oriented culture. These findings imply that the organisational culture in many departments is not yet fully in line with the staff members' preferences. [ABSTRACT FROM AUTHOR]

Background and aims: The learning environment of undergraduate research internships has received little attention, compared to postgraduate research training. This study investigates students' experiences with research internships, particularly the quality of supervision, development of research skills, the intellectual and social climate, infrastructure support, and the clarity of goals and the relationship between the experiences and the quality of students' research reports and their overall satisfaction with internships. Method: A questionnaire (23 items, a 5-point Likert scale) was administered to 101 Year five veterinary students after completion of a research internship. Multiple linear regression analyses were conducted with quality of supervision, development of research skills, climate, infrastructure and clarity of goals as independent variables and the quality of students' research reports and students' overall satisfaction as dependent variables. Results: The response rate was 79.2%. Students' experiences are generally positive. Students' experiences with the intellectual and social climate are significantly correlated with the quality of research reports whilst the quality of supervision is significantly correlated with both the quality of research reports and students' overall satisfaction with the internship. Conclusion: Both the quality of supervision and the climate are found to be crucial factors in students' research learning and satisfaction with the internship. [ABSTRACT FROM AUTHOR]

Background: Changing a curriculum raises the question whether the results, new curriculum student outcomes, are different from old curriculum student outcomes. Aims: To see whether different curricula produce different outcomes, we compared test and questionnaire results of two cohorts. We wanted to know if there is a difference on knowledge and skills test results, and on the number of times students practiced their skills during their final internships. Method: We used two validated test instruments: the Dutch Progress Test (PT) and the Objective Structured Clinical Examination (OSCE). For reporting their skills practice, we asked students of both cohorts to fill out a basic skills questionnaire. Results: We found no significant difference between both cohorts on the results of their knowledge test and their report on skills practice. On the OSCE, students from the new curriculum cohort scored significantly higher than old curriculum students. Conclusion: Curriculum change can lead to improvements in graduating students' outcome results. [ABSTRACT FROM AUTHOR]

Background: Introducing innovative curricular designs can be evaluating by scrutinizing the learning patterns students use. Aim: Studying the potential of Vermunt's Inventory of Learning Styles (ILS) in detecting differences in student learning patterns in different medical curricula. Methods: Cross-sectional between-subjects comparison of ILS-scores in third-year medical students in a conventional, an integrated contextual and a PBL-curriculum using one-way post hoc ANOVA. Results: Response rate was 85%: 197 conventional, 130 integrated contextual and 301 PBL students. The results show a differential impact from the three curricula. In relation to processing strategies, the students in the problem-based curriculum showed less rote learning and rehearsing, greater variety in sources of knowledge used and less ability to express study content in a personal manner than did the students in the conventional curriculum. The students of the integrated contextual curriculum showed more structuring of subject matter by integrating different aspects into a whole. In relation to regulation strategies, the students in the problem-based curriculum showed significantly more self-regulation of learning content and the students in the integrated contextual curriculum showed lower levels of regulation. As to learning orientations, the students in the problem-based curriculum showed less ambivalence and the students of the conventional curriculum were less vocationally oriented. Conclusion: The study provides empirical support for expected effects of traditional and innovative curricula which thus far were not well supported by empirical studies. [ABSTRACT FROM AUTHOR]

Background: Research indicates that the quality of supervision strongly influences the learning of medical students in clinical practice. Clinical teachers need feedback to improve their supervisory skills. The available instruments either lack a clear theoretical framework or are not suitable for providing feedback to individual teachers. We developed an evaluation instrument based on the 'cognitive apprenticeship model'. Aim: The aim was to estimate the content validity of the developed instrument. Method: Item relevance was rated on a five-point scale (1 = highly irrelevant, 5 = highly relevant) by three groups of stakeholders in undergraduate clinical teaching: educationalists (N = 12), doctors (N = 16) and students (N = 12). Additionally, stakeholders commented on content, wording and omission of items. Results: The items were generally rated as very relevant (Mean = 4.3, SD = 0.38, response = 95%) and any differences between the stakeholder groups were small. The results led to elimination of 4 items, rewording of 13 items and addition of 1 item. Discussion: The cognitive apprenticeship model appears to offer a useful framework for the development of an evaluation instrument aimed at providing feedback to individual clinical teachers on the quality of student supervision. Further studies in larger populations will have to establish the instrument's statistical validity and generalizability. [ABSTRACT FROM AUTHOR]

Background: There is a lack of explicit criteria for the assessment of students' research reports. Teachers tend to use their own, idiosyncratic sets of implicit criteria. A well-defined set of criteria could foster reliable and valid assessment of students' research reports. Aim: The aim of this study was twofold: to detect the strengths and weaknesses of students' research reports and to develop an assessment tool for students' research papers. Methods: On the basis of the literature and advice from experts, we developed a list of 19 criteria, comprising 15 specific items and 4 global items. Three raters, using the list, independently scored a sample of 18 research reports. Results: The strengths of the reports were the structure of the results and the description of materials and methods. The weaknesses were the research questions, the grounding of the study in the research literature, the analysis, the statistics, and the clarity of visuals. With three raters, the reliability of the rating scale was reasonable. Conclusion: The rating scale seems to be a useful tool for judging and giving feedback on students' research reports. These findings will have to be confirmed in studies with larger samples. [ABSTRACT FROM AUTHOR]

The practice of assessment is governed by an interesting paradox. On the one hand good assessment requires substantial resources which may exceed the capacity of a single institution and we have reason to doubt the quality of our in-house examinations. On the other hand, our parsimonity with regard to our resources makes us reluctant to pool efforts and share our test material. This paper reports on an initiative to share test material across different medical schools. Three medical schools in The Netherlands have successfully set up a partnership for a specific testing method: progress testing. At present, these three schools collaboratively produce high-quality test items. The jointly produced progress tests are administered concurrently by these three schools and one other school, which buys the test. The steps taken in establishing this partnership are described and results are presented to illustrate the unique sort of information that is obtained by cross-institutional assessment. In addition, plans to improve test content and procedure and to expand the partnership are outlined. Eventually, the collaboration may even extend to other test formats. This article is intended to give evidence of the feasibility and exciting potential of between school collaboration in test development and test administration. Our experiences have demonstrated that such collaboration has excellent potential to combine economic benefit with educational advantages, which exceed what is achievable by individual schools. [ABSTRACT FROM AUTHOR]

Several standard-setting methods can be used for authentic performance-based assessment using direct observation. Rational methods (e.g., the Angoff method) use judgments of test items to set a cutscore, whereas empirical methods (e.g., the borderline group or contrasting groups procedure) use judgments of examinees to obtain a cutscore. When comparing standard-setting methods, a critical concern is the precision of the obtained cutscore. With the Angoff method, cutscore precision can be estimated by applying generalizability analysis to the item judgment data. The empirical methods, however, lack a generally applicable procedure to estimate the precision. In this study a method is developed to estimate the cutscore precision for empirical methods by using resampling. The method was illustrated for a regression based standard-setting method applied to two actual datasets. The results show the applicability of the method: The precision estimate obtained with the resampling method was in reasonable agreement with the estimate obtained by using standard linear regression equations. [ABSTRACT FROM AUTHOR]

Background: This research focuses on the relation between individual study and group discussion. In a problem-based curriculum, it is expected that the way students prepare themselves during individual study (i.e., search and prepare the literature) will influence the quality of the reporting phase. Purpose: To investigate whether searching for different literature resources and preparing the literature (by making summaries to explain the literature) affects the quality of the reporting phase. Method : A 23-item questionnaire was developed, reflecting 2 factors of the search phase and 1 factor of the preparing phase of the individual study. Two factors (breadth and depth of the discussion) reflected the reporting phase. Participants were 1st-year students (N = 195, 90%) at the Medical School of the Maastricht University in the Netherlands during the academic year 1997-1998. Results: Regression analyses show that the search phase and the preparing phase explained 29% of the variance of the breadth and 38% of the depth of the reporting phase. Conclusions: Searching different literature resources has a small and negative impact on the quality of the reporting phase. However, preparing extensively for the next tutorial meeting is very important, especially for the depth of the reporting phase. Therefore, when new knowledge must be integrated in the tutorial group, students must be taught to prepare themselves by making summaries that are useful and effective during the discussion. [ABSTRACT FROM AUTHOR]

Discusses some theoretical issues in the context of nonparametric item response theory modeling. Example using data from a test measuring medical knowledge; Existing group-based statistics to detect improbable item score patterns; Practical use of various cut scores; Demonstration that person-fit statistics can be used to detect persons with a deficiency of knowledge on an achievement test.

BACKGROUND: Changing concepts of education have led many medical schools to adopt student-centred approaches to teaching, requiring different teaching competencies than more traditional approaches. AIMS: The aim of this study was to investigate whether participation in a faculty development (FD) programme had a positive effect on veterinary medical teachers' perceptions of their competence in seven different teaching roles. METHOD: All teaching staff of a veterinary medical school (251) were invited to complete a questionnaire asking about their perceived competence in seven teaching roles. A stepwise multiple linear regression analysis examined the effect of the completion of a FD programme on perceived teaching competence. RESULTS: There was a significant positive effect of the FD programme on teachers' perceived competence in majority of the teaching roles at issue. CONCLUSIONS: FD appeared to have a positive effect on the perceived competence of veterinary medical teachers. Further research should investigate whether FD actually results in improved teaching performance. [ABSTRACT FROM AUTHOR]

A letter to the editor is presented in response to the progress testing material within a School of Medicine and between schools by Doctor Van der Vleuten and his colleagues.

Background: Feedback to aid teachers in improving their teaching requires validated evaluation instruments. When implementing an evaluation instrument in a different context, it is important to collect validity evidence from multiple sources. Aim: We examined the validity and reliability of the Maastricht Clinical Teaching Questionnaire (MCTQ) as an instrument to evaluate individual clinical teachers during short clinical rotations in veterinary education. Methods: We examined four sources of validity evidence: (1) Content was examined based on theory of effective learning. (2) Response process was explored in a pilot study. (3) Internal structure was assessed by confirmatory factor analysis using 1086 student evaluations and reliability was examined utilizing generalizability analysis. (4) Relations with other relevant variables were examined by comparing factor scores with other outcomes. Results: Content validity was supported by theory underlying the cognitive apprenticeship model on which the instrument is based. The pilot study resulted in an additional question about supervision time. A five-factor model showed a good fit with the data. Acceptable reliability was achievable with 10--12 questionnaires per teacher. Correlations between the factors and overall teacher judgement were strong. Conclusions: The MCTQ appears to be a valid and reliable instrument to evaluate clinical teachers'' performance during short rotations. [ABSTRACT FROM AUTHOR]

Background: Medical students increasingly participate in rotations in primary care settings, such as general practices. Although general practices can differ markedly from hospital settings, research on the instructional quality of general practice rotations is relatively scarce. Aim: We conducted a study to identify which aspects make a significant contribution to the instructional quality of general practice clerkships, as perceived by students. Method: After completing their general practice attachment, 155 fifth-year medical students filled out an evaluation questionnaire. Exploratory factor analysis and reliability analysis identified clusters of correlated independent variables. Subsequent regression analysis revealed the relations between the reduced set of independent variables and the dependent variable ‘Instructional quality’. Results: Both the quality of supervision and the size and diversity of the patient mix substantially affected students’ experienced instructional quality. Opportunities and facilities to perform independently were correlated with instructional quality, but did not affect the instructiveness directly. Conclusions: Supervision, patient mix and independence are crucial factors for learning in general practice. This is consistent with findings in hospital settings. The perceived quality of instruction hinges on supervision, which is not only the variable most strongly related to instructional quality, but which also affects both the patient mix and students’ independence. [ABSTRACT FROM AUTHOR]