Search

Growing demand for accountability, transparency, and efficiency in health professions education is expected to drive increased demand for, and use of, cost and value analyses. In this AMEE Guide, we introduce key concepts, methods, and literature that will enable novices in economics to conduct simple cost and value analyses, hold informed discussions with economic specialists, and undertake further learning on more advanced economic topics. The practical structure for conducting analyses provided in this guide will enable researchers to produce robust results that are meaningful and useful for improving educational practice. Key steps include defining the economic research question, identifying an appropriate economic study design, carefully identifying cost ingredients, quantifying, and pricing the ingredients consumed, and conducting sensitivity analyses to explore uncertainties in the results. [ABSTRACT FROM AUTHOR]

Context: The design of selection methods must balance, amongst a range of factors, the desire to select the best possible future doctors with the reality of our resource‐constrained environment. Examining the cost of selection processes enables us to identify areas in which efficiencies may be gained. Methods: A cost description study was conducted based on selection for 2018 entry into medical school directly from secondary school. The perspectives of applicants, volunteer interviewers and the admitting institution were considered. Costs were modelled based on the Monash University (Australia) selection process, which uses a combination of secondary school matriculation score, aptitude test score (Undergraduate Medicine and Health Sciences Admission Test) and multiple mini‐interview score. A variety of data sources were utilised, including bespoke surveys, audit data and existing literature. All costs are expressed in 2018 Australian dollars (AU$). Applicant behaviours in preparing for selection tests were also evaluated. Results: A total of 381 of 383 applicants returned the survey. Over 70% of applicants had utilised commercial preparation materials. The median total cost to applicants was AU$2586 (interquartile range [IQR] AU$1574‐3999), including costs to both prepare for and attend selection tests. Of 217 volunteer interviewers, 108 returned the survey. These were primarily health professional clinicians at a mid‐career stage. The median total cost to interviewers was AU$452 (IQR AU$252‐715) for participation in a half‐day interview session, largely due to the loss of income. The cost to the admitting institution was AU$269 per applicant, accounted for by the costs of equipment and consumables (52%), personnel (34%) and facilities (14%). Conclusions: The costs of student selection for medical school are substantial. Understanding costs facilitates achievement of the objective of selecting the desired future medical workforce within the constraints of the resources available. Opportunities for change may arise from changes in applicant preparation behaviours, opportunities for economies of scale, and efficiencies driven by technological solutions. What does it take to get into medical school? Money for one thing. The authors demonstrate that, in Australia, students spend AU$2, 586 on preparation, attendance. The universities to which they apply spend AU$269 per applicant, money that is likely to be recouped if selection can avoid student attrition. [ABSTRACT FROM AUTHOR]

Background: Journal clubs are an educational activity in which individuals meet to critique and discuss research articles. They are an established part of the medical education system and are considered to be a practical way to improve the content knowledge of health professionals. Aims: To determine the effectiveness of journal clubs for increasing health professionals' competency in EBM. Methods: A systematic review of randomized controlled trials (RCTs) was performed. Electronic searches were conducted in October 2019 across MEDLINE, ERIC and Scopus databases. Two authors independently reviewed articles, and extracted data. A risk‐of‐bias tool, based on the Cochrane Collaboration's tool for assessing risk of bias in RCTs, was used to assess internal validity. Results: A total of 151 citations were returned, from which five studies (n = 378 individuals) were included in the final review. No overall statistical difference in knowledge scores was observed between health professionals participating in journal clubs compared to other professional education modes (SMD 0.15, 95% CI –0.09, 0.39). Similarly, no significant difference in attitudes or implementation of evidence‐based medicine practices was observed across studies. Conclusions: There is insufficient evidence to support the effectiveness of journal clubs in improving the knowledge, attitudes, and implementation of evidence‐based skills by health professionals in clinical practice. Further research is needed to test the effectiveness of other interventions to increase uptake of EBM in real world settings. Such interventions may include interactive components with auditing and feedback to facilitate more effective learning. [ABSTRACT FROM AUTHOR]

Context: High‐quality research into education costs can inform better decision making. Improvements to cost research can be guided by information about the research questions, methods and reporting of studies evaluating costs in health professions education (HPE). Our objective was to appraise the overall state of the field and evaluate temporal trends in the methods and reporting quality of cost evaluations in HPE research. Methods: We searched the MEDLINE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), EMBASE, Business Source Complete and ERIC (Education Resources Information Centre) databases on 31 July 2017. To evaluate trends over time, we sampled research reports at 5‐year intervals (2001, 2006, 2011 and 2016). All original research studies in HPE that reported a cost outcome were included. The Medical Education Research Study Quality Instrument (MERSQI) and the BMJ economic checklist were used to appraise methodological and reporting quality, respectively. Trends in quality over time were analysed. Results: A total of 78 studies were included, of which 16 were published in 2001, 15 in 2006, 20 in 2011 and 27 in 2016. The region most commonly represented was the USA (n = 43). The profession most commonly referred to was that of the physician (n = 46). The mean ± standard deviation (SD) MERSQI score was 10.9 ± 2.6 out of 18, with no significant change over time (p = 0.55). The mean ± SD BMJ score was 13.5 ± 7.1 out of 35, with no significant change over time (p = 0.39). A total of 49 (63%) studies stated a cost‐related research question, 23 (29%) stated the type of cost evaluation used, and 31 (40%) described the method of estimating resource quantities and unit costs. A total of 16 studies compared two or more interventions and reported both cost and learning outcomes. Conclusions: The absolute number of cost evaluations in HPE is increasing. However, there are shortcomings in the quality of methodology and reporting, and these are not improving over time. Cost may be most influential yet least studied determinant of practice in health professional education. In this systematic review, Foo et al. explore the state of the literature regarding formal use of cost‐evaluations. [ABSTRACT FROM AUTHOR]

Background: Student failure creates additional economic costs. Knowing the cost of failure helps to frame its economic burden relative to other educational issues, providing an evidence-base to guide priority setting and allocation of resources. The Ingredients Method is a cost-analysis approach which has been previously applied to health professions education research. In this study, the Ingredients Method is introduced, and applied to a case study, investigating the cost of pre-clinical student failure. Methods: The four step Ingredients Method was introduced and applied: (1) identify and specify resource items, (2) measure volume of resources in natural units, (3) assign monetary prices to resource items, and (4) analyze and report costs. Calculations were based on a physiotherapy program at an Australian university. Results: The cost of failure was £5991 per failing student, distributed across students (70%), the government (21%), and the university (8%). If the cost of failure and attrition is distributed among the remaining continuing cohort, the cost per continuing student educated increases from £9923 to £11,391 per semester. Conclusions: The economics of health professions education is complex. Researchers should consider both accuracy and feasibility in their costing approach, toward the goal of better informing cost-conscious decision-making. [ABSTRACT FROM AUTHOR]

The estimation of cost and value in health professions education should involve robust methodologies and decision tools. These methods and tools should be applied consistently and transparently, but more importantly, employed in the appropriate context depending on the availability of data, target estimates and stakeholder focus. Best practice methodology and application of decision tools will allow for a clearer and more comprehensive understanding of the return on investment of health profession education interventions by shedding greater light on the full opportunity cost of providing programs and the value of such programs to learners and recipients of health care services more broadly. This article provides insight into the basic methods, decision tools and key evaluation concepts that might be considered for the purpose of undertaking cost and value research in continuing education for health professionals. These methods and decision tools are based on consolidated discussions by the organizing delegates of the inaugural Symposium of the Society for Cost and Value of Health Professions Education, held in Prato, Italy, October 2015. In summary, the application of economic methods and tools used for analyzing health professions education is currently inconsistent. This article provides an overview and recommendations on the use of certain economic methods and tools when evaluating health profession education programs. It also provides a clear understanding of key evaluation concepts important for undertaking an economic review of a program. [ABSTRACT FROM AUTHOR]

CONTEXT Failure by students in health professional clinical education intertwines the health and education sectors, with actions in one having potential downstream effects on the other. It is unknown what economic costs are associated with failure, how these costs are distributed, and the impacts these have on students, clinicians and workplace productivity. An understanding of cost drivers and cost boundaries will enable evidence-based targeting of strategic investments into clinical education, including where they should be made and by whom. OBJECTIVES This study was designed to determine the additional economic costs associated with failure by students in health professional clinical education. METHODS A cost analysis study involving cost identification, measurement, valuation and the calculation of total cost was conducted. Costs were considered from the perspective of the student, the education institution, the clinical educator, the health service placement provider organisation and the government. Data were based on a 5-week clinical education programme at Monash University,Australia. Data were collected using quantitative surveys and interviews conducted with health professional students, clinical educators and education institute staff. Reference group representation was also sought at various education institution and health service organisation levels. A transferable model with sensitivity analysis was developed. RESULTS There is a total additional cost of US $9371 per student failing in clinical education from the perspective of all stakeholders considered. Students bear the majority of this burden, incurring 49% of costs, followed by the government (22%), the education institution (18%), the health service organisation (10%) and the clinical educator (1%). CONCLUSIONS Strong economic links for multiple stakeholders as a result of failure by students in clinical education have been identified. The cost burden is skewed in the direction of students. Any generalisation of these results should be made with consideration for the unique clinical education context in which each health professional education programme operates. [ABSTRACT FROM AUTHOR]

The authors present the Prato Statement on cost and value in professional and interprofessional health education and workforce planning and training. They discuss the elements of educational research knowledge that include promotion of economic literacy and standards for published literature, as well as policy that include expectation and multiple perspectives of cost and value in decision making, and use of economic reasoning to advocate for culture change and demonstrate educational value.

Health professional education is experiencing a cultural shift towards student-centered education. Although we are now challenging our traditional training methods, our methods for evaluating the impact of the training on the learner remains largely unchanged. What is not typically measured is student-centered value; whether it was ‘worth’ what the learner paid. The primary aim of this study was to apply a method of calculating student-centered value, applied to the context of a change in teaching methods within a health professional program. This study took place over the first semester of the third year of the Bachelor of Physiotherapy at Monash University, Victoria, Australia, in 2014. The entire third year cohort (n = 78) was invited to participate. Survey based design was used to collect the appropriate data. A blended learning model was implemented; subsequently students were only required to attend campus three days per week, with the remaining two days comprising online learning. This was compared to the previous year’s format, a campus-based face-to-face approach where students attended campus five days per week, with the primary outcome—Value to student. Value to student incorporates, user costs associated with transportation and equipment, the amount of time saved, the price paid and perceived gross benefit. Of the 78 students invited to participate, 76 completed the post-unit survey (non-participation rate 2.6%). Based on Value to student the blended learning approach provided a $1,314.93 net benefit to students. Another significant finding was that the perceived gross benefit for the blended learning approach was $4014.84 compared to the campus-based face-to-face approach of $3651.72, indicating that students would pay more for the blended learning approach. This paper successfully applied a novel method of calculating student-centered value. This is the first step in validating the value to student outcome. Measuring economic value to the student may be used as a way of evaluating effective change in a modern health professional curriculum. This could extend to calculate total value, which would incorporate the economic implications for the educational providers. Further research is required for validation of this outcome. [ABSTRACT FROM AUTHOR]

Objective: To identify whether the clinical maturity of medical trainees impacts upon the level of trainee competency in evidence-based medicine (EBM).Materials and Methods: Undergraduate and graduate-entry medical trainees entering their first year of training in the clinical environment were recruited for this study. Competency in EBM was measured using a psychometrically validated instrument. EBM competency scores were analysed using Student's t tests, in order to differentiate between undergraduate and graduate-entry trainee performance. Ten focus group discussions were conducted with undergraduate and graduate-entry trainees. Audio transcripts were thematically analysed.Results: Data on a total of 885 medical trainees were collected over a 5-year period. Undergraduate trainees had significantly higher EBM competency scores during years in which the programme was presented in a didactic format (mean difference (MD)=1.24 (95% CI)CI 0.21 to 2.26; 1.78 (0.39 to 3.17); 2.13 (1.16 to 3.09)). Graduate trainee EBM competency scores increased when a blended learning approach to EBM was adopted, demonstrating no significant difference in EBM competency scores between undergraduate and graduate cohorts (-0.27 (-1.38 to 0.85); -0.39 (-1.57 to 0.79). Qualitative findings indicated that differences in learning and teaching preference among undergraduate and graduate-entry trainees influenced the level of competency obtained in EBM.Conclusions: Clinical maturity is the only one factor that may influence medical trainees' competency in EBM. Other predictors of EBM competency may include previous training and exposure to epidemiology, biostatistics and information literacy. While graduate-entry medical students may have more 'life' experience, or maturity, it does not necessarily translate into clinical maturity and integration into the clinical environment. [ABSTRACT FROM AUTHOR]

Background: Health professionals need to be integrated more effectively in clinical research to ensure that research addresses clinical needs and provides practical solutions at the coal face of care. In light of limited evidence on how best to achieve this, evaluation of strategies to introduce, adapt and sustain evidence-based practices across different populations and settings is required. This project aims to address this gap through the co-design, development, implementation, evaluation, refinement and ultimately scale-up of a clinical research engagement and leadership capacity building program in a clinical setting with little to no co-ordinated approach to clinical research engagement and education. Methods/Design: The protocol is based on principles of research capacity building and on a six-step framework, which have previously led to successful implementation and long-term sustainability. A mixed methods study design will be used. Methods will include: (1) a review of the literature about strategies that engage health professionals in research through capacity building and/or education in research methods; (2) a review of existing local research education and support elements; (3) a needs assessment in the local clinical setting, including an online cross-sectional survey and semi-structured interviews; (4) co-design and development of an educational and support program; (5) implementation of the program in the clinical environment; and (6) pre- and post-implementation evaluation and ultimately program scale-up. The evaluation focuses on research activity and knowledge, attitudes and preferences about clinical research, evidence-based practice and leadership and post implementation, about their satisfaction with the program. The investigators will evaluate the feasibility and effect of the program according to capacity building measures and will revise where appropriate prior to scale-up. Discussion: It is anticipated that this clinical research engagement and leadership capacity building program will enable and enhance clinically relevant research to be led and conducted by health professionals in the health setting. This approach will also encourage identification of areas of clinical uncertainty and need that can be addressed through clinical research within the health setting. [ABSTRACT FROM AUTHOR]

Objectives: The purpose of this study was to examine the attributes that students and educators believe are important to being a good health educator in a non-clinical setting. Methods: A cross-sectional survey of first-year health science students and educators involved with a Health Science course in Melbourne, Australia was performed. A convenience sampling approach was implemented, with participants were required to rate the importance of teaching attributes on a previously developed 15-item written questionnaire. Descriptive statistics were generated, with Pearson's chi-square statistics used to examine differences between groups. In total 94/147 (63.9%) of students and 15/15 (100%) of educators participated in the study. Results: Of the 15 attributes, only 'scholarly activity' was not deemed to be not as an important attribute to define a good educator. Knowledge base (50% vs. 13.3%) and feedback skills (22.3% vs. 0%) were rated as important attributes by students in comparison to educators. Professionalism (20% vs. 5.3%), scholarly activity (20% vs. 3.2%) and role modelling (26.7% vs. 3.2%) were rated as the most important attributes by educators in comparison to students. Conclusions: No single attribute makes a good health educator; rather health educators are required to have a rounded approach to teaching. Students have greater focus on the educator providing a transfer of knowledge. Educators are additionally focussed on professionalism attributes, which may not be valued by students. Students and educators must enter into a clearer understanding of expectations, from both parties, to obtain optimal education outcomes. [ABSTRACT FROM AUTHOR]

Background: Few studies have been performed to inform how best to teach evidence-based medicine (EBM) to medical trainees. Current evidence can only conclude that any form of teaching increases EBM competency, but cannot distinguish which form of teaching is most effective at increasing student competency in EBM. This study compared the effectiveness of a blended learning (BL) versus didactic learning (DL) approach of teaching EBM to medical students with respect to competency, self-efficacy, attitudes and behaviour toward EBM. Methods: A mixed methods study consisting of a randomised controlled trial (RCT) and qualitative case study was performed with medical students undertaking their first clinical year of training in EBM. Students were randomly assigned to receive EBM teaching via either a BL approach or the incumbent DL approach. Competency in EBM was assessed using the Berlin questionnaire and the 'Assessing Competency in EBM' (ACE) tool. Students' self-efficacy, attitudes and behaviour was also assessed. A series of focus groups was also performed to contextualise the quantitative results. Results: A total of 147 students completed the RCT, and a further 29 students participated in six focus group discussions. Students who received the BL approach to teaching EBM had significantly higher scores in 5 out of 6 behaviour domains, 3 out of 4 attitude domains and 10 out of 14 self-efficacy domains. Competency in EBM did not differ significantly between students receiving the BL approach versus those receiving the DL approach [Mean Difference (MD)=-0.68, (95% CI-1.71, 0.34), p=0.19]. No significant difference was observed between sites (p=0.89) or by student type (p=0.58). Focus group discussions suggested a strong student preference for teaching using a BL approach, which integrates lectures, online learning and small group activities. Conclusions: BL is no more effective than DL at increasing medical students' knowledge and skills in EBM, but was significantly more effective at increasing student attitudes toward EBM and self-reported use of EBM in clinical practice. Given the various learning styles preferred by students, a multifaceted approach (incorporating BL) may be best suited when teaching EBM to medical students. Further research on the cost-effectiveness of EBM teaching modalities is required. [ABSTRACT FROM AUTHOR]

In health professional education, reflective practice is seen as a potential means for self-improvement from everyday clinical encounters. This study aims to examine the level of student honesty in critical reflection, and barriers and facilitators for students engaging in honest reflection. Third year physiotherapy students, completing summative reflective essays on clinical encounters using the modified Gibbs cycle, were invited to participate in an anonymous online survey. Student knowledge and beliefs about reflective practice, and disclosure of the truthfulness of their reflections, were assessed using a mixed method approach. A total of 34 students, from a maximum possible of 48 (71 %), participated in the study activities. A total of 68 % stated that they were at least 80 % truthful about their experiences. There was general student consensus that reflective practice was important for their growth as a clinician. Students questioned the belief that the reflection needed to be based on a factual experience. Reflective practice can be a valuable addition to the clinical education of health care professionals, although this value can be diminished through dishonest reflections if it is not carefully implemented. Student influences on honest reflection include; (1) the design of any assessment criteria, and (2) student knowledge and competency in applying critical reflection. [ABSTRACT FROM AUTHOR]

Background Evidence Based Medicine (EBM) is a core unit delivered across many medical schools. Few studies have investigated the most effective method of teaching a course in EBM to medical students. The objective of this study was to identify whether a blended-learning approach to teaching EBM is more effective a didactic-based approach at increasing medical student competency in EBM. Methods A mixed-methods study was conducted consisting of a controlled trial and focus groups with second year graduate medical students. Students received the EBM course delivered using either a didactic approach (DID) to learning EBM or a blended-learning approach (BL). Student competency in EBM was assessed using the Berlin tool and a criterion-based assessment task, with student perceptions on the interventions assessed qualitatively. Results A total of 61 students (85.9 %) participated in the study. Competency in EBM did not differ between the groups when assessed using the Berlin tool (p = 0.29). Students using the BL approach performed significantly better in one of the criterion-based assessment tasks (p = 0.01) and reported significantly higher self-perceived competence in critical appraisal skills. Qualitative analysis identified that students had a preference for the EBM course to be delivered using the BL approach. Conclusions Implementing a blended-learning approach to EBM teaching promotes greater student appreciation of EBM principles within the clinical setting. Integrating a variety of teaching modalities and approaches can increase student self-confidence and assist in bridging the gap between the theory and practice of EBM. [ABSTRACT FROM AUTHOR]

Background: Blended learning describes a combination of teaching methods, often utilizing digital technologies. Research suggests that learner outcomes can be improved through some blended learning formats. However, the cost-effectiveness of delivering blended learning is unclear. [ABSTRACT FROM AUTHOR]