Search

Background Free Open-Access Medical education (FOAM) use among residents continues to rise. However, it often lacks quality assurance processes and residents receive little guidance on quality assessment. The Academic Life in Emergency Medicine Approved Instructional Resources tool (AAT) was created for FOAM appraisal by and for expert educators and has demonstrated validity in this context. It has yet to be evaluated in other populations. Objectives We assessed the AAT's usability in a diverse population of practicing emergency medicine (EM) physicians, residents, and medical students; solicited feedback; and developed a revised tool. Methods As part of the Medical Education Translational Resources: Impact and Quality (METRIQ) study, we recruited medical students, EM residents, and EM attendings to evaluate five FOAM posts with the AAT and provide quantitative and qualitative feedback via an online survey. Two independent analysts performed a qualitative thematic analysis with discrepancies resolved through discussion and negotiated consensus. This analysis informed development of an initial revised AAT, which was then further refined after pilot testing among the author group. The final tool was reassessed for reliability. Results Of 330 recruited international participants, 309 completed all ratings. The Best Evidence in Emergency Medicine (BEEM) score was the component most frequently reported as difficult to use. Several themes emerged from the qualitative analysis: for ease of use-understandable, logically structured, concise, and aligned with educational value. Limitations include deviation from questionnaire best practices, validity concerns, and challenges assessing evidence-based medicine. Themes supporting its use include evaluative utility and usability. The author group pilot tested the initial revised AAT, revealing a total score average measure intraclass correlation coefficient (ICC) of moderate reliability (ICC = 0.68, 95% confidence interval [CI] = 0 to 0.962). The final AAT's average measure ICC was 0.88 (95% CI = 0.77 to 0.95). Conclusions We developed the final revised AAT from usability feedback. The new score has significantly increased usability, but will need to be reassessed for reliability in a broad population.

Introduction: The number of educational resources created for emergency medicine and criticalcare (EMCC) that incorporate social media has increased dramatically. With no way to assess theirimpact or quality, it is challenging for educators to receive scholarly credit and for learners to identifyrespected resources. The Social Media index (SMi) was developed to help address this. Methods: We used data from social media platforms (Google PageRanks, Alexa Ranks, FacebookLikes, Twitter Followers, and Google+ Followers) for EMCC blogs and podcasts to derive threenormalized (ordinal, logarithmic, and raw) formulas. The most statistically robust formula wasassessed for 1) temporal stability using repeated measures and website age, and 2) correlationwith impact by applying it to EMCC journals and measuring the correlation with known journalimpact metrics. Results: The logarithmic version of the SMi containing four metrics was the most statistically robust.It correlated significantly with website age (Spearman r=0.372; p

Background Historically, trainees in undergraduate and graduate health professions education have relied on secondary resources, such as textbooks and lectures, for core learning activities. Recently, blogs and podcasts have entered into mainstream usage, especially for residents and educators. These low-cost, widely available resources have many characteristics of disruptive innovations and, if they continue to improve in quality, have the potential to reinvigorate health professions education. One potential limitation of further growth in the use of these resources is the lack of information on their quality and effectiveness. Objective To identify quality indicators for secondary resources that are described in the literature, which might be applicable to blogs and podcasts. Methods Using a blended research methodology, we performed a systematic literature review using Google Scholar, MEDLINE, Embase, Web of Science, and ERIC to identify quality indicators for secondary resources. A qualitative analysis of these indicators resulted in the organization of this information into themes and subthemes. Expert focus groups were convened to triangulate these findings and ensure that no relevant quality indicators were missed. Results The literature search identified 4530 abstracts, and quality indicators were extracted from 157 articles. The qualitative analysis produced 3 themes (credibility, content, and design), 13 subthemes, and 151 quality indicators. Conclusions The list of quality indicators resulting from our analysis can be used by stakeholders, including learners, educators, academic leaders, and blog/podcast producers. Further studies are being conducted, which will refine the list into a form that is more structured and stratified for use by these stakeholders.

Study objective This study identified the most important quality indicators for online educational resources such as blogs and podcasts. Methods A modified Delphi process that included 2 iterative surveys was used to build expert consensus on a previously defined list of 151 quality indicators divided into 3 themes: credibility, content, and design. Aggregate social media indicators were used to identify an expert population of editors from a defined list of emergency medicine and critical care blogs and podcasts. Survey 1 consisted of the quality indicators and a 7-point Likert scale. The mean score for each quality indicator was included in survey 2, which asked participants whether to "include" or "not include" each quality indicator. The cut point for consensus was defined at greater than 70% "include." Results Eighty-three percent (20/24) of bloggers and 90.9% (20/22) of podcasters completed survey 1 and 90% (18/20) of bloggers and podcasters completed survey 2. The 70% inclusion criteria were met by 44 and 80 quality indicators for bloggers and podcasters, respectively. Post hoc, a 90% cutoff was used to identify a list of 14 and 26 quality indicators for bloggers and podcasters, respectively. Conclusion The relative importance of quality indicators for emergency medicine blogs and podcasts was determined. This will be helpful for resource producers trying to improve their blogs or podcasts and for learners, educators, and academic leaders assessing their quality. These results will inform broader validation studies and attempts to develop user-friendly assessment instruments for these resources.