Search

BackgroundThe implementation of clinical decision support systems (CDSSs) as an intervention to foster clinical practice change is affected by many factors. Key factors include those associated with behavioral change and those associated with technology acceptance. However, the literature regarding these subjects is fragmented and originates from two traditionally separate disciplines: implementation science and technology acceptance. ObjectiveOur objective is to propose an integrated framework that bridges the gap between the behavioral change and technology acceptance aspects of the implementation of CDSSs. MethodsWe employed an iterative process to map constructs from four contributing frameworks—the Theoretical Domains Framework (TDF); the Consolidated Framework for Implementation Research (CFIR); the Human, Organization, and Technology-fit framework (HOT-fit); and the Unified Theory of Acceptance and Use of Technology (UTAUT)—and the findings of 10 literature reviews, identified through a systematic review of reviews approach. ResultsThe resulting framework comprises 22 domains: agreement with the decision algorithm; attitudes; behavioral regulation; beliefs about capabilities; beliefs about consequences; contingencies; demographic characteristics; effort expectancy; emotions; environmental context and resources; goals; intentions; intervention characteristics; knowledge; memory, attention, and decision processes; patient–health professional relationship; patient’s preferences; performance expectancy; role and identity; skills, ability, and competence; social influences; and system quality. We demonstrate the use of the framework providing examples from two research projects. ConclusionsWe proposed BEAR (BEhavior and Acceptance fRamework), an integrated framework that bridges the gap between behavioral change and technology acceptance, thereby widening the view established by current models.

Introduction: When performance data are provided as feedback to healthcare professionals, they may use it to significantly improve care quality. However, the question of how to provide effective feedback remains unanswered, as decades of evidence have produced a consistent pattern of effects—with wide variation. From a coaching perspective, feedback is often based on a learner's objectives and goals. Furthermore, when coaches provide feedback, it is ideally informed by their understanding of the learner's needs and motivation. We anticipate that a "coaching"‐informed approach to feedback may improve its effectiveness in two ways. First, by aligning feedback with healthcare professionals' chosen goals and objectives, and second, by enabling large‐scale feedback systems to use new types of data to learn what kind of performance information is motivating in general. Our objective is to propose a conceptual model of precision feedback to support these anticipated enhancements to feedback interventions. Methods: We iteratively represented models of feedback's influence from theories of motivation and behavior change, visualization, and human‐computer interaction. Through cycles of discussion and reflection, application to clinical examples, and software development, we implemented and refined the models in a software application to generate precision feedback messages from performance data for anesthesia providers. Results: We propose that precision feedback is feedback that is prioritized according to its motivational potential for a specific recipient. We identified three factors that influence motivational potential: (1) the motivating information in a recipient's performance data, (2) the surprisingness of the motivating information, and (3) a recipient's preferences for motivating information and its visual display. Conclusions: We propose a model of precision feedback that is aligned with leading theories of feedback interventions to support learning about the success of feedback interventions. We plan to evaluate this model in a randomized controlled trial of a precision feedback system that enhances feedback emails to anesthesia providers. [ABSTRACT FROM AUTHOR]

Computable biomedical knowledge (CBK) is: "the result of an analytic and/or deliberative process about human health, or affecting human health, that is explicit, and therefore can be represented and reasned upon using logic, formal standards, and mathematical approaches." Representing biomedical knowledge in a machine-interpretable, computable form increases its ability to be discovered, accessed, understood, and deployed. Computable knowledge artifacts can greatly advance the potential for implementation, reproducibility, or extension of the knowledge by users, who may include practitioners, researchers, and learners. Enriching computable knowledge artifacts may help facilitate reuse and translation into practice. Following the examples of 10 Simple Rules papers for scientific code, software, and applications, we present 10 Simple Rules intended to make shared computable knowledge artifacts more useful and reusable. These rules are mainly for researchers and their teams who have decided that sharing their computable knowledge is important, who wish to go beyond simply describing results, algorithms, or models via traditional publication pathways, and who want to both make their research findings more accessible, and to help others use their computable knowledge. These rules are roughly organized into 3 categories: planning, engineering, and documentation. Finally, while many of the following examples are of computable knowledge in biomedical domains, these rules are generalizable to computable knowledge in any research domain. Author summary: Computable biomedical knowledge, including algorithms, models, and evidence, is already impacting—and will likely transform—health research and healthcare delivery. Representing this knowledge in formats that can be discovered and processed by computers increases its potential for use at scale. Similarly, making these algorithms, models, etc. (aka computable knowledge artifacts) understandable and reusable by others can help facilitate reuse and translation into practice. However, as with other types of scientific software or research data, creating artifacts that are usable by and useful to others requires intentionality, and often additional time and effort. We've developed over 100 computable knowledge artifacts in diverse areas of health research and healthcare and we've learned a few things along the way. We've distilled these lessons learned into 10 Simple Rules to help researchers who would like to make their computational research products available and useful to others. Our work centers on knowledge in the biomedical domain, but we believe these 10 Simple Rules are generalizable and valuable as design principles for any researcher wishing to share computational research products. [ABSTRACT FROM AUTHOR]

We present our work to develop digital objects to represent and convey a specific category of scientific knowledge: computable biomedical knowledge (CBK). Properly developed, validated, implemented and stewarded, CBK has the potential to accelerate the translation of actionable knowledge from scientific discovery to clinical application. Our research takes an infrastructural approach to CBK, initially by focusing on the creation of a conceptual model for packaging computable biomedical knowledge - the Knowledge Object (KO) - and on corresponding efforts to create an architecture for KO management and implementation. Additionally, our work is grounded in the FAIR principles, such that KO artefacts should be findable, accessible, interoperable and reusable and we are exploring aligning KOs with emerging best practices for FAIR Digital Objects (FDO). The outcomes of this work resonate in clinical contexts, health professions education, healthcare quality improvement, biomedical and translational research and population care. Our KO model is also of interest to researchers and practitioners interested in knowledge science, including those working with semantic technologies and other forms of digital objects. [ABSTRACT FROM AUTHOR]

Introduction: We present current work to develop and define a class of digital objects that facilitates patient cohort identification for clinical studies, such that these objects are Findable, Accessible, Interoperable, and Reusable (FAIR) (Wilkinson et al. 2016). Developing this class of FAIR Digital Objects (FDOs) builds on the work of several years to develop the Knowledge Grid (https://kgrid.org/), which facilitates the development, description and implementation of biomedical knowledge packaged in machine-readable and machineexecutable formats (Flynn et al. 2018). Additionally, this work aligns with the goals of the Mobilizing Computable Biomedical Knowledge (MCBK) community (https://mobilizecbk.med.umich.edu/) (Mobilizing Computable Biomedical Knowledge 2018). In this abstract, we describe our work to develop a FDO carrying a computable phenotype. Defining computable phenotypes: In biomedical informatics, 'phenotyping' describes a data-driven approach to identifying a group of individuals sharing observable characteristics of interest, generally related to a disease or condition, and a 'computable phenotype' (CP) is a machine-processable expression of a phenotypic pattern of these characteristics (Hripcsak and Albers 2018). For the purposes of this work, we are interested in CPs derived from data contained in electronic health record (EHR) systems. This includes both structured data, e.g. codes for diseases, diagnoses, procedures, or laboratory tests, and unstructured data, e.g. free text including patient histories, clinical observations, discharge summaries, and reports. Thus, we define computable phenotype FDOs (CP-FDOs) as a class of FDO that packages an executable EHR-derived CP together with documentation needed to implement and use it effectively for creating cohorts of individuals with similar observable characteristics from EHR data sets. Importance of portable and FAIR CPs: There is tremendous excitement for using real-world EHR data to discover important findings about human health and well-being. However, for discovery to happen, researchers need mechanisms like CPs to identify study cohorts for analysis. Beginning in the early 2010s, a growing literature explores various methods for the secondary use of EHR data for patient phenotyping to arrive at consistent study cohorts (Shivade et al. 2014, Banda et al. 2018). The heterogeneous nature of EHR data has inspired a wide variety of phenotyping methods, from those which rely solely on documented codes linked to terms in existing vocabularies to those which combine such codes with other concepts extracted from free text using natural language processing. Our current focus is on packaging CPs inside FDOs for classifying patients as having or not having a phenotype of interest. This can be done within an individual health system, or at scale across a clinical data research network. Using CPs for cohort identification can reduce the time and expense of traditional data set building and clincal trial recruitment, and expand the potential scope of a study population(Boland et al. 2013). Creating and validating CPs requires time, resources, and both clinical and technical expertise. One estimate is that it can take 6-10 months to develop and validate a CP (Shang et al. 2019). And, as there is no standard data model within EHRs in the United States, many CPs are designed for performance at a single site, rather than for portability, which is understood as the ability to implement a phenotype at a different site with similar performance (Shang et al. 2019). While portability is increasingly recognized as an important element of phenotyping, and there have been recent efforts to develop more portable CPs, many of these processes still require significant technical expertise at the implementation site to adapt the phenotype for use on local data. There may also be significant advantages to making CPs FAIR. These include transparency in cohort selection, and better generalizability of results. FAIR CPs may also increase the potential for robust comparisons of data from related studies, leading to better evidence synthesis to improve delivery of care and ultimately human health. Defining a new class of FDOs to hold and convey CPs: We believe that packaging validated CPs inside digital objects may alleviate many of the pressures mentioned above, and contributes to making both the processes and products of clinical research more FAIR. To this end, our current work focuses on packaging a validated CP inside a machine-processable FDO. The phenotype of interest identifies pediatric and adult patients with a rare disease (Oliverio et al. 2021), and has several features which make it ideal for transformation to an executable FDO. First, the phenotype utilizes standards to define the clinical characteristics of interest, and is based on a common data model; these features increase the potential for both interoperability and reuse. Additionally, because the phenotype has been validated across three sites, its portability has already been demonstrated. Finally, the full computable phenotype has been shared as a series of SQL queries, including scripts for patient identification, deriving statistics, and validation, which have been annotated with instructions for implementation at other sites. The goals of this work are: 1. To develop CPs as executable DOs, leveraging previous work to develop executable Knowledge Objects (KO) (Flynn et al. 2018) 2. To advance our understanding of how to define computable phenotypes as a class of FDO, including what is needed to meet the requirements of binding, abstraction, and encapsulation (Wittenburg et al. 2019) Conclusion: Computable phenotypes, packaged as FDOs, may increase the potential both for the portability of a phenotype and the reusability of data resulting from its implementation. Providing CPs as executable FDOs may also reduce barriers to portability and local implementation. In this presentation, we describe our work to develop a FDO computable phenotype from an existing validated phenotype. Lessons learned from this process will increase our understanding of both the technical requirements, and how to address necessary components of abstraction, binding, and encapsulation so that these can function as FAIR Digital Objects. [ABSTRACT FROM AUTHOR]

Introduction: To advance the goals of the Mobilizing Computable Biomedical Knowledge (MCBK) Movement, we are exploring the use of FAIR Digital Objects (FDOs) (De Smedt et al. 2020, Williams et al. 2021). First, we are beginning to clarify the full range of metadata for FDOs that carry bit sequences expressing knowledge in machine readable or executable formats. We view knowledge through an empirical lens as the reliable, valid, and valued results of analytic or deliberative data analysis. Computability of knowledge refers to the degree to which knowledge is formally represented for use by computing machines. Second, we are figuring out how to apply linked data principles to FDO metadata records (Bizer et al. (2008)). Linked data are structured data with openly defined and uniquely identified concepts. We are developing linked metadata that conform to the Resource Description Format (RDF), where domains of interest are represented using a pattern of subject-predicate-object "triples." RDF triples give rise to machine actionable FDO metadata records that can be visualized as directed graphs. In keeping with the FAIR Digital Object Framework (FDOF), we value linked metadata as a general method of bringing consistency to FDO metadata records, making it so that artificial agents can act on them in predictable ways. Five other benefits of linked metadata are that they are divisible, aggregable, extensible, queryable (using SPARQL), and support logical inferencing. With a focus specifically on FDOs that carry computable knowledge artifacts at their core, here we present our recent metadata work completed between 2019 and mid-2022. Metadata Scope for FDOs Carrying Computable Knowledge This section summarizes previously published work to specify and scope FDO metadata. This work was completed by members of our team and the larger MCBK Movement. Through many dialogs over a period of more than a year, thirteen high-level categories of metadata for FDOs carrying computable knowledge were described (Alper et al. (2021)). These categories are listed in Table 1 below. For detailed explanations and examples of each metadata category above, see our full publication. Next, we briefly discuss six categories marked with an asterisk (*) in Table 1. These six categories are somewhat specific to FDOs that contain computable knowledge. For Knowledge Domain metadata, a large and growing number of biomedical vocabularies or schema exist. For clinical terms, the Standardized Nomenclature of Medicine (SNOMED) includes more than 350K RDF classes and 200 properties. Many bioscience vocabularies spanning a wide range of terms from human biology also exist. Purpose metadata are critical for FDOs that convey computable knowledge about the prevention, diagnosis, treatment, amelioration, and monitoring of disease. Interestingly, we have yet to find vocabularies for representing clinically-oriented FDO purposes as linked metadata. We anticipate needing FDO-to-FDO Relation metadata. Going beyond citations that relate knowledge to its antecedents, FDOs containing computable biomedical knowledge may relate sequentially (diagnostic knowledge preceding treatment knowledge), dependently (stratification depends on measurement), or comparatively (multiple models estimate the same factor). More work is needed to formalize these relations. For technical metadata about FDOs carrying computable knowledge, we emphasize existing vocabularies, including software ontologies like the function ontology. Moreover, for certain FDO operations, webservices are a way of leveraging the decentralized web. As Technical FDO metadata, we can describe FDO-backed webservices semantically by building on the work of the OpenAPI and AsyncAPI initiatives. Finally, we need FDO metadata about two different kinds of evidence. First, there are Evidential Basis metadata that describe features and details about how computable knowledge contained FDOs was generated. Second, there are Evidence from Use metadata that describe the effects of applying the computable knowledge contained in FDOs to simulated or real cases. Linked Metadata for actual FDOs Carrying Computable Knowledge: This section shares new work. Since 2016, we have built and tested several hundred compound Digital Objects (DOs) carrying executable biomedical knowledge in the form of pure functions (e.g., math functions for estimating a health risk) (Beck et al. 2022). Our particular DOs - called Knowledge Objects (KOs) - conform to a common design pattern we created (Fig. 1). We have demonstrated how these DOs can be rapidly implemented in several technical environments to enable RESTful webservice requests and responses to and from pure functions of interest in biomedicine. In a move towards having a specific type of FDOs for carrying computable knowledge, we have started the process of developing linked metadata records for FDOs using a prototype metadata schema. An example of an early FDO linked data record appears in Example 1.... The KO described in the linked metadata record above is available here for inspection. As Example 1 shows in bold text, our initial prototype linked metadata record for KOs relies on three vocabularies, Dublin Core Terms, the Function Ontology, and our own Knowledge Object Implementation Ontology (KOIO). As its FDO identifier, the KO uses an Archival Resource Key (ARK). ARKs are attractive because they support a suffix passthrough mechanism for consistently identifying the common parts of a KO, such as Deployment and Service Descriptions. This linked metadata record in Example 1 has been successfully loaded into several RDF systems, including the JSON-LD Playground and an instance of the Blue Brain Nexus knowledge graph system. We have used SPARQL queries to extract and filter elements from this linked metadata record. Conclusion: For FDOs containing computable knowledge to have high-degrees of FAIRness, extensive metadata records are required. Some metadata content specified to date is specific to this type of FDO and payload. It is possible to represent FDO metadata as linked metadata, making the metadata richer semantically and potentially easier to manage with artificial agents and machines. In biomedicine especially, more work is needed to identify more vocabularies for use as controlled terminologies to arrive at suitably comprehensive linked metadata for this important new type of FDO. [ABSTRACT FROM AUTHOR]

Background: Health care delivery organizations lack evidence-based strategies for using quality measurement data to improve performance. Audit and feedback (A&F), the delivery of clinical performance summaries to providers, demonstrates the potential for large effects on clinical practice but is currently implemented as a blunt one size fits most intervention. Each provider in a care setting typically receives a performance summary of identical metrics in a common format despite the growing recognition that precisionizing interventions hold significant promise in improving their impact. A precision approach to A&F prioritizes the display of information in a single metric that, for each recipient, carries the highest value for performance improvement, such as when the metric's level drops below a peer benchmark or minimum standard for the first time, thereby revealing an actionable performance gap. Furthermore, precision A&F uses an optimal message format (including framing and visual displays) based on what is known about the recipient and the intended gist meaning being communicated to improve message interpretation while reducing the cognitive processing burden. Well-established psychological principles, frameworks, and theories form a feedback intervention knowledge base to achieve precision A&F. From an informatics perspective, precision A&F requires a knowledge-based system that enables mass customization by representing knowledge configurable at the group and individual levels. Objective: This study aims to implement and evaluate a demonstration system for precision A&F in anesthesia care and to assess the effect of precision feedback emails on care quality and outcomes in a national quality improvement consortium. Methods: We propose to achieve our aims by conducting 3 studies: a requirements analysis and preferences elicitation study using human-centered design and conjoint analysis methods, a software service development and implementation study, and a cluster randomized controlled trial of a precision A&F service with a concurrent process evaluation. This study will be conducted with the Multicenter Perioperative Outcomes Group, a national anesthesia quality improvement consortium with >60 member hospitals in >20 US states. This study will extend the Multicenter Perioperative Outcomes Group quality improvement infrastructure by using existing data and performance measurement processes. Results: The proposal was funded in September 2021 with a 4-year timeline. Data collection for Aim 1 began in March 2022. We plan for a 24-month trial timeline, with the intervention period of the trial beginning in March 2024. Conclusions: The proposed aims will collectively demonstrate a precision feedback service developed using an open-source technical infrastructure for computable knowledge management. By implementing and evaluating a demonstration system for precision feedback, we create the potential to observe the conditions under which feedback interventions are effective. [ABSTRACT FROM AUTHOR]

Background: Health care organizations increasingly depend on business intelligence tools, including "dashboards," to capture, analyze, and present data on performance metrics. Ideally, dashboards allow users to quickly visualize actionable data to inform and optimize clinical and organizational performance. In reality, dashboards are typically embedded in complex health care organizations with massive data streams and end users with distinct needs. Thus, designing effective dashboards is a challenging task and theoretical underpinnings of health care dashboards are poorly characterized; even the concept of the dashboard remains ill-defined. Researchers, informaticists, clinical managers, and health care administrators will benefit from a clearer understanding of how dashboards have been developed, implemented, and evaluated, and how the design, end user, and context influence their uptake and effectiveness. Objective: This scoping review first aims to survey the vast published literature of "dashboards" to describe where, why, and for whom they are used in health care settings, as well as how they are developed, implemented, and evaluated. Further, we will examine how dashboard design and content is informed by intended purpose and end users. Methods: In July 2020, we searched MEDLINE, Embase, Web of Science, and the Cochrane Library for peer-reviewed literature using a targeted strategy developed with a research librarian and retrieved 5188 results. Following deduplication, 3306 studies were screened in duplicate for title and abstract. Any abstracts mentioning a health care dashboard were retrieved in full text and are undergoing duplicate review for eligibility. Articles will be included for data extraction and analysis if they describe the development, implementation, or evaluation of a dashboard that was successfully used in routine workflow. Articles will be excluded if they were published before 2015, the full text is unavailable, they are in a non-English language, or they describe dashboards used for public health tracking, in settings where direct patient care is not provided, or in undergraduate medical education. Any discrepancies in eligibility determination will be adjudicated by a third reviewer. We chose to focus on articles published after 2015 and those that describe dashboards that were successfully used in routine practice to identify the most recent and relevant literature to support future dashboard development in the rapidly evolving field of health care informatics. Results: All articles have undergone dual review for title and abstract, with a total of 2019 articles mentioning use of a health care dashboard retrieved in full text for further review. We are currently reviewing all full-text articles in duplicate. We aim to publish findings by mid-2022. Findings will be reported following guidance from the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. Conclusions: This scoping review will provide stakeholders with an overview of existing dashboard tools, highlighting the ways in which dashboards have been developed, implemented, and evaluated in different settings and for different end user groups, and identify potential research gaps. Findings will guide efforts to design and use dashboards in the health care sector more effectively. [ABSTRACT FROM AUTHOR]

Background Errors in reasoning are a common cause of diagnostic error. However, it is difficult to improve performance partly because providers receive little feedback on diagnostic performance. Examining means of providing consistent feedback and enabling continuous improvement may provide novel insights for diagnostic performance. Methods We developed a model for improving diagnostic performance through feedback using a six-step qualitative research process, including a review of existing models from within and outside of medicine, a survey, semistructured interviews with individuals working in and outside of medicine, the development of the new model, an interdisciplinary consensus meeting, and a refinement of the model. Results We applied theory and knowledge from other fields to help us conceptualise learning and comparison and translate that knowledge into an applied diagnostic context. This helped us develop a model, the Diagnosis Learning Cycle, which illustrates the need for clinicians to be given feedback about both their confidence and reasoning in a diagnosis and to be able to seamlessly compare diagnostic hypotheses and outcomes. This information would be stored in a repository to allow accessibility. Such a process would standardise diagnostic feedback and help providers learn from their practice and improve diagnostic performance. This model adds to existing models in diagnosis by including a detailed picture of diagnostic reasoning and the elements required to improve outcomes and calibration. Conclusion A consistent, standard programme of feedback that includes representations of clinicians' confidence and reasoning is a common element in non-medical fields that could be applied to medicine. Adapting this approach to diagnosis in healthcare is a promising next step. This information must be stored reliably and accessed consistently. The next steps include testing the Diagnosis Learning Cycle in clinical settings. [ABSTRACT FROM AUTHOR]

Automation, workflow, knowledge management, health information technology, infrastructure Keywords: automation; workflow; knowledge management; health information technology; infrastructure EN automation workflow knowledge management health information technology infrastructure 1222 1223 2 05/24/23 20230601 NES 230601 Dear Editors, We recognize that inefficient and idiosyncratic workflows in health care contribute to a myriad of obstacles for all healthcare stakeholders, including misuse of resources, provider burnout, and increased burden on patients and their caregivers.[1] Therefore, we were pleased to read Zayas-Cabán et al's[2] article, "Priorities to accelerate workflow automation in healthcare.". [Extracted from the article]

Background: Evidence shows that clinical audit and feedback can significantly improve compliance with desired practice, but it is unclear when and how it is effective. Audit and feedback is likely to be more effective when feedback messages can influence barriers to behavior change, but barriers to change differ across individual health-care providers, stemming from differences in providers' individual characteristics. Discussion: The purpose of this article is to invite debate and direct research attention towards a novel audit and feedback component that could enable interventions to adapt to barriers to behavior change for individual health-care providers: computer-supported tailoring of feedback messages. We argue that, by leveraging available clinical data, theory-informed knowledge about behavior change, and the knowledge of clinical supervisors or peers who deliver feedback messages, a software application that supports feedback message tailoring could improve feedback message relevance for barriers to behavior change, thereby increasing the effectiveness of audit and feedback interventions. We describe a prototype system that supports the provision of tailored feedback messages by generating a menu of graphical and textual messages with associated descriptions of targeted barriers to behavior change. Supervisors could use the menu to select messages based on their awareness of each feedback recipient's specific barriers to behavior change. We anticipate that such a system, if designed appropriately, could guide supervisors towards giving more effective feedback for health-care providers. Summary: A foundation of evidence and knowledge in related health research domains supports the development of feedback message tailoring systems for clinical audit and feedback. Creating and evaluating computer-supported feedback tailoring tools is a promising approach to improving the effectiveness of clinical audit and feedback. [ABSTRACT FROM AUTHOR]

The objective of this study was to determine the relative efficiency of novices compared to a prediction of skilled use when performing tasks using the touchscreen interface of an EMR developed in Malawi. We observed novice users performing touchscreen tasks and recorded timestamp data from their performances. Using a predictive human performance modeling tool, the authors predicted the skilled task performance time for each task. Efficiency and rates of error were evaluated with respect to user interface design. Nineteen participants performed 31 EMR tasks seven times for a total of 4,123 observed performances. We analyzed twelve representative tasks leaving 1,596 performances featuring six user interface designs. Mean novice performance time was significantly slower than mean predicted skilled performance time (p < 0.001). However, novices performed faster than the predicted skilled level in 208 (13%) of successful task performances. These findings suggest the user interface design supports a primary design goal of the EMR – to allow novice users to perform tasks efficiently and effectively. [ABSTRACT FROM AUTHOR]

Background: Audit and feedback (A&F) is a common quality improvement strategy with highly variable effects on patient care. It is unclear how A&F effectiveness can be maximised. Since the core mechanism of action of A&F depends on drawing attention to a discrepancy between actual and desired performance, we aimed to understand current and best practices in the choice of performance comparator.Methods: We described current choices for performance comparators by conducting a secondary review of randomised trials of A&F interventions and identifying the associated mechanisms that might have implications for effective A&F by reviewing theories and empirical studies from a recent qualitative evidence synthesis.Results: We found across 146 trials that feedback recipients' performance was most frequently compared against the performance of others (benchmarks; 60.3%). Other comparators included recipients' own performance over time (trends; 9.6%) and target standards (explicit targets; 11.0%), and 13% of trials used a combination of these options. In studies featuring benchmarks, 42% compared against mean performance. Eight (5.5%) trials provided a rationale for using a specific comparator. We distilled mechanisms of each comparator from 12 behavioural theories, 5 randomised trials, and 42 qualitative A&F studies.Conclusion: Clinical performance comparators in published literature were poorly informed by theory and did not explicitly account for mechanisms reported in qualitative studies. Based on our review, we argue that there is considerable opportunity to improve the design of performance comparators by (1) providing tailored comparisons rather than benchmarking everyone against the mean, (2) limiting the amount of comparators being displayed while providing more comparative information upon request to balance the feedback's credibility and actionability, (3) providing performance trends but not trends alone, and (4) encouraging feedback recipients to set personal, explicit targets guided by relevant information. [ABSTRACT FROM AUTHOR]

Background: The distribution of printed materials is the most frequently used strategy to disseminate and implement clinical practice guidelines, although several studies have shown that the effectiveness of this approach is modest at best. Nevertheless, there is insufficient evidence to support the use of other strategies. Recent research has shown that the use of computerized decision support presents a promising approach to address some aspects of this problem. Objective: The aim of this study is to provide qualitative evidence on the potential effect of mobile decision support systems to facilitate the implementation of evidence-based recommendations included in clinical practice guidelines. Methods: We will conduct a qualitative study with two arms to compare the experience of primary care physicians while they try to implement an evidence-based recommendation in their clinical practice. In the first arm, we will provide participants with a printout of the guideline article containing the recommendation, while in the second arm, we will provide participants with a mobile app developed after formalizing the recommendation text into a clinical algorithm. Data will be collected using semistructured and open interviews to explore aspects of behavioral change and technology acceptance involved in the implementation process. The analysis will be comprised of two phases. During the first phase, we will conduct a template analysis to identify barriers and facilitators in each scenario. Then, during the second phase, we will contrast the findings from each arm to propose hypotheses about the potential impact of the system. Results: We have formalized the narrative in the recommendation into a clinical algorithm and have developed a mobile app. Data collection is expected to occur during 2018, with the first phase of analysis running in parallel. The second phase is scheduled to conclude in July 2019. Conclusions: Our study will further the understanding of the role of mobile decision support systems in the implementation of clinical practice guidelines. Furthermore, we will provide qualitative evidence to aid decisions made by low- and middle-income countries' ministries of health about investments in these technologies. [ABSTRACT FROM AUTHOR]

Correction: The authors would like to correct errors in the original article [1] that may have lead readers to misinterpret the scope, evidence base and target population of VHA Handbook 1004.03 "Life-Sustaining Treatment (LST) Decisions: Eliciting, Documenting, and Honoring Patients' Values, Goals, and Preferences". [ABSTRACT FROM AUTHOR]

Background: Health care professionals must learn continuously as a core part of their work. As the rate of knowledge production in biomedicine increases, better support for health care professionals' continuous learning is needed. In health systems, feedback is pervasive and is widely considered to be essential for learning that drives improvement. Clinical quality dashboards are one widely deployed approach to delivering feedback, but engagement with these systems is commonly low, reflecting a limited understanding of how to improve the effectiveness of feedback about health care. When coaches and facilitators deliver feedback for improving performance, they aim to be responsive to the recipient's motivations, information needs, and preferences. However, such functionality is largely missing from dashboards and feedback reports. Precision feedback is the delivery of high-value, motivating performance information that is prioritized based on its motivational potential for a specific recipient, including their needs and preferences. Anesthesia care offers a clinical domain with high-quality performance data and an abundance of evidence-based quality metrics., Objective: The objective of this study is to explore anesthesia provider preferences for precision feedback., Methods: We developed a test set of precision feedback messages with balanced characteristics across 4 performance scenarios. We created an experimental design to expose participants to contrasting message versions. We recruited anesthesia providers and elicited their preferences through analysis of the content of preferred messages. Participants additionally rated their perceived benefit of preferred messages to clinical practice on a 5-point Likert scale., Results: We elicited preferences and feedback message benefit ratings from 35 participants. Preferences were diverse across participants but largely consistent within participants. Participants' preferences were consistent for message temporality (α=.85) and display format (α=.80). Ratings of participants' perceived benefit to clinical practice of preferred messages were high (mean rating 4.27, SD 0.77)., Conclusions: Health care professionals exhibited diverse yet internally consistent preferences for precision feedback across a set of performance scenarios, while also giving messages high ratings of perceived benefit. A "one-size-fits-most approach" to performance feedback delivery would not appear to satisfy these preferences. Precision feedback systems may hold potential to improve support for health care professionals' continuous learning by accommodating feedback preferences., (© Zach Landis-Lewis, Chris A Andrews, Colin A Gross, Charles P Friedman, Nirav J Shah. Originally published in JMIR Medical Education (https://mededu.jmir.org).)

Background: National guidelines recommend that maternity systems provide patient-centered access to immediate postpartum long-acting reversible contraception (ie, insertion of an intrauterine device or implant during the delivery hospitalization). Hospitals face significant barriers to offering these services, and efforts to improve peripartum contraception care quality have met with mixed success. Implementation toolkits-packages of resources and strategies to facilitate the implementation of new services-are a promising approach for guiding clinical practice change., Objective: This study aimed to develop a theory-informed toolkit, evaluate the feasibility of toolkit-based implementation of immediate postpartum long-acting reversible contraception care in a single site, and refine the toolkit and implementation process for future effectiveness testing., Study Design: We conducted a single-site feasibility study of the toolkit-based implementation of immediate postpartum contraception services at a large academic medical center in 2017 to 2020. Based on previous qualitative work, we developed a theory-informed implementation toolkit. A stakeholder panel selected toolkit resources to use in a multicomponent implementation intervention at the study site. These resources included tools and strategies designed to optimize implementation conditions (ie, implementation leadership, planning, and evaluation; the financial environment; engagement of key stakeholders; patient needs; compatibility with workflow; and clinician and staff knowledge, skills, and attitudes). The implementation intervention was executed from January 2018 to April 2019. Study outcomes included implementation outcomes (ie, provider perceptions of the implementation process and implementation tools [assessed via online provider survey]) and healthcare quality outcomes (ie, trends in prenatal contraceptive counseling, trends in immediate postpartum long-acting reversible contraceptive utilization [both ascertained by institutional administrative data], and the patient experience of contraceptive care [assessed via serial, cross-sectional, online patient survey items adapted from the National Quality Forum-endorsed, validated Person-Centered Contraceptive Counseling measure])., Results: In the implementation process, among 172 of 401 eligible clinicians (43%) participating in surveys, 70% were "extremely" or "somewhat" satisfied with the implementation process overall. In the prenatal contraceptive counseling, among 4960 individuals undergoing childbirth at the study site in 2019, 1789 (36.1%) had documented prenatal counseling about postpartum contraception. Documented counseling rates increased overall throughout 2019 (Q1, 12.5%; Q4, 51.0%) but varied significantly by clinic site (Q4, range 30%-79%). Immediate postpartum long-acting reversible contraception utilization increased throughout the study period (before implementation, 5.46% of deliveries; during implementation, 8.95%; after implementation, 8.58%). In the patient experience of contraceptive care, patient survey respondents (response rate, 15%-29%) were largely White (344/425 [81%]) and highly educated (309/425 [73%] with at least a 4-year college degree), reflecting the study site population. Scores were poor across settings, with modest improvements in the hospital setting from 2018 to 2020 (prenatal visits, 67%-63%; hospitalization, 45%-58%; outpatient after delivery, 69%-65%). Based on these findings, toolkit refinements included additional resources designed to routinize prenatal contraceptive counseling and support a more patient-centered experience of contraceptive care., Conclusion: A toolkit-based process to implement immediate postpartum long-acting reversible contraceptive services at a single academic center was associated with high acceptability but mixed healthcare quality outcomes. Toolkit resources were added to optimize counseling rates and the patient experience of contraceptive care. Future research should formally test the effectiveness of the refined toolkit in a multisite, prospective trial., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Improving visualizations in clinical quality reports and dashboards may improve the visualization influence on clinical practice. Tailored displays could accommodate individual and situational differences, but these diplays introduce complex requirements across healthcare professionals and teams. We applied user stories, a method for managing complex software requirements, to a user-centered design process for tailored visual displays about postpartum contraception care. We mapped user stories to tailored displays to identify the quantity of displays that were supported by each user story. We developed 9 tailored displays and 11 user stories. Displays varied in their mappings to user stories (mean 5, max 9, min 0), revealing differences in healthcare professionals and teams preferences and information needs. User stories and user-centered design may be useful for healthcare organizations to manage complex requirements of tailored displays in clinical practice feedback.

Background: The distribution of printed materials is the most frequently used strategy to disseminate and implement clinical practice guidelines, although several studies have shown that the effectiveness of this approach is modest at best. Nevertheless, there is insufficient evidence to support the use of other strategies. Recent research has shown that the use of computerized decision support presents a promising approach to address some aspects of this problem., Objective: The aim of this study is to provide qualitative evidence on the potential effect of mobile decision support systems to facilitate the implementation of evidence-based recommendations included in clinical practice guidelines., Methods: We will conduct a qualitative study with two arms to compare the experience of primary care physicians while they try to implement an evidence-based recommendation in their clinical practice. In the first arm, we will provide participants with a printout of the guideline article containing the recommendation, while in the second arm, we will provide participants with a mobile app developed after formalizing the recommendation text into a clinical algorithm. Data will be collected using semistructured and open interviews to explore aspects of behavioral change and technology acceptance involved in the implementation process. The analysis will be comprised of two phases. During the first phase, we will conduct a template analysis to identify barriers and facilitators in each scenario. Then, during the second phase, we will contrast the findings from each arm to propose hypotheses about the potential impact of the system., Results: We have formalized the narrative in the recommendation into a clinical algorithm and have developed a mobile app. Data collection is expected to occur during 2018, with the first phase of analysis running in parallel. The second phase is scheduled to conclude in July 2019., Conclusions: Our study will further the understanding of the role of mobile decision support systems in the implementation of clinical practice guidelines. Furthermore, we will provide qualitative evidence to aid decisions made by low- and middle-income countries' ministries of health about investments in these technologies., (©Jhon Camacho, Ana María Medina Ch, Zach Landis-Lewis, Gerald Douglas, Richard Boyce. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 13.04.2018.)

Although performance feedback has the potential to help clinicians improve the quality and safety of care, healthcare organizations generally lack knowledge about how this guidance is best provided. In low-resource settings, tools for theory-informed feedback tailoring may enhance limited clinical supervision resources. Our objectives were to establish proof-of-concept for computer-supported feedback message tailoring in Malawi, Africa. We conducted this research in five stages: clinical performance measurement, modeling the influence of feedback on antiretroviral therapy (ART) performance, creating a rule-based message tailoring process, generating tailored messages for recipients, and finally analysis of performance and message tailoring data. We retrospectively generated tailored messages for 7,448 monthly performance reports from 11 ART clinics. We found that tailored feedback could be routinely generated for four guideline-based performance indicators, with 35% of reports having messages prioritized to optimize the effect of feedback. This research establishes proof-of-concept for a novel approach to improving the use of clinical performance feedback in low-resource settings and suggests possible directions for prospective evaluations comparing alternative designs of feedback messages.