Search

Objectives To identify interventions implemented in hospital electronic prescribing systems and the outcome measures used to monitor their impact. Methods We systematically searched CINAHL, EMBASE, Google Scholar and Medline using keywords in three strands: (i) population: hospital inpatient or emergency department; (ii) intervention: electronic prescribing functionality; and (iii) outcome: antimicrobial stewardship. The interventions were grouped into six themes: alerts, order sets, restriction of access, mandated documentation, embedded guidelines and automatic prescription stop. The outcome measures were organized into those that measure the quality or quantity of prescribing or clinical decision support (CDS) activity. The impact of each intervention reported was grouped into a positive, negative or no change. Results A total of 28 studies were eligible for inclusion. There were 28 different interventions grouped into the six themes. Alerts visible to the practitioner in the electronic health record (EHR) were most frequently implemented (n = 11/28). Twenty different outcome measures were identified, divided into quality (n = 13/20) and quantity outcomes (n = 4/20) and CDS activity (n = 3/20). One-third of outcomes reported across the 28 studies showed positive change (34.4%, n = 42/122) and 61.4% (n = 75/122) showed no change. Conclusions The most frequently implemented interventions were alerts, the majority of which were to influence behaviour or decision-making of the practitioner within the EHR. Quality outcomes were most frequently selected by researchers. The review supports previous research that larger well-designed randomized studies are needed to investigate the impact of interventions on AMS and outcome measures to be standardized.

Background Despite the increase in use and high expectations of digital health solutions, scientific evidence about the effectiveness of electronic health (eHealth) and other aspects such as usability and accuracy is lagging behind. eHealth solutions are complex interventions, which require a wide array of evaluation approaches that are capable of answering the many different questions that arise during the consecutive study phases of eHealth development and implementation. However, evaluators seem to struggle in choosing suitable evaluation approaches in relation to a specific study phase. Objective The objective of this project was to provide a structured overview of the existing eHealth evaluation approaches, with the aim of assisting eHealth evaluators in selecting a suitable approach for evaluating their eHealth solution at a specific evaluation study phase. Methods Three consecutive steps were followed. Step 1 was a systematic scoping review, summarizing existing eHealth evaluation approaches. Step 2 was a concept mapping study asking eHealth researchers about approaches for evaluating eHealth. In step 3, the results of step 1 and 2 were used to develop an “eHealth evaluation cycle” and subsequently compose the online “eHealth methodology guide.” Results The scoping review yielded 57 articles describing 50 unique evaluation approaches. The concept mapping study questioned 43 eHealth researchers, resulting in 48 unique approaches. After removing duplicates, 75 unique evaluation approaches remained. Thereafter, an “eHealth evaluation cycle” was developed, consisting of six evaluation study phases: conceptual and planning, design, development and usability, pilot (feasibility), effectiveness (impact), uptake (implementation), and all phases. Finally, the “eHealth methodology guide” was composed by assigning the 75 evaluation approaches to the specific study phases of the “eHealth evaluation cycle.” Conclusions Seventy-five unique evaluation approaches were found in the literature and suggested by eHealth researchers, which served as content for the online “eHealth methodology guide.” By assisting evaluators in selecting a suitable evaluation approach in relation to a specific study phase of the “eHealth evaluation cycle,” the guide aims to enhance the quality, safety, and successful long-term implementation of novel eHealth solutions.