Search

BACKGROUND The proliferation of health care data presented in electronic health records (EHRs) is fueling the need to consolidate and present the salient aspects in ways that reduce cognitive processing time and promote accurate use. The format in which data are presented can play a key role in achieving the desired patient outcomes while reducing burden to clinicians. OBJECTIVE To compare care planning time (CPT) and adoption of evidence-based clinical decision support (CDS) recommendations by registered nurses (RNs) randomly assigned to one of four CDS format groups; 1) text only (TO); 2) text+table (TT); 3) text+graph (TG); 4) tailored (based on RN’s graph literacy score). We hypothesize that the tailored CDS group will have faster CPT (primary) and higher adoption rates (secondary) than the three non-tailored CDS groups. METHODS Eligible RNs employed in an adult hospital unit within the past 2 years were recruited randomly from 10 state nursing licensure lists representing the five regions of the United States (NE, SE, MW, SW, W) to participate in a randomized controlled trial (RCT). RNs were randomly assigned to one of four CDS format groups; (1) text only; 2) text+table; 3) text+graph; 4) tailored (based on RN’s graph literacy score) and interacted with the intervention on their personal computers. Regression analysis was performed to estimate the effect of tailoring and the association between CDS format and RN characteristics. RESULTS The differences between the tailored (n=46) and the non-tailored (TO [n=55], TT [n=54], TG [n=48]) CDS groups were not significant for either the CPT or the CDS adoption rate. RNs with low graph literacy had longer CPT interacting with text+graph CDS format than text only CDS format (p=.01). CPT under text+graph CDS format was associated with age (p=.02), graph literacy (p=.02), and comfort with EHR (p CONCLUSIONS We demonstrated the feasibility of an innovative approach for studying EHR CDS in a simulation environment that enables preliminary testing of EHR innovations with minimum unintended risk to patients. Our study findings provided further evidence of the effectiveness of CDS in supporting the adoption of evidence-based recommendations. Although our specific approach of tailoring the CDS format to a participant’s graph literacy did not lead to improvement in either care planning time or CDS adoption rate, we found a robust association between graph literacy and time needed to process graph based CDS. The study also revealed additional important factors to consider when tailoring CDS formats for nurse care planning (i.e., age, comfort with EHR).

BACKGROUND Poor usability is a primary cause of unintended consequences related to the use of electronic health record (EHR) systems, which negatively impacts patient safety. Due to the cost and time needed to carry out iterative evaluations, many EHR components, such as clinical decision support systems (CDSSs), have not undergone rigorous usability testing prior to their deployment in clinical practice. Usability testing in the predeployment phase is crucial to eliminating usability issues and preventing costly fixes that will be needed if these issues are found after the system’s implementation. OBJECTIVE This study presents an example application of a systematic evaluation method that uses clinician experts with human-computer interaction (HCI) expertise to evaluate the usability of an electronic clinical decision support (CDS) intervention prior to its deployment in a randomized controlled trial. METHODS We invited 6 HCI experts to participate in a heuristic evaluation of our CDS intervention. Each expert was asked to independently explore the intervention at least twice. After completing the assigned tasks using patient scenarios, each expert completed a heuristic evaluation checklist developed by Bright et al based on Nielsen’s 10 heuristics. The experts also rated the overall severity of each identified heuristic violation on a scale of 0 to 4, where 0 indicates no problems and 4 indicates a usability catastrophe. Data from the experts’ coded comments were synthesized, and the severity of each identified usability heuristic was analyzed. RESULTS The 6 HCI experts included professionals from the fields of nursing (n=4), pharmaceutical science (n=1), and systems engineering (n=1). The mean overall severity scores of the identified heuristic violations ranged from 0.66 (flexibility and efficiency of use) to 2.00 (user control and freedom and error prevention), in which scores closer to 0 indicate a more usable system. The heuristic principle user control and freedom was identified as the most in need of refinement and, particularly by nonnursing HCI experts, considered as having major usability problems. In response to the heuristic match between system and the real world, the experts pointed to the reversed direction of our system’s pain scale scores (1=severe pain) compared to those commonly used in clinical practice (typically 1=mild pain); although this was identified as a minor usability problem, its refinement was repeatedly emphasized by nursing HCI experts. CONCLUSIONS Our heuristic evaluation process is simple and systematic and can be used at multiple stages of system development to reduce the time and cost needed to establish the usability of a system before its widespread implementation. Furthermore, heuristic evaluations can help organizations develop transparent reporting protocols for usability, as required by Title IV of the 21st Century Cures Act. Testing of EHRs and CDSSs by clinicians with HCI expertise in heuristic evaluation processes has the potential to reduce the frequency of testing while increasing its quality, which may reduce clinicians’ cognitive workload and errors and enhance the adoption of EHRs and CDSSs.

Background Poor usability is a primary cause of unintended consequences related to the use of electronic health record (EHR) systems, which negatively impacts patient safety. Due to the cost and time needed to carry out iterative evaluations, many EHR components, such as clinical decision support systems (CDSSs), have not undergone rigorous usability testing prior to their deployment in clinical practice. Usability testing in the predeployment phase is crucial to eliminating usability issues and preventing costly fixes that will be needed if these issues are found after the system’s implementation. Objective This study presents an example application of a systematic evaluation method that uses clinician experts with human-computer interaction (HCI) expertise to evaluate the usability of an electronic clinical decision support (CDS) intervention prior to its deployment in a randomized controlled trial. Methods We invited 6 HCI experts to participate in a heuristic evaluation of our CDS intervention. Each expert was asked to independently explore the intervention at least twice. After completing the assigned tasks using patient scenarios, each expert completed a heuristic evaluation checklist developed by Bright et al based on Nielsen’s 10 heuristics. The experts also rated the overall severity of each identified heuristic violation on a scale of 0 to 4, where 0 indicates no problems and 4 indicates a usability catastrophe. Data from the experts’ coded comments were synthesized, and the severity of each identified usability heuristic was analyzed. Results The 6 HCI experts included professionals from the fields of nursing (n=4), pharmaceutical science (n=1), and systems engineering (n=1). The mean overall severity scores of the identified heuristic violations ranged from 0.66 (flexibility and efficiency of use) to 2.00 (user control and freedom and error prevention), in which scores closer to 0 indicate a more usable system. The heuristic principle user control and freedom was identified as the most in need of refinement and, particularly by nonnursing HCI experts, considered as having major usability problems. In response to the heuristic match between system and the real world, the experts pointed to the reversed direction of our system’s pain scale scores (1=severe pain) compared to those commonly used in clinical practice (typically 1=mild pain); although this was identified as a minor usability problem, its refinement was repeatedly emphasized by nursing HCI experts. Conclusions Our heuristic evaluation process is simple and systematic and can be used at multiple stages of system development to reduce the time and cost needed to establish the usability of a system before its widespread implementation. Furthermore, heuristic evaluations can help organizations develop transparent reporting protocols for usability, as required by Title IV of the 21st Century Cures Act. Testing of EHRs and CDSSs by clinicians with HCI expertise in heuristic evaluation processes has the potential to reduce the frequency of testing while increasing its quality, which may reduce clinicians’ cognitive workload and errors and enhance the adoption of EHRs and CDSSs.