Search

Objective The growing trend to use wearable devices to track activity and health data has the potential to positively impact the patient experience with their health care at home and with their care team. As part of a pilot program, the U.S. Department of Veterans Affairs (VA) distributed Fitbits to Veterans through four VA medical centers. Our objective was to assess the program from both Veterans' and clinicians' viewpoints. Specifically, we aimed to understand barriers to Fitbit setup and use for Veterans, including syncing devices with a VA mobile application (app) to share data, and assess the perceived value of the device functions and ability to share information from the Fitbit with their care team. In addition, we explored the clinicians' perspective, including how they expected to use the patient-generated health data (PGHD). Methods We performed semi-structured interviews with 26 Veterans and 16 VA clinicians to assess the program. Responses to each question were summarized in order of frequency of occurrence across participants and audited by an independent analyst for accuracy. Results Our findings reveal that despite setup challenges, there is support for the use of Fitbits to engage Veterans and help manage their health. Clinicians believed there were benefits for having Veterans use the Fitbits and expected to use the PGHD in a variety of ways as part of the Veterans' care plans, including monitoring progress toward health behavior goals. Veterans were overwhelmingly enthusiastic about using the Fitbits; this enthusiasm seems to extend beyond the 3 month “novelty period.” Conclusion The pilot program for distributing Fitbits to Veterans appears to be successful from both Veterans' and clinicians' perspectives and suggests that expanded use of wearable devices should be considered. Future studies will need to carefully consider how to incorporate the PGHD into the electronic health record and clinical workflow.

While there is an extensive and established history of research that demonstrates the unfortunate capacity of exam room computing and electronic health records (EHRs) to negatively impact provider-patient communication and interaction, recent trends in exam room computing are promising in that there may be an opportunity for the EHR to improve patient engagement. The logical evolution of this research is to flip the narrative to understand strategies for using exam room computing as a mediator or facilitator of provider-patient communication and interaction, rather than simply establishing ways to mitigate the documented barriers. Panelists will present and discuss their latest research and experiences that may contribute to the evolution of using exam room computing as a tool to enhance provider-patient communication and interaction. We will also discuss how a thoughtful exam room design with patient-centered exam room computing technologies and practices may positively impact specific human factors, safety, and bias outcomes.

This practice-oriented paper presents a human-centered design (HCD) framework that we developed to perform a comprehensive evaluation of a new health information technology (HIT) system under development, intended to replace a legacy system. The Department of Veterans Affairs (VA) Veteran Crisis Line (VCL) program provides a vital service in crisis intervention and suicide prevention. VCL staff rely on a Customer Relationship Management (CRM) legacy system, Medora. VCL intended to replace Medora with Microsoft Dynamics 365 (D365) CRM system. Due to wide-spread criticism of D365, the VA Human Factors Engineering (HFE) team engaged in a multi-study, mixed-method HCD evaluation to investigate the legacy system and intended replacement in terms of ability to support VCL staff needs. The HCD framework we developed to perform this evaluation may be adapted for other large-scale HIT transitions and may provide human factors practitioners with guidance to make evidence-based decisions to support (or abandon) such transitions.

Over the last 15 years, healthcare delivery has experienced enormous changes across every aspect of clinical care, both for participants in it and affiliated stakeholders. The plethora and breadth of these changes and ongoing struggles with their incorporation into the work of healthcare begs the question: In 2035, what emerging innovations and considerations will we wish we had prepared for with regard to human factors engineering research, technology development, and system redesign efforts? Panelists and co-chairs have deep experience with steering, and being impacted by, innovations in information technology in healthcare. The panelists represent diverse academic, clinical, government contract research and development, and small entrepreneurial business. Perhaps the biggest benefit of this session will be the explicit inclusion of the audience in brainstorming and discussion activities, which will be facilitated digitally and verbally during the session and shared afterwards electronically with participants.

Provider burnout has reached epidemic levels, especially with primary care-oriented specialties such as Family Medicine. Guided by a sociotechnical systems perspective, we investigated contributing burnout factors that relate to the electronic health record (EHR) and clinical workflow in an academic healthcare institution. We conducted semi-structured interviews with 10 family medicine and geriatrics providers and administered EHR usability and workflow integration surveys. Findings are organized around recurrent, overarching themes: (1) Clinic Workflow, (2) Documentation, (3) EHR Workflow and Usability, (4) Patient Complexity, (5) Staffing, and (6) Technical Issues. The most consistent finding across all provider interviews was poor EHR usability as a contributing factor to burnout; especially the number of clicks needed to complete EHR tasks. This finding is supported by low usability and workflow integration survey ratings. Using a sociotechnical systems framework, we demonstrate social, technological, and environmental contributors to burnout and discuss potential interventions to mitigate these contributing factors.

Given the substantial increase in interest and activity in the virtual care space, the Department of Veterans Affairs (VA) is making a concerted effort to increase opportunities for Veterans to adopt virtual care applications. We investigated Veteran attitudes, interests, and needs for virtual care applications, including privacy considerations and specific types of data they would be comfortable sharing with their providers through various Veteran-facing tools. We administered a 28-item questionnaire to 40 Veterans and performed follow-up interviews with a subset of nine Veterans. Study results revealed broad support by Veterans for using virtual care applications, including wearable devices, for sharing all types of health information with their providers. Further, Veterans expressed a desire for consolidation across VA-provided virtual care tools. Our findings will inform a strategy for Veteran-facing tools. Other health care systems may be interested in exploring these topics, based on our study results.

Coping with stressors related to the coronavirus disease (COVID) pandemic has caused major mental health challenges. Text message interventions are identified as helpful for promoting health behavior self-management. Drawing from cognitive-behavioral theory (CBT), U.S. Veterans Health Administration staff developed a 20-week automated text messaging protocol called "Coping During COVID" to support veterans' self-care during the pandemic. This project evaluated attitudes about and perceived helpfulness of the protocol among 651 veterans. A cross-sectional evaluation quantitatively assessed COVID-related stressors as well as reported use, acceptability, attitudes, and perceived helpfulness of the protocol via a self-administered online questionnaire. Open-ended questions assessed perceived helpfulness/unhelpfulness of the protocol. Of the 651 veterans (22% of those enrolled in the protocol) who responded to the survey, 73.1% were male and 88.5% were 50 years or over. Ninety percent found the protocol messages helpful and about half endorsed various additional benefits, including feeling better, being more educated about coping behaviors, and being better able to focus on positive thoughts. Qualitative feedback was overwhelmingly positive. For example, veterans stated that the protocol "was like having a personal coach giving hints and tips to cope with the boredom every day," that it "gave me reminders of skills I was not using. It helped me focus on the positives," and brought relief from "many hopeless and sad thoughts … the messages from Annie have been a Godsend to me." Overall results suggested that the Coping During COVID text messaging protocol successfully supported the majority of respondents' self-care efforts during the COVID pandemic. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The Veterans Health Administration (VHA) within the Department of Veterans Affairs (VA) was interested in determining how mobile tablets may be perceived by clinicians for use in clinical settings. Therefore, we conducted an evaluation to assess how performance might differ between two devices and to gather feedback on device use. We conducted a within-subject comparison with 32 clinicians involving a usability test and two questionnaires. Qualitative data was organized around eight themes: facilitators and barriers to tablet use, observations of physical use, device specific advantages and disadvantages, and the Patient Viewer mobile application. Clinicians envisioned many facilitators to tablet use, but also voiced some concerns. Participants rated one device significantly better than the other for questionnaire items related to ‘Access and Efficiency’, ‘Introduction into the Clinical Environment’, ‘Usability and Usefulness’, and ‘Desire to Use’. Results can be used by mobile health app developers, healthcare organizations considering device purchases, and researchers conducting studies on tablets to inform respective work.

The US Department of Veterans Affairs (VA) is using an automated short message service application named “Annie” as part of its coronavirus disease 2019 (COVID-19) response with a protocol for coronavirus precautions, which can help the veteran monitor symptoms and can advise the veteran when to contact his or her VA care team or a nurse triage line. We surveyed 1134 veterans on their use of the Annie application and coronavirus precautions protocol. Survey results support what is likely a substantial resource savings for the VA, as well as non-VA community healthcare. Moreover, the majority of veterans reported at least 1 positive sentiment (felt more connected to VA, confident, or educated and/or felt less anxious) by receiving the protocol messages. The findings from this study have implications for other healthcare systems to help manage a patient population during the coronavirus pandemic.

The coordination of site visits to execute human factors methods, such as onsite usability tests, interviews, or observations, in clinical settings requires a high level of management to attain successful data collection outcomes. Members of the Department of Veterans Affairs (VA) Veterans Health Administration (VHA) Human Factors Engineering (HFE) team occasionally visit VHA medical centers or outpatient clinics to complete our work. We have developed a site visit checklist as a practice innovation to facilitate logistical coordination when gathering data onsite. This Practice-Oriented paper includes the full checklist, as well as discussion of its use to enable other groups to benefit from lessons we have learned in conducting onsite work in health care settings.

This paper reports the results of a literature review of health care organizations that have transitioned from one electronic health record (EHR) to another. Ten different EHR to EHR transitions are documented in the academic literature. In eight of the 10 transitions, the health care organization transitioned to Epic, a commercial EHR which is dominating the market for large and medium hospitals and health care systems. The focus of the articles reviewed falls into two main categories: (1) data migration from the old to new EHR and (2) implementation of the new EHR as it relates to patient safety, provider satisfaction, and other measures pre-and post-transition. Several conclusions and recommendations are derived from this review of the literature, which may be informative for healthcare organizations preparing to replace an existing EHR. These recommendations are likely broadly relevant to EHR to EHR transitions, regardless of the new EHR vendor.

Although numerous healthcare organizations have transitioned from one electronic health record (EHR) to another or are currently planning a transition, there are few documented artifacts, such as published studies or operationalizable resources, that offer guidance on such transitions. This panel seeks to begin a conversation about human factors considerations in EHR transitions from a legacy system. Panel members will discuss current literature and research on the topic as well as experiences with and lessons learned from transitions within their organizations. Panel discussion can be expected to identify new research opportunities, needed resources, and guidance for EHR vendors or healthcare facilities in the midst of or preparing for an EHR transition. Panelists will also lay out systemic issues that need to be addressed at the national policy and regulatory level. This topic is relevant not only to full-scale EHR transitions, but also has applicability for significant EHR version changes.

Background During medical referrals, communication barriers between referring and consulting outpatient clinics delay patients’ access to health care. One notable opportunity for reducing these barriers is improved usefulness and usability of electronic medical consultation order forms. The cognitive systems engineering (CSE) design approach focuses on supporting humans in managing cognitive complexity in sociotechnical systems. Cognitive complexity includes communication, decision-making, problem solving, and planning. Objective The objective of this research was to implement a CSE design approach to develop a template that supports the cognitive needs of referring clinicians and improves referral communication. Methods We conducted interviews and observations with primary care providers and specialists at two major tertiary, urban medical facilities. Using qualitative analysis, we identified cognitive requirements and design guidelines. Next, we designed user interface (UI) prototypes and compared their usability with that of a currently implemented UI at a major Midwestern medical facility. Results Physicians’ cognitive challenges were summarized in four cognitive requirements and 13 design guidelines. As a result, two UI prototypes were developed to support order template search and completion. To compare UIs, 30 clinicians (referrers) participated in a consultation ordering simulation complemented with the think-aloud elicitation method. Oral comments about the UIs were coded for both content and valence (i.e., positive, neutral, or negative). Across 619 comments, the odds ratio for the UI prototype to elicit higher-valenced comments than the implemented UI was 13.5 (95% CI = [9.2, 19.8]), p Conclusion This study reinforced the significance of applying a CSE design approach to inform the design of health information technology. In addition, knowledge elicitation methods enabled identification of physicians’ cognitive requirements and challenges when completing electronic medical consultation orders. The resultant knowledge was used to derive design guidelines and UI prototypes that were more useful and usable for referring physicians. Our results support the implementation of a CSE design approach for electronic medical consultation orders.

Objective To create an operational definition and framework to study diagnostic error in the emergency department setting. Methods We convened a 17-member multidisciplinary panel with expertise in general and pediatric emergency medicine, nursing, patient safety, informatics, cognitive psychology, social sciences, human factors, and risk management and a patient/caregiver advocate. We used a modified nominal group technique to develop a shared understanding to operationally define diagnostic errors in emergency care and modify the National Academies of Sciences, Engineering, and Medicine's conceptual process framework to this setting. Results The expert panel defined diagnostic errors as "a divergence from evidence-based processes that increases the risk of poor outcomes despite the availability of sufficient information to provide a timely and accurate explanation of the patient's health problem(s)." Diagnostic processes include tasks related to (a) acuity recognition, information and synthesis, evaluation coordination, and (b) communication with patients/caregivers and other diagnostic team members. The expert panel also modified the National Academies of Sciences, Engineering, and Medicine's diagnostic process framework to incorporate influence of mode of arrival, triage level, and interventions during emergency care and underscored the importance of outcome feedback to emergency department providers to promote learning and improvement related to diagnosis. Conclusions The proposed operational definition and modified diagnostic process framework can potentially inform the development of measurement tools and strategies to study the epidemiology and interventions to improve emergency care diagnosis.

Background Previous studies have evaluated the effectiveness of high fidelity patient simulators (HFPS) on nursing training; however, a gap exists on the effects of role assignment on critical thinking, self-efficacy, and situation awareness skills in team-based simulation scenarios. Objectives This study aims to determine if role assignment and the involvement level related to the roles yields significant effects and differences in critical thinking, situation awareness and self-efficacy scores in team-based high-fidelity simulation scenarios. Design A single factorial design with five levels and random assignment was utilized. Setting A public university-sponsored simulation center in the United States of America. Participants A convenience sample of 69 junior-level baccalaureate nursing students was recruited for participation. Methods Participants were randomly assigned one of five possible roles and completed pre-simulation critical thinking and self-efficacy assessments prior to the simulation beginning. Playing within their assigned roles, participants experienced post-partum hemorrhaging scenario using an HFPS. After completing the simulation, participants completed a situation awareness assessment and a post-simulation critical thinking and self-efficacy assessment. Results Role assignment was found to have a statistically significant effect on critical thinking skills and a statistically significant difference in various areas of self-efficacy was also noted. However, no statistical significance in situation awareness abilities was found. Conclusions Results support the notion that certain roles required the participant to be more involved with the simulation scenario, which may have yielded higher critical thinking and self-efficacy scores than roles that required a lesser level of involvement.

Objective The Veterans Health Administration (VHA) has deployed a large number of tablet computers in the last several years. However, little is known about how clinicians may use these devices with a newly planned Web-based electronic health record (EHR), as well as other clinical tools. The objective of this study was to understand the types of use that can be expected of tablet computers versus desktops. Methods Semi-structured interviews were conducted with 24 clinicians at a Veterans Health Administration (VHA) Medical Center. Results An inductive qualitative analysis resulted in findings organized around recurrent themes of: (1) Barriers, (2) Facilitators, (3) Current Use, (4) Anticipated Use, (5) Patient Interaction, and (6) Connection. Conclusions Our study generated several recommendations for the use of tablet computers with new health information technology tools being developed. Continuous connectivity for the mobile device is essential to avoid interruptions and clinician frustration. Also, making a physical keyboard available as an option for the tablet was a clear desire from the clinicians. Larger tablets (e.g., regular size iPad as compared to an iPad mini) were preferred. Being able to use secure messaging tools with the tablet computer was another consistent finding. Finally, more simplicity is needed for accessing patient data on mobile devices, while balancing the important need for adequate security.

Wearable heart rate (HR) monitors typically implement photoplethysmography (PPG) technology and are used in research as an alternative to electrocardiogram (ECG). However, questions surrounding the accuracy of PPG technology exist. To provide an answer regarding the question of accuracy, we conducted a study to compare average HR readings of two different HR technologies (PPG vs. ECG) after an interval style cardio-based workout. A total of 30 trials were conducted and average HR readings from the two HR technologies were compared using an ANOVA. Results revealed no significant difference between the two technologies. However, when HR reached around 155-160 beats per minute, a difference of +/− 5 beats per minute was observed between the two technologies with PPG HR readings being less than ECG. As a result, future research could consider the wearable PPG HR technology as accurate, but with certain experimental design implications.

Previous studies have evaluated the effectiveness of high fidelity patient simulators (HFPS) on nursing training. However, a gap exists on the effects of role assignment in multiple-student simulation scenarios. This study explored the effects of role assignment on self-efficacy development in baccalaureate-level nursing students. Using a researcher-developed tool, based upon a previously developed and utilized tool to fit the simulation scenario, the effects of role assignment were determined. Role assignment was found to have a significant effect on self-efficacy development. Furthermore, roles that require the participant to be more involved with the simulation scenario yielded more confident scores than roles that did not require the participant to be as involved. With this study, future multi-student simulation scenarios can be adjusted knowing the impact different roles can have on self-efficacy development.

Training can be expensive, dangerous, or impractical for certain situations. Virtual reality (VR) technology could be utilized to reduce the negative aspects of real-life training and the consequences incurred from inadequate training. However, for VR to be an effective training method, it must reflect reality to a certain extent. We measured task performance and situation awareness for parking situations with 15 participants in a real-world environment, and in a virtual environment using a VR headset and a flat screen computer monitor separately. Results revealed no significant difference in driver situation awareness between the reality, VR, and flat screen conditions. Performance in terms of task time was significantly less with the reality condition compared to the others. Therefore, the VR device was not equivalent to the real-world environment for training purposes. We discuss ways in which improvements to the VR training condition may increase the effectiveness of VR-based training.

Previous research on personal health records (PHRs) has focused on applications that are “tethered” to a specific electronic health record (EHR). However, there is a gap in research on the usability of unaffiliated, independent PHRs, as well as research on college-aged PHR users. Therefore, we performed a comparative usability study with 18 college-aged participants on three popular, freely available, independent PHRs. Participants completed the same six tasks in three different PHRs. Dependent variables included task time, mouse movement, mouse clicks, keystrokes, errors, and user satisfaction ratings based on the Computer System Usability Questionnaire (CSUQ). Analysis of variance (ANOVA) was used to determine the significance in the difference of the means for each dependent variable. Results showed statistically significant differences in CSUQ survey categories, errors, and keystrokes. Results supported one of the three PHRs (“System A”) as having better usability than its tested counterparts;...

Background The introduction of the electronic health record (EHR) has had a significant impact on provider–patient interactions, particularly revolving around patient-centeredness. More research is needed to understand the provider perspective of this interaction. Objectives Our objective was to obtain provider feedback on a new exam room design compared with the one already in use with respect to the computing layout, which included a wall-mounted monitor for ease of (re)-positioning. An additional objective was to understand elements of exam room design and computing that were highly valued. Methods Semistructured interviews were conducted with 28 providers from several health care organizations. Interviews were audio recorded and transcribed for analysis. We used an inductive coding approach to abstract recurrent themes from the data. Results Our analysis revealed several themes organized around exam room layout, exam room computing, and provider workflow. We report frequency of occurrence of the coded data for computer accessories, computing usefulness, computer mobility, documentation habits, form factor, layout preference, patient interaction, screen sharing, and work habits. Conclusion Providers in our study preferred exam room design to promote flexibility, mobility, and body orientation directed toward the patient. Providers also expressed the need for exam room design to support varying work habits and preferences, including whether to share the computer screen or not.

BACKGROUND: Challenges persist regarding how to integrate computing effectively into the exam room, while maintaining patient-centered care. PURPOSE: Our objective was to evaluate a new exam room design with respect to the computing layout, which included a wall-mounted monitor for ease of (re)-positioning. METHODS: In a lab-based experiment, 28 providers used prototypes of the new and older “legacy” outpatient exam room layouts in a within-subject comparison using simulated patient encounters. We measured efficiency, errors, workload, patient-centeredness (proportion of time the provider was focused on the patient), amount of screen sharing with the patient, workflow integration, and provider situation awareness. RESULTS: There were no statistically significant differences between the exam room layouts for efficiency, errors, or time spent focused on the patient. However, when using the new layout providers spent 75% more time in screen sharing activities with the patient, had 31% lower workload, and gave higher ratings for situation awareness (14%) and workflow integration (17%). CONCLUSIONS: Providers seemed to be unwilling to compromise their focus on the patient when the computer was in a fixed position in the corner of the room and, as a result, experienced greater workload, lower situation awareness, and poorer workflow integration when using the old “legacy” layout. A thoughtful design of the exam room with respect to the computing may positively impact providers’ workload, situation awareness, time spent in screen sharing activities, and workflow integration.

This research evaluated factors that first-year engineering students took into consideration when deciding on their major. The study was done at a large, public, urban institution located i...

SummaryObjective Pharmacogenomic-guided dosing has the potential to improve patient outcomes but its implementation has been met with clinical challenges. Our objective was to develop and evaluate a clinical decision support system (CDSS) for pharmacogenomic-guided warfarin dosing designed for physicians and pharmacists.Methods Twelve physicians and pharmacists completed 6 prescribing tasks using simulated patient scenarios in two iterations (development and validation phases) of a newly developed pharmacogenomic-driven CDSS prototype. For each scenario, usability was measured via efficiency, recorded as time to task completion, and participants’ perceived satisfaction which were compared using Kruskal-Wallis and Mann Whitney U tests, respectively. Debrief interviews were conducted and qualitatively analyzed. Usability findings from the first (i.e. development) iteration were incorporated into the CDSS design for the second (i.e. validation) iteration.Results During the CDSS validation iteration, participants took more time to complete tasks with a median (IQR) of 183 (124–247) seconds versus 101 (73.5–197) seconds in the development iteration (p=0.01). This increase in time on task was due to the increase in time spent in the CDSS corresponding to several design changes. Efficiency differences that were observed between pharmacists and physicians in the development iteration were eliminated in the validation iteration. The increased use of the CDSS corresponded to a greater acceptance of CDSS recommended doses in the validation iteration (4% in the first iteration vs. 37.5% in the second iteration, pConclusions A pharmacogenomic-guided CDSS has been developed using warfarin as the test drug. The final CDSS prototype was trusted by prescribers and significantly increased the time using the tool and acceptance of the recommended doses. This study is an important step toward incorporating pharmacogenomics into CDSS design for clinical testing.Citation: Melton BL, Zillich AJ, Saleem JJ, Russ AL, Tisdale JE, Overholser BR. Iterative development and evaluation of a pharmacogenomic-guided clinical decision support system for warfarin dosing.

This practice-oriented paper provides a collection of design principles that are specific to certain functions within the electronic health record (EHR). Design principles for EHRs tend to be broad rules of thumb rather than specific and actionable because the relevant literature is organized by specific EHR functions. That is, a good amount of research has been conducted on specific functions, rather than EHRs as a whole. Based on the relevant literature, we provide design principles with underlying rationale for progress notes, problem list, consults, clinical reminders, clinical decision support, medication list, medication alerts, and medication reconciliation. This paper is meant to offer a collection of practical guidelines for designers, grounded in the academic literature, that are more actionable than broad usability heuristics. Future work should include refinement of these principles through systematic literature review and the inclusion of additional EHR functions.

This study tested the effects of listening to fast vocal music, fast instrumental music, and no music on nine runners’ distance, average speed, average and maximum heart rate, stride length, and perceived exertion. The data collected during this study revealed music had an effect on maximum heart rate, perceived body tiredness, and perceived mood. This study supports the following conclusions: (1) to lower one’s maximum heart rate, one should listen to instrumental music rather than vocal music; and (2) to reduce the feeling of tiredness while running, one should listen to instrumental music rather than no music. The study also demonstrated that listening to vocal music rather than instrumental music improved runners’ moods. Further research should be conducted to investigate other factors and trends and their potential application to running performance.

With an increase in the number of nursing students and the limited number of open clinical positions, high-fidelity patient simulators (HFPS) have become the new norm. Multiple studies have evaluated HFPS effectiveness and several suggest that HFPS does has an effect on critical thinking. This study intends to provide data to support that suggestion. In addition, this study was designed to identify a possible correlation between role assignment and improvements in critical thinking after completion of a HFPS scenario. Analysis revealed that role assignment, for most of the roles, did have a statistically significant effect on the post-simulation critical thinking assessment scores. The relationship between role assignments and HFPS scenario outcomes (such as critical thinking), as well as the nature of the correlation, may help scenario developers better understand how critical thinking improvement can be affected by the involvement of the participant based on the roles assigned to them.

Medication management is a complex and mentally demanding endeavor performed by multiple individuals in diverse settings and at variable points in time. Achieving the goal of safe care delivery with optimized patient outcomes requires ongoing communication, coordination, and interaction between an array of technologies and roles across the health care continuum. Human factors principles and methods have the potential to substantially improve patient safety throughout this high-risk cycle. The purpose of this interactive discussion panel is to explore the medication management process from a sociotechnical perspective to identify specific challenges and vulnerabilities, and recommend strategies that leverage human factors processes and design principles to efficiently and effectively address safety critical issues.

Several factors contribute to the divide between applied research and operations for health information technology. These differentiating factors include: work performance metrics, project interests, funding, timelines, goals, incentives, and interpretation and dissemination of findings. We propose specific recommendations for addressing each of these factors to help reduce this gap. Factors and corresponding recommendations are drawn from experiences working in both research and operations positions within the Veterans Health Administration. The lessons learned and associated recommendations are likely to be relevant to other healthcare institutions.

The Department of Veterans Affairs (VA) has developed a new exam room design standard that is intended to facilitate a greater degree of patient centeredness. This new design includes a wall-mounted monitor on an armature system and a moveable table workspace. To date, however, this design has not been formally evaluated in a field setting. We conducted observations and interviews with primary care providers and their patients from three locations within the Phoenix VA Health Care System, in a pilot study comparing the new exam room design standard with the older legacy exam rooms. When using the new exam room layout, providers spent a greater proportion of time focused on the patient, spent more time in screen-sharing activities with the patient, and had a higher degree of self-reported situation awareness. However, the legacy exam rooms were perceived as better facilitating workflow integration. Provider and patient debrief interviews were supportive of the new exam room design. Overall, our field study results suggest that the new exam room design does contribute to a greater degree of patient centeredness, though more thorough evaluations are required to support these preliminary results.

Although much research has been conducted on the impact of the electronic health record (EHR) and health information technology (HIT) on provider-patient interaction, challenges persist on how to effectively integrate these tools into healthcare environments, while maintaining patient-centered care. Panelists will present and discuss their experience relating to: the introduction of new technology and how it changes the nature of work; the positive impact of greater focus on the patient; the nature of interruptions and strategies to reduce their negative impact; enhancing the workflow of medication ordering; and designing collaborative spaces considering all relevant stakeholders (patient care team, patients, family members). These topics represent some of the latest human factors research and design trends with a common goal: advancing the design of the modern healthcare environment, with consideration of the additional complexity introduced by evolutionary changes in clinical information systems and HIT.

The quality of usability testing is highly dependent upon the associated usability scenarios. To promote usability testing as part of electronic health record (EHR) certification, the Office of the National Coordinator (ONC) for Health Information Technology requires that vendors test specific capabilities of EHRs with clinical end-users and report their usability testing process - including the test scenarios used - along with the results. The ONC outlines basic expectations for usability testing, but there is little guidance in usability texts or scientific literature on how to develop usability scenarios for healthcare applications. The objective of this article is to outline key factors to consider when developing usability scenarios and tasks to evaluate computer-interface based health information technologies. To achieve this goal, we draw upon a decade of our experience conducting usability tests with a variety of healthcare applications and a wide range of end-users, to include healthcare professionals as well as patients. We discuss 10 key factors that influence scenario development: objectives of usability testing; roles of end-user(s); target performance goals; evaluation time constraints; clinical focus; fidelity; scenario-related bias and confounders; embedded probes; minimize risks to end-users; and healthcare related outcome measures. For each factor, we present an illustrative example. This article is intended to aid usability researchers and practitioners in their efforts to advance health information technologies. The article provides broad guidance on usability scenario development and can be applied to a wide range of clinical information systems and applications.

Adoption of clinical decision support has been limited. Important barriers include an emphasis on algorithmic approaches to decision support that do not align well with clinical work flow and human decision strategies, and the expense and challenge of developing, implementing, and refining decision support features in existing electronic health records (EHRs). We applied decision-centered design to create a modular software application to support physicians in managing and tracking colorectal cancer screening. Using decision-centered design facilitates a thorough understanding of cognitive support requirements from an end user perspective as a foundation for design. In this project, we used an iterative design process, including ethnographic observation and cognitive task analysis, to move from an initial design concept to a working modular software application called the Screening & Surveillance App. The beta version is tailored to work with the Veterans Health Administration’s EHR Computerized Patient Record System (CPRS). Primary care providers using the beta version Screening & Surveillance App more accurately answered questions about patients and found relevant information more quickly compared to those using CPRS alone. Primary care providers also reported reduced mental effort and rated the Screening & Surveillance App positively for usability.

Display Omitted This method was feasible to evaluate users information processing of alerts.This method detected significant differences between two alert displays.The study method may be useful for other health information technology displays. BackgroundPrescribers commonly receive alerts during medication ordering. Prescribers work in a complex, time-pressured environment; to enhance the effectiveness of safety alerts, the effort needed to cognitively process these alerts should be minimized. Methods to evaluate the extent to which computerized alerts support prescribers information processing are lacking. ObjectiveTo develop a methodological protocol to assess the extent to which alerts support prescribers information processing at-a-glance; specifically, the incorporation of information into their working memory. We hypothesized that the method would be feasible and that we would be able to detect a significant difference in prescribers information processing with a revised alert display that incorporates warning design guidelines compared to the original alert display. MethodsA counterbalanced, within-subject study was conducted with 20 prescribers in a human-computer interaction laboratory. We tested a single alert that was displayed in two different ways. Prescribers were informed that an alert would appear for 10s. After the alert was shown, a white screen was displayed, and prescribers were asked to verbally describe what they saw; indicate how many total warnings; and describe anything else they remembered about the alert. We measured information processing via the accuracy of prescribers free recall and their ability to identify that three warning messages were present. Two analysts independently evaluated participants responses against a comprehensive catalog of alert elements and then discussed discrepancies until reaching consensus. ResultsThis feasibility study demonstrated that the method seemed to be effective for evaluating prescribers information processing of medication alert displays. With this method, we were able to detect significant differences in prescribers recall of alert information. The proportion of total data elements that prescribers were able to accurately recall was significantly greater for the revised versus original alert display (p=0.006). With the revised display, more prescribers accurately reported that three warnings were shown (p=0.002). ConclusionsThe methodological protocol was feasible for evaluating the alert display and yielded important findings on prescribers information processing. Study methods supplement traditional usability evaluation methods and may be useful for evaluating information processing of other healthcare technologies.

Background: Computerized medication alerts can often be bypassed by entering an override rationale, but prescribers’ override reasons are frequently ambiguous to pharmacists who review orders. Objective: To develop and evaluate a new override mechanism for adverse reaction and drug-drug interaction alerts. We hypothesized that the new mechanism would improve usability for prescribers and increase the clinical appropriateness of override reasons. Methods: A counterbalanced, crossover study was conducted with 20 prescribers in a simulated prescribing environment. We modified the override mechanism timing, navigation, and text entry. Instead of free-text entry, the new mechanism presented prescribers with a predefined set of override reasons. We assessed usability (learnability, perceived efficiency, and usability errors) and used a priori criteria to evaluate the clinical appropriateness of override reasons entered. Results: Prescribers rated the new mechanism as more efficient (Wilcoxon signed-rank test, P = 0.032). When first using the new design, 5 prescribers had difficulty finding the new mechanism, and 3 interpreted the navigation to mean that the alert could not be overridden. The number of appropriate override reasons significantly increased with the new mechanism compared with the original mechanism (median change of 3.0; interquartile range = 3.0; P < 0.0001). Conclusions: When prescribers were given a menu-based choice for override reasons, clinical appropriateness of these reasons significantly improved. Further enhancements are necessary, but this study is an important first step toward a more standardized menu of override choices. Findings may be used to improve communication through e-prescribing systems between prescribers and pharmacists.

Objective This study evaluated the current use of commercial-off-the-shelf Clinical Information Systems (CIS) for intensive care units (ICUs) and Anesthesia Record Keeping (ARK) for operating rooms and post-anesthesia care recovery settings at three Veterans Affairs Medical Centers (VAMCs). Clinicians and administrative staff use these applications at bedside workstations, in operating rooms, at nursing stations, in physician's rooms, and in other various settings. The intention of a CIS or an ARK system is to facilitate creation of electronic records of data, assessments, and procedures from multiple medical devices. The US Department of Veterans Affairs (VA) Office of the Chief of Nursing Informatics sought to understand usage barriers and facilitators to optimize these systems in the future. Therefore, a human factors study was carried out to observe the CIS and ARK systems in use at three VAMCs in order to identify best practices and suggested improvements to currently implemented CIS and ARK systems. Methods We conducted a rapid ethnographic study of clinical end-users interacting with the CIS and ARK systems in the critical care and anesthesia care areas in each of three geographically distributed VAMCs. Two observers recorded interactions and/or interview responses from 88 CIS and ARK end-users. We coded and sorted into logical categories field notes from 69 shadowed participants. The team transcribed and combined data from key informant interviews with 19 additional participants with the observation data. We then integrated findings across observations into meaningful patterns and abstracted the data into themes, which translated directly to barriers to effective adoption and optimization of the CIS and ARK systems. Results Effective optimization of the CIS and ARK systems was impeded by: (1) integration issues with other software systems; (2) poor usability; (3) software challenges; (4) hardware challenges; (5) training concerns; (6) unclear roles and lack of coordination among stakeholders; and (7) insufficient technical support. Many of these barriers are multi-faceted and have associated sub-barriers, which are described in detail along with relevant quotes from participants. In addition, regionalized purchases of different CIS and ARK systems, as opposed to enterprise level purchases, contributed to some of the identified barriers. Facilitators to system use included (1) automation and (2) a dedicated facility-level CIS-ARK Coordinator. Conclusions We identified barriers that explain some of the challenges with the optimization of the CIS and ARK commercial systems across the Veterans Health Administration (VHA). To help address these barriers, and evolve them into facilitators, we categorized report findings as (1) interface and system-level changes that vendors or VA healthcare systems can implement; (2) implementation factors under VA control and not under VA control; and (3) factors that may be used to inform future application purchases. We outline several recommendations for improved adoption of CIS and ARK systems and further recommend that human factors engineering and usability requirements become an integral part of VA health information technology (HIT) application procurement, customization, and implementation in order to help eliminate or mitigate some of the barriers of use identified in this study. Human factors engineering methods can be utilized to apply a user-centered approach to application requirements specification, application evaluation, system integration, and application implementation.

Previous research on personal health records (PHRs) has focused on applications that are “tethered” to a specific electronic health record (EHR). However, there is a gap in research on the usability of unaffiliated, independent PHRs, as well as research on college-aged PHR users. Therefore, we performed a comparative usability study with 18 college-aged participants on three popular, freely available, independent PHRs. We chose a within-subject design to allow for comparative feedback from each participant in the post-experiment interview. In order to control for a learning effect between PHRs potentially affecting the usability performance measures, the order that participants used the PHRs was randomized across participants. We used analysis of variance (ANOVA) to determine if there were statistically significant differences in the usability measures. Based on variability of PHR use and design reported in previous research, we hypothesized that there would be a significant difference for each of the measures previously described and that one PHR would be the most usable product based on these measures. This hypothesis is based on the PHR’s extensive resources and company history. Participants completed the same six tasks in three different PHRs. Dependent variables included task time, mouse movement, mouse clicks, keystrokes, errors, and user satisfaction ratings based on the Computer System Usability Questionnaire (CSUQ). Analysis of variance (ANOVA) was used to determine the significance in the difference of the means for each dependent variable. Results showed statistically significant differences in CSUQ survey categories, errors, and keystrokes. Results supported one of the three PHRs as having better usability than its tested counterparts. While the initial purpose of this study was comparative usability testing of PHRs for college-aged students, the study provided other insights as well. Similar to other usability studies found in the literature review, the study used multiple methods, including objective task metrics, a survey, and an interview to solicit feedback on the systems. This provides a new addition to the literature in that it analyzes the usability of a system with a new user group and completed a comparative analysis of three leading Web-based, untethered PHRs. The initial hypothesis that there would be a significant difference in usability for each of the dependent measures and that one PHR would have better usability based on these measures was partially supported by the results. While not all of the criteria had statistically significant results for the three different systems, such as task time, mouse movement, mouse clicks, and interface quality, many of the measures did have significant differences in their means, and one of the PHRs had the best results in the majority of metrics analyzed. Though one PHR seemed to more usable than the other systems, it does not imply that this PHR is without error. There are still improvements that could be made to enhance the usability of the system. For example, several participants commented on how they liked the interface of one of the competing PHRs. Several participants also commented that some of the drop-downs limit the options and are not representative of the information they were trying to enter. Expanded dropdowns or typing options could be added to improve this. As found in the a different study, there are still barriers to the adoption of the PHR that was found to be most usable. Using the survey and open-ended survey results as a guide to improvement, the PHR that was found to be most usable in our study has the potential to further improve its usability.

A future trend is the of adoption of a multiscreen and multiple device approach to cope with distractions and multiple tasks. Although this may seem counterintuitive, more screens and possibly more devices may help focus one’s attention rather than serve as a distraction, making multiple tasks viewable at a glance across multiple device screens (Thompson, 2014). Potential benefits of multiscreen and multiple device environments were assessed using three different computing environments. A single factor, within-subject study was conducted with 18 engineering students in a laboratory experiment. Three levels for the computing environment factor included one with a desktop computer with a single monitor (control, condition A); one with a desktop with dual monitors, as well as a single tablet computer (condition B); and one with a desktop with a single monitor, as well as two tablet computers (condition C). There was no statistically significant difference in efficiency when completing scenarios for the three computing environments. However, a dual monitor desktop with a single tablet computer (B) was the ideal computing environment for the information-rich engineering problem given to participants, supported by significantly fewer errors compared to condition C and significantly higher usability ratings compared to conditions A and C. A single desktop monitor with two tablet computers (C) did not provide any advantage compared to a single desktop monitor (A). A dual monitor desktop with a single tablet computer (condition B) was the ideal computing environment for the “information-rich” engineering problem given to participants. This is supported by converging evidence from the dependent measures as well as qualitative debrief interviews. A single desktop monitor with two tablet computers (condition C) did not provide any advantage compared to a single desktop monitor (condition A). Overall, these findings provide only marginal support for the concept we set out to investigate, which was the notion that more screens and possibly more devices may help focus one’s attention rather than serve as a distraction, making multiple tasks viewable at a glance across multiple device screens (Thompson, 2014). The finding of a performance and usability advantage of the dual monitors in condition B is consistent with previous studies (Anderson et al., 2004; Poder et al., 2011; Russell & Wong, 2005). A key difference in our study is that we provided a tablet computer in addition to the dual monitors. However, the debrief interviews were mixed as to the usefulness of the third screen provided by the tablet; some participants thought it was not helpful whereas other did find it useful. The complete lack of performance, workload, and usability differences between condition C (single monitor and two tablet computers) and condition A (single monitor) does not support the notion that a multiscreen environment can help focus one’s attention. Indeed, some participants noted that using multiple screens provided by the tablet computer(s) was distracting. Others noted that while they did not hinder their tasks, they did not help. Based on these results, we recommend a computing work environment with dual screen monitors, with an optional tablet computer, for complex and information-rich computing tasks.

This paper describes a project employing decision-centered design to develop a decision support application called the Screening & Surveillance App (SSA) to aid primary care providers in tracking and managing colorectal cancer (CRC) screening for their patients. This paper details the path from cognitive task analysis (CTA) to a decision support application. We conducted ethnographic observations and CTA interviews to identify common elements of clinical workflow across health systems and articulate decision requirements. We describe three design iterations that emerged over the course of CTA and software development. Findings from a beta test evaluating the SSA suggest that it saves time, increases accuracy in responding to CRC-related patient-specific questions, and is perceived to reduce workflow as compared to participants’ current electronic health records (EHR). Usability and usefulness ratings for the SSA as measured by the Health ITUES were above 4 on a 5-point scale where 5 was most positive.

This practice-oriented presentation describes several factors that contribute to the divide between applied research and operations for health information technology (HIT). Factors and corresponding recommendations were drawn from experiences working in both research and operations positions as human factors professionals within a large healthcare system, the Veterans Health Administration (VHA). We offer specific recommendations for addressing each of these factors to help reduce the gap. Although our findings and recommendations were derived from our VHA experience, we expect many of the lessons learned are relevant to other major healthcare organizations. Indeed, we strongly suspect that the factors presented represent fundamental differentiators between researchers and operational-based professionals. Understanding these differences and findings ways to bridge them is essential for advancing HIT.