Search

Over the last 15 years, healthcare delivery has experienced enormous changes across every aspect of clinical care, both for participants in it and affiliated stakeholders. The plethora and breadth of these changes and ongoing struggles with their incorporation into the work of healthcare begs the question: In 2035, what emerging innovations and considerations will we wish we had prepared for with regard to human factors engineering research, technology development, and system redesign efforts? Panelists and co-chairs have deep experience with steering, and being impacted by, innovations in information technology in healthcare. The panelists represent diverse academic, clinical, government contract research and development, and small entrepreneurial business. Perhaps the biggest benefit of this session will be the explicit inclusion of the audience in brainstorming and discussion activities, which will be facilitated digitally and verbally during the session and shared afterwards electronically with participants.

Given the substantial increase in interest and activity in the virtual care space, the Department of Veterans Affairs (VA) is making a concerted effort to increase opportunities for Veterans to adopt virtual care applications. We investigated Veteran attitudes, interests, and needs for virtual care applications, including privacy considerations and specific types of data they would be comfortable sharing with their providers through various Veteran-facing tools. We administered a 28-item questionnaire to 40 Veterans and performed follow-up interviews with a subset of nine Veterans. Study results revealed broad support by Veterans for using virtual care applications, including wearable devices, for sharing all types of health information with their providers. Further, Veterans expressed a desire for consolidation across VA-provided virtual care tools. Our findings will inform a strategy for Veteran-facing tools. Other health care systems may be interested in exploring these topics, based on our study results.

The Veterans Health Administration (VHA) within the Department of Veterans Affairs (VA) was interested in determining how mobile tablets may be perceived by clinicians for use in clinical settings. Therefore, we conducted an evaluation to assess how performance might differ between two devices and to gather feedback on device use. We conducted a within-subject comparison with 32 clinicians involving a usability test and two questionnaires. Qualitative data was organized around eight themes: facilitators and barriers to tablet use, observations of physical use, device specific advantages and disadvantages, and the Patient Viewer mobile application. Clinicians envisioned many facilitators to tablet use, but also voiced some concerns. Participants rated one device significantly better than the other for questionnaire items related to ‘Access and Efficiency’, ‘Introduction into the Clinical Environment’, ‘Usability and Usefulness’, and ‘Desire to Use’. Results can be used by mobile health app developers, healthcare organizations considering device purchases, and researchers conducting studies on tablets to inform respective work.

The US Department of Veterans Affairs (VA) is using an automated short message service application named “Annie” as part of its coronavirus disease 2019 (COVID-19) response with a protocol for coronavirus precautions, which can help the veteran monitor symptoms and can advise the veteran when to contact his or her VA care team or a nurse triage line. We surveyed 1134 veterans on their use of the Annie application and coronavirus precautions protocol. Survey results support what is likely a substantial resource savings for the VA, as well as non-VA community healthcare. Moreover, the majority of veterans reported at least 1 positive sentiment (felt more connected to VA, confident, or educated and/or felt less anxious) by receiving the protocol messages. The findings from this study have implications for other healthcare systems to help manage a patient population during the coronavirus pandemic.

Background The hospitalist workday is cognitively demanding and dominated by activities away from patients’ bedsides. Although mobile technologies are offered as solutions, clinicians report lower expectations of mobile technology after actual use. Objective The purpose of this study is to better understand opportunities for integrating mobile technology and apps into hospitalists’ workflows. We aim to identify difficult tasks and contextual factors that introduce inefficiencies and characterize hospitalists’ perspectives on mobile technology and apps. Methods We conducted a workflow analysis based on semistructured interviews. At a Midwestern US medical center, we recruited physicians and nurse practitioners from hospitalist and inpatient teaching teams and internal medicine residents. Interviews focused on tasks perceived as frequent, redundant, and difficult. Additionally, participants were asked to describe opportunities for mobile technology interventions. We analyzed contributing factors, impacted workflows, and mobile app ideas. Results Over 3 months, we interviewed 12 hospitalists. Participants collectively identified chart reviews, orders, and documentation as the most frequent, redundant, and difficult tasks. Based on those tasks, the intake, discharge, and rounding workflows were characterized as difficult and inefficient. The difficulty was associated with a lack of access to electronic health records at the bedside. Contributing factors for inefficiencies were poor usability and inconsistent availability of health information technology combined with organizational policies. Participants thought mobile apps designed to improve team communications would be most beneficial. Based on our analysis, mobile apps focused on data entry and presentation supporting specific tasks should also be prioritized. Conclusions Based on our results, there are prioritized opportunities for mobile technology to decrease difficulty and increase the efficiency of hospitalists’ workflows. Mobile technology and task-specific mobile apps with enhanced usability could decrease overreliance on hospitalists’ memory and fragmentation of clinical tasks across locations. This study informs the design and implementation processes of future health information technologies to improve continuity in hospital-based medicine.

The coordination of site visits to execute human factors methods, such as onsite usability tests, interviews, or observations, in clinical settings requires a high level of management to attain successful data collection outcomes. Members of the Department of Veterans Affairs (VA) Veterans Health Administration (VHA) Human Factors Engineering (HFE) team occasionally visit VHA medical centers or outpatient clinics to complete our work. We have developed a site visit checklist as a practice innovation to facilitate logistical coordination when gathering data onsite. This Practice-Oriented paper includes the full checklist, as well as discussion of its use to enable other groups to benefit from lessons we have learned in conducting onsite work in health care settings.

This paper reports the results of a literature review of health care organizations that have transitioned from one electronic health record (EHR) to another. Ten different EHR to EHR transitions are documented in the academic literature. In eight of the 10 transitions, the health care organization transitioned to Epic, a commercial EHR which is dominating the market for large and medium hospitals and health care systems. The focus of the articles reviewed falls into two main categories: (1) data migration from the old to new EHR and (2) implementation of the new EHR as it relates to patient safety, provider satisfaction, and other measures pre-and post-transition. Several conclusions and recommendations are derived from this review of the literature, which may be informative for healthcare organizations preparing to replace an existing EHR. These recommendations are likely broadly relevant to EHR to EHR transitions, regardless of the new EHR vendor.

Although numerous healthcare organizations have transitioned from one electronic health record (EHR) to another or are currently planning a transition, there are few documented artifacts, such as published studies or operationalizable resources, that offer guidance on such transitions. This panel seeks to begin a conversation about human factors considerations in EHR transitions from a legacy system. Panel members will discuss current literature and research on the topic as well as experiences with and lessons learned from transitions within their organizations. Panel discussion can be expected to identify new research opportunities, needed resources, and guidance for EHR vendors or healthcare facilities in the midst of or preparing for an EHR transition. Panelists will also lay out systemic issues that need to be addressed at the national policy and regulatory level. This topic is relevant not only to full-scale EHR transitions, but also has applicability for significant EHR version changes.

Background During medical referrals, communication barriers between referring and consulting outpatient clinics delay patients’ access to health care. One notable opportunity for reducing these barriers is improved usefulness and usability of electronic medical consultation order forms. The cognitive systems engineering (CSE) design approach focuses on supporting humans in managing cognitive complexity in sociotechnical systems. Cognitive complexity includes communication, decision-making, problem solving, and planning. Objective The objective of this research was to implement a CSE design approach to develop a template that supports the cognitive needs of referring clinicians and improves referral communication. Methods We conducted interviews and observations with primary care providers and specialists at two major tertiary, urban medical facilities. Using qualitative analysis, we identified cognitive requirements and design guidelines. Next, we designed user interface (UI) prototypes and compared their usability with that of a currently implemented UI at a major Midwestern medical facility. Results Physicians’ cognitive challenges were summarized in four cognitive requirements and 13 design guidelines. As a result, two UI prototypes were developed to support order template search and completion. To compare UIs, 30 clinicians (referrers) participated in a consultation ordering simulation complemented with the think-aloud elicitation method. Oral comments about the UIs were coded for both content and valence (i.e., positive, neutral, or negative). Across 619 comments, the odds ratio for the UI prototype to elicit higher-valenced comments than the implemented UI was 13.5 (95% CI = [9.2, 19.8]), p Conclusion This study reinforced the significance of applying a CSE design approach to inform the design of health information technology. In addition, knowledge elicitation methods enabled identification of physicians’ cognitive requirements and challenges when completing electronic medical consultation orders. The resultant knowledge was used to derive design guidelines and UI prototypes that were more useful and usable for referring physicians. Our results support the implementation of a CSE design approach for electronic medical consultation orders.

Objective To create an operational definition and framework to study diagnostic error in the emergency department setting. Methods We convened a 17-member multidisciplinary panel with expertise in general and pediatric emergency medicine, nursing, patient safety, informatics, cognitive psychology, social sciences, human factors, and risk management and a patient/caregiver advocate. We used a modified nominal group technique to develop a shared understanding to operationally define diagnostic errors in emergency care and modify the National Academies of Sciences, Engineering, and Medicine's conceptual process framework to this setting. Results The expert panel defined diagnostic errors as "a divergence from evidence-based processes that increases the risk of poor outcomes despite the availability of sufficient information to provide a timely and accurate explanation of the patient's health problem(s)." Diagnostic processes include tasks related to (a) acuity recognition, information and synthesis, evaluation coordination, and (b) communication with patients/caregivers and other diagnostic team members. The expert panel also modified the National Academies of Sciences, Engineering, and Medicine's diagnostic process framework to incorporate influence of mode of arrival, triage level, and interventions during emergency care and underscored the importance of outcome feedback to emergency department providers to promote learning and improvement related to diagnosis. Conclusions The proposed operational definition and modified diagnostic process framework can potentially inform the development of measurement tools and strategies to study the epidemiology and interventions to improve emergency care diagnosis.

Objective The Veterans Health Administration (VHA) has deployed a large number of tablet computers in the last several years. However, little is known about how clinicians may use these devices with a newly planned Web-based electronic health record (EHR), as well as other clinical tools. The objective of this study was to understand the types of use that can be expected of tablet computers versus desktops. Methods Semi-structured interviews were conducted with 24 clinicians at a Veterans Health Administration (VHA) Medical Center. Results An inductive qualitative analysis resulted in findings organized around recurrent themes of: (1) Barriers, (2) Facilitators, (3) Current Use, (4) Anticipated Use, (5) Patient Interaction, and (6) Connection. Conclusions Our study generated several recommendations for the use of tablet computers with new health information technology tools being developed. Continuous connectivity for the mobile device is essential to avoid interruptions and clinician frustration. Also, making a physical keyboard available as an option for the tablet was a clear desire from the clinicians. Larger tablets (e.g., regular size iPad as compared to an iPad mini) were preferred. Being able to use secure messaging tools with the tablet computer was another consistent finding. Finally, more simplicity is needed for accessing patient data on mobile devices, while balancing the important need for adequate security.

Wearable heart rate (HR) monitors typically implement photoplethysmography (PPG) technology and are used in research as an alternative to electrocardiogram (ECG). However, questions surrounding the accuracy of PPG technology exist. To provide an answer regarding the question of accuracy, we conducted a study to compare average HR readings of two different HR technologies (PPG vs. ECG) after an interval style cardio-based workout. A total of 30 trials were conducted and average HR readings from the two HR technologies were compared using an ANOVA. Results revealed no significant difference between the two technologies. However, when HR reached around 155-160 beats per minute, a difference of +/− 5 beats per minute was observed between the two technologies with PPG HR readings being less than ECG. As a result, future research could consider the wearable PPG HR technology as accurate, but with certain experimental design implications.

Previous studies have evaluated the effectiveness of high fidelity patient simulators (HFPS) on nursing training. However, a gap exists on the effects of role assignment in multiple-student simulation scenarios. This study explored the effects of role assignment on self-efficacy development in baccalaureate-level nursing students. Using a researcher-developed tool, based upon a previously developed and utilized tool to fit the simulation scenario, the effects of role assignment were determined. Role assignment was found to have a significant effect on self-efficacy development. Furthermore, roles that require the participant to be more involved with the simulation scenario yielded more confident scores than roles that did not require the participant to be as involved. With this study, future multi-student simulation scenarios can be adjusted knowing the impact different roles can have on self-efficacy development.

Previous research on personal health records (PHRs) has focused on applications that are “tethered” to a specific electronic health record (EHR). However, there is a gap in research on the usability of unaffiliated, independent PHRs, as well as research on college-aged PHR users. Therefore, we performed a comparative usability study with 18 college-aged participants on three popular, freely available, independent PHRs. Participants completed the same six tasks in three different PHRs. Dependent variables included task time, mouse movement, mouse clicks, keystrokes, errors, and user satisfaction ratings based on the Computer System Usability Questionnaire (CSUQ). Analysis of variance (ANOVA) was used to determine the significance in the difference of the means for each dependent variable. Results showed statistically significant differences in CSUQ survey categories, errors, and keystrokes. Results supported one of the three PHRs (“System A”) as having better usability than its tested counterparts;...

Background The introduction of the electronic health record (EHR) has had a significant impact on provider–patient interactions, particularly revolving around patient-centeredness. More research is needed to understand the provider perspective of this interaction. Objectives Our objective was to obtain provider feedback on a new exam room design compared with the one already in use with respect to the computing layout, which included a wall-mounted monitor for ease of (re)-positioning. An additional objective was to understand elements of exam room design and computing that were highly valued. Methods Semistructured interviews were conducted with 28 providers from several health care organizations. Interviews were audio recorded and transcribed for analysis. We used an inductive coding approach to abstract recurrent themes from the data. Results Our analysis revealed several themes organized around exam room layout, exam room computing, and provider workflow. We report frequency of occurrence of the coded data for computer accessories, computing usefulness, computer mobility, documentation habits, form factor, layout preference, patient interaction, screen sharing, and work habits. Conclusion Providers in our study preferred exam room design to promote flexibility, mobility, and body orientation directed toward the patient. Providers also expressed the need for exam room design to support varying work habits and preferences, including whether to share the computer screen or not.

BACKGROUND: Challenges persist regarding how to integrate computing effectively into the exam room, while maintaining patient-centered care. PURPOSE: Our objective was to evaluate a new exam room design with respect to the computing layout, which included a wall-mounted monitor for ease of (re)-positioning. METHODS: In a lab-based experiment, 28 providers used prototypes of the new and older “legacy” outpatient exam room layouts in a within-subject comparison using simulated patient encounters. We measured efficiency, errors, workload, patient-centeredness (proportion of time the provider was focused on the patient), amount of screen sharing with the patient, workflow integration, and provider situation awareness. RESULTS: There were no statistically significant differences between the exam room layouts for efficiency, errors, or time spent focused on the patient. However, when using the new layout providers spent 75% more time in screen sharing activities with the patient, had 31% lower workload, and gave higher ratings for situation awareness (14%) and workflow integration (17%). CONCLUSIONS: Providers seemed to be unwilling to compromise their focus on the patient when the computer was in a fixed position in the corner of the room and, as a result, experienced greater workload, lower situation awareness, and poorer workflow integration when using the old “legacy” layout. A thoughtful design of the exam room with respect to the computing may positively impact providers’ workload, situation awareness, time spent in screen sharing activities, and workflow integration.

SummaryObjective Pharmacogenomic-guided dosing has the potential to improve patient outcomes but its implementation has been met with clinical challenges. Our objective was to develop and evaluate a clinical decision support system (CDSS) for pharmacogenomic-guided warfarin dosing designed for physicians and pharmacists.Methods Twelve physicians and pharmacists completed 6 prescribing tasks using simulated patient scenarios in two iterations (development and validation phases) of a newly developed pharmacogenomic-driven CDSS prototype. For each scenario, usability was measured via efficiency, recorded as time to task completion, and participants’ perceived satisfaction which were compared using Kruskal-Wallis and Mann Whitney U tests, respectively. Debrief interviews were conducted and qualitatively analyzed. Usability findings from the first (i.e. development) iteration were incorporated into the CDSS design for the second (i.e. validation) iteration.Results During the CDSS validation iteration, participants took more time to complete tasks with a median (IQR) of 183 (124–247) seconds versus 101 (73.5–197) seconds in the development iteration (p=0.01). This increase in time on task was due to the increase in time spent in the CDSS corresponding to several design changes. Efficiency differences that were observed between pharmacists and physicians in the development iteration were eliminated in the validation iteration. The increased use of the CDSS corresponded to a greater acceptance of CDSS recommended doses in the validation iteration (4% in the first iteration vs. 37.5% in the second iteration, pConclusions A pharmacogenomic-guided CDSS has been developed using warfarin as the test drug. The final CDSS prototype was trusted by prescribers and significantly increased the time using the tool and acceptance of the recommended doses. This study is an important step toward incorporating pharmacogenomics into CDSS design for clinical testing.Citation: Melton BL, Zillich AJ, Saleem JJ, Russ AL, Tisdale JE, Overholser BR. Iterative development and evaluation of a pharmacogenomic-guided clinical decision support system for warfarin dosing.

This practice-oriented paper provides a collection of design principles that are specific to certain functions within the electronic health record (EHR). Design principles for EHRs tend to be broad rules of thumb rather than specific and actionable because the relevant literature is organized by specific EHR functions. That is, a good amount of research has been conducted on specific functions, rather than EHRs as a whole. Based on the relevant literature, we provide design principles with underlying rationale for progress notes, problem list, consults, clinical reminders, clinical decision support, medication list, medication alerts, and medication reconciliation. This paper is meant to offer a collection of practical guidelines for designers, grounded in the academic literature, that are more actionable than broad usability heuristics. Future work should include refinement of these principles through systematic literature review and the inclusion of additional EHR functions.

Medication management is a complex and mentally demanding endeavor performed by multiple individuals in diverse settings and at variable points in time. Achieving the goal of safe care delivery with optimized patient outcomes requires ongoing communication, coordination, and interaction between an array of technologies and roles across the health care continuum. Human factors principles and methods have the potential to substantially improve patient safety throughout this high-risk cycle. The purpose of this interactive discussion panel is to explore the medication management process from a sociotechnical perspective to identify specific challenges and vulnerabilities, and recommend strategies that leverage human factors processes and design principles to efficiently and effectively address safety critical issues.

Several factors contribute to the divide between applied research and operations for health information technology. These differentiating factors include: work performance metrics, project interests, funding, timelines, goals, incentives, and interpretation and dissemination of findings. We propose specific recommendations for addressing each of these factors to help reduce this gap. Factors and corresponding recommendations are drawn from experiences working in both research and operations positions within the Veterans Health Administration. The lessons learned and associated recommendations are likely to be relevant to other healthcare institutions.

The Department of Veterans Affairs (VA) has developed a new exam room design standard that is intended to facilitate a greater degree of patient centeredness. This new design includes a wall-mounted monitor on an armature system and a moveable table workspace. To date, however, this design has not been formally evaluated in a field setting. We conducted observations and interviews with primary care providers and their patients from three locations within the Phoenix VA Health Care System, in a pilot study comparing the new exam room design standard with the older legacy exam rooms. When using the new exam room layout, providers spent a greater proportion of time focused on the patient, spent more time in screen-sharing activities with the patient, and had a higher degree of self-reported situation awareness. However, the legacy exam rooms were perceived as better facilitating workflow integration. Provider and patient debrief interviews were supportive of the new exam room design. Overall, our field study results suggest that the new exam room design does contribute to a greater degree of patient centeredness, though more thorough evaluations are required to support these preliminary results.

Potential benefits of multiscreen and multiple device environments were assessed using three different computing environments. A single factor, within-subject study was conducted with 18 engineering students in a laboratory experiment. Three levels for the computing environment factor included one with a desktop computer with a single monitor (control, condition A); one with a desktop with dual monitors, as well as a single tablet computer (condition B); and one with a desktop with a single monitor, as well as two tablet computers (condition C). There was no statistically significant difference in efficiency or workload when completing scenarios for the three computing environments. However, a dual monitor desktop with a single tablet computer (B) was the ideal computing environment for the information-rich engineering problem given to participants, supported by significantly fewer errors compared to condition C and significantly higher usability ratings compared to conditions A and C. A single desktop monitor with two tablet computers (C) did not provide any advantage compared to a single desktop monitor (A).

Although much research has been conducted on the impact of the electronic health record (EHR) and health information technology (HIT) on provider-patient interaction, challenges persist on how to effectively integrate these tools into healthcare environments, while maintaining patient-centered care. Panelists will present and discuss their experience relating to: the introduction of new technology and how it changes the nature of work; the positive impact of greater focus on the patient; the nature of interruptions and strategies to reduce their negative impact; enhancing the workflow of medication ordering; and designing collaborative spaces considering all relevant stakeholders (patient care team, patients, family members). These topics represent some of the latest human factors research and design trends with a common goal: advancing the design of the modern healthcare environment, with consideration of the additional complexity introduced by evolutionary changes in clinical information systems and HIT.

The quality of usability testing is highly dependent upon the associated usability scenarios. To promote usability testing as part of electronic health record (EHR) certification, the Office of the National Coordinator (ONC) for Health Information Technology requires that vendors test specific capabilities of EHRs with clinical end-users and report their usability testing process - including the test scenarios used - along with the results. The ONC outlines basic expectations for usability testing, but there is little guidance in usability texts or scientific literature on how to develop usability scenarios for healthcare applications. The objective of this article is to outline key factors to consider when developing usability scenarios and tasks to evaluate computer-interface based health information technologies. To achieve this goal, we draw upon a decade of our experience conducting usability tests with a variety of healthcare applications and a wide range of end-users, to include healthcare professionals as well as patients. We discuss 10 key factors that influence scenario development: objectives of usability testing; roles of end-user(s); target performance goals; evaluation time constraints; clinical focus; fidelity; scenario-related bias and confounders; embedded probes; minimize risks to end-users; and healthcare related outcome measures. For each factor, we present an illustrative example. This article is intended to aid usability researchers and practitioners in their efforts to advance health information technologies. The article provides broad guidance on usability scenario development and can be applied to a wide range of clinical information systems and applications.

This paper describes an effort to design and evaluate persuasive educational materials for colorectal cancer (CRC) screening. Although CRC screening is highly effective, screening rates in the US remain low. Educational materials represent one strategy for educating patients about screening options and increasing openness to screening. We developed a one-page brochure, leveraging factual information from the Centers for Disease Control and Prevention (CDC) and national guidelines, and strategies for persuasion from the human factors and behavioral economics literatures. We evaluated the resulting brochure with adults over the age of 50. Findings suggest that the educational brochure increases knowledge of CRC and screening options, and increases openness to screening. Furthermore, no significant difference was found between the new one-page brochure and an existing multi-page Screen for Life brochure recommended by the CDC. We interpret these findings as indication that the more practical and potentially less intimidating one-page brochure is as effective as the existing multi-page Screen for Life brochure.

Patient safety issues continue to be prevalent in delivering care with the electronic health record (EHR) and other health information technology (HIT) tools. Defining the unit of analysis for the study of clinical information systems is important based on the focus of the research. This study applies a conceptual framework with three levels of units of analysis for human-computer interaction (HCI) to identify and guide investigations at each level. Ethnographic observations and semi-structured, key-informant interviews were conducted with 40 healthcare workers across five primary care clinics, nine specialty clinics, and one inpatient ward at a tertiary-care Veterans Affairs Medical Center (VAMC). Two researchers recorded the interactions and interview responses of 40 healthcare workers related to their work with computerized consultations, clinical documentation, and the EHR in general. Patient safety issues from these data were categorized using three different units of analysis: EHR interface-, team coordination / workflow-, and organizational-levels. A total of 30 patient safety issues were identified; 17 emerged from the observation and interview data on computerized consults, 10 from the data on clinical documentation, and 3 were related to the EHR in general. Patient safety issues were organized by level of unit of analysis and mapped to specific methods that could be used for further investigation. Relevant concepts are discussed to help guide investigations at each level.

Challenges persist on how to effectively integrate the electronic health record (EHR) into patient visits and clinical workflow, while maintaining patient-centered care. Our goal was to identify variations in, barriers to, and facilitators of the use of the US Department of Veterans Affairs (VA) EHR in ambulatory care workflow in order better to understand how to integrate the EHR into clinical work. We observed and interviewed 20 ambulatory care providers across three geographically distinct VA medical centers. Analysis revealed several variations in, associated barriers to, and facilitators of EHR use corresponding to different units of analysis: computer interface, team coordination/workflow, and organizational. We discuss our findings in the context of different units of analysis and connect variations in EHR use to various barriers and facilitators. Findings from this study may help inform the design of the next generation of EHRs for the VA and other healthcare systems.

This article reports redesign strategies identified to create a Web-based user-interface for the Self-management TO Prevent (STOP) Stroke Tool. Members of a Stroke Quality Improvement Network (N = 12) viewed a visualization video of a proposed prototype and provided feedback on implementation barriers/facilitators. Stroke-care providers (N = 10) tested the Web-based prototype in think-aloud sessions of simulated clinic visits. Participants’ dialogues were coded into themes. Access to comprehensive information and the automated features/systematized processes were the primary accessibility and usability facilitator themes. The need for training, time to complete the tool, and computer-centric care were identified as possible usability barriers. Patient accountability, reminders for best practice, goal-focused care, and communication/counseling themes indicate that the STOP Stroke Tool supports the paradigm of patient-centered care. The STOP Stroke Tool was found to prompt clinicians on secondary stroke-prevention clinical-practice guidelines, facilitate comprehensive documentation of evidence-based care, and support clinicians in providing patient-centered care through the shared decision-making process that occurred while using the action-planning/goal-setting feature of the tool.

This article identifies sources of variation in clinical workflow and implications for the design and implementation of electronic clinical decision support. Sources of variation in workflow were identified via rapid ethnographic observation, focus groups, and interviews across a total of eight medical centers in both the Veterans Health Administration and academic medical centers nationally regarded as leaders in developing and using clinical decision support. Data were reviewed for types of variability within the social and technical subsystems and the external environment as described in the sociotechnical systems theory. Two researchers independently identified examples of variation and their sources, and then met with each other to discuss them until consensus was reached. Sources of variation were categorized as environmental (clinic staffing and clinic pace), social (perception of health information technology and real-time use with patients), or technical (computer access and information access). Examples of sources of variation within each of the categories are described and discussed in terms of impact on clinical workflow. As technologies are implemented, barriers to use become visible over time as users struggle to adapt workflow and work practices to accommodate new technologies. Each source of variability identified has implications for the effective design and implementation of useful health information technology. Accommodating moderate variability in workflow is anticipated to avoid brittle and inflexible workflow designs, while also avoiding unnecessary complexity for implementers and users.

Background Interest in human factors has increased across healthcare communities and institutions as the value of human centred design in healthcare becomes increasingly clear. However, as human factors is becoming more prominent, there is growing evidence of confusion about human factors science, both anecdotally and in scientific literature. Some of the misconceptions about human factors may inadvertently create missed opportunities for healthcare improvement. Methods The objective of this article is to describe the scientific discipline of human factors and provide common ground for partnerships between healthcare and human factors communities. Results The primary goal of human factors science is to promote efficiency, safety and effectiveness by improving the design of technologies, processes and work systems. As described in this article, human factors also provides insight on when training is likely (or unlikely) to be effective for improving patient safety. Finally, we outline human factors specialty areas that may be particularly relevant for improving healthcare delivery and provide examples to demonstrate their value. Conclusions The human factors concepts presented in this article may foster interdisciplinary collaborations to yield new, sustainable solutions for healthcare quality and patient safety.

Adoption of clinical decision support has been limited. Important barriers include an emphasis on algorithmic approaches to decision support that do not align well with clinical work flow and human decision strategies, and the expense and challenge of developing, implementing, and refining decision support features in existing electronic health records (EHRs). We applied decision-centered design to create a modular software application to support physicians in managing and tracking colorectal cancer screening. Using decision-centered design facilitates a thorough understanding of cognitive support requirements from an end user perspective as a foundation for design. In this project, we used an iterative design process, including ethnographic observation and cognitive task analysis, to move from an initial design concept to a working modular software application called the Screening & Surveillance App. The beta version is tailored to work with the Veterans Health Administration’s EHR Computerized Patient Record System (CPRS). Primary care providers using the beta version Screening & Surveillance App more accurately answered questions about patients and found relevant information more quickly compared to those using CPRS alone. Primary care providers also reported reduced mental effort and rated the Screening & Surveillance App positively for usability.

Display Omitted This method was feasible to evaluate users information processing of alerts.This method detected significant differences between two alert displays.The study method may be useful for other health information technology displays. BackgroundPrescribers commonly receive alerts during medication ordering. Prescribers work in a complex, time-pressured environment; to enhance the effectiveness of safety alerts, the effort needed to cognitively process these alerts should be minimized. Methods to evaluate the extent to which computerized alerts support prescribers information processing are lacking. ObjectiveTo develop a methodological protocol to assess the extent to which alerts support prescribers information processing at-a-glance; specifically, the incorporation of information into their working memory. We hypothesized that the method would be feasible and that we would be able to detect a significant difference in prescribers information processing with a revised alert display that incorporates warning design guidelines compared to the original alert display. MethodsA counterbalanced, within-subject study was conducted with 20 prescribers in a human-computer interaction laboratory. We tested a single alert that was displayed in two different ways. Prescribers were informed that an alert would appear for 10s. After the alert was shown, a white screen was displayed, and prescribers were asked to verbally describe what they saw; indicate how many total warnings; and describe anything else they remembered about the alert. We measured information processing via the accuracy of prescribers free recall and their ability to identify that three warning messages were present. Two analysts independently evaluated participants responses against a comprehensive catalog of alert elements and then discussed discrepancies until reaching consensus. ResultsThis feasibility study demonstrated that the method seemed to be effective for evaluating prescribers information processing of medication alert displays. With this method, we were able to detect significant differences in prescribers recall of alert information. The proportion of total data elements that prescribers were able to accurately recall was significantly greater for the revised versus original alert display (p=0.006). With the revised display, more prescribers accurately reported that three warnings were shown (p=0.002). ConclusionsThe methodological protocol was feasible for evaluating the alert display and yielded important findings on prescribers information processing. Study methods supplement traditional usability evaluation methods and may be useful for evaluating information processing of other healthcare technologies.

In order to create user-centered design information to guide the development of personal health records (PHRs), 24 patients participated in usability assessments of VA's MyHealtheVet program. Observational videos and efficiency measures were collected among users performing four PHR scenarios: registration and log-in, prescription refill, tracking health, and searching for health information. Twenty-five percent of users successfully completed registration. Individuals preferred prescription numbers over names, sometimes due to privacy concerns. Only efficiency in prescription refills was significantly better than target values. Users wanted to print their information to share with their doctors, and questioned the value of MyHealtheVet search functions over existing online health information. In summary, PHR registration must balance simplicity and security, usability tests guide how PHRs can tailor functions to individual preferences, PHRs add value to users' data by making information more accessible and understandable, and healthcare organizations should build trust for PHR health content.

With the United States national goal and incentive program to transition from paper to electronic health records (EHRs), healthcare organizations are increasingly implementing EHRs and other related health information technology (IT). However, in institutions which have long adopted these computerized systems, such as the Veterans Health Administration, healthcare workers continue to rely on paper to complete their work. Furthermore, insufficient EHR design also results in computer-based workarounds. Using direct observation with opportunistic interviewing, we investigated the use of paper- and computerbased workarounds to the EHR with a multi-site study of 54 healthcare workers, including primary care providers, nurses, and other healthcare staff. Our analysis revealed several paper- and computer-based workarounds to the VA’s EHR. These workarounds, including clinician-designed information tools, provide evidence for how to enhance the design of the EHR to better support the needs of clinicians.

Recent nursing graduates often find it challenging to prioritize competing tasks in the increasingly complex hospital environment. In order to develop a normative framework for task prioritization, Registered Nurses from a variety of settings were interviewed about how they prioritized their activities in the workplace. From 30 interviews, 422 prioritization decisions were coded when study participants were confronted with two tasks that could not be done simultaneously. The findings suggest a seven-level prioritization hierarchy of nursing activities: 1) addressing imminent clinical concerns, 2) high uncertainty activities, 3) significant, core clinical caregiving and managing pain, 4) relationship management, 5) documenting, helping others, and patient support, 6) system improvement and cleaning/preparing supplies, and 7) personal breaks and social interactions. Explicitly providing a normative framework for task prioritization during nursing education may help to accelerate learning this skill. For more experienced nurses, we believe that flexibility to deviate from this framework will be important for providing high-quality, personalized care that best matches the particular context. Relevancy to industry Nursing personnel comprises a significant portion of the human capital and operating budget for hospitals. Knowledge of what tasks are likely to be shed under severe workload conditions aids with optimizing hiring and allocation of nursing personnel.

The aim of this investigation was to assess helpful and challenging aspects of electronic health information with respect to clinical workflow and identify a set of characteristics that support patient care processes. We conducted 20 semi-structured interviews at a Veterans Affairs Medical Center, with a fully implemented electronic health record (EHR), and elicited positive and negative examples of how information technology (IT) affects the work of healthcare employees. Responses naturally shed light on information characteristics that aid work processes. We performed a secondary analysis on interview data and inductively identified characteristics of electronic information that support healthcare workflow. Participants provided 199 examples of how electronic information affects workflow. Seventeen characteristics emerged along with four primary domains: trustworthy and reliable; ubiquitous; effectively displayed; and adaptable to work demands. Each characteristic may be used to help evaluate health information technology pre- and post-implementation. Results provide several strategies to improve EHR design and implementation to better support healthcare workflow.

While the potential benefits of computerized provider order entry (CPOE) are well recognized, little is known about how prescribers have integrated electronic medication ordering into other aspects of patient care. As part of a larger investigation of computerized medication alerts, we observed and opportunistically interviewed 20 primary care prescribers at a major Veterans Affairs Medical Center (VAMC). Participants were recruited from each of the VAMC's 5 primary care clinics and included physicians, nurse practitioners, and clinical pharmacists. Prescribers were observed as they ordered mediations via the CPOE system during routine patient care. In total, observations included 91 routine patient encounters across 66.5 hrs of observation. This paper provides illustrative case examples of how prescribers have integrated CPOE into medication decision-making processes and other patient care tasks. Results demonstrate that CPOE workflow varies widely among prescribers. In addition, results indicate that there are trade-offs associated with using CPOE with the patient in the exam room versus outside of the exam room. Findings may have implications for provider-patient relationships, workflow efficiency, and medication safety, and may ultimately enhance the effectiveness of CPOE in primary care.

We developed a User-Testing Database to be able to process a greater amount of user data, from multiple sources of data, at a much finer level of granularity, and to be able to aid in a more sophisticated analysis, including specific queries of usability data than a typical “manual”-based usability analysis. In this paper, we demonstrate our User-Testing Database as applied to a usability assessment of the Veterans Affairs (VA) My HealtheVet personal health record. The usability test included 24 Veterans who completed a series of scenarios in the usability lab at a Midwest Veterans Affairs Medical Center (VAMC). The User-Testing Database facilitated reduction of data gathered from video review, facilitator notes, and debrief notes into 1160 observations that were sorted into conceptual bins and summarized for the designers of the personal health record. From creation of the database to completion of the reports took four months and did not require extensive knowledge of qualitative analysis techniques. We argue a User-Testing Database can allow other usability studies to increase the number of participants and the granularity of the data without prohibitively increasing the amount of time and experience required to process the data gathered.

Personal health records (PHRs) are designed to help people manage information about their health. Over the past decade, there has been a proliferation of PHRs, but research regarding their effects on clinical, behavioral, and financial outcomes remains limited. The potential for PHRs to facilitate patient-centered care and health system transformation underscores the importance of embracing a broader perspective on PHR research.Drawing from the experiences of VA staff to evaluate the My HealtheVet (MHV) PHR, this article advocates for a health services research perspective on the study of PHR systems.We describe an organizing framework and research agenda, and offer insights that have emerged from our ongoing efforts regarding the design of PHR-related studies, the need to address PHR data ownership and consent, and the promotion of effective PHR research collaborations.These lessons are applicable to other PHR systems and the conduct of PHR research across different organizational contexts.

Summary Objective: Computerized clinical reminder (CCR) systems can improve preventive service delivery by providing patient-specific reminders at the point of care. However, adherence varies between individual CCRs and is correlated to resolution time amongst other factors. This study aimed to evaluate how a proposed CCR redesign providing information explaining why the CCRs occurred would impact providers’ prioritization of individual CCRs. Design: Two CCR designs were prototyped to represent the original and the new design, respectively. The new CCR design incorporated a knowledge-based risk factor repository, a prioritization mechanism, and a role-based filter. Sixteen physicians participated in a controlled experiment to compare the use of the original and the new CCR systems. The subjects individually simulated a scenario-based patient encounter, followed by a semi-structured interview and survey. Measurements: We collected and analyzed the order in which the CCRs were prioritized, the perceived usefulness of each design feature, and semi-structured interview data. Results: We elicited the prioritization heuristics used by the physicians, and found a CCR system needed to be relevant, easy to resolve, and integrated with workflow. The redesign impacted 80% of physicians and 44% of prioritization decisions. Decisions were no longer correlated to resolution time given the new design. The proposed design features were rated useful or very useful. Conclusion: This study demonstrated that the redesign of a CCR system using a knowledge-based risk factor repository, a prioritization mechanism, and a role-based filter can impact clinicians’ decision making. These features are expected to ultimately improve the quality of care and patient safety.

Electronic health record (EHR) systems and health information technology (IT) hold unrealized potential for improving the quality, continuity, and safety of medical care; they can also introduce new gaps in care and present unique challenges for healthcare workers. We conducted 14 key informant, semi-structured interviews at a large Veterans Affairs Medical Center and asked healthcare employees why paper use persists despite a fully implemented EHR. In this investigation, we performed a secondary analysis on interview data to identify characteristics of information technology (IT) that are important to support healthcare workflow. As a result of this study, 17 distinct information characteristics emerged; in this document, we provide detail on five characteristics that were often cited as desirable for workflow but insufficiently supported by health IT: 1) customizable, 2) prioritized, 3) trendable, 4) locatable, and 5) accessible. Results from this study reveal key electronic information characteristics for healthcare workflow and have implications for patient safety and future health IT designs.

Computerized medication alerts (e.g., drug-drug interactions, drug-allergy interactions), which are intended to protect patient safety, need to match the mental models of medication prescribers in order to aid medication ordering. To maximally protect patient safety, the programmer mental model, system image, and prescriber mental model should work seamlessly together to fully support prescriber decision-making. In this study, we examined prescribing processes in the context of routine patient care to understand how the design of medication alerts can be enhanced for prescribers. We shadowed prescribers, including physicians, pharmacists, and nurse practitioners, across five outpatient primary care clinics at a large Veterans Affairs Medical Center (VAMC). In addition, prescribers were opportunistically interviewed as they ordered mediations via a computerized order entry system and resolved any subsequent medication alerts. This investigation is one of the few to examine medication alerts by directly observing prescribers during patient care. Altogether, 191 medication alerts occurred across 63.5 total hrs of observation, 19 prescribers, and 86 patients during routine patient care tasks. Results reveal problematic system images and mismatches between programmer and prescriber mental models. Findings can help inform medication alert redesigns, which may promote safer, more effective prescribing practices.

Objective Healthcare organizations are increasingly implementing electronic health records (EHRs) and other related health information technology (IT). Even in institutions which have long adopted these computerized systems, employees continue to rely on paper to complete their work. The objective of this study was to explore and understand human-technology integration factors that may be causing employees to rely on paper alternatives to the EHR. Methods We conducted semi-structured interviews with 20 key-informants in a large Veterans Affairs Medical Center (VAMC), with a fully implemented EHR, to understand the use of paper-based alternatives. Participants included clinicians, administrators, and IT specialists across several service areas in the medical center. Results We found 11 distinct categories of paper-based workarounds to the use of the EHR. Paper use related to the following: (1) efficiency; (2) knowledge/skill/ease of use; (3) memory; (4) sensorimotor preferences; (5) awareness; (6) task specificity; (7) task complexity; (8) data organization; (9) longitudinal data processes; (10) trust; and (11) security. We define each of these and provide examples that demonstrate how these categories promoted paper use in spite of a fully implemented EHR. Conclusions In several cases, paper served as an important tool and assisted healthcare employees in their work. In other cases, paper use circumvented the intended EHR design, introduced potential gaps in documentation, and generated possible paths to medical error. We discuss implications of these findings for EHR design and implementation.

Summary Objectives Clinical information system (CIS) developers and implementers have begun to look to other scientific disciplines for new methods, tools, and techniques to help them better understand clinicians and their organizational structures, clinical work environments, capabilities of clinical information and communications technology, and the way these structures and processes interact. The goal of this article is to help CIS researchers, developers, implementers, and evaluators better understand the methods, tools, techniques, and literature of the field of human factors. Methods We developed a framework that explains how six key human factors topics relate to the design, implementation, and evaluation of CISs. Results Using this framework we discuss the following six topics: 1) informatics and patient safety; 2) user interface design and evaluation; 3) workflow and task analysis; 4) clinical decision making and decision support; 5) distributed cognition; and 6) mental workload and situation awareness. Conclusions Integrating the methods, tools, and lessons learned from each of these six areas of human factors research early in CIS design and incorporating them iteratively during development can improve user performance, user satisfaction, and integration into clinical workflow. Ultimately, this approach will improve clinical information systems and healthcare delivery.

Healthcare organizations are increasingly implementing electronic medical records (EMRs) and other related health information technology (IT). Even in institutions which have long adopted these computerized systems, there are still instances where employees rely on paper to complete their work. The use of paper suggests that parts of the EMR may not be sufficiently designed to support clinicians and their work processes. To understand the use of paper-based alternatives, we conducted 14 key-informant interviews in a large Veterans Affairs Medical Center (VAMC), with a fully implemented EMR. We found nine distinct categories of paper-based workarounds to the use of the EMR. In several cases, paper served as an important tool and assisted healthcare employees in their work. In other cases, paper use circumvented the intended EMR design, introduced potential gaps in documentation, and generated possible paths to medical error. We discuss implications of these findings for EMR design and implementation.

OBJECTIVE: We examined the willingness and ability of general aviation pilots to execute steep approaches in low-visibility conditions into nontowered airports. BACKGROUND: Executing steep approaches in poor weather is required for a proposed Small Aircraft Transportation System (SATS) that consists of small aircraft flying direct routes to a network of regional airports. METHOD: Across two experiments, 17 pilots rated for Instrument Flight Rules at George Mason University or Virginia Tech flew a Cessna 172R simulator into Blacksburg, Virginia. Pilots were familiarized with the simulator and asked to fly approaches with either a 200- or 400-foot ceiling (at approach angles of 3 degrees, 5 degrees, and 7 degrees in the first experiment, 3 degrees and 6 degrees in the second). Pilots rated subjective workload and the simulator recorded flight parameters for each set of approaches. RESULTS: Approaches with a 5 degree approach angle produced safe landings with minimal deviations from normal descent control configurations and were rated as having a moderate level of workload. Approaches with 6 degree and 7 degree approach angles produced safe landings but high workload ratings. Pilots reduced power to control the speed of descent and flew the aircraft slightly above the glide path to gain time to control the landing. CONCLUSION: Although the 6 degree and 7 degree approaches may not be practical for routine approaches, they may be achievable in the event of an emergency. APPLICATION: Further work using other aircraft flying under a wider variety of conditions is needed before implementing SATS-type flights into airports intended to supplant or complement commercial operations in larger airports. Language: en

In this paper we report on an in-depth study of engineering design processes. Specifically, we extend our previous research on engineering student design processes to compare the design behavior of students and expert engineers. Nineteen experts from a variety of engineering disciplines and industries each designed a playground in a lab setting, and gave verbal reports of their thoughts during the design task. Measures of their design processes and solution quality were compared to pre-existing data from 26 freshmen and 24 seniors. The experts spent significantly more time on the task overall and in each stage of engineering design, including significantly more time problem scoping. The experts also gathered significantly more information covering more categories. Results support the argument that problem scoping and information gathering are major differences between advanced engineers and students, and important competencies for engineering students to develop. Timeline representations of the expert designers' processes illustrate characteristic distinctions we found and may help students gain insights into their own design processes.

Objective: Computerized clinical reminders (CRs) were designed to reduce clinicians' reliance on their memory and to present evidence-based guidelines at point of care. However, the literature indicates that CR adoption and effectiveness has been variable. We examined the impact of four design modifications to CR software on learnability, efficiency, usability, and workload for intake nursing personnel in an outpatient clinic setting. These modifications were included in a redesign primarily to address barriers to effective CR use identified during a previous field study. Design: In a simulation experiment, 16 nurses used prototypes of the current and redesigned system in a within- subject comparison for five simulated patient encounters. Prior to the experimental session, participants completed an exploration session, where "learnability" of the current and redesigned systems was assessed. Measurements: Time, performance, and survey data were analyzed in conjunction with semi-structured debrief interview data. Results: The redesign was found to significantly increase learnability for first-time users as measured by time to complete the first CR, efficiency as measured by task completion time for two of five patient scenarios, usability as determined by all three groupings of questions taken from a commonly used survey instrument, and two of six workload subscales of the NASA Task Load Index (TLX) survey: mental workload and frustration. Conclusion: Modest design modifications to existing CR software positively impacted variables that likely would increase the willingness for first-time nursing personnel to adopt and consistently use CRs.

Background: Computerized medication alerts can often be bypassed by entering an override rationale, but prescribers’ override reasons are frequently ambiguous to pharmacists who review orders. Objective: To develop and evaluate a new override mechanism for adverse reaction and drug-drug interaction alerts. We hypothesized that the new mechanism would improve usability for prescribers and increase the clinical appropriateness of override reasons. Methods: A counterbalanced, crossover study was conducted with 20 prescribers in a simulated prescribing environment. We modified the override mechanism timing, navigation, and text entry. Instead of free-text entry, the new mechanism presented prescribers with a predefined set of override reasons. We assessed usability (learnability, perceived efficiency, and usability errors) and used a priori criteria to evaluate the clinical appropriateness of override reasons entered. Results: Prescribers rated the new mechanism as more efficient (Wilcoxon signed-rank test, P = 0.032). When first using the new design, 5 prescribers had difficulty finding the new mechanism, and 3 interpreted the navigation to mean that the alert could not be overridden. The number of appropriate override reasons significantly increased with the new mechanism compared with the original mechanism (median change of 3.0; interquartile range = 3.0; P < 0.0001). Conclusions: When prescribers were given a menu-based choice for override reasons, clinical appropriateness of these reasons significantly improved. Further enhancements are necessary, but this study is an important first step toward a more standardized menu of override choices. Findings may be used to improve communication through e-prescribing systems between prescribers and pharmacists.

Objective This study evaluated the current use of commercial-off-the-shelf Clinical Information Systems (CIS) for intensive care units (ICUs) and Anesthesia Record Keeping (ARK) for operating rooms and post-anesthesia care recovery settings at three Veterans Affairs Medical Centers (VAMCs). Clinicians and administrative staff use these applications at bedside workstations, in operating rooms, at nursing stations, in physician's rooms, and in other various settings. The intention of a CIS or an ARK system is to facilitate creation of electronic records of data, assessments, and procedures from multiple medical devices. The US Department of Veterans Affairs (VA) Office of the Chief of Nursing Informatics sought to understand usage barriers and facilitators to optimize these systems in the future. Therefore, a human factors study was carried out to observe the CIS and ARK systems in use at three VAMCs in order to identify best practices and suggested improvements to currently implemented CIS and ARK systems. Methods We conducted a rapid ethnographic study of clinical end-users interacting with the CIS and ARK systems in the critical care and anesthesia care areas in each of three geographically distributed VAMCs. Two observers recorded interactions and/or interview responses from 88 CIS and ARK end-users. We coded and sorted into logical categories field notes from 69 shadowed participants. The team transcribed and combined data from key informant interviews with 19 additional participants with the observation data. We then integrated findings across observations into meaningful patterns and abstracted the data into themes, which translated directly to barriers to effective adoption and optimization of the CIS and ARK systems. Results Effective optimization of the CIS and ARK systems was impeded by: (1) integration issues with other software systems; (2) poor usability; (3) software challenges; (4) hardware challenges; (5) training concerns; (6) unclear roles and lack of coordination among stakeholders; and (7) insufficient technical support. Many of these barriers are multi-faceted and have associated sub-barriers, which are described in detail along with relevant quotes from participants. In addition, regionalized purchases of different CIS and ARK systems, as opposed to enterprise level purchases, contributed to some of the identified barriers. Facilitators to system use included (1) automation and (2) a dedicated facility-level CIS-ARK Coordinator. Conclusions We identified barriers that explain some of the challenges with the optimization of the CIS and ARK commercial systems across the Veterans Health Administration (VHA). To help address these barriers, and evolve them into facilitators, we categorized report findings as (1) interface and system-level changes that vendors or VA healthcare systems can implement; (2) implementation factors under VA control and not under VA control; and (3) factors that may be used to inform future application purchases. We outline several recommendations for improved adoption of CIS and ARK systems and further recommend that human factors engineering and usability requirements become an integral part of VA health information technology (HIT) application procurement, customization, and implementation in order to help eliminate or mitigate some of the barriers of use identified in this study. Human factors engineering methods can be utilized to apply a user-centered approach to application requirements specification, application evaluation, system integration, and application implementation.