Search

The COVID-19 pandemic has rapidly accelerated the adoptionof on-demand telehealth as a way to keep patients and providers safe from viral transmission. However, theconstrained timelines and challenging conditions under which telehealth was implemented along with the factthat many patientsand providers are interacting in a new modality raisesconcerns about the potential for unintended patient safety risks that have yet to be fully understood. With the surge in telehealth patient volumeduringthe pandemic,on demand telehealth has served as a way to triageand refer patients who need additional in-person evaluation and treatment. One safety concern is whether patients who are referred are successfully following through on in-person care referrals as thefailure to seekin-person care may lead to increased risk of delayed or missed diagnosis. This study aimsto evaluate whether patientsare successfully fulfilling their in -person referralsfollowingareferralfromon-demandtelehealth through retrospective analysis of a subset of telehealth referral datafrom a large healthcare system. Of the 911 on-demand telehealth visitswith a referral to in-person care, only 689showed anin-person care encounter following the telehealthvisit and only 75-85% ofmore immediate urgency in-person referrals were fulfilled within the recommended time period of 24 hours. This preliminary data highlights theneed for amore comprehensiveanalysisof the telehealth referral processand the application of human factors methodsto understand and address barriers and risks associated with telehealth referrals and successful follow up.

The National Academies Board on Human Systems Integration (BOHSI) has organized this session exploring the state of the art and research and design frontiers relating to the application of Artificial Intelligence (AI) technologies to Healthcare. Advances in AI, machine learning, and robotics promise to revolutionize access to and quality of healthcare. At the same time lessons learned from earlier applications of AI technologies suggest a need to carefully consider the user and use context and the need for effective human-machine teaming design to avoid brittle systems that fail to be used or worse that negatively impact patient safety. The Panel brings together prominent researchers from both the Human Factors and Computer Science and AI communities to discuss promising applications, challenges and short-falls and ways forward for achieving the promise of enhanced delivery of health care through application of AI technologies.

Computerized Provider Order Entry (CPOE) is a critical component of electronic health records (EHR). Although widely adopted, the usability of these systems is not well understood. A cross-sectional usability study was conducted at four healthcare systems using two commercially available EHRs. This research aims to predict the accuracy of an order placed in the EHR by emergency medicine physicians. Physician-EHR interactions were recorded during six simulated task scenarios in a test environment. The data from 222 interactions were used to predict the accuracy of the order placed. Multiple models were developed, and their performance was compared. The Random Forest model had the best performance with F1 score 0.82 and AUC of 0.89. These results demonstrate the utility of using keystroke, time and user demographics as variables in predicting accurate order placement. Applying such models to passively monitor EHR usage in real world clinical environments may help improve patient safety and care.

Introduction: The objective of this paper is to describe a manual annotation process to identify likely health information technology (IT) related patient safety event (PSE) reports and the descriptive analysis of the self-reported event type categories of the resulting likely health IT related events. Methods: 5287 PSE reports were manually coded as likely or unlikely related to health IT and the structured general and specific event type categories were analyzed. Results: Of the 2435 likely health IT related events, 1200 were categorized as medication error events, 709 described an error related to a procedure, treatment, or test, and the remaining events were distributed among 19 different general event types. Discussion: The variety of self-reported general and specific event categories indicates a potential need to provide additional support for individuals reporting events to capture contextual nuances and incorporating advanced techniques to assist researchers and safety officers in identifying health IT related events.

Care for cancer patients is very complex with many treatments involving drugs that can be toxic outside of the therapeutic range. As such, cancer patients are a particularly vulnerable population within the healthcare system, with any error potentially being harmful to a population whose health is already compromised. Self-reported patient safety events can provide a window into healthcare, revealing areas of care that require improvement. We identified events occurring in oncology units that both caused harm and were involved cancer patients for both out-patients and in-patients. Across all events, adverse drug reactions and intravenous-related events were the most common. Other relatively frequent events included those related to poor care coordination and falling. These results illustrate the safety risks patients’ face when being treated with cytotoxic drugs and offer insight to areas in which patient care can be improved.

In recent years, there has been tremendous growth in both the number and diversity of wearable sensors, including sensors for monitoring mental state. Understanding physiological metrics such as fatigue and stress is an important aspect of human factors research, yet collecting and analyzing these measures can be resource intensive. This paper explores the usability and applicability of four off-the-shelf wearable sensors (Emotiv Epoc, Melon Headband, Spire Stone, and Muse™ Headband) for applied healthcare research. We perform a heuristic usability evaluation of the four sensors and discuss the extent to which each device can be applied to human factors healthcare research in clinical settings.

Stationary eye-tracking technology has been used extensively in human-computer interaction to both understand how humans interact with computers and as an interaction mechanism. Mobile eye-tracking technology is becoming more prevalent, yet the analysis and annotation of mobile eye-tracking data remains challenging. We present a novel human-in-the-loop approach for mobile eye-tracking data analysis that dramatically reduces resource requirements. This method incorporates human insight in a semi-automatic decision making process, leveraging both computational power and human decision making abilities. We demonstrate the accuracy of this approach with eye movement data from two real-world use cases. Average accuracy across the two environments is 82.3%. Our approach holds tremendous promise and has the potential to open the door to more robust eye movement studies in the real-world.

The relatively rapid transition from a paper-based system to a digital system in healthcare has not always employed a sophisticated integration of usability concepts. Yet usability is critical to safety and to effectiveness of the electronic health record, and regulators and policy makers have been increasingly focused on this area. This panel will provide a variety of perspectives on this important issue, ranging from a description of the problem based on current vendor usability practices; recommendations regarding domain content rich usability processes including use cases, assessments, and scenarios; and the extension of usability assessments and design improvements to post-system implementation.

Over the past two decades, the focus on capturing and understanding patient safety events has led to the development of several initiatives and programs in hospitals across the country to increase the reporting of these events. Computerized reporting systems are an integral part of these programs. With these systems going live, hospitals now have databases that contain large numbers of patient safety event reports. Now, the challenge is to make sense of and effectively analyze the data by examining trends and predicting future events. The purpose of this practice-orientated paper is to describe and demonstrate how to apply advances in information visualization techniques and methods to patient safety event data with the goal of making these data more understandable and actionable. We will discuss the process of how we developed patient safety event visualizations for a multi-hospital healthcare system and provide visualization techniques, resources, guidelines, and practical examples.

While there are several theoretical accounts of interruptions and numerous empirical studies that have examined the impact of interruptions in laboratory settings, our understanding of interruptions in healthcare is in its infancy. Understanding and managing interruptions in healthcare can be particularly challenging because of the diversity of clinical environments and the importance of interruptions in delivering effective care in some settings. Our expert panel will review the theoretical and practical aspects of interruptions and facilitate a meaningful discussion on how to advance the study of interruptions in healthcare. The discussion will focus on patient safety implications of interruptions and methods to better manage interruptions in complex clinical environments.

Interruptions can adversely affect human performance, particularly in fast-paced and high-risk environments. Much of the research on interruptions has been laboratory based and the extension of these methods to real-world settings has been challenging and limited. This paper discusses the development and usage of a new tool, TaskTracker, to increase understanding of interruptions in the emergency department. With the data collected from this tool we identified several temporal groupings of interruptions, what we define as interruption clusters. We found significantly more clusters during self-initiated computer tasks. In this setting, we also observed the tendencies of assistants, technicians, students, and nurses to interrupt attending physicians in clusters. A deeper understanding of who engages in interruption clusters and why may provide insights for future systemic strategies that could facilitate better communication patterns.

A postcompletion error occurs when the final step of a task is omitted because the main goal of the task is thought to be completed (Byrne & Bovair, 1997). Postcompletion errors are more likely to occur after interruptions (Ratwani, McCurry & Trafton, 2008). Global placekeeping cues (Gray, 2000) allow a user to track their progress in a task and may be a method for reducing the rate of postcompletion errors. A computer-based procedural task with a postcompletion step was used in this experiment to determine how the interaction of global placekeeping cues with interruptions would affect postcompletion errors. These results suggest that global placekeeping cues reduce the postcompletion error rate after interruptions, but that global placekeeping does not completely eliminate postcompletion errors.

One method for reducing the disruptiveness of interruptions is to present an alert prior to an interruption. Based on the Memory for Goals theory (Altmann and Trafton, 2002), this alert period provides an opportunity to maintain an associative link between the suspended primary task goal and relevant environmental cues which facilitates resumption. This theory does not, however, describe what types of alerts are most effective. Using reaction time and eye movement measures, three different types of alerts were examined to determine which alerts were the most effective and to determine which afforded the greatest opportunity to form an associative link between the suspended primary task goal and relevant environmental cues. Alert conditions resulted in faster resumption times than a no alert condition. There were no differences between the alert conditions themselves, despite eye movement differences reflecting cue association processes. The eye movement data suggest that one fixation is enough to form an associative link which reduces the disruptiveness of interruptions. Keywords: Driver distraction; Language: en

The majority of empirical papers investigating the effect of interruption modality on primary task resumption have been grounded in Multiple Resource theory; this theory stresses the benefits of cross-modal information presentation. Alternatively, Altmann and Trafton's (2002) Memory for Goals theory suggests that maintaining an association between the suspended primary task goal and relevant environmental cues is critical to the task resumption process. Using reaction time and eye movement measures, the theoretical predictions of these two frameworks were empirically examined to determine whether interruption modality influences primary task resumption.

Theories accounting for the process of primary task resumption following an interruption have focused on the suspension and retrieval of a specific goal (Altmann & Trafton, 2002). The ability to recall the spatial location of where in the task one was prior to being interrupted may also be important. We show that being able to maintain a spatial representation of the primary task facilitates task resumption. Participants were interrupted by an instant message window that either partially or fully occluded the primary task interface. Reaction time measures show that participants were faster at resuming in the partial occlusion condition. In addition, eye track data suggest that participants were more accurate at returning to where they left off, suggesting that they were able to maintain a spatial representation of the task and use this information to resume more quickly.

Most research on interruptions has shown that they can be disruptive by causing a longer time to complete the primary task and by causing more errors on the primary task. However, a limited amount of research has shown that interruptions can actually be beneficial to simple primary tasks, a benefit that has been explained by arousal. We sought to replicate the finding that simple tasks can benefit from interruptions and to examine the specific processes that are actually improving as a result of the interruptions. More specifically, reaction time data and eye movement data were collected to account for motor actions and perceptual processes. Results indicate that participants' immediate action following the interruption was disrupted. However, participants' other actions during the interruption trials were actually performed faster and with fewer errors as compared to the control. This speed-up is not attributed to faster motor responses, but actually to faster perceptual processing.

We examined how graph readers extracted specific information, integrated information and made inferences from choropleth graphs. We present a hierarchical framework of graph comprehension suggesting how graph readers extract these different types of information. Our framework suggests the cognitive operations required to extract these different types of information build in a hierarchical fashion as the complexity of the type of extraction increases. Empirical data gathered in our laboratory is reviewed in support of our hierarchical framework and implications are discussed.