Search

Objective Situation awareness (SA) refers to people’s perception and understanding of their dynamic environment. In primary care, reduced SA among physicians increases errors in clinical decision-making and, correspondingly, patients’ risk of experiencing adverse outcomes. Our objective was to understand the extent to which electronic health records (EHRs) support primary care physicians (PCPs)’ SA during clinical decision-making. Method We conducted a metanarrative review of papers in selected academic databases, including CINAHL and MEDLINE. Eligible studies included original peer-reviewed research published between January 2012 and August 2020 on PCP–EHR interactions. We iteratively queried, screened, and summarized literature focused on EHRs supporting PCPs’ clinical decision-making and care management for adults. Then, we mapped findings to an established SA framework to classify external factors (individual, task, and system) affecting PCPs’ levels of SA (1–Perception, 2–Comprehension, and 3–Projection) and identified SA barriers. Results From 1504 articles identified, we included and synthesized 19 studies. Study designs were largely noninterventional. Studies described EHR workflow misalignments, usability issues, and communication challenges. EHR information, including lab results and care plans, was characterized as incomplete, untimely, or irrelevant. Unmet information needs made it difficult for PCPs to obtain even basic SA, Level 1 SA. Prevalent barriers to PCPs developing SA with EHRs were errant mental models, attentional tunneling, and data overload. Conclusion Based on our review, EHRs do not support the development of higher levels of SA among PCPs. Review findings suggest SA-oriented design processes for health information technology could improve PCPs’ SA, satisfaction, and decision-making.

Background As telemedicine adoption increases, so does the importance of building cohesion among physicians in telemedicine teams. For example, in acute telestroke services, stroke specialists provide rapid remote stroke assessment and treatment to patients at hospitals without stroke specialty care. In the National Telestroke Program (NTSP) of the U.S. Department of Veterans Affairs, a virtual (distributed) hub of stroke specialists throughout the country provides 24/7 consultations nationwide. We examined how these specialists adapted to distributed teamwork, and we identified cohesion-related factors in program development and support. Methods We studied the virtual hub of stroke specialists employed by the NTSP. Semi-structured, confidential interviews with stroke specialists in the virtual hub were recorded and transcribed. We explored the extent to which these specialists had developed a sense of shared identity and team cohesion, and we identified factors in this development. Using a qualitative approach with constant comparison methods, two researchers coded each interview transcript independently using a shared codebook. We used matrix displays to identify themes, with special attention to team cohesion, communication, trust, and satisfaction. Results Of 13 specialists with at least 8 months of NTSP practice, 12 completed interviews; 7 had previously practiced in telestroke programs in other healthcare systems. Interviewees reported high levels of trust and team cohesion, sometimes even more with their virtual colleagues than with co-located colleagues. Factors facilitating perceived team cohesion included a weekly case conference call, a sense of transparency in discussing challenges, engagement in NTSP development tasks, and support from the NTSP leadership. Although lack of in-person contact was associated with lower cohesion, annual in-person NTSP meetings helped mitigate this issue. Despite technical challenges in establishing a new telehealth system within existing national infrastructure, providers reported high levels of satisfaction with the NTSP. Conclusion A virtual telestroke hub can provide a sense of team cohesion among stroke specialists at a level comparable with a standard co-located practice. Engaging in transparent discussion of challenging cases, reviewing new clinical evidence, and contributing to program improvements may promote cohesion in distributed telemedicine teams.

Patients with chronic obstructive pulmonary disease (COPD) commonly experience dyspnea, which may limit activities of daily living. Pursed-lip breathing improves dyspnea for COPD patients; however, access to pursed-lip breathing training is limited.The proposed MELodica Orchestra for DYspnea (MELODY) study will be a single-site pilot study to assess the safety, feasibility, and efficacy of a music-based approach to teach pursed-lip breathing. Patients with COPD and moderate-severe dyspnea are randomized to intervention, education-control, or usual care control groups. Intervention patients meet twice weekly for eight weeks for melodica instruction, group music-making, and COPD education. Safety, feasibility, and efficacy is assessed qualitatively and quantitatively.This manuscript describes the rationale and methods of the MELODY pilot project.If pilot data demonstrate efficacy, then a multi-site randomized control trial will be conducted to evaluate program effectiveness and implementation.

Background During medical referrals, communication barriers between referring and consulting outpatient clinics delay patients’ access to health care. One notable opportunity for reducing these barriers is improved usefulness and usability of electronic medical consultation order forms. The cognitive systems engineering (CSE) design approach focuses on supporting humans in managing cognitive complexity in sociotechnical systems. Cognitive complexity includes communication, decision-making, problem solving, and planning. Objective The objective of this research was to implement a CSE design approach to develop a template that supports the cognitive needs of referring clinicians and improves referral communication. Methods We conducted interviews and observations with primary care providers and specialists at two major tertiary, urban medical facilities. Using qualitative analysis, we identified cognitive requirements and design guidelines. Next, we designed user interface (UI) prototypes and compared their usability with that of a currently implemented UI at a major Midwestern medical facility. Results Physicians’ cognitive challenges were summarized in four cognitive requirements and 13 design guidelines. As a result, two UI prototypes were developed to support order template search and completion. To compare UIs, 30 clinicians (referrers) participated in a consultation ordering simulation complemented with the think-aloud elicitation method. Oral comments about the UIs were coded for both content and valence (i.e., positive, neutral, or negative). Across 619 comments, the odds ratio for the UI prototype to elicit higher-valenced comments than the implemented UI was 13.5 (95% CI = [9.2, 19.8]), p Conclusion This study reinforced the significance of applying a CSE design approach to inform the design of health information technology. In addition, knowledge elicitation methods enabled identification of physicians’ cognitive requirements and challenges when completing electronic medical consultation orders. The resultant knowledge was used to derive design guidelines and UI prototypes that were more useful and usable for referring physicians. Our results support the implementation of a CSE design approach for electronic medical consultation orders.

Objective An electronic medication reconciliation tool was previously developed by another research team to aid provider-patient communication for medication reconciliation. To evaluate the usability of this tool, we integrated artificial safety probes into standard usability methods. The objective of this article is to describe this method of using safety probes, which enabled us to evaluate how well the tool supports users’ detection of medication discrepancies. Materials and methods We completed a mixed-method usability evaluation in a simulated setting with 30 participants: 20 healthcare professionals (HCPs) and 10 patients. We used factual scenarios but embedded three artificial safety probes: (1) a missing medication (i.e., omission); (2) an extraneous medication (i.e., commission); and (3) an inaccurate dose (i.e., dose discrepancy). We measured users’ detection of each probe to estimate the probability that a HCP or patient would detect these discrepancies. Additionally, we recorded participants’ detection of naturally occurring discrepancies. Results Each safety probe was detected by ≤50% of HCPs. Patients’ detection rates were generally higher. Estimates indicate that a HCP and patient, together, would detect 44.8% of these medication discrepancies. Additionally, HCPs and patients detected 25 and 45 naturally-occurring discrepancies, respectively. Discussion Overall, detection of medication discrepancies was low. Findings indicate that more advanced interface designs are warranted. Future research is needed on how technologies can be designed to better aid HCPs’ and patients’ detection of medication discrepancies. Conclusion This is one of the first studies to evaluate the usability of a collaborative medication reconciliation tool and assess HCPs’ and patients’ detection of medication discrepancies. Results demonstrate that embedded safety probes can enhance standard usability methods by measuring additional, clinically-focused usability outcomes. The novel safety probes we used may serve as an initial, standard set for future medication reconciliation research. More prevalent use of safety probes could strengthen usability research for a variety of health information technologies.

BackgroundA pharmacogenomic clinical decision support tool (PGx-CDS) for thiopurine medications can help physicians incorporate pharmacogenomic results into prescribing decisions by providing up-to-date, real-time decision support. However, the PGx-CDS user interface may introduce errors and promote alert fatigue. The objective of this study was to develop and evaluate a prototype of a PGx-CDS user interface for thiopurine medications with user-centered design methods.MethodsThis study had two phases: In phase I, we conducted qualitative interviews to assess providers’ information needs. Interview transcripts were analyzed through a combination of inductive and deductive qualitative analysis to develop design requirements for a PGx-CDS user interface. Using these requirements, we developed a user interface prototype and evaluated its usability (phase II).ResultsIn total, 14 providers participated: 10 were interviewed in phase I, and seven providers completed usability testing in phase II (3 providers participated in both phases). Most (90%) participants were interested in PGx-CDS systems to help improve medication efficacy and patient safety. Interviews yielded 11 themes sorted into two main categories: 1) health care providers’ views on PGx-CDS and 2) important design features for PGx-CDS. We organized these findings into guidance for PGx-CDS content and display. Usability testing of the PGx-CDS prototype showed high provider satisfaction.ConclusionThis is one of the first studies to utilize a user-centered design approach to develop and assess a PGx-CDS interface prototype for Thiopurine Methyltransferase (TPMT). This study provides guidance for the development of a PGx-CDS, and particularly for biomarkers such as TPMT.

Background: In most hub-and-spoke telestroke systems, geographically co-located hub stroke specialists support regional spoke sites. In the VA’s National Telestroke Program (NTSP), a virtual hub of stroke specialists located around the country provides 24/7 consults nationwide. We examined how stroke specialists adapted to virtual teamwork, and identified factors important in developing and sustaining a high-functioning virtual team. Methods: Semi-structured, confidential interviews with hub stroke specialists were audiotaped and transcribed. Probes were used to explore the extent to which providers had developed a sense of a teamness or a community of practice, and what factors helped or hindered this development. Core elements of a high-functioning team were defined using Mitchell's taxonomy, developed as part of the IOM's Best Practices Innovation Collaborative. Each interview transcript was independently coded by two investigators using NVivo11. The constant comparative method and matrix displays were used to identify themes, with special attention to themes about team, communication, trust, and satisfaction. Results: Of 13 hub providers with > 8 months NTSP participation, 12 were interviewed; 7 had prior telestroke experience. Participants reported high levels of trust and sense of teamwork with their virtual colleagues, sometimes even more than with local colleagues. Factors facilitating perceived teamness included communicating via a weekly case conference call, a sense of transparency in discussing challenges, engagement in NTSP development tasks, and leadership support. Lack of in-person contact decreased perceived teamness, but having an in-person NTSP meeting helped mitigate this issue. Despite technical challenges, providers reported high levels of satisfaction with the NTSP. Conclusions: Practicing as a virtual Telestroke hub provider can provide an equal or greater sense of trust and sense of teamwork with colleagues compared with traditional practice. Engaging in transparent discussion of challenging cases and contributing to program improvements may be key to promoting high-functioning virtual teams. Ongoing surveys will assess providers’ satisfaction with program outcomes over time.

Additional file 1: S1. Moderator script. S2. Task sheet for oncologist participants. S3. Example of a case for gastroenterologists. S4. Interview questions. S5. Screenshot of a mock-up electronic health record. S6. Definitions and comments from each group.

Background Web-based patient portals feature secure messaging systems that enable health care providers and patients to communicate information. However, little is known about the usability of these systems for clinical document sharing. Objective This article evaluates the usability of a secure messaging system for providers and patients in terms of its ability to support sharing of electronic clinical documents. Methods We conducted usability testing with providers and patients in a human–computer interaction laboratory at a Midwestern U.S. hospital. Providers sent a medication list document to a fictitious patient via secure messaging. Separately, patients retrieved the clinical document from a secure message and returned it to a fictitious provider. We collected use errors, task completion, task time, and satisfaction. Results Twenty-nine individuals participated: 19 providers (6 physicians, 6 registered nurses, and 7 pharmacists) and 10 patients. Among providers, 11 (58%) attached and sent the clinical document via secure messaging without requiring assistance, in a median (range) of 4.5 (1.8–12.7) minutes. No patients completed tasks without moderator assistance. Patients accessed the secure messaging system within 3.6 (1.2–15.0) minutes; retrieved the clinical document within 0.8 (0.5–5.7) minutes; and sent the attached clinical document in 6.3 (1.5–18.1) minutes. Although median satisfaction ratings were high, with 5.8 for providers and 6.0 for patients (scale, 0–7), we identified 36 different use errors. Physicians and pharmacists requested additional features to support care coordination via health information technology, while nurses requested features to support efficiency for their tasks. Conclusion This study examined the usability of clinical document sharing, a key feature of many secure messaging systems. Our results highlight similarities and differences between provider and patient end-user groups, which can inform secure messaging design to improve learnability and efficiency. The observations suggest recommendations for improving the technical aspects of secure messaging for clinical document sharing.

Just as physical appearance affects social influence in human communication, it may also affect the processing of advice conveyed through avatars, computer-animated characters, and other human-like interfaces. Although the most persuasive computer interfaces are often the most human-like, they have been predicted to incur the greatest risk of falling into the uncanny valley, the loss of empathy attributed to characters that appear eerily human. Previous studies compared interfaces on the left side of the uncanny valley, namely, those with low human likeness. To examine interfaces with higher human realism, a between-groups factorial experiment was conducted through the internet with 426 midwestern U.S. undergraduates. This experiment presented a hypothetical ethical dilemma followed by the advice of an authority figure. The authority was manipulated in three ways: depiction (digitally recorded or computer animated), motion quality (smooth or jerky), and advice (disclose or refrain from disclosing sensitive information). Of these, only the advice changed opinion about the ethical dilemma, even though the animated depiction was significantly eerier than the human depiction. These results indicate that compliance with an authority persists even when using an uncannily realistic computer-animated double.

Communication breakdowns in the referral process negatively impact clinical workflow and patient safety. There is a lack of evidence demonstrating the impact of published design recommendations addressing contributing issues with consultation order templates. This study translated the recommendations into a computer-based prototype and conducted a comparative usability evaluation. With a scenario-based simulation, 30 clinicians (referrers) participated in a within-group, counterbalanced experiment comparing the prototype with their present electronic order entry system. The prototype significantly increased satisfaction (Cohen's d = 1.80, 95% CI [1.19, 2.41], p .001), and required significantly less mental effort (d = 0.67 [0.14, 1.20], p .001). Regarding efficiency, the prototype required significantly fewer mouse clicks (mean difference = 29 clicks, p .001). Although overall task time did not differ significantly (d = -0.05 [-0.56, 0.47]), the prototype significantly quickened identification of the appropriate specialty clinic (mean difference = 12 s, d = 0.98 [0.43, 1.52], p .001). The experimental evidence demonstrated that clinician-centered interfaces significantly improved system usability during ordering of consultations.

We assessed the usability of consultation order templates and identified problems to prioritize in design efforts for improving referral communication. With a sample of 26 consultation order templates, three evaluators performed a usability heuristic evaluation. The evaluation used 14 domain-independent heuristics and the following three supplemental references: 1 new domain-specific heuristic, 6 usability goals, and coded clinicians' statements regarding ease of use for 10 sampled templates. Evaluators found 201 violations, a mean of 7.7 violations per template. Minor violations outnumbered major violations almost twofold, 115 (57%) to 62 (31%). Approximately 68% of violations were linked to 5 heuristics: aesthetic and minimalist design (17%), error prevention (16%), consistency and standards (14%), recognition rather than recall (11%), and meet referrers' information needs (10%). Severe violations were attributed mostly to meet referrers' information needs and recognition rather than recall. Recorded violations yielded potential negative consequences for efficiency, effectiveness, safety, learnability, and utility. Evaluators and clinicians demonstrated 80% agreement in usability assessment. Based on frequency and severity of usability heuristic violations, the consultation order templates reviewed may impede clinical efficiency and risk patient safety. Results support the following design considerations: communicate consultants' requirements, facilitate information seeking, and support communication. While the most frequent heuristic violations involved interaction design and presentation, the most severe violations lacked information desired by referring clinicians. Violations related to templates' inability to support referring clinicians' information needs had the greatest potential negative impact on efficiency and safety usability goals. Heuristics should be prioritized in future design efforts.

When used deliberately in television and film, jerky motion captures attention. However, it can be distracting in the movements of characters in digital video. To what extent does this kind of jerkiness influence message processing? Based on a limited-capacity model of message processing, jerky character motion was predicted to increase compliance to a persuasive message. The present experiment manipulated the jerkiness of an actor's movements in a computer-delivered video to examine its effect on responses to a hypothetical medical scenario. Jerkiness, whether subtle or obvious, increased self-reported compliance. It also decreased heart rate variability, indicating attentional mediation. Though counterintuitive, these findings indicate that jerky character motion can make computer-mediated messages more persuasive.

The novice–expert ratio method (NEM) pinpoints user interface design problems by identifying the steps in a task that have a high ratio of novice to expert completion time. This study tested the construct validity of NEM's ratio measure against common alternatives. Data were collected from 337 participants who separately performed 10 word-completion tasks on a cellular phone interface. The logarithm, ratio, Cohen's d, and Hedges's ĝ measures had similar construct validity, but Hedges's ĝ provided the most accurate measure of effect size. All these measures correlated more strongly with self-reported interface usability and interface knowledge when applied to the number of actions required to complete a task than when applied to task completion time. A weighted average of both measures had the highest correlation. The relatively high correlation between self-reported interface usability and a weighted Hedges's ĝ measure as compared to the correlations found in the literature indicates the usefulness of the...

Do people treat computers as social actors? To answer this question, researchers have measured the extent to which computers elicit social responses in people, such as impression management strategies for influencing the perceptions of others. But on this question findings in the literature conflict. To make sense of these findings, the present study proposes a dual-process model of impression management in human-computer interaction. The model predicts that, although machines may elicit nonconscious impression management strategies, they do not generally elicit conscious impression management strategies. One such strategy is presenting oneself favorably to others, which can be measured as social desirability bias when comparing self-reported preferences with implicit preferences. The current study uses both a questionnaire and an implicit association test (IAT) to compare attitudes toward human and machine speech. Although past studies on social desirability bias have demonstrated people's tendency to underreport their preference for the preferred group when comparing two human groups, the current study found that, when comparing human speech and machine-synthesized speech, participants instead overreported their preference for the preferred (human) group. This finding supports the proposed dual-process model of impression management, because participants did not consciously treat computers as social actors.

Simulated humans in computer interfaces are increasingly taking on roles that were once reserved for real humans. The presentation of simulated humans is affected by their appearance, motion quality, and interactivity. These presentational factors can influence the decisions of those who interact with them. This is of concern to interface designers and users alike, because these decisions often have moral and ethical consequences. However, the impact of presentational factors on decisions in ethical dilemmas has not been explored. This study is intended as a first effort toward filling this gap. In a between-groups experiment, a female character presented participants with an ethical dilemma. The character's human photorealism and motion quality were varied to generate four stimulus conditions: real human versus computer-generated character × fluid versus jerky movement. The results indicate that the stimulus condition had no significant effect on female participants, while male participants were significantly more likely to rule against the character when her visual appearance was computer generated and her movements were jerky.

This poster describes a project to improve understanding of the challenges associated with managing consultations in the Veterans Health Administration (VHA). We conducted interviews and observations with primary care providers and specialists at two VHA facilities. Using qualitative analysis, we identified cognitive requirements, challenges associated with each, and design seeds. During the poster session, we will present design concepts exploring interventions to support management of consultations.

To decrease medication errors, a common cause of injury to patients, we developed a novel electronic tool to facilitate asynchronous communication between healthcare professionals (HCPs) and patients for medication reconciliation. However, it was unknown whether the tool adequately supported HCPs’ usability needs. Our objective was to conduct an iterative usability evaluation of the tool with physicians, nurses, and pharmacists, in preparation for a randomized controlled trial. We hypothesized that we would identify design weaknesses that could be addressed via interface modifications prior to the trial. We completed a mixed-method, formative usability evaluation with 20 HCPs in the Veterans Affairs (VA) Health Services Research and Development, Human-Computer Interaction and Simulation Laboratory located within a major medical center. The tool in this study is formally known as the Secure Messaging for Medication Reconciliation Tool (SMMRT). The evaluation consisted of four sequential steps: 1) phase I usability testing to assess the baseline tool along with small, iterative design changes throughout testing; 2) heuristic evaluation; 3) implement major design changes that incorporate findings from previous steps; and 4) phase II usability testing to assess the implemented design changes and further refine the tool. This presentation focuses on steps 1 and 4 related to usability testing. During testing, HCPs worked through a real case consisting of a patient discharged from the hospital within the past 30 days who had at least 5 outpatient medications. We collected data on efficiency, usability errors, and participants’ satisfaction, along with participants’ ability to detect and address three distinct types of medication errors via the tool. For the latter, we inserted three safety probes into the simulation: 1) a missing medication (i.e., omission); 2) an extraneous medication (i.e., commission); and 3) an inaccurate dose (i.e., dose discrepancy). Data were analyzed descriptively, rather than via statistical comparisons, due to the formative and iterative nature of this research. There was no indication of efficiency gains during iterative prototyping and testing. Highlights of usability errors included confusion about medication entry fields; incorrect assumptions regarding medication list accuracy; inadequate medication information sorting and organization; and premature closure. Additionally, HCPs described usability errors that might occur in clinical practice. For example, medication images on the tool may not match what is dispensed to patients. HCPs also expressed concern that medication updates made via the tool may not be consistently updated in the electronic health record. In terms of satisfaction, HCPs’ ratings tended to increase as design modifications were implemented. After phase II usability testing, their overall satisfaction was favorable. Finally, for each of the three safety probes, 50% or fewer of HCPs identified the associated medication error. This research illustrates the importance of usability evaluations as a precursor to randomized trials of health information technology. Our multi-step approach to usability testing, with heuristic evaluation at the midpoint, may inform the design of other usability evaluations. While efficiency gains were not realized, user satisfaction improved. The inclusion of safety probes was especially valuable, since probes allowed us to assess error detection rates. There may be opportunities for human factors professionals to expand the sophistication and types of probes used in future healthcare research. Future studies are needed to develop more advanced design approaches that facilitate healthcare professionals’ detection of medication errors.

When a computer-animated human character looks eerily realistic, viewers report a loss of empathy; they have difficulty taking the character's perspective. To explain this perspective-taking impairment, known as the uncanny valley, a novel theory is proposed: The more human or less eerie a character looks, the more it interferes with level 1 visual perspective taking when the character's perspective differs from that of the human observer (e.g., because the character competitively activates shared circuits in the observer's brain). The proposed theory is evaluated in three experiments involving a dot-counting task in which participants either assumed or ignored the perspective of characters varying in their human photorealism and eeriness. Although response times and error rates were lower when the number of dots faced by the observer and character were the same (congruent condition) than when they were different (incongruent condition), no consistent pattern emerged between the human photorealism or eeriness of the characters and participants' response times and error rates. Thus, the proposed theory is unsupported for level 1 visual perspective taking. As the effects of the uncanny valley on empathy have not previously been investigated systematically, these results provide evidence to eliminate one possible explanation.

Socially assistive robots have the potential to improve the quality of life of older adults by encouraging and guiding their performance of rehabilitation exercises while offering cognitive stimulation and companionship. This study focuses on the early stages of developing and testing an interactive personal trainer robot to monitor and increase exercise adherence in older adults. The robot physically demonstrates exercises for the user to follow and monitors the user's progress using a vision-processing unit that detects face and hand movements. When the user successfully completes a move, the robot gives positive feedback and begins the next repetition. The results of usability testing with 10 participants support the feasibility of this approach. Further extensions are planned to evaluate a complete exercise program for improving older adults' physical range of motion in a controlled experiment with three conditions: a personal trainer robot, a personal trainer on-screen character, and a pencil-and-paper exercise plan.