Search

Purpose: We attempt to understand the relationship between surgeon technical skills, cognitive workload, and errors during a simulated robotic dissection task. Materials and Methods: Participant surgeons performed a robotic surgery dissection exercise. Participants were grouped based on surgical experience. Technical skills were evaluated utilizing the validated Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. The dissection task was evaluated for errors during active dissection or passive retraction maneuvers. We quantified cognitive workload of surgeon participants as an index of cognitive activity (ICA), derived from task-evoked pupillary response metrics; ICA ranged 0 to 1, with 1 representing maximum ICA. Generalized estimating equation (GEE) was used for all modelings to establish relationships between surgeon technical skills, cognitive workload, and errors. Results: We found a strong association between technical skills as measured by multiple GEARS domains (depth perception, force sensitivity, and robotic control) and passive errors, with higher GEARS scores associated with a lower relative risk of errors (all p

Background Health care and well-being are 2 main interconnected application areas of conversational agents (CAs). There is a significant increase in research, development, and commercial implementations in this area. In parallel to the increasing interest, new challenges in designing and evaluating CAs have emerged. Objective This study aims to identify key design, development, and evaluation challenges of CAs in health care and well-being research. The focus is on the very recent projects with their emerging challenges. Methods A review study was conducted with 17 invited studies, most of which were presented at the ACM (Association for Computing Machinery) CHI 2020 conference workshop on CAs for health and well-being. Eligibility criteria required the studies to involve a CA applied to a health or well-being project (ongoing or recently finished). The participating studies were asked to report on their projects’ design and evaluation challenges. We used thematic analysis to review the studies. Results The findings include a range of topics from primary care to caring for older adults to health coaching. We identified 4 major themes: (1) Domain Information and Integration, (2) User-System Interaction and Partnership, (3) Evaluation, and (4) Conversational Competence. Conclusions CAs proved their worth during the pandemic as health screening tools, and are expected to stay to further support various health care domains, especially personal health care. Growth in investment in CAs also shows the value as a personal assistant. Our study shows that while some challenges are shared with other CA application areas, safety and privacy remain the major challenges in the health care and well-being domains. An increased level of collaboration across different institutions and entities may be a promising direction to address some of the major challenges that otherwise would be too complex to be addressed by the projects with their limited scope and budget.

BACKGROUND Healthcare and wellbeing are two main interconnected application areas of conversational agents (CAs). There is a significant increase in research, development, and commercial implementations in this area. In parallel to the increasing interest, new challenges in designing and evaluating CAs have emerged. OBJECTIVE This study aims to identify key design, development, and evaluation challenges of CAs in healthcare and wellbeing research. The focus is on the very recent projects with their emerging challenges. METHODS A review study was conducted with 17 invited studies, most of which were presented at the ACM CHI2020 conference workshop on CAs for health and wellbeing. Eligibility criteria required the studies to involve a CA applied to a health or wellbeing project in an ongoing or recently finished project. The participating studies were asked to report on their projects' design and evaluation challenges. We used thematic analysis to review the studies. RESULTS The findings include a range of topics from primary care to caring for older adults to health coaching. We identified four major themes: i) domain information and integration, ii) user-system interaction and partnership, iii) evaluation, and iv) conversational competence. CONCLUSIONS While some challenges are shared with other CA application areas, safety and privacy remain the major challenges in the healthcare and wellbeing domains. An increased level of collaboration across different institutions and entities may be a promising direction to address some of the major challenges which otherwise would be too complex to be addressed by the projects with their limited scope and budget.

The ever-increasing pace of scientific publication necessitates methods for quickly identifying relevant papers. While neural recommenders trained on user interests can help, they still result in long, monotonous lists of suggested papers. To improve the discovery experience we introduce multiple new methods for \em augmenting recommendations with textual relevance messages that highlight knowledge-graph connections between recommended papers and a user's publication and interaction history. We explore associations mediated by author entities and those using citations alone. In a large-scale, real-world study, we show how our approach significantly increases engagement -- and future engagement when mediated by authors -- without introducing bias towards highly-cited authors. To expand message coverage for users with less publication or interaction history, we develop a novel method that highlights connections with proxy authors of interest to users and evaluate it in a controlled lab study. Finally, we synthesize design implications for future graph-based messages., Comment: to be published in ACM SIGCHI 2022

Conversational User Interfaces (CUI) are becoming increasingly utilized in Health applications due to their ability to engage patients and support clinical workflows. Yet recent reviews show that our understanding of CUI performance and user preferences towards them is still lacking. This work examines factors that explain people’s preference for engaging with a social needs screening CUI in a clinical context with 41 emergency department visitors. We demonstrate that people with low health literacy and high attitude towards emotional interaction (AEI) prefer responding to questions via CUI rather than a form-based survey. Specifically, participants with low health literacy appreciate the improved understandability offered by the CUI, whereas participants with high AEI appreciate the added level of engagement offered through conversational interactions. Our work advances the understanding of the benefits of CUI for different user groups in health contexts and beyond.

Conversational agents have increasingly been deployed in healthcare applications. However, significant challenges remain in developing this technology. Recent research in this area has highlighted that: i) patient safety was rarely evaluated; ii) health outcomes were poorly measured, and iii) no standardised evaluation methods were employed. The conversational agents in healthcare are lagging behind the developments in other domains. This one-day workshop aims to create a roadmap for healthcare conversational agents to develop standardised design and evaluation frameworks. This will prioritise health outcomes and patient safety while ensuring a high-quality user experience. In doing so, this workshop will bring together researchers and practitioners from HCI, healthcare and related speech and chatbot domains to collaborate on these key challenges.

Accessing patients’ social needs is a critical challenge at emergency departments (EDs). However, most EDs do not have extra staff to administer screeners, and without personnel administration, response rates are low especially for low health literacy patients. To facilitate engagement with such low health literacy patients, we designed a chatbot - HarborBot for social needs screening. Through a study with 30 participants, where participants took a social needs screener both via a traditional survey platform and HarborBot, we found that the two platforms resulted in comparable data (equivalent in 87% of the responses). We also found that while the high health literate participants preferred the traditional survey platform because of efficiency (allowing participants to proceed at their own pace), the low health literate participants preferred HarborBot as it was more engaging, personal, and more understandable. We conclude with a discussion on the design implications for chatbots for social needs screening.

Critical flicker frequency (CFF) is the minimum frequency at which a flickering light source appears fused to an observer. Measuring CFF can support early diagnosis of minimal hepatic encephalopathy (MHE), a condition affecting up to 80% of people with cirrhosis of the liver. However, adoption of CFF measurement in clinical practice has been hampered by the cost of a device for measuring CFF and the need for specialized training to administer the test. This paper presents Beacon, a portable, inexpensive device that enables people to measure their own critical flicker frequency. We adopt a mixed-methods approach to informing and evaluating the design of and potential opportunities for Beacon. We first report on a two-part formative study with 10 participants to evaluate the choice of certain parameters in the design of Beacon. We then report on a study of 41 healthy adults ranging from 18 to 99 years of age, finding that Beacon performs on par with Lafayette Flicker Fusion System, an established medical device, achieving a pearson correlation coefficient of 0.88. We finally report on a focus group with five hepatoligists who work with patients with cirrhosis of the liver, using our initial prototype development to examine their perspectives on potential opportunities and challenges in adoption of a device like Beacon. We discuss Beacon as an exploration of reframing critical flicker frequency measurement from a clinical screening tool into a self-administered self-tracking measure, thereby drawing upon and contributing to research in the health and personal informatics.

Machine learning (ML) has become increasingly influential to human society, yet the primary advancements and applications of ML are driven by research in only a few computational disciplines. Even applications that affect or analyze human behaviors and social structures are often developed with limited input from experts outside of computational fields. Social scientists—experts trained to examine and explain the complexity of human behavior and interactions in the world—have considerable expertise to contribute to the development of ML applications for human-generated data, and their analytic practices could benefit from more human-centered ML methods. Although a few researchers have highlighted some gaps between ML and social sciences [51, 57, 70], most discussions only focus on quantitative methods. Yet many social science disciplines rely heavily on qualitative methods to distill patterns that are challenging to discover through quantitative data. One common analysis method for qualitative data is qualitative coding . In this article, we highlight three challenges of applying ML to qualitative coding. Additionally, we utilize our experience of designing a visual analytics tool for collaborative qualitative coding to demonstrate the potential in using ML to support qualitative coding by shifting the focus to identifying ambiguity. We illustrate dimensions of ambiguity and discuss the relationship between disagreement and ambiguity. Finally, we propose three research directions to ground ML applications for social science as part of the progression toward human-centered machine learning.