Search

Objective The objective of this study is to characterize the dynamic structure of primary care consultations by identifying typical activities and their inter-relationships to inform the design of automated approaches to clinical documentation using natural language processing and summarization methods. Materials and Methods This is an observational study in Australian general practice involving 31 consultations with 4 primary care physicians. Consultations were audio-recorded, and computer interactions were recorded using screen capture. Physical interactions in consultation rooms were noted by observers. Brief interviews were conducted after consultations. Conversational transcripts were analyzed to identify different activities and their speech content as well as verbal cues signaling activity transitions. An activity transition analysis was then undertaken to generate a network of activities and transitions. Results Observed activity classes followed those described in well-known primary care consultation models. Activities were often fragmented across consultations, did not flow necessarily in a defined order, and the flow between activities was nonlinear. Modeling activities as a network revealed that discussing a patient’s present complaint was the most central activity and was highly connected to medical history taking, physical examination, and assessment, forming a highly interrelated bundle. Family history, allergy, and investigation discussions were less connected suggesting less dependency on other activities. Clear verbal signs were often identifiable at transitions between activities. Discussion Primary care consultations do not appear to follow a classic linear model of defined information seeking activities; rather, they are fragmented, highly interdependent, and can be reactively triggered. Conclusion The nonlinearity of activities has significant implications for the design of automated information capture. Whereas dictation systems generate literal translation of speech into text, speech-based clinical summary systems will need to link disparate information fragments, merge their content, and abstract coherent information summaries.

What if things had a voice? What if we could talk directly to things instead of using a mediating voice interface such as an Alexa or a Google Assistant? In this paper, we share our insights from talking to a pair of boots, a tampon, a perfume bottle, and toilet paper among other everyday things to explore their conversational capabilities. We conducted Thing Interviews using a more-than-human design approach to discover a thing’s perspectives, worldviews and its relations to other humans and nonhumans. Based on our analysis of the speculative conversations, we identified some themes characterizing the emergent qualities of people’s relationships with everyday things. We believe the themes presented in the paper may inspire future research on designing everyday things with conversational capabilities at home.

Given that the one-size-fits-all approach to mobile health interventions have limited effects, a personalized approach might be necessary to promote healthy behaviors and prevent chronic conditions. Our systematic review aims to evaluate the effectiveness of personalized mobile interventions on lifestyle behaviors (i.e., physical activity, diet, smoking and alcohol consumption), and identify the effective key features of such interventions. We included any experimental trials that tested a personalized mobile app or fitness tracker and reported any lifestyle behavior measures. We conducted a narrative synthesis for all studies, and a meta-analysis of randomized controlled trials. Thirty-nine articles describing 31 interventions were included (n = 77,243, 64% women). All interventions personalized content and rarely personalized other features. Source of data included system-captured (12 interventions), user-reported (11 interventions) or both (8 interventions). The meta-analysis showed a moderate positive effect on lifestyle behavior outcomes (standardized difference in means [SDM] 0.663, 95% CI 0.228 to 1.10). A meta-regression model including source of data found that interventions that used system-captured data for personalization were associated with higher effectiveness than those that used user-reported data (SDM 1.48, 95% CI 0.76 to 2.19). In summary, the field is in its infancy, with preliminary evidence of the potential efficacy of personalization in improving lifestyle behaviors. Source of data for personalization might be important in determining intervention effectiveness. To fully exploit the potential of personalization, future high-quality studies should investigate the integration of multiple data from different sources and include personalized features other than content.

ObjectiveTo determine the effectiveness of physical activity interventions involving mobile applications (apps) or trackers with automated and continuous self-monitoring and feedback.DesignSystematic review and meta-analysis.Data sourcesPubMed and seven additional databases, from 2007 to 2020.Study selectionRandomised controlled trials in adults (18–65 years old) without chronic illness, testing a mobile app or an activity tracker, with any comparison, where the main outcome was a physical activity measure. Independent screening was conducted.Data extraction and synthesisWe conducted random effects meta-analysis and all effect sizes were transformed into standardised difference in means (SDM). We conducted exploratory metaregression with continuous and discrete moderators identified as statistically significant in subgroup analyses.Main outcome measuresPhysical activity: daily step counts, min/week of moderate-to-vigorous physical activity, weekly days exercised, min/week of total physical activity, metabolic equivalents.ResultsThirty-five studies met inclusion criteria and 28 were included in the meta-analysis (n=7454 participants, 28% women). The meta-analysis showed a small-to-moderate positive effect on physical activity measures (SDM 0.350, 95% CI 0.236 to 0.465, I2=69%, T2=0.051) corresponding to 1850 steps per day (95% CI 1247 to 2457). Interventions including text-messaging and personalisation features were significantly more effective in subgroup analyses and metaregression.ConclusionInterventions using apps or trackers seem to be effective in promoting physical activity. Longer studies are needed to assess the impact of different intervention components on long-term engagement and effectiveness.

We are all together in a fight against the COVID-19 pandemic. Chatbots, if effectively designed and deployed, could help us by sharing up-to-date information quickly, encouraging desired health impacting behaviors, and lessening the psychological damage caused by fear and isolation. Despite this potential, the risk of amplifying misinformation and the lack of prior effectiveness research is cause for concern. Immediate collaborations between healthcare workers, companies, academics and governments are merited and may aid future pandemic preparedness efforts.

Background People with low back pain (LBP) in the community often do not receive evidence-based advice and management. Community pharmacists can play an important role in supporting people with LBP as pharmacists are easily accessible to provide first-line care. However, previous research suggests that pharmacists may not consistently deliver advice that is concordant with guideline recommendations and may demonstrate difficulty determining which patients require prompt medical review. A clinical decision support system (CDSS) may enhance first-line care of LBP, but none exists to support the community pharmacist–client consultation. Objective This study aimed to develop a CDSS to guide first-line care of LBP in the community pharmacy setting and to evaluate the pharmacist-reported usability and acceptance of the prototype system. Methods A cross-platform Web app for the Apple iPad was developed in conjunction with academic and clinical experts using an iterative user-centered design process during interface design, clinical reasoning, program development, and evaluation. The CDSS was evaluated via one-to-one user-testing with 5 community pharmacists (5 case vignettes each). Data were collected via video recording, screen capture, survey instrument (system usability scale), and direct observation. Results Pharmacists’ agreement with CDSS-generated self-care recommendations was 90% (18/20), with medicines recommendations was 100% (25/25), and with referral advice was 88% (22/25; total 70 recommendations). Pharmacists expressed uncertainty when screening for serious pathology in 40% (10/25) of cases. Pharmacists requested more direction from the CDSS in relation to automated prompts for user input and page navigation. Overall system usability was rated as excellent (mean score 92/100, SD 6.5; 90th percentile compared with similar systems), with acceptance rated as good to excellent. Conclusions A novel CDSS (high-fidelity prototype) to enhance pharmacist care of LBP was developed, underpinned by clinical practice guidelines and informed by a multidisciplinary team of experts. User-testing revealed a high level of usability and acceptance of the prototype system, with suggestions to improve interface prompts and information delivery. The small study sample limits the generalizability of the findings but offers important insights to inform the next stage of system development.

Conversational systems are inherently disadvantaged when indicating either what capabilities they have or the state they are in. The notion of habitability, the appropriate balancing in design between the language people use and the language a system can accept, emerged out of these early difficulties with conversational systems. This literature review aims to summarize progress in habitability research and explore implications for the design of current AI-enabled conversational systems. We found that i) the definitions of habitability focus mostly on matching between user expectations and system capabilities by employing well-balanced restrictions on language use; ii) there are two comprehensive design perspectives on different domains of habitability; iii) there is one standardized questionnaire with a sub-scale to measure habitability in a limited way. The review has allowed us to propose a working definition of habitability and some design implications that may prove useful for guiding future research and practice in this field.

Conversational agents have increasingly been deployed in healthcare applications. However, significant challenges remain in developing this technology. Recent research in this area has highlighted that: i) patient safety was rarely evaluated; ii) health outcomes were poorly measured, and iii) no standardised evaluation methods were employed. The conversational agents in healthcare are lagging behind the developments in other domains. This one-day workshop aims to create a roadmap for healthcare conversational agents to develop standardised design and evaluation frameworks. This will prioritise health outcomes and patient safety while ensuring a high-quality user experience. In doing so, this workshop will bring together researchers and practitioners from HCI, healthcare and related speech and chatbot domains to collaborate on these key challenges.

Speech summarization techniques take human speech as input and then output an abridged version as text or speech. Speech summarization has applications in many domains from information technology to health care, for example improving speech archives or reducing clinical documentation burden. This scoping review maps close to 2 decades of speech summarization literature, spanning from the early machine learning works up to ensemble models, with no restrictions on the language summarized, research method, or paper type. We reviewed a total of 110 papers out of a set of 188 found through a literature search and extracted speech features used, methods, scope, and training corpora. Most studies employ one of four speech summarization architectures: (1) Sentence extraction and compaction; (2) Feature extraction and classification or rank-based sentence selection; (3) Sentence compression and compression summarization; and (4) Language modelling. We also discuss the strengths and weaknesses of these different methods and speech features. Overall, supervised methods (e.g. Hidden Markov support vector machines, Ranking support vector machines, Conditional random fields) performed better than unsupervised methods. As supervised methods require manually annotated training data which can be costly, there was more interest in unsupervised methods. Recent research into unsupervised methods focusses on extending language modelling, for example by combining Uni-gram modelling with deep neural networks. This review does not include recent work in deep learning.

Background The personalization of conversational agents with natural language user interfaces is seeing increasing use in health care applications, shaping the content, structure, or purpose of the dialogue between humans and conversational agents. Objective The goal of this systematic review was to understand the ways in which personalization has been used with conversational agents in health care and characterize the methods of its implementation. Methods We searched on PubMed, Embase, CINAHL, PsycInfo, and ACM Digital Library using a predefined search strategy. The studies were included if they: (1) were primary research studies that focused on consumers, caregivers, or health care professionals; (2) involved a conversational agent with an unconstrained natural language interface; (3) tested the system with human subjects; and (4) implemented personalization features. Results The search found 1958 publications. After abstract and full-text screening, 13 studies were included in the review. Common examples of personalized content included feedback, daily health reports, alerts, warnings, and recommendations. The personalization features were implemented without a theoretical framework of customization and with limited evaluation of its impact. While conversational agents with personalization features were reported to improve user satisfaction, user engagement and dialogue quality, the role of personalization in improving health outcomes was not assessed directly. Conclusions Most of the studies in our review implemented the personalization features without theoretical or evidence-based support for them and did not leverage the recent developments in other domains of personalization. Future research could incorporate personalization as a distinct design factor with a more careful consideration of its impact on health outcomes and its implications on patient safety, privacy, and decision-making.

Background Conversational agents (CAs) are systems that mimic human conversations using text or spoken language. Their widely used examples include voice-activated systems such as Apple Siri, Google Assistant, Amazon Alexa, and Microsoft Cortana. The use of CAs in health care has been on the rise, but concerns about their potential safety risks often remain understudied. Objective This study aimed to analyze how commonly available, general-purpose CAs on smartphones and smart speakers respond to health and lifestyle prompts (questions and open-ended statements) by examining their responses in terms of content and structure alike. Methods We followed a piloted script to present health- and lifestyle-related prompts to 8 CAs. The CAs’ responses were assessed for their appropriateness on the basis of the prompt type: responses to safety-critical prompts were deemed appropriate if they included a referral to a health professional or service, whereas responses to lifestyle prompts were deemed appropriate if they provided relevant information to address the problem prompted. The response structure was also examined according to information sources (Web search–based or precoded), response content style (informative and/or directive), confirmation of prompt recognition, and empathy. Results The 8 studied CAs provided in total 240 responses to 30 prompts. They collectively responded appropriately to 41% (46/112) of the safety-critical and 39% (37/96) of the lifestyle prompts. The ratio of appropriate responses deteriorated when safety-critical prompts were rephrased or when the agent used a voice-only interface. The appropriate responses included mostly directive content and empathy statements for the safety-critical prompts and a mix of informative and directive content for the lifestyle prompts. Conclusions Our results suggest that the commonly available, general-purpose CAs on smartphones and smart speakers with unconstrained natural language interfaces are limited in their ability to advise on both the safety-critical health prompts and lifestyle prompts. Our study also identified some response structures the CAs employed to present their appropriate responses. Further investigation is needed to establish guidelines for designing suitable response structures for different prompt types.

© 2019 The Author(s) 2019. Published by Oxford University Press on behalf of The British Computer Society. All rights reserved. For Permissions, please email: journals.permissions@oup.com. Although various methods have been developed to evaluate conversational interfaces, there has been a lack of methods specifically focusing on evaluating user experience. This paper reviews the understandings of user experience (UX) in conversational interfaces literature and examines the six questionnaires commonly used for evaluating conversational systems in order to assess the potential suitability of these questionnaires to measure different UX dimensions in that context. The method to examine the questionnaires involved developing an assessment framework for main UX dimensions with relevant attributes and coding the items in the questionnaires according to the framework. The results show that (i) the understandings of UX notably differed in literature; (ii) four questionnaires included assessment items, in varying extents, to measure hedonic, aesthetic and pragmatic dimensions of UX; (iii) while the dimension of affect was covered by two questionnaires, playfulness, motivation, and frustration dimensions were covered by one questionnaire only. The largest coverage of UX dimensions has been provided by the Subjective Assessment of Speech System Interfaces (SASSI). We recommend using multiple questionnaires to obtain a more complete measurement of user experience or improve the assessment of a particular UX dimension. RESEARCH HIGHLIGHTS: Varying understandings of UX in conversational interfaces literature. A UX assessment framework with UX dimensions and their relevant attributes. Descriptions of the six main questionnaires for evaluating conversational interfaces. A comparison of the six questionnaires based on their coverage of UX dimensions.

Clinicians spend a large amount of time on clinical documentation of patient encounters, often impacting quality of care and clinician satisfaction, and causing physician burnout. Advances in artificial intelligence (AI) and machine learning (ML) open the possibility of automating clinical documentation with digital scribes, using speech recognition to eliminate manual documentation by clinicians or medical scribes. However, developing a digital scribe is fraught with problems due to the complex nature of clinical environments and clinical conversations. This paper identifies and discusses major challenges associated with developing automated speech-based documentation in clinical settings: recording high-quality audio, converting audio to transcripts using speech recognition, inducing topic structure from conversation data, extracting medical concepts, generating clinically meaningful summaries of conversations, and obtaining clinical data for AI and ML algorithms.

Current generation electronic health records suffer a number of problems that make them inefficient and associated with poor clinical satisfaction. Digital scribes or intelligent documentation support systems, take advantage of advances in speech recognition, natural language processing and artificial intelligence, to automate the clinical documentation task currently conducted by humans. Whilst in their infancy, digital scribes are likely to evolve through three broad stages. Human led systems task clinicians with creating documentation, but provide tools to make the task simpler and more effective, for example with dictation support, semantic checking and templates. Mixed-initiative systems are delegated part of the documentation task, converting the conversations in a clinical encounter into summaries suitable for the electronic record. Computer-led systems are delegated full control of documentation and only request human interaction when exceptions are encountered. Intelligent clinical environments permit such augmented clinical encounters to occur in a fully digitised space where the environment becomes the computer. Data from clinical instruments can be automatically transmitted, interpreted using AI and entered directly into the record. Digital scribes raise many issues for clinical practice, including new patient safety risks. Automation bias may see clinicians automatically accept scribe documents without checking. The electronic record also shifts from a human created summary of events to potentially a full audio, video and sensor record of the clinical encounter. Digital scribes promisingly offer a gateway into the clinical workflow for more advanced support for diagnostic, prognostic and therapeutic tasks.

Background: Context-aware systems, also known as context-sensitive systems, are computing applications designed to capture, interpret, and use contextual information and provide adaptive services according to the current context of use. Context-aware systems have the potential to support patients with chronic conditions; however, little is known about how such systems have been utilized to facilitate patient work. Objective: This study aimed to characterize the different tasks and contexts in which context-aware systems for patient work were used as well as to assess any existing evidence about the impact of such systems on health-related process or outcome measures. Methods: A total of 6 databases (MEDLINE, EMBASE, CINAHL, ACM Digital, Web of Science, and Scopus) were scanned using a predefined search strategy. Studies were included in the review if they focused on patients with chronic conditions, involved the use of a context-aware system to support patients’ health-related activities, and reported the evaluation of the systems by the users. Studies were screened by independent reviewers, and a narrative synthesis of included studies was conducted. Results: The database search retrieved 1478 citations; 6 papers were included, all published from 2009 onwards. The majority of the papers were quasi-experimental and involved pilot and usability testing with a small number of users; there were no randomized controlled trials (RCTs) to evaluate the efficacy of a context-aware system. In the included studies, context was captured using sensors or self-reports, sometimes involving both. Most studies used a combination of sensor technology and mobile apps to deliver personalized feedback. A total of 3 studies examined the impact of interventions on health-related measures, showing positive results. Conclusions: The use of context-aware systems to support patient work is an emerging area of research. RCTs are needed to evaluate the effectiveness of context-aware systems in improving patient work, self-management practices, and health outcomes in chronic disease patients.

Motivation -- The main motivation of this research is to gain a better understanding of dynamic agency between human, machine and environment relations mediated by a synthesis of wearable computing and smart environments technologies. Research approach -- The study follows a research through design approach. There are two main stages of the study involving a series of workshops involving designed prototype systems with different configurations. The prototype systems are designed based on the idea of "Wearable Environments" combining wearable computing and smart environments approaches to ubiquitous computing together. The interactions between prototype systems and human participants are analysed from a post-phenomenological perspective. Findings/Design -- The preliminary workshop study showed that the perception and interpretation of sonic and tactile feedbacks and consequently the strategies of subjects were highly dependent on the places of wearable devices attached to. Research limitations/Implications -- The study deals with only low-level cognitive actions and micro-perception shaping the machine-mediated human agency. Originality/Value -- The research will clarify some critical dimensions and aspects of complex phenomenon of agency in service of designing wearable environments by synthesizing the approaches of the fields of wearable computing and smart environments. Take away message -- Wearable environments with enactive interfaces can provide unique opportunities for investigating and reconfiguring various forms of human-machine-environment relations.

Technological artefacts can mediate the relations between humans and the environment: mediation changes our agency, which can be defined as our capacity for action. There can be different types of technological mediation and each type shapes our agency differently. Our model of wearable environments, which combines wearable computing and smart environment approaches, is useful for exploring new types of relations and, by extension, new forms of agency. In this paper, we present the first stage of developing a wearable environment system involving a series of workshops using two prototype devices. We evaluated the workshop activities according to a post-phenomenological account: this has allowed us to analyse the transformation of machine-mediated agency vis-a-vis two dimensions: perception and praxis. Our findings showed that interpretations of sonic and tactile feedback were highly dependent upon the placement of the sensing and effecting capacities of the system.

In order to be context-aware, a system or application should adapt its behaviour according to current context, acquired by various context provision mechanisms. After acquiring current context, this information should be matched against the previously defined context sets. In this paper, a granular best match algorithm dealing with the subjective, fuzzy, multi-granular and multi-dimensional characteristics of contextual information is introduced. The CAPRA - Context-Aware Personal Reminder Agent tool is used to show the applicability of the new context matching algorithm. The obtained outputs showed that proposed algorithm produces the results which are more sensitive to the user's intention, and more adaptive to the aforementioned characteristics of the contextual information than the traditional exact match method.