Search

Abstract Artificial intelligence (AI)-based computational tools for deriving digital behavioral markers are increasingly able to automatically detect clinically relevant patterns in mood and behavior through algorithmic analysis of continuously and passively collected data. The integration of these technologies into clinical care is imminent, most notably in clinical psychology and psychiatry but also other disciplines (e.g., cardiology, neurology, neurosurgery, pain management). Meanwhile, ethical guidelines for implementation are lacking, as are insights into what patients and caregivers want and need to know about these technologies to ensure acceptability and informed consent. In this work, we present qualitative findings from interviews with 40 adolescent patients and their caregivers examining ethical and practical considerations for translating these technologies into clinical care. We observed seven key domains (in order of salience) in stakeholders’ informational needs: (1) clinical utility and value; (2) evidence, explainability, evaluation and contestation; (3) accuracy and trustworthiness; (4) data security, privacy, and misuse; (5) patient consent, control, and autonomy; (6) physician-patient relationship; and (7) patient safety, well-being, and dignity. Drawing from these themes, we provide a checklist of questions, as well as suggestions and key challenges, to help researchers and practitioners respond to what stakeholders want to know when integrating these technologies into clinical care and research. Our findings inform participatory approaches to co-designing treatment roadmaps for using these AI-based tools for enhanced patient engagement, acceptability and informed consent.

BackgroundArtificial intelligence (AI)-based computer perception technologies (e.g., digital phenotyping and affective computing) promise to transform clinical approaches to personalized care in psychiatry and beyond by offering more objective measures of emotional states and behavior, enabling precision treatment, diagnosis, and symptom monitoring. At the same time, passive and continuous nature by which they often collect data from patients in non-clinical settings raises ethical issues related to privacy and self-determination. Little is known about how such concerns may be exacerbated by the integration of neural data, as parallel advances in computer perception, AI, and neurotechnology enable new insights into subjective states. Here, we present findings from a multi-site NCATS-funded study of ethical considerations for translating computer perception into clinical care and contextualize them within the neuroethics and neurorights literatures.MethodsWe conducted qualitative interviews with patients (n = 20), caregivers (n = 20), clinicians (n = 12), developers (n = 12), and clinician developers (n = 2) regarding their perspective toward using PC in clinical care. Transcripts were analyzed in MAXQDA using Thematic Content Analysis.ResultsStakeholder groups voiced concerns related to (1) perceived invasiveness of passive and continuous data collection in private settings; (2) data protection and security and the potential for negative downstream/future impacts on patients of unintended disclosure; and (3) ethical issues related to patients’ limited versus hyper awareness of passive and continuous data collection and monitoring. Clinicians and developers highlighted that these concerns may be exacerbated by the integration of neural data with other computer perception data.DiscussionOur findings suggest that the integration of neurotechnologies with existing computer perception technologies raises novel concerns around dignity-related and other harms (e.g., stigma, discrimination) that stem from data security threats and the growing potential for reidentification of sensitive data. Further, our findings suggest that patients’ awareness and preoccupation with feeling monitored via computer sensors ranges from hypo- to hyper-awareness, with either extreme accompanied by ethical concerns (consent vs. anxiety and preoccupation). These results highlight the need for systematic research into how best to implement these technologies into clinical care in ways that reduce disruption, maximize patient benefits, and mitigate long-term risks associated with the passive collection of sensitive emotional, behavioral and neural data.