Search

Abstract AI tools intend to transform mental healthcare by providing remote estimates of depression risk using behavioral data collected by sensors embedded in smartphones. While these tools accurately predict elevated depression symptoms in small, homogenous populations, recent studies show that these tools are less accurate in larger, more diverse populations. In this work, we show that accuracy is reduced because sensed-behaviors are unreliable predictors of depression across individuals: sensed-behaviors that predict depression risk are inconsistent across demographic and socioeconomic subgroups. We first identified subgroups where a developed AI tool underperformed by measuring algorithmic bias, where subgroups with depression were incorrectly predicted to be at lower risk than healthier subgroups. We then found inconsistencies between sensed-behaviors predictive of depression across these subgroups. Our findings suggest that researchers developing AI tools predicting mental health from sensed-behaviors should think critically about the generalizability of these tools, and consider tailored solutions for targeted populations.

Abstract While studies show links between smartphone data and affective symptoms, we lack clarity on the temporal scale, specificity (e.g., to depression vs. anxiety), and person-specific (vs. group-level) nature of these associations. We conducted a large-scale (n = 1013) smartphone-based passive sensing study to identify within- and between-person digital markers of depression and anxiety symptoms over time. Participants (74.6% female; M age = 40.9) downloaded the LifeSense app, which facilitated continuous passive data collection (e.g., GPS, app and device use, communication) across 16 weeks. Hierarchical linear regression models tested the within- and between-person associations of 2-week windows of passively sensed data with depression (PHQ-8) or generalized anxiety (GAD-7). We used a shifting window to understand the time scale at which sensed features relate to mental health symptoms, predicting symptoms 2 weeks in the future (distal prediction), 1 week in the future (medial prediction), and 0 weeks in the future (proximal prediction). Spending more time at home relative to one’s average was an early signal of PHQ-8 severity (distal β = 0.219, p = 0.012) and continued to relate to PHQ-8 at medial (β = 0.198, p = 0.022) and proximal (β = 0.183, p = 0.045) windows. In contrast, circadian movement was proximally related to (β = −0.131, p = 0.035) but did not predict (distal β = 0.034, p = 0.577; medial β = −0.089, p = 0.138) PHQ-8. Distinct communication features (i.e., call/text or app-based messaging) related to PHQ-8 and GAD-7. Findings have implications for identifying novel treatment targets, personalizing digital mental health interventions, and enhancing traditional patient-provider interactions. Certain features (e.g., circadian movement) may represent correlates but not true prospective indicators of affective symptoms. Conversely, other features like home duration may be such early signals of intra-individual symptom change, indicating the potential utility of prophylactic intervention (e.g., behavioral activation) in response to person-specific increases in these signals.

Abstract Conversational artificial intelligence (AI), particularly AI-based conversational agents (CAs), is gaining traction in mental health care. Despite their growing usage, there is a scarcity of comprehensive evaluations of their impact on mental health and well-being. This systematic review and meta-analysis aims to fill this gap by synthesizing evidence on the effectiveness of AI-based CAs in improving mental health and factors influencing their effectiveness and user experience. Twelve databases were searched for experimental studies of AI-based CAs’ effects on mental illnesses and psychological well-being published before May 26, 2023. Out of 7834 records, 35 eligible studies were identified for systematic review, out of which 15 randomized controlled trials were included for meta-analysis. The meta-analysis revealed that AI-based CAs significantly reduce symptoms of depression (Hedge’s g 0.64 [95% CI 0.17–1.12]) and distress (Hedge’s g 0.7 [95% CI 0.18–1.22]). These effects were more pronounced in CAs that are multimodal, generative AI-based, integrated with mobile/instant messaging apps, and targeting clinical/subclinical and elderly populations. However, CA-based interventions showed no significant improvement in overall psychological well-being (Hedge’s g 0.32 [95% CI –0.13 to 0.78]). User experience with AI-based CAs was largely shaped by the quality of human-AI therapeutic relationships, content engagement, and effective communication. These findings underscore the potential of AI-based CAs in addressing mental health issues. Future research should investigate the underlying mechanisms of their effectiveness, assess long-term effects across various mental health outcomes, and evaluate the safe integration of large language models (LLMs) in mental health care.

Abstract Background The COVID-19 pandemic involved a rapid change to the working conditions of all healthcare workers (HCW), including those in primary care. Organizational responses to the pandemic, including a shift to virtual care, changes in staffing, and reassignments to testing-related work, may have shifted more burden to these HCWs, increasing their burnout and turnover intent, despite their engagement to their organization. Our objectives were (1) to examine changes in burnout and intent to leave rates in VA primary care from 2017–2020 (before and during the pandemic), and (2) to analyze how individual protective factors and organizational context affected burnout and turnover intent among VA primary care HCWs during the early months of the pandemic. Methods We analyzed individual- and healthcare system-level data from 19,894 primary care HCWs in 139 healthcare systems in 2020. We modeled potential relationships between individual-level burnout and turnover intent as outcomes, and individual-level employee engagement, perceptions of workload, leadership, and workgroups. At healthcare system-level, we assessed prior-year levels of burnout and turnover intent, COVID-19 burden (number of tests and deaths), and the extent of virtual care use as potential determinants. We conducted multivariable analyses using logistic regression with standard errors clustered by healthcare system controlled for individual-level demographics and healthcare system complexity. Results In 2020, 37% of primary care HCWs reported burnout, and 31% reported turnover intent. Highly engaged employees were less burned out (OR = 0.57; 95% CI 0.52–0.63) and had lower turnover intent (OR = 0.62; 95% CI 0.57–0.68). Pre-pandemic healthcare system-level burnout was a major predictor of individual-level pandemic burnout (p = 0.014). Perceptions of reasonable workload, trustworthy leadership, and strong workgroups were also related to lower burnout and turnover intent (p

Background There is increasing recognition of the need to focus on the health and well-being of healthcare employees given high rates of burnout and turnover. Employee wellness programs are effective at addressing these issues; however, participation in these programs is often a challenge and requires large scale organizational transformation. The Veterans Health Administration (VA) has begun to roll out their own employee wellness program—Employee Whole Health (EWH)—focused on the holistic needs of all employees. This evaluation’s goal was to use the Lean Enterprise Transformation (LET) model for organizational transformation to identify key factors—facilitators and barriers—affecting the implementation of VA EWH. Methods This cross-sectional qualitative evaluation based on the action research model reflects on the organizational implementation of EWH. Semi-structured 60-minute phone interviews were conducted in February-April 2021 with 27 key informants (e.g., EWH coordinator, wellness/occupational health staff) knowledgeable about EWH implementation across 10 VA medical centers. Operational partner provided a list of potential participants, eligible because of their involvement in EWH implementation at their site. The interview guide was informed by the LET model. Interviews were recorded and professionally transcribed. Constant comparative review with a combination of a priori coding based on the model and emergent thematic analysis was used to identify themes from transcripts. Matrix analysis and rapid turnaround qualitative methods were used to identify cross-site factors to EWH implementation. Results Eight common factors in the conceptual model were found to facilitate and/or hinder EWH implementation efforts: [1] EWH initiatives, [2] multilevel leadership support, [3] alignment, [4] integration, [5] employee engagement, [6] communication, [7] staffing, and [8] culture. An emergent factor was [9] the impact of the COVID-19 pandemic on EWH implementation. Conclusions As VA expands its EWH cultural transformation nationwide, evaluation findings can (a) enable existing programs to address known implementation barriers, and (b) inform new sites to capitalize on known facilitators, anticipate and address barriers, and leverage evaluation recommendations through concerted implementation at the organization, process, and employee levels to jump-start their EWH program implementation.

Background: Prior literature links passively sensed information about a person's location, movement, and communication with social anxiety. These findings hold promise for identifying novel treatment targets, informing clinical care, and personalizing digital mental health interventions. However, social anxiety symptoms are heterogeneous; to identify more precise targets and tailor treatments, there is a need for personal sensing studies aimed at understanding differential predictors of the distinct subdomains of social anxiety. Our objective was to conduct a large-scale smartphone-based sensing study of fear, avoidance, and physiological symptoms in the context of trait social anxiety over time. Methods: Participants (n = 1013; 74.6 % female; M age = 40.9) downloaded the LifeSense app, which collected continuous passive data (e.g., GPS, communication, app and device use) over 16 weeks. We tested a series of multilevel linear regression models to understand within- and between-person associations of 2-week windows of passively sensed smartphone data with fear, avoidance, and physiological distress on the self-reported Social Phobia Inventory (SPIN). A shifting sensor lag was applied to examine how smartphone features related to SPIN subdomains 2 weeks in the future (distal prediction), 1 week in the future (medial prediction), and 0 weeks in the future (proximal prediction). Results: A decrease in time visiting novel places was a strong between-person predictor of social avoidance over time (distal β = −0.886, p = .002; medial β = −0.647, p = .029; proximal β = −0.818, p = .007). Reductions in call- and text-based communications were associated with social avoidance at both the between- (distal β = −0.882, p = .002; medial β = −0.932, p = .001; proximal β = −0.918, p = .001) and within- (distal β = −0.191, p = .046; medial β = −0.213, p = .028) person levels, as well as between-person fear of social situations (distal β = −0.860, p

As digital mental health interventions (DMHIs) proliferate, there is a growing need to understand the complexities of moving these tools from concept and design to service-ready products. We highlight five case studies from a center that specializes in the design and evaluation of digital mental health interventions to illustrate pragmatic approaches to the development of digital mental health interventions, and to make transparent some of the key decision points researchers encounter along the design-to-product pipeline. Case studies cover different key points in the design process and focus on partnership building, understanding the problem or opportunity, prototyping the product or service, and testing the product or service. We illustrate lessons learned and offer a series of questions researchers can use to navigate key decision points in the digital mental health intervention (DMHI) development process.

Background: Young adults have high rates of mental health conditions, but most do not want or cannot access treatment. By leveraging a medium that young adults routinely use, text messaging programs have potential to keep young adults engaged with content supporting self-management of mental health issues and can be delivered inexpensively at scale. We designed an intervention that imparts strategies for self-managing mental health symptoms through interactive text messaging dialogues and engages users through novelty and variety in strategies (from cognitive behavioral therapy, acceptance and commitment therapy, and positive psychology) and styles of interaction (e.g., prompts, peer stories, writing tasks). Methods: The aim of this mixed-methods study was to pilot 1- and 2-week versions of an interactive text messaging intervention among young adults (ages 18–25), and to obtain feedback to guide intervention refinements. Young adults were recruited via a mental health advocacy website and snowball sampling at a North American University. We used Wizard-of-Oz methods in which study staff sent messages based on a detailed script. Transcripts of interviews were subject to qualitative analysis to identify aspects of the program that need improvements, and to gather participant perspectives on possible solutions. Results: Forty-eight individuals ages 18–25 participated in the study (mean age: 22.0). 85 % responded to the program at least once. Among those who ever responded, they replied to messages on 85 % of days, and with engagement sustained over the study period. Participants endorsed the convenience of text messaging, the types of interactive dialogues, and the variety of content. They also identified needed improvements to message volume, scheduling, and content. Conclusions: Young adults showed high levels of engagement and satisfaction with a texting program supporting mental health self-management. The program may be improved through refining personalization, timing, and message volume, and extending content to support use over a longer timeframe. If shown to be effective in randomized trials, this program has potential to help address a substantial treatment gap in young adults' mental health.

Abstract Recent growth in digital technologies has enabled the recruitment and monitoring of large and diverse populations in remote health studies. However, the generalizability of inference drawn from remotely collected health data could be severely impacted by uneven participant engagement and attrition over the course of the study. We report findings on long-term participant retention and engagement patterns in a large multinational observational digital study for depression containing active (surveys) and passive sensor data collected via Android smartphones, and Fitbit devices from 614 participants for up to 2 years. Majority of participants (67.6%) continued to remain engaged in the study after 43 weeks. Unsupervised clustering of participants’ study apps and Fitbit usage data showed 3 distinct engagement subgroups for each data stream. We found: (i) the least engaged group had the highest depression severity (4 PHQ8 points higher) across all data streams; (ii) the least engaged group (completed 4 bi-weekly surveys) took significantly longer to respond to survey notifications (3.8 h more) and were 5 years younger compared to the most engaged group (completed 20 bi-weekly surveys); and (iii) a considerable proportion (44.6%) of the participants who stopped completing surveys after 8 weeks continued to share passive Fitbit data for significantly longer (average 42 weeks). Additionally, multivariate survival models showed participants’ age, ownership and brand of smartphones, and recruitment sites to be associated with retention in the study. Together these findings could inform the design of future digital health studies to enable equitable and balanced data collection from diverse populations.

Abstract The use of remote measurement technologies (RMTs) across mobile health (mHealth) studies is becoming popular, given their potential for providing rich data on symptom change and indicators of future state in recurrent conditions such as major depressive disorder (MDD). Understanding recruitment into RMT research is fundamental for improving historically small sample sizes, reducing loss of statistical power, and ultimately producing results worthy of clinical implementation. There is a need for the standardisation of best practices for successful recruitment into RMT research. The current paper reviews lessons learned from recruitment into the Remote Assessment of Disease and Relapse- Major Depressive Disorder (RADAR-MDD) study, a large-scale, multi-site prospective cohort study using RMT to explore the clinical course of people with depression across the UK, the Netherlands, and Spain. More specifically, the paper reflects on key experiences from the UK site and consolidates these into four key recruitment strategies, alongside a review of barriers to recruitment. Finally, the strategies and barriers outlined are combined into a model of lessons learned. This work provides a foundation for future RMT study design, recruitment and evaluation.

BackgroundAdolescence is a vulnerable developmental period, characterized by high rates of mental health concerns, yet few adolescents receive treatment. Public libraries support adolescents by providing them with access to teen programming, technological resources, and have recently been providing mental health services. Digital mental health (DMH) services may help libraries provide scalable mental health solutions for their adolescent patrons and could be well positioned to address the mental health needs of historically underrepresented racial and ethnic (HURE) adolescents; however, little research has been conducted on the compatibility of DMH services with adolescent patron mental health needs or resource needs of library workers supporting them.MethodsThe research team formed a partnership with a public library, which serves a large HURE adolescent population. We conducted needs assessment and implementation readiness interviews with 17 library workers, including leadership, librarians, and workers with specialized areas of practice. Interview questions focused on library infrastructure, as well as library needs and preferences around the design and implementation of DMH services for adolescents. We used the Consolidated Framework for Implementation Research as guiding implementation determinant framework to code and analyze the interview transcripts.ResultsOur findings revealed library workers play an important role in guiding patrons to desired resources and share a goal of implementing adolescent DMH resources into the library and elevating marginalized adolescents’ voices. Existing library resources, such as the library's role as a safe space for adolescents in the community, close relationships with external and community organizations, and availability of no-cost technological resources, could help facilitate the implementation of DMH services. Barriers related to community buy-in, mental health stigma, and library worker confidence in supporting adolescent mental health could affect service implementation.ConclusionsOur findings suggest public libraries are highly promising settings to deploy DMH services for adolescents. We identified important determinants that may impact the implementation of DMH services in public library settings. Special considerations are needed to design services to meet the mental health needs of HURE adolescent populations and those adolescents’ most experiencing health inequities.

Introduction Care coordinators (CCs) are specialized healthcare providers and often the primary point of contact for patients with multiple medical and mental health comorbidities in integrated healthcare settings. Prior work shows CCs have lower comfort addressing mental health than physical health concerns. Digital mental health interventions can support CCs’ management of patient mental health needs, but training gaps must be addressed prior to a digital mental health intervention's implementation. Methods As part of a quality improvement initiative, a 1-hour training focused on the assessment and management of depression and suicide-related thoughts and behaviors was delivered to CCs within a large midwestern healthcare system's Division of Ambulatory Care Coordination. CCs completed online surveys prior to and following the training. Conclusion Training resulted in increased comfort working with clinical populations, including patients who experience suicide-related thoughts and behaviors. Gains around screening for suicide risk were modest. Brief trainings for CCs can address training gaps, however, ongoing training and case consultation may also be indicated.

Abstract Background Major Depressive Disorder (MDD) is prevalent, often chronic, and requires ongoing monitoring of symptoms to track response to treatment and identify early indicators of relapse. Remote Measurement Technologies (RMT) provide an opportunity to transform the measurement and management of MDD, via data collected from inbuilt smartphone sensors and wearable devices alongside app-based questionnaires and tasks. A key question for the field is the extent to which participants can adhere to research protocols and the completeness of data collected. We aimed to describe drop out and data completeness in a naturalistic multimodal longitudinal RMT study, in people with a history of recurrent MDD. We further aimed to determine whether those experiencing a depressive relapse at baseline contributed less complete data. Methods Remote Assessment of Disease and Relapse – Major Depressive Disorder (RADAR-MDD) is a multi-centre, prospective observational cohort study conducted as part of the Remote Assessment of Disease and Relapse – Central Nervous System (RADAR-CNS) program. People with a history of MDD were provided with a wrist-worn wearable device, and smartphone apps designed to: a) collect data from smartphone sensors; and b) deliver questionnaires, speech tasks, and cognitive assessments. Participants were followed-up for a minimum of 11 months and maximum of 24 months. Results Individuals with a history of MDD (n = 623) were enrolled in the study,. We report 80% completion rates for primary outcome assessments across all follow-up timepoints. 79.8% of people participated for the maximum amount of time available and 20.2% withdrew prematurely. We found no evidence of an association between the severity of depression symptoms at baseline and the availability of data. In total, 110 participants had > 50% data available across all data types. Conclusions RADAR-MDD is the largest multimodal RMT study in the field of mental health. Here, we have shown that collecting RMT data from a clinical population is feasible. We found comparable levels of data availability in active and passive forms of data collection, demonstrating that both are feasible in this patient group.

Abstract The use of digital tools to measure physiological and behavioural variables of potential relevance to mental health is a growing field sitting at the intersection between computer science, engineering, and clinical science. We summarised the literature on remote measuring technologies, mapping methodological challenges and threats to reproducibility, and identified leading digital signals for depression. Medical and computer science databases were searched between January 2007 and November 2019. Published studies linking depression and objective behavioural data obtained from smartphone and wearable device sensors in adults with unipolar depression and healthy subjects were included. A descriptive approach was taken to synthesise study methodologies. We included 51 studies and found threats to reproducibility and transparency arising from failure to provide comprehensive descriptions of recruitment strategies, sample information, feature construction and the determination and handling of missing data. The literature is characterised by small sample sizes, short follow-up duration and great variability in the quality of reporting, limiting the interpretability of pooled results. Bivariate analyses show consistency in statistically significant associations between depression and digital features from sleep, physical activity, location, and phone use data. Machine learning models found the predictive value of aggregated features. Given the pitfalls in the combined literature, these results should be taken purely as a starting point for hypothesis generation. Since this research is ultimately aimed at informing clinical practice, we recommend improvements in reporting standards including consideration of generalisability and reproducibility, such as wider diversity of samples, thorough reporting methodology and the reporting of potential bias in studies with numerous features.

Background: Mental health conditions are common among adolescents and young adults, yet few receive adequate mental health treatment. Many young people seek support and information online through social media, and report preferences for digital interventions. Thus, digital interventions deployed through social media have promise to reach a population not yet engaged in treatment, and at risk of worsening symptoms. Objective: In this scoping review, we aimed to identify and review empirical research on social media-based interventions aimed at improving adolescent and young adult mental health. A secondary objective was to identify the features and functionalities of platforms described as social media. Methods: Adhering to the PRISMA-ScR guidelines for scoping reviews, the search was conducted in PubMed MEDLINE; Embase Central Register of Controlled Trials (Wiley); PsycINFO (Ebsco); Scopus; Web of Science; IEEE Xplore; ACM Digital Library; and ClinicalTrials.gov from inception until November 2021. Studies were included if they involved adolescents or young adults (10–26 years of age) that meet clinical, or subclinical, levels of a mental health condition and include a pre- and post-assessment of mental health outcomes. Results: Among the 18,380 references identified, 15 met full inclusion criteria and were published between 2017 and 2021–this included four randomized controlled trials, seven non-randomized pre-post trials, and four were experimental or quasi-experimental designs. Just five studies were delivered through an existing social media site (Facebook or Pixtori), with the remainder focused on purpose-built networks. Three studies involved adolescents or young adults who self-reported a mental health condition, seven involved young people diagnosed with a mental health condition by a clinician or who scored above a clinical threshold on valid clinical measure, three involved college students without a mental health inclusion criterion, and two studies focused on young people with a cancer diagnosis. Conclusions: The review highlights innovations in the delivery of mental health interventions, provides preliminary evidence of the ability of social media interventions to improve mental health outcomes, and underscores the need for, and merit of, future work in this area. We discuss opportunities and challenges for future research, including the potential to leveragei existing peer networks, the use of just-in-time interventions, and scaling interventions to meet need.

Abstract Background The outbreak of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which causes a clinical illness Covid-19, has had a major impact on mental health globally. Those diagnosed with major depressive disorder (MDD) may be negatively impacted by the global pandemic due to social isolation, feelings of loneliness or lack of access to care. This study seeks to assess the impact of the 1st lockdown – pre-, during and post – in adults with a recent history of MDD across multiple centres. Methods This study is a secondary analysis of an on-going cohort study, RADAR-MDD project, a multi-centre study examining the use of remote measurement technology (RMT) in monitoring MDD. Self-reported questionnaire and passive data streams were analysed from participants who had joined the project prior to 1st December 2019 and had completed Patient Health and Self-esteem Questionnaires during the pandemic (n = 252). We used mixed models for repeated measures to estimate trajectories of depressive symptoms, self-esteem, and sleep duration. Results In our sample of 252 participants, 48% (n = 121) had clinically relevant depressive symptoms shortly before the pandemic. For the sample as a whole, we found no evidence that depressive symptoms or self-esteem changed between pre-, during- and post-lockdown. However, we found evidence that mean sleep duration (in minutes) decreased significantly between during- and post- lockdown (− 12.16; 95% CI − 18.39 to − 5.92; p

BackgroundThere is growing interest in the design of digital interventions to improve conditions for young people who engage in high-risk behaviors, like nonsuicidal self-injury (NSSI). However, few studies have focused on how young people self-manage NSSI, or their existing, and historic, use of technologies to support their goals related to NSSI behavior change. Such an understanding has the potential to inform the design of digital interventions that meet this population's unique needs.ObjectivesThis study aims to (a) understand the self-management practices of young adults who engage in NSSI, (b) explore how they currently use technologies for self-injury self-management, and (c) identify the ways they can envision an app-based technology supporting their self-management.Methods and MaterialsTwenty young adults (aged 18–24) with lived experience of NSSI, and who were not currently enrolled in therapy, were recruited from online venues. Participants completed baseline measures to assess mental health and NSSI characteristics, followed by a virtual 1-h semi-structured interview where they were invited to share their experience of self-management, their goals, and their thoughts on supportive technology. Interview scripts were transcribed and analyzed via thematic analysis.ResultsThemes and sub-themes are organized under two broad domain areas: (1) How young adults self-manage NSSI thoughts and behaviors and (2) Opportunities and challenges for digital interventions to assist young adults in their recovery process. We found that young adults had varied experiences with, and goals related to, NSSI. Participants reported a lack of effective strategies to reduce NSSI urges and a desire for an app-based technology to track patterns and deliver personalized suggestions for self-management. Participants reported existing use of technologies as part of self-management, as well as early information and support seeking for NSSI online.ConclusionsThis study contributes a greater understanding of young people's experiences with self-injury, their self-management practices, and their desire to engage with technology. Our findings highlight the need for design flexibility in developing digital interventions that support individual goals, unique presentations of NSSI, and needs at different phases of recovery. Implications for the design of highly personalized and relevant digital interventions to address NSSI are discussed.

Background The implementation of eHealth applications in primary care remains challenging. Enhancing knowledge and awareness of implementation determinants is critical to build evidence-based implementation strategies and optimise uptake and sustainability. Objectives We consider how evidence-based implementation strategies can be built to support eHealth implementation. Discussion What implementation strategies to consider depends on (potential) barriers and facilitators to eHealth implementation in a given situation. Therefore, we first discuss key barriers and facilitators following the five domains of the Consolidated Framework for Implementation Research (CFIR). Cost is identified as a critical barrier to eHealth implementation. Privacy, security problems, and a lack of recognised standards for eHealth applications also hinder implementation. Engagement of key stakeholders in the implementation process, planning the implementation of the intervention, and the availability of training and support are important facilitators. To support care professionals and researchers, we provide a stepwise approach to develop and apply evidence-based implementation strategies for eHealth in primary care. It includes the following steps: (1) specify the eHealth application, (2) define problem, (3) specify desired implementation behaviour, and (4) choose and (5) evaluate the implementation strategy. To improve the fit of the implementation strategy with the setting, the stepwise approach considers the phase of the implementation process and the specific context. Conclusion Applying an approach, as provided here, may help to improve the implementation of eHealth applications in primary care.

The dataset summarized in this article includes a nationwide prevalence sample of U.S. military Veterans who were aged 65 years or older, dually enrolled in the Veterans Health Administration and traditional Medicare and had a previous diagnosis of diabetes (diabetes mellitus) as of December 2005 (N = 275,190) [1]. Our data were originally used to develop and validate prognostic indices of 5- and 10-year mortality among older Veterans with diabetes. We include various potential predictors including demographics (e.g., sex, age, marital status, and VA priority group), healthcare utilization (e.g., # of outpatient visits, # days of inpatient stays), medication history, and major comorbidities. This novel dataset provides researchers with an opportunity to study the associations between a large variety of individual-level risk factors and longevity for patients living with diabetes.

Digital biomarkers of mental health, created using data extracted from everyday technologies including smartphones, wearable devices, social media and computer interactions, have the opportunity to revolutionise mental health diagnosis and treatment by providing near-continuous unobtrusive and remote measures of behaviours associated with mental health symptoms. Machine learning models process data traces from these technologies to identify digital biomarkers. In this editorial, we caution clinicians against using digital biomarkers in practice until models are assessed for equitable predictions (‘model equity’) across demographically diverse patients at scale, behaviours over time, and data types extracted from different devices and platforms. We posit that it will be difficult for any individual clinic or large-scale study to assess and ensure model equity and alternatively call for the creation of a repository of open de-identified data for digital biomarker development.

Background: College students endorse high rates of mental health problems. While many colleges offer on-campus services, many students who could benefit from mental health services do not receive care. Indeed, nearly half of students who screen positive for depression, for example, do not receive treatment. Digital mental health programs, such as those delivered via mobile apps, may help expand access to mental health care and resources. This mixed-methods study aims to examine the uptake and effectiveness of an implementation of IntelliCare for College Students, a self-guided app-based mental health platform, on two university campuses. Methods: Data on counseling center utilization was collected prior to the implementation of the app (pre-implementation phase) and while the app was available on campus (implementation phase). Data on app usage was collected throughout the implementation phase. A subset of participants (n = 20), along with counseling center staff members (n = 10), completed feedback interviews. Results: Overall, uptake of the app platform was low. A total of 117 participants downloaded the app and registered their study ID during the implementation phase. Approximately 24% (28/117) of participants used the app only once. The number of days between the first and last day of app use ranged from 0 to 299, with a mean of 35.01 days and a median of 14 days. A relatively small portion of the sample (26.5%; 31/117) downloaded one or more of the IntelliCare interactive apps. In examining counseling center utilization, there were no significant changes in intake appointments, individual therapy sessions, or crisis appointments observed from the pre-implementation phase to the implementation phase of the study. Feedback interviews highlighted the significant level of disruption caused by the COVID-19 pandemic and shift to remote learning, including challenges disseminating information to students and a preference to spend less time with digital devices outside of class time. Conclusions: Findings from this study indicate that there is an ongoing need to identify ways to reach college students and support student mental health and wellness for the remainder of the COVID-19 pandemic and beyond.

Objective Given the increasing number of publicly available mental health apps, we need independent advice to guide adoption. This paper discusses the challenges and opportunities of current mental health app rating systems and describes the refinement process of one prominent system, the One Mind PsyberGuide Credibility Rating Scale (PGCRS). Methods PGCRS Version 1 was developed in 2013 and deployed for 7 years, during which time a number of limitations were identified. Version 2 was created through multiple stages, including a review of evaluation guidelines and consumer research, input from scientific experts, testing, and evaluation of face validity. We then re-reviewed 161 mental health apps using the updated rating scale, investigated the reliability and discrepancy of initial scores, and updated ratings on the One Mind PsyberGuide public app guide. Results Reliabilities across the scale's 9 items ranged from −0.10 to 1.00, demonstrating that some characteristics of apps are more difficult to rate consistently. The average overall score of the 161 reviewed mental health apps was 2.51/5.00 (range 0.33–5.00). Ratings were not strongly correlated with app store star ratings, suggesting that credibility scores provide different information to what is contained in star ratings. Conclusion PGCRS summarizes and weights available information in 4 domains: intervention specificity, consumer ratings, research, and development. Final scores are created through an iterative process of initial rating and consensus review. The process of updating this rating scale and integrating it into a procedure for evaluating apps demonstrates one method for determining app quality.

Background: One of the most widely used coaching models is Supportive Accountability (SA) which aims to provide intervention users with clear expectations for intervention use, regular monitoring, and a sense that coaches are trustworthy, benevolent, and have domain expertise. However, few measures exist to study the role of the SA model on coached digital interventions. We developed the Supportive Accountability Inventory (SAI) and evaluated the underlying factor structure and psychometric properties of this brief self-report measure. Method: Using data from a two-arm randomized trial of a remote intervention for major depressive disorder (telephone CBT [tCBT] or a stepped care model of web-based CBT [iCBT] and tCBT), we conducted an Exploratory Factor Analysis on the SAI item pool and explored the final SAI's relationship to iCBT engagement as well as to depression outcomes. Participants in our analyses (n = 52) included those randomized to a receive iCBT, but were not stepped up to tCBT due to insufficient response to iCBT, had not remitted prior to the 10-week assessment point, and completed the pool of 8 potential SAI items. Results: The best fitting EFA model included only 6 items from the original pool of 8 and contained two factors: Monitoring and Expectation. Final model fit was mixed, but acceptable (χ2(4) = 5.24, p = 0.26; RMSR = 0.03; RMSEA = 0.091; TLI = 0.967). Internal consistency was acceptable at α = 0.68. The SAI demonstrated good convergent and divergent validity. The SAI at the 10-week/mid-treatment mark was significantly associated with the number of days of iCBT use (r = 0.29, p = .037), but, contrary to expectations, was not predictive of either PHQ-9 scores (F(2,46) = 0.14, p = .89) or QIDS-C scores (F(2,46) = 0.84, p = .44) at post-treatment. Conclusion: The SAI is a brief measure of the SA framework constructs. Continued development to improve the SAI and expand the constructs it assesses is necessary, but the SAI represents the first step towards a measure of a coaching protocol that can support both coached digital mental health intervention adherence and improved outcomes.

Engagement is a multifaceted construct and a likely mechanism by which digital interventions achieve clinical improvements. To date, clinical research on digital mental health interventions (DMHIs) has overwhelmingly defined engagement and assessed its association with clinical outcomes through the objective/behavioral metrics of use of or interactions with a DMHI, such as number of log-ins or time spent using the technology. However, engagement also entails users' subjective experience. Research is largely lacking that tests the relationship between subjective metrics of engagement and clinical outcomes. The purpose of this study is to present a proof-of-concept exploratory evaluation of the association between subjective engagement measures of a mobile DMHI with changes in depression and anxiety. Adult primary care patients (N = 146) who screened positive for depression or anxiety were randomized to receive a DMHI, IntelliCare, immediately or following an 8-week waitlist. Subjective engagement was measured via the Usefulness, Satisfaction, and Ease of Use (USE) Questionnaire. Across both conditions, results showed that individuals who perceived a mobile intervention as more useful, easy to use and learn, and satisfying had greater improvements in depression and anxiety over eight weeks. Findings support our proposed experimental therapeutics framework that hypothesizes objective/behavioral and subjective engagement metrics as mechanisms that lead to changes in clinical outcomes, as well as support directing intervention design efforts for DMHIs to target the user experience.

Background: Depression and anxiety are leading causes of disability worldwide but often remain undetected and untreated. Smartphone and wearable devices may offer a unique source of data to detect moment by moment changes in risk factors associated with mental disorders that overcome many of the limitations of traditional screening methods.Objective: The current study aimed to explore the extent to which data from smartphone and wearable devices could predict symptoms of depression and anxiety.Methods: A total of N = 60 adults (ages 24–68) who owned an Apple iPhone and Oura Ring were recruited online over a 2-week period. At the beginning of the study, participants installed the Delphi data acquisition app on their smartphone. The app continuously monitored participants' location (using GPS) and smartphone usage behavior (total usage time and frequency of use). The Oura Ring provided measures related to activity (step count and metabolic equivalent for task), sleep (total sleep time, sleep onset latency, wake after sleep onset and time in bed) and heart rate variability (HRV). In addition, participants were prompted to report their daily mood (valence and arousal). Participants completed self-reported assessments of depression, anxiety and stress (DASS-21) at baseline, midpoint and the end of the study.Results: Multilevel models demonstrated a significant negative association between the variability of locations visited and symptoms of depression (beta = −0.21, p = 0.037) and significant positive associations between total sleep time and depression (beta = 0.24, p = 0.023), time in bed and depression (beta = 0.26, p = 0.020), wake after sleep onset and anxiety (beta = 0.23, p = 0.035) and HRV and anxiety (beta = 0.26, p = 0.035). A combined model of smartphone and wearable features and self-reported mood provided the strongest prediction of depression.Conclusion: The current findings demonstrate that wearable devices may provide valuable sources of data in predicting symptoms of depression and anxiety, most notably data related to common measures of sleep.

As access is the lowest rated dimension in surveys of outpatient experience, we sought to identify patient, practice, and provider factors associated with positive ratings of timeliness of primary care appointments. A cross-sectional study with multivariable, multilevel logistic regression was performed using survey responses from 236 695 individuals receiving care in the Veterans Health Administration (VA). Top box ratings (response of “always”) for whether the patient reported receiving an appointment as soon as they needed in primary care for routine care and for care needed right away were the main outcomes. Independent variables capturing patient, practice, and provider factors were obtained from survey responses and VA databases. Degree of continuity with primary care provider and duration of relationship were strongly associated with higher ratings. Shorter primary care appointment wait times for both new and returning patients were associated with higher ratings. Independent wait times for mental health and specialty appointments had no effect. Older age, better self-reported physical and mental health, lower disease complexity, and rural residence were patient factors associated with higher ratings while gender, race, ethnicity, and education had little effect. Measures of continuity with primary care provider as well as appointment wait times have strong association with positive patient ratings of appointment timeliness. Patients treated in Veterans Affairs clinics may value continuity with their primary care provider over longer times. Initiatives to improve access could focus on improving continuity and ensuring efforts to improve access do not impact continuity.

Since the emergence of psychological interventions delivered via the Internet they have differed in numerous ways. The wealth of formats, methods, and technological solutions has led to increased availability and cost-effectiveness of clinical care, however, it has simultaneously generated a multitude of terms. With this paper, we first aim to establish whether a terminology issue exists in the field of Internet-delivered psychological interventions. If so, we aim to determine its implications for research, education, and practice. Furthermore, we intend to discuss solutions to mitigate the problem; in particular, we propose the concept of a common glossary. We invited 23 experts in the field of Internet-delivered interventions to respond to four questions, and employed the Delphi method to facilitate a discussion. We found that experts overwhelmingly agreed that there were terminological challenges, and that it had significant consequences for conducting research, treating patients, educating students, and informing the general public about Internet-delivered interventions. A cautious agreement has been reached that formulating a common glossary would be beneficial for the field to address the terminology issue. We end with recommendations for the possible formats of the glossary and means to disseminate it in a way that maximizes the probability of broad acceptance for a variety of stakeholders.

Abstract Background US healthcare organizations increasingly use physician satisfaction and attitudes as a key performance indicator. Further, many health care organizations also have an academically oriented mission. Physician involvement in research and teaching may lead to more positive workplace attitudes, with subsequent decreases in turnover and beneficial impact on patient care. This article aimed to understand the influence of time spent on academic activities and perceived quality of care in relation to job attitudes among internal medicine physicians in the Veterans Health Administration (VHA). Methods A cross-sectional survey was conducted with inpatient attending physicians from 36 Veterans Affairs Medical Centers. Participants were surveyed regarding demographics, practice settings, workplace staffing, perceived quality of care, and job attitudes. Job attitudes consisted of three measures: overall job satisfaction, intent to leave the organization, and burnout. Analysis used a two-level hierarchical model to account for the nesting of physicians within medical centers. The regression models included organizational-level characteristics: inpatient bed size, urban or rural location, hospital teaching affiliation, and performance-based compensation. Results A total of 373 physicians provided useable survey responses. The majority (72%) of respondents reported some level of teaching involvement. Almost half (46%) of the sample reported some level of research involvement. Degree of research involvement was a significant predictor of favorable ratings on physician job satisfaction and intent to leave. Teaching involvement did not have a significant impact on outcomes. Perceived quality of care was the strongest predictor of physician job satisfaction and intent to leave. Perceived levels of adequate physician staffing was a significant contributor to all three job attitude measures. Conclusions Expanding opportunities for physician involvement with research may lead to more positive work experiences, which could potentially reduce turnover and improve system performance.

Background: There is an ongoing need for effective and accessible preventive interventions for adolescent depression and substance abuse. This paper reports on a field trial of an online indicated preventive intervention, ProjectTECH, which is based on cognitive-behavioral therapy (CBT) techniques. The study aims to gather information about the feasibility and acceptability of this program. Secondary aims of this study were to examine the impact of the program on depression symptoms, perceived stress, positive affect, and substance use and to compare differences between groups that were led by a peer versus those that were led by a licensed clinician. Methods: High school students (n = 39) were recruited primarily through social media advertisements, and assigned to four groups of 8–12 individuals to collaboratively participate in an 8 week peer network-based online preventive intervention which were led by a trained peer guide or a licensed clinician. Participants were provided with didactic lessons, CBT-based mood management tools, and peer networking features, and completed quantitative and qualitative feedback at baseline, midpoint, end of intervention, and 1 month follow-up. Results: The program attracted and retained users primarily from social media and was used frequently by many of the participants (system login M = 25.62, SD = 16.58). Participants rated the program as usable, and offered several suggestions for improving the program, including allowing for further personalization by the individual user, and including more prompts to engage with the social network. From baseline to end of intervention, significant decreases were observed in depressive symptoms and perceived stress (p's

Abstract Data from networked sensors, such as those in our phones, are increasingly being explored and used to identify behaviors related to health and mental health. While computer scientists have referred to this field as context sensing, personal sensing, or mobile sensing, medicine has more recently adopted the term digital phenotyping. This paper discusses the implications of these labels in light of privacy concerns, arguing language that is transparent and meaningful to the people whose data we are acquiring.

Objective: To determine whether parents of Very Low Birth Weight (VLBW) infants in the Neonatal Intensive Care Unit (NICU) transitioning home with the NICU-2-Home smartphone application have greater parenting self-efficacy, are better prepared for discharge and have shorter length of stay (LOS) than control parents. Methods: A four-week pilot randomized controlled trial during the transition home with 90 VLBW parents randomized to usual care (n = 44) or usual care plus NICU-2-Home (n = 46), a smartphone application designed for VLBW parents. Parenting Sense of Competence Scale (PSOC) was assessed at baseline, day after discharge, and two weeks post-discharge. Preparedness for discharge and length of stay (LOS) were secondary outcomes. Analyses by usage were also included. Results: While parents of VLBW infants in the intervention group did not show an improvement in PSOC during the transition when compared directly to controls, after accounting for actual mean app usage, PSOC improved 7% (2.71 points/time greater; 95% CI = 1.45, 6.27) for intervention versus controls. Compared to controls, above-average users increased their PSOC score by 14% (6.84 points/time; 95% CL = 5.02, 8.67), average users by 11% (4.58 points/time; 95% CL = 2.89, 6.27) and below-average users by 6% (2.41 points/time; 95% CL = 0.04, 4.79). Moderate evidence showed LOS was shorter for above-average users compared to the control group (β = 12.2. SE = 6.9, p = 0.085). Conclusion: A smartphone application used by parents of VLBW infants during the transition home from the NICU can improve parenting self-efficacy, discharge preparedness, and LOS with improved benefits based on usage.

Background: Treatments for depression and anxiety have several behavioral and psychological targets and rely on varied strategies. Digital mental health treatments often employ feature-rich approaches addressing several targets and strategies. These treatments, often optimized for desktop computer use, are at odds with the ways people use smartphone applications. Smartphone use tends to focus on singular functions with easy navigation to desired tools. The IntelliCare suite of apps was developed to address the discrepancy between need for diverse behavioral strategies and constraints imposed by typical app use. Each app focuses on one strategy for a limited subset of clinical aims all pertinent to depression and anxiety. This study presents the uptake and usage of apps from the IntelliCare suite following an open deployment on a large app marketplace. Methods: Thirteen lightweight apps, including 12 interactive apps and one Hub app that coordinates use across those interactive apps, were developed and made free to download on the Google Play store. De-identified app usage data from the first year of IntelliCare suite deployment were analyzed for this study. Results: In the first year of public availability, 5210 individuals downloaded one or more of the IntelliCare apps, for a total of 10,131 downloads. Nearly a third of these individuals (31.8%) downloaded more than one of these apps. The modal number of launches for each of the apps was 1, however the mean number of app launches per app ranged from 3.10 to 16.98, reflecting considerable variability in the use of each app. Conclusions: The use rate of the IntelliCare suite of apps is higher than public deployments of other comparable digital resources. Our findings suggest that people will use multiple apps and provides support for the concept of app suites as a useful strategy for providing diverse behavioral strategies.

Background Smartphones offer the hope that depression can be detected using passively collected data from the phone sensors. The aim of this study was to replicate and extend previous work using geographic location (GPS) sensors to identify depressive symptom severity. Methods We used a dataset collected from 48 college students over a 10-week period, which included GPS phone sensor data and the Patient Health Questionnaire 9-item (PHQ-9) to evaluate depressive symptom severity at baseline and end-of-study. GPS features were calculated over the entire study, for weekdays and weekends, and in 2-week blocks. Results The results of this study replicated our previous findings that a number of GPS features, including location variance, entropy, and circadian movement, were significantly correlated with PHQ-9 scores (r’s ranging from −0.43 to −0.46, p-values < .05). We also found that these relationships were stronger when GPS features were calculated from weekend, compared to weekday, data. Although the correlation between baseline PHQ-9 scores with 2-week GPS features diminished as we moved further from baseline, correlations with the end-of-study scores remained significant regardless of the time point used to calculate the features. Discussion Our findings were consistent with past research demonstrating that GPS features may be an important and reliable predictor of depressive symptom severity. The varying strength of these relationships on weekends and weekdays suggests the role of weekend/weekday as a moderating variable. The finding that GPS features predict depressive symptom severity up to 10 weeks prior to assessment suggests that GPS features may have the potential as early warning signals of depression.

Objective. There is limited theory regarding the real-world implementation of mental health care in the primary care setting: a type of organizational coordination intervention. The purpose of this study was to develop a theory to conceptualize the potential causes of barriers and facilitators to how local sites responded to this mandated intervention to achieve coordinated mental health care. Methods. Data from 65 primary care and mental health staff interviews across 16 sites were analyzed to identify how coordination was perceived one year after an organizational mandate to provide integrated mental health care in the primary care setting. Results. Standardized referral procedures and communication practices between primary care and mental health were influenced by the organizational factors of resources, training, and work design, as well as provider-experienced organizational boundaries between primary care and mental health, time pressures, and staff participation. Organizational factors and provider experiences were in turn influenced by leadership. Conclusions. Our emergent theory describes how leadership, organizational factors, and provider experiences affect the implementation of a mandated mental health coordination intervention. This framework provides a nuanced understanding of the potential barriers and facilitators to implementing interventions designed to improve coordination between professional groups.