Search

Evaluating digital behavior change intervention engagement is complex and requires multidimensional and novel approaches that are emerging. The relationship and interdependence between engagement with the technology and engagement with the psychosocial or behavior change process often presents conceptual and evaluative challenges. Large objective data sets detailing technology use are plentiful but meaningful interpretation can be challenging at granular levels. Affiliation network analysis which describes two-mode network data may provide a novel approach to evaluate engagement of digital behavior change interventions. The purpose of this paper is to use affiliation network analysis as an exploratory method to describe, assess and visualize content-specific patterns underlying psychosocial characteristics related to HPV vaccine safety concerns of parents using the HPVcancerFree intervention. Results indicate that affiliation network analysis shows promise in supplementing existing methods to assess engagement of digital interventions.

Evaluating digital behavior change intervention engagement is complex and requires multidimensional and novel approaches that are emerging. The relationship and interdependence between engagement with the technology and engagement with the psychosocial or behavior change process often presents conceptual and evaluative challenges. Large objective data sets detailing technology use are plentiful but meaningful interpretation can be challenging at granular levels. Affiliation network analysis which describes two-mode network data may provide a novel approach to evaluate engagement of digital behavior change interventions. The purpose of this paper is to use affiliation network analysis as an exploratory method to describe, assess and visualize content-specific patterns underlying psychosocial characteristics related to HPV vaccine safety concerns of parents using the HPVcancerFree intervention. Results indicate that affiliation network analysis shows promise in supplementing existing methods to assess engagement of digital interventions.

BACKGROUND Human papillomavirus (HPV)-attributed cancers are preventable, yet HPV vaccination rates severely lag behind other adolescent vaccinations. HPVcancerFree (HPVCF) is a mobile health (mHealth) intervention developed to influence parental HPV vaccination decision making by raising awareness of HPV, reducing HPV vaccination barriers, and enabling HPV vaccination scheduling and reminders through a smartphone app. Evaluating the user experience of mHealth interventions is a vital component in assessing their quality and success but tends to be underreported in mHealth intervention evaluation. OBJECTIVE We aimed to evaluate the user experience of HPVCF, an HPV cancer prevention app designed for a pediatric clinic network, using mixed methods data collected from log files, survey measures, and qualitative feedback. METHODS Study data were evaluated from parents in a large US pediatric clinic network using HPVCF in the treatment study condition of a group randomized controlled trial. Log data captured HPVCF retention and use. Postintervention rating scales and items assessed HPVCF utility, usefulness, understandability, appeal, credibility, and perceived impact. Overall quality was evaluated using the user version of the Mobile Application Rating Scale (uMars). Open-ended responses assessed parent recommendations for HPVCF enhancement. RESULTS The 98 parents were mainly female (n=94, 96%), 41 (5.67) years of age, college educated (n=55, 56%), and White and non-Hispanic (n=55, 56%) and had private health insurance for their children (n=75, 77%). Parents used HPVCF 197 times, with the average visit duration approximating 3.5 minutes. The uMARS app quality score was positively skewed (4.2/5.0). Mean ratings were highest for information (4.46 [SD 0.53]) and lowest for engagement (3.74 [SD 0.69]). In addition, of 95 parents, 45 (47%) rated HPVCF as helpful in HPV vaccination decision making and 16 (17%) attributed HPV vaccine initiation to HPVCF. Parents reported that HPVCF increased their awareness (84/95, 88%), knowledge (84/95, 88%), and HPV vaccination intentions (64/95, 67%). Most of the 98 parents rated the 4 HPVCF components as useful (72-92 [73%-94%]). Parents also agreed that HPVCF is clear (86/95, 91%), accurate (86/95, 91%), and more helpful than other HPV vaccine information they had received (89/95, 94%) and that they would recommend it to others (81/95, 85%). In addition, parents suggested ways to increase awareness and engagement with the app, along with opportunities to enhance the content and functionality. CONCLUSIONS HPVCF was well received by parents and performed well on indicators of quality, usefulness, utility, credibility, and perceived impact. This study contributes a multimethod and multimeasure evaluation to the growing body of literature focused on assessing the user experience of patient-focused technology-mediated applications for HPV education.

BACKGROUND Human papillomavirus (HPV) is a common and preventable sexually transmitted infection; however, vaccination rates in the United States among the target age group, which is 11-12 years, are lower than national goals. Interventions that address the barriers to and facilitators of vaccination are important for improving HPV vaccination rates. Web-based, text-based focus groups are becoming a promising method that may be well suited for conducting formative research to inform the design of digital behavior change intervention (DBCI) content and features that address HPV vaccination decision-making. OBJECTIVE This study aims to explore parental HPV vaccination decision-making processes using a web-based, text-based focus group protocol to inform content and feature recommendations for an HPV prevention DBCI. METHODS We conducted 4 web-based, text-based synchronous focus groups via Skype with the parents of patients aged 11-13 years within a large urban US pediatric clinic network. RESULTS The 22 parents were mostly female, White, non-Hispanic college graduates, and they mostly had private health insurance for their children. Approximately half (14/25, 56%) of the parents' 11-13 year old children had initiated HPV vaccination. Most parents had experience using Skype (19/22, 86%). Approximately half (8/17, 47%) of parents expressed no preference for the focus group format, whereas 47% (8/17) requested a text-only chat format and 6% (1/17) requested an audiovisual format. The three main themes from the qualitative data were barriers to HPV vaccination, facilitators of HPV vaccination, and suggestions for improving the HPV vaccination clinic experience. A total of 11 intervention content and feature recommendations emerged from the themes, including addressing HPV knowledge barriers using trusted sources, designing for a family audience, focusing on the framing of messages, reporting reputable HPV research in a comprehensible format, and expanding the clinic visit experience. CONCLUSIONS Synchronous text-based focus groups are feasible for conducting formative research on HPV vaccination decision-making. Among well-educated and well-resourced parents, there are barriers such as misinformation and facilitators such as pediatrician recommendations that influence HPV vaccination decision-making. Parents want to conduct their own HPV research as well as receive relevant HPV vaccination advice from their child’s pediatrician. In addition, parents want an enhanced clinic visit experience that lets them access and connect to tailored information before and after clinic visits. The results gathered provide guidance for content and features that may inform a more responsive DBCI to address HPV vaccination decision-making among parents.

Background Human papillomavirus (HPV)-attributed cancers are preventable, yet HPV vaccination rates severely lag behind other adolescent vaccinations. HPVcancerFree (HPVCF) is a mobile health (mHealth) intervention developed to influence parental HPV vaccination decision making by raising awareness of HPV, reducing HPV vaccination barriers, and enabling HPV vaccination scheduling and reminders through a smartphone app. Evaluating the user experience of mHealth interventions is a vital component in assessing their quality and success but tends to be underreported in mHealth intervention evaluation. Objective We aimed to evaluate the user experience of HPVCF, an HPV cancer prevention app designed for a pediatric clinic network, using mixed methods data collected from log files, survey measures, and qualitative feedback. Methods Study data were evaluated from parents in a large US pediatric clinic network using HPVCF in the treatment study condition of a group randomized controlled trial. Log data captured HPVCF retention and use. Postintervention rating scales and items assessed HPVCF utility, usefulness, understandability, appeal, credibility, and perceived impact. Overall quality was evaluated using the user version of the Mobile Application Rating Scale (uMars). Open-ended responses assessed parent recommendations for HPVCF enhancement. Results The 98 parents were mainly female (n=94, 96%), 41 (5.67) years of age, college educated (n=55, 56%), and White and non-Hispanic (n=55, 56%) and had private health insurance for their children (n=75, 77%). Parents used HPVCF 197 times, with the average visit duration approximating 3.5 minutes. The uMARS app quality score was positively skewed (4.2/5.0). Mean ratings were highest for information (4.46 [SD 0.53]) and lowest for engagement (3.74 [SD 0.69]). In addition, of 95 parents, 45 (47%) rated HPVCF as helpful in HPV vaccination decision making and 16 (17%) attributed HPV vaccine initiation to HPVCF. Parents reported that HPVCF increased their awareness (84/95, 88%), knowledge (84/95, 88%), and HPV vaccination intentions (64/95, 67%). Most of the 98 parents rated the 4 HPVCF components as useful (72-92 [73%-94%]). Parents also agreed that HPVCF is clear (86/95, 91%), accurate (86/95, 91%), and more helpful than other HPV vaccine information they had received (89/95, 94%) and that they would recommend it to others (81/95, 85%). In addition, parents suggested ways to increase awareness and engagement with the app, along with opportunities to enhance the content and functionality. Conclusions HPVCF was well received by parents and performed well on indicators of quality, usefulness, utility, credibility, and perceived impact. This study contributes a multimethod and multimeasure evaluation to the growing body of literature focused on assessing the user experience of patient-focused technology-mediated applications for HPV education.

Background Human papillomavirus (HPV) is a common and preventable sexually transmitted infection; however, vaccination rates in the United States among the target age group, which is 11-12 years, are lower than national goals. Interventions that address the barriers to and facilitators of vaccination are important for improving HPV vaccination rates. Web-based, text-based focus groups are becoming a promising method that may be well suited for conducting formative research to inform the design of digital behavior change intervention (DBCI) content and features that address HPV vaccination decision-making. Objective This study aims to explore parental HPV vaccination decision-making processes using a web-based, text-based focus group protocol to inform content and feature recommendations for an HPV prevention DBCI. Methods We conducted 4 web-based, text-based synchronous focus groups via Skype with the parents of patients aged 11-13 years within a large urban US pediatric clinic network. Results The 22 parents were mostly female, White, non-Hispanic college graduates, and they mostly had private health insurance for their children. Approximately half (14/25, 56%) of the parents' 11-13 year old children had initiated HPV vaccination. Most parents had experience using Skype (19/22, 86%). Approximately half (8/17, 47%) of parents expressed no preference for the focus group format, whereas 47% (8/17) requested a text-only chat format and 6% (1/17) requested an audiovisual format. The three main themes from the qualitative data were barriers to HPV vaccination, facilitators of HPV vaccination, and suggestions for improving the HPV vaccination clinic experience. A total of 11 intervention content and feature recommendations emerged from the themes, including addressing HPV knowledge barriers using trusted sources, designing for a family audience, focusing on the framing of messages, reporting reputable HPV research in a comprehensible format, and expanding the clinic visit experience. Conclusions Synchronous text-based focus groups are feasible for conducting formative research on HPV vaccination decision-making. Among well-educated and well-resourced parents, there are barriers such as misinformation and facilitators such as pediatrician recommendations that influence HPV vaccination decision-making. Parents want to conduct their own HPV research as well as receive relevant HPV vaccination advice from their child’s pediatrician. In addition, parents want an enhanced clinic visit experience that lets them access and connect to tailored information before and after clinic visits. The results gathered provide guidance for content and features that may inform a more responsive DBCI to address HPV vaccination decision-making among parents.

Background Accurate and valid measures for implementation constructs are critical to advance research and guide implementation efforts. However, there is a continued need for valid and reliable measures for implementation research. The purpose of this study was to assess the psychometric properties of measures for the Inner Setting domain of the Consolidated Framework for Implementation Research (CFIR) in a network of pediatric clinics. Methods This study used cross-sectional survey data collected from physicians, advanced practice providers, clinic managers, and clinical staff (n = 546) working in a pediatric clinic network (n = 51). Surveys included measures assessing Inner Setting constructs from CFIR (culture, learning climate, leadership engagement, and available resources). We used a series multilevel confirmatory factor analysis (CFA) models to assess factorial validity. We also examined measure correlations to test discriminant validity and intraclass correlation coefficients, ICC(1) and ICC(2), to assess inter-rater reliability. Results Factor loadings were high (≥0.60) for all but one of the measurement items. Most CFA models for respective constructs demonstrated adequate or good model fit (CFI > 0.90, TLI > 0.90, RMSEA