1. Impact of Social Stories on social and emotional health of autism spectrum primary school children: the ASSSIST2 RCT with economic evaluation
- Author
-
Barry Wright, Kerry Jane Bell, Jane E Blackwell, Catarina Teige, Laura Mandefield, Han I Wang, Charlie Welch, Arabella Scantlebury, Jude Watson, Dean McMillan, Emma Standley, Leah Attwell, Hayley Carrick, Amelia Taylor, Olivia Taylor, Rachel Hodkinson, Hannah Edwards, Hannah Pearson, Steve Parrott, David Marshall, Danielle Varley, Rebecca Hargate, Ann Mclaren, and Catherine Elizabeth Hewitt
- Subjects
randomised controlled trialchild ,autism ,mental health ,intervention ,social stories ,school ,randomised controlled trial ,Medical technology ,R855-855.5 - Abstract
Background Differences in the way autistic children experience the world can contribute to anxiety and stress. Carol Gray’s Social Stories™ are a highly personalised intervention to support children by providing social information about specific situations in an individual story. Objectives This randomised controlled trial aimed to establish whether Social Stories are clinically effective and cost-effective in improving social responsiveness and social and emotional health in children on the autism spectrum in schools. Design A multisite pragmatic cluster randomised controlled trial comparing Social Stories with care as usual. Setting Eighty-seven schools (clusters) across Yorkshire and the Humber. Participants Two hundred and forty-nine children were randomised via a bespoke system hosted at York Trials Unit (129 Social Stories and 120 care as usual). Recruitment was completed in May 2021. Participants were children aged 4–11 years with a diagnosis of autism, alongside teachers, interventionists and caregivers. Recruitment was via schools, NHS trusts, support groups and local publicity. Intervention The intervention included training for educational professionals and caregivers covering psychoeducation and implementation of Social Stories. Stories were written around contextualised goals around the child’s need for social information. Interventionists read the Social Story™ with the child at least six times over 4 weeks during school. Main outcome measure The primary outcome was the Social Responsiveness Scale-2 completed by teachers at 6 months (the primary end point), which measures social awareness, cognition, communication and behaviour. Data were collected from caregivers and educational professionals at 6 weeks and 6 months through questionnaires. Blinding of participants was not possible. Results At 6 months, the estimated difference in expected teacher-reported Social Responsiveness Scale-2 T-score (the primary end point) was −1.61 (95% confidence interval −4.18 to 0.96, p = 0.220), slightly favouring the intervention group. The estimated differences for the parent-reported secondary outcomes at 6 months were small and generally favoured the control group except the measure of children’s quality-adjusted life-year (+ 0.001, 95% confidence interval −0.032 to 0.035) and parental stress (−1.49, 95% confidence interval −5.43 to 2.46, p = 0.460), which favoured the intervention group. Children in the intervention group met their individual goals more frequently than children who received usual care alone (0.97 confidence interval 0.21 to 1.73, p = 0.012). The intervention is likely to save small costs (−£191 per child, 95% confidence interval −767.7 to 337.7) and maintain a similar quality of life compared to usual care. The probability of Social Stories being a preferred option is 75% if the society is willing to pay £20,000 per quality-adjusted life-year gained. Limitations include considerable disruptions during the coronavirus disease 2019 pandemic. Conclusion Social Stories are used in schools and represent a low-cost intervention. There is no clinically evident impact on social responsiveness, anxiety and/or depression, parental stress or general health. Benefits were observed for specific behavioural goals as assessed by the teacher, and Social Stories may serve as a useful tool for facilitating dialogue between children and school staff to address specific behavioural challenges. Usage should be at the school’s discretion. Future work Given the uncertainty of the results in light of coronavirus disease 2019, further work to establish the impact of Social Stories is merited. Trial registration This trial is registered as ISRCTN11634810. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 16/111/91) and is published in full in Health Technology Assessment; Vol. 28, No. 39. See the NIHR Funding and Awards website for further award information. Plain language summary Autism affects the way children experience the world, and some children find social situations stressful. We wanted to know whether Social Stories™, developed by Carol Gray, helped children with their social skills and behaviour in school and whether they offered value for money. A randomised controlled trial design was used, which gave schools an equal chance of being asked to deliver Social Stories or to continue providing care as usual. Two hundred and forty-nine children from 87 schools took part and we trained school staff and parents to write and deliver Social Stories. We agreed with teachers and parents, what each child needed help with and wrote stories with this in mind. Trained staff read the Social Story with the child at least six times over 4 weeks. Follow-up information was collected from parents and school staff at the start of the study, after 6 weeks and 6 months. After 6 months, teachers completed a questionnaire called the Social Responsiveness Scale-2 which measures the child’s social skills. Using these measures, the results suggest that Social Stories do not lead to any significant changes in social skills, mental health, parent stress, general health or quality of life but children in schools allocated to Social Stories met their goal more frequently and incurred less costs than children who did not. Parents and educational professionals found the Social Stories intervention and training beneficial. Based on our findings, Social Stories do not appear to improve general social skills in primary-aged autistic children. Benefits were observed for specific goals, and school-based costs were reduced. Scientific summary Sections of this summary have been adapted from the trial protocol by Wright et al. (Wright B, Teige C, Watson J, Hodkinson R, Marshall D, Varley D, et al. Autism Spectrum Social Stories In Schools Trial 2 (ASSSIST2): study protocol for a randomised controlled trial analysing clinical and cost effectiveness of Social Stories™ in primary schools. BMC Psychol 2020;8:60. https://doi.org/10.1186/s40359-020-00427-z). This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution and reproduction in any medium provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article unless otherwise stated. Background Children on the autism spectrum experience a higher prevalence of mental health problems than typically developing children, including anxiety and low mood. Many children on the autism spectrum struggle to manage social anxiety and feelings of frustration, which can lead to behaviours that challenge them. The International Society for Autism Research (INSAR) has highlighted that more research evaluating early interventions for children on the autism spectrum is needed to ensure practitioners and policy-makers have robust data on intervention effectiveness and implementation to secure optimal outcomes for children. One intervention that attempts to alleviate social difficulties while not being intrusive, time-consuming or requiring extensive involvement from experts is Carol Gray’s Social Stories™. Social Stories are a highly personalised intervention aiming to share accurate, meaningful information about a particular goal or topic that the child needs help with in a positive and reassuring way. Social Stories can be written and delivered by both parents and professionals in a range of settings and represent a less time-consuming and intrusive intervention than alternatives. Previous studies examining the use of Social Stories have yielded mostly positive results but have largely been single-case studies with a lack of evidence from randomised controlled trials (RCTs). Despite the lack of rigorous evidence, numerous schools and families are already accessing Social Stories training and delivering the intervention with children and therefore a fully powered trial is timely. Schools have limited resources and limited access to specialist practitioner interventions, and therefore it is important that interventions such as Social Stories undergo robust evaluation. If they are found to be clinically effective and cost-effective, they can be delivered within schools on a day-to-day basis. Objectives The aim was to assess whether Social Stories alongside care as usual is clinically effective and cost-effective in improving child social impairment, reducing anxiety and improving social and emotional health in children on the autism spectrum in primary and special educational needs (SEN) schools when compared with care as usual alone. Primary objective The primary objective of the study was to establish whether Social Stories can improve social responsiveness in children on the autism spectrum in primary schools across Yorkshire and the Humber, when compared to children who have received care as usual only. Social responsiveness can be broadly defined as social awareness, social cognition, social communication, social motivation and mannerisms. Secondary objectives The secondary objectives of this trial were: to investigate whether Social Stories can reduce behaviours that challenge children on the autism spectrum in primary schools to investigate whether Social Stories can improve social and emotional health in children on the autism spectrum in primary schools to assess the cost effectiveness of Social Stories to examine the effects of Social Stories delivered in the classroom on general measures of health-related quality of life to examine whether Social Stories improve classroom attendance to assess sustainability of Social Stories in an educational setting across a 6-month period to examine any changes in parental stress to examine any associations between treatment preference and outcomes to examine how elements of session delivery (e.g. session frequency, length and any associated problems/adverse events) are associated with outcomes. Methods This trial was a multisite pragmatic cluster RCT comparing Social Stories and care as usual with a control group receiving care as usual alone. Care as usual is defined as the existing support routinely provided for a child with autism spectrum condition (ASC) from educational and health services such as specialist autism teacher teams, mental health teams or other associated professionals. The trial included an internal pilot, economic evaluation and a nested process evaluation. Setting The Social Stories intervention was primarily delivered within educational settings by educational professionals. Educational settings included both mainstream primary schools and SEN schools. Parents/caregivers were invited to receive Social Stories training and had the option of also delivering the intervention within the home. Participants Inclusion criteria The child was aged 4–11 years at the time of recruitment. The child attended a participating primary or SEN school within Yorkshire and the Humber. The child has a clinical diagnosis of ASC and daily challenging behaviour. Parents/guardians of the child were able to self-complete the English language outcome measures (with assistance if necessary). Exclusion criteria The school had used Social Stories for any pupil in the current or preceding school term. The child or interventionist teacher had taken part in the previous Social Stories feasibility study (ASSSIST-1). Schools that have taken part were not excluded. Where families were recruited but the child’s school was unwilling or unable to participate, families were notified by a telephone call and/or an e-mail. Where children were confirmed eligible and consent had been taken from all relevant parties, baseline questionnaires were distributed to parents/carers and educational professionals. A school was considered ‘ready to randomise’ once all consent forms and baseline questionnaires had been received by the trial team. Interventions Children in the intervention arm of the trial received the Social Stories intervention in addition to their care as usual. The Social Stories intervention was delivered by a trained educational professional (the interventionist) who was employed within each school allocated to the intervention arm. The interventionist varied between the schools [e.g. a teacher, teaching assistant (TA) or Special Educational Needs Coordinator (SENCO)] but was most typically a TA. A core aspect of the intervention was to first agree a goal around which the story would be set. The goal was typically a behavioural challenge the child was struggling with, for example, sharing with their teacher how they were feeling. This goal was agreed during a collaborative ‘goal-setting meeting’ attended by the child’s teacher, a parent/caregiver, a member of the research team and sometimes the child’s TA. Occasionally, the children themselves were able to feed into the goal-setting process if deemed appropriate. The intervention is designed to provide social information to the child, and so the goal sought to reflect this. In this way, many goals sought to reduce the child’s anxiety or frustration by equipping them with information or providing reassurance around appropriate behaviours within a given circumstance. We assessed goal attainment in terms of how frequently a child was able to implement the desired behaviours, for example, if a goal for a particular child was to use calming strategies when they were upset, we asked teachers to rate how often a child was able to do this. Interventionists were trained by members of the research team, who had received training via a cascade model overseen by child psychiatrist, Professor Barry Wright. The training of interventionists included some psychoeducation around the differences in children on the autism spectrum to facilitate understanding around what factors may be driving the observed behaviours associated with the behavioural goal. Training also provided key information on the design and implementation of Social Stories, with materials based on those developed in the preceding feasibility study with the support of Carol Gray and a Social Stories manual produced by Professor Barry Wright and a Clinical Psychologist with expertise in autism. During the training session, interventionists constructed a Social Story. Parents/guardians were also invited to attend these sessions. Following training, all Social Stories were assessed against a fidelity checklist by a member of the research team to ensure they conformed to the 10 established criteria central to Carol Gray’s Social Stories. They were then delivered to children in the intervention arm by the interventionist at least six times over a 4-week period. Sample size The primary outcome was that the teacher completed the Social Responsiveness Scale, Second Edition (SRS-2) T-score at 6 months. Within the pilot data, outcomes were measured at 6 and 16 weeks. The correlation between baseline and 6 weeks [r = 0.67, 95% confidence interval (CI) 0.44 to 0.80] was lower than that at 16 weeks (r = 0.83, 95% CI 0.68 to 0.91) for the pilot data. To be conservative, the lower 95% confidence limit was chosen for the lowest correlation between baseline and follow-up that we observed within our pilot data (r = 0.44). Assuming a difference of 3 points, standard deviation (SD) = 7, 5% alpha, 90% power, average cluster size 1.35, intracluster correlation coefficient (ICC) = 0.34, correlation = 0.44 and 25% attrition, a total sample size of 278 was required. Support was gained from the trial steering committee and funder to stop recruitment with 249 children, which we modelled would still retain at least 80% power under a variety of different scenarios. Randomisation Randomisation was completed by unblinded members of the trial team via a bespoke trial management system. Stratified blocked randomisation was used to allocate school clusters, with randomly varying block sizes (4, 6 and 8), stratification by school type (SEN school or mainstream school) and the number of participating children within a school (≤ 5 or > 5 participating children). There were no stipulations regarding the minimum number of participants per school cluster (i.e. any school with ≥ 1 eligible/consented child with available baseline data was eligible for randomisation). Main outcome measure The primary outcome of the trial was the SRS-2 at 6 months post randomisation, as reported by the child’s teacher. This was also collected at 6 weeks post randomisation. The SRS-2 identifies the presence and severity of social difficulty within the autism spectrum condition and consists of 65 questions. For each question, the person completing the form picks a score from 1 to 4 (1 = not true, 2 = sometimes true, 3 = often true, 4 = almost always true) that best describes the child’s behaviour. A T-score is calculated based on the sex of the child and the person completing the form (teacher). Economic evaluation The primary analysis for the economic evaluation was a within-trial cost-utility analysis conducted from a societal perspective. Combining costs and quality-adjusted life-years (QALYs), an incremental cost-effectiveness ratio (ICER) of cost per QALY was calculated and evaluated against the willingness-to-pay threshold of £20,000–30,000 per QALY gained to assess the cost effectiveness of Social Stories compared to usual care. Regression models on an intention-to-treat basis were used to compare mean costs and QALYs. To take uncertainty into consideration, a non-parametric bootstrap resampling method was used to produce CIs around the cost and QALY differences and ICER. The following sensitivity analyses were conducted to test assumptions made in the primary analysis: a complete case analysis; a sensitivity analysis from the UK NHS and personal social services (PSS) perspective; and a sensitivity analysis from the joint perspectives of the NHS/PSS and education perspectives to representative a global public sector perspective. Process evaluation The process evaluation was cross-sectional and longitudinal, encompassing all aspects of the Social Stories intervention. The aim of the process evaluation was to assess the fidelity of the programme, consider the views of various stakeholders and identify barriers and facilitators to successful implementation. We aimed to achieve this through a combination of data collection techniques, including interviews, focus groups, questionnaires, surveys and diaries (session logs). Statistical analyses For the primary outcome, differences in expected SRS-2 scores were estimated using a linear mixed-effect covariance pattern model with both post-randomisation time points (6 weeks and 6 months) included as outcomes and fixed effects for treatment group, time point and their interaction. Further fixed effects were included for the following cluster and participant-level baseline covariates: school SEN status (binary, SEN/non-SEN), number of consented children attending school (binary, ≤ 5/> 5), baseline score (linear term), age at randomisation (linear term) and sex (binary, female/male). Dependence between participants within a cluster was modelled using school cluster-level random intercepts, and dependence between repeated measurements within participants was modelled using an unstructured correlation matrix for the residual errors. Similar models were used to analyse the secondary outcomes. Results The primary analysis included all 249 randomised children. After 6 months, a reduction of 1.61 points was found on the SRS-2 in children on the autism spectrum in the intervention group (95% CI −4.18 to 0.96; p = 0.220). No statistically significant differences were found in overall symptoms of anxiety and/or depression, parental stress or general health. Children in the intervention group met their individual goals more frequently than children who received usual care alone, and this difference was statistically significant (0.97, CI 0.21 to 1.73; p = 0.012). The primary analysis was conducted under the principles of intention to treat. Coronavirus disease 2019 (COVID-19) impacted upon delivery in schools and school routines, and only 62.8% of the sample is known to have received the intervention as per protocol. Sensitivity analyses suggested that there may be a dose effect with improved clinical effects in those receiving the intervention as per protocol compared to those with fewer sessions. The economic evaluation showed that, compared to usual care, Social Stories slightly decreased the service use costs over the 6-month period by £191 (95% CI −337.7 to 767.7) per child and maintained similar QALYs. If society is willing to pay £20,000 for extra QALY gained, then the probability of Social Stories being a dominant and preferred option is 75%. The results of both primary and sensitivity analyses, which considered costs derived from various perspectives, are consistent but limited to the given data within the short study time frame, number of missing data and disruption of COVID-19. The qualitative results suggest parents and educational professionals found the Social Stories training and intervention beneficial. Limitations include considerable disruptions during the COVID-19 pandemic. Conclusion We found no impact of Social Stories on autistic children’s overall social responsiveness skills. There was some evidence that Social Stories are well supported by parents and teachers, and it may be effective at addressing a wider set of individual goals and could reduce costs. Based on the evidence generated through this trial, we cannot recommend Social Stories for the purposes of improving social skills, anxiety and/or depression, parental stress, general health and quality of life in autistic children. The COVID-19 pandemic had a large impact on schools during this trial, and sensitivity analyses suggested that poor compliance may have reduced the effectiveness of the intervention and that effects were better in those receiving Social Stories as per protocol. We did not find any negative effects, and Social Stories are already frequently used in schools to support autistic children and represent a low-cost and potentially cost-saving intervention. Despite limited evident impact on global social skills, based on the data elicited through the process evaluation, it appears that Social Stories may serve as a useful tool for facilitating dialogue between children and school staff to better understand the needs of autistic children, and usage should be at the school’s discretion. Trial registration This trial was registered as ISRCTN11634810. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 16/111/91) and is published in full in Health Technology Assessment; Vol. 28, No. 39. See the NIHR Funding and Awards website for further award information.
- Published
- 2024
- Full Text
- View/download PDF