Search

This is a report of a pilot programme of the SHINE in Secondaries Saturday school transition intervention. The programme aims to improve attainment by focusing on literacy and numeracy and revisiting areas where pupils are struggling through a creative curriculum approach and enrichment opportunities. The Saturday programme is run by teachers from the school for 25 weeks throughout Year 7. Schools are responsible for developing their own curriculum of activities to suit the abilities and needs of their pupils following guiding principles specified by SHINE. The programme is designed to run for approximately 60 pupils staffed by four qualified teachers, three teaching assistants and three peer mentors all recruited and employed by the host secondary school. This evaluation was set up as a pilot study with four schools and 613 pupils participating. The study had three aims: (1) To explore the impact of SHINE in Secondaries on literacy, numeracy and attitudes to school and learning outcomes; (2) To explore the organisational implications and lessons for future wider roll-out; and (3) To explore the feasibility of using regression discontinuity design (RDD) as a pilot study. The evaluation showed a very small impact on literacy outcomes across all four schools using the Progress in English and Progress in Maths tests. These findings are limited evidence of promise of the intervention. A larger RDD study or randomised controlled trial would be required to investigate the impact of the intervention and the size of the effect (if any) with confidence. A subgroup analysis of children eligible for free school meals (FSM) was conducted. This showed no evidence of the intervention differentially benefiting pupils eligible for free school meals in any of the schools. These findings should be treated with caution because, due to the small sample size, the study was unable to detect the small effects with confidence. The pilot demonstrated that it was feasible to design, implement and analyse the RDD successfully; and the design could be used again to evaluate similar interventions where randomisation is not possible or desirable. Key conclusions include: (1) Impact of the intervention: Very small effect sizes on the primary and secondary outcomes of literacy and numeracy were detected. As this was an exploratory evaluation design and the number of schools involved in the project was small, this made it unlikely that any impact of SHINE could be detected, should the true estimate of effect be small; (2) Implementation and organisation issues: Feedback from the teachers and pupils suggested that the intervention was perceived to be valuable in providing an opportunity to consolidate literacy and numeracy skills; and (3) Feasibility of evaluation design: The purpose in piloting this design was to explore the feasibility of using this approach when randomly allocating pupils to an intervention group was not possible or desirable. The regression discontinuity design is an appropriate evaluation method for education interventions. However, because it is less powerful than a randomised controlled trial, the number of pupils involved must be two and a half times greater if a small effect is to be detected.