Evaluation of Playground Physics: Implementation and Outcomes, 2020-21

Playground Physics is a supplemental middle school science program designed to support embodied learning by enabling students to connect physics concepts to their own playful activities. The purpose of this study was to examine a model for scaling up the Playground Physics program and to determine the program's impact on middle school students' knowledge of physics concepts and science-related attitudes (engagement in science class, perceived utility of physics, and interest in science). Because the study took place during the COVID-19 pandemic, another objective was to examine whether the influence of remote or blended learning environments on program implementation. Researchers randomly assigned 46 schools in New York State either to use Playground Physics to teach middle school physics concepts (motion, force, and energy) to students in grades 6-8 or to teach these topics using their regular curriculum. The study used pre- and post-test student knowledge assessments and surveys to collect student outcome data. In addition, the study used teacher surveys and program records and artifacts to measure fidelity of implementation as well as teacher perceptions of program components. Attrition of schools from schools occurred primarily in the treatment condition, with 13 of 23 treatment schools and 4 of 23 control schools choosing to leave the study. To account for baseline differences in measures of engagement in science class and interest in physics, researchers matched students in treatment schools with students in control schools using a propensity score matching method. This study did not find a significant impact of the program on students' physics knowledge or science-related attitudes. Most of the 12 treatment teachers responding to a post-program survey stated that the professional development workshop and online community of practice prepared them somewhat well to teach Playground Physics. Most teachers participated in the online community of practice 1-3 times. Teachers typically reported implementing at least one of the three units in its entirely. Most teachers reported that Playground Physics at least moderately supported instruction, and the majority reported that students were more engaged when using Playground Physics than when using their regular physics curriculum. Teachers in remote or blended-learning instructional environments reported implementing fewer Playground Physics lessons than teachers whose schools had resumed in-person instruction. Teachers described remote learning as a barrier to program implementation. These findings are discussed in the context of the COVID-19 pandemic and the shifts in teaching and learning environments that it required. (With five appendices describing additional findings and the technical approach to the impact analysis. 9 references.)