Search

In a rigorous evaluation of ASSISTments as an online homework support conducted in the state of Maine, SRI International reported that "the intervention significantly increased student scores on an end-of-the-year standardized mathematics assessment as compared with a control group that continued with existing homework practices." (Roschelle, Feng, Murphy & Mason, 2016). Naturally, education stakeholders want to know how big the improvement was.To answer this type of question, researchers report an effect size as a simple way of quantifying the difference between two groups. We reported an effect size of g = 0.18 of a standard deviation (t(20) = 2.992, p = 0.007) based on a two level hierarchical linear model (Roschelle et al., 2016). An effect size is calculated by dividing the difference in scores between the two groups by the pooled standard deviation (Hedges, 1981). The underlying idea is that the strength of an effect depends both on the magnitude of the score difference and on how much the scores vary naturally. Consider this analogy to a commute: If it takes exactly 25 minutes to get to work every day, then a reduction to 22 minutes might mean a lot. Yet if the commute time varies between 10 minutes and 60 minutes, a reduction of the average time of 25 minutes to 22 minutes might not feel like much. Roschelle and colleagues (2016) also reported an improvement index corresponding to the effect size: "Students at the 50th percentile without the intervention would improve to the 58th percentile if they received the ASSISTments treatment." An improvement index is the expected percentile gain for the average student in the control group--the student who scored at the 50th percentile on the outcome measure--if that student had attended a school where the intervention was implemented. Reporting the effect size or an improvement index does not appear to answer educators'questions completely, however. To an educator, the implications of whether such numbers are high or low may not be unclear. In this technical report, we present alternatives for explaining the effect size, building on the guidance of Lipsey et al. (2012), a leading researcher who developed broad recommendations for effect size reporting. First, we provide additional detail on how we calculated the effect size and highlight the range of values that might be considered valid for this study. Second, we give comparisons with conventional benchmarks, a strategy that Lipsey and colleagues criticized but that still bears reporting. Third, we offer comparisons based on the recommendations of Lipsey et al. The report closes with a discussion of the challenges of interpreting effect sizes. The sidebar at the end of the report provides sample statements that educators may use to describe the study.

This report presents the findings from interviews with principals about their schools' participation in the ASSISTments efficacy study on seventh-grade mathematics. The purpose of the interviews was to explore the following areas: To what extent schools had policies or expectations regarding homework practice and completion; To what extent schools had concerns about homework completion rates; Whether the school had existing practices or norms for teachers to discuss student learning outcomes, and how often teachers met to focus on data; The extent to which principals met with teachers to discuss data; What other initiatives in the district or school could impact curriculum, instruction, assessment, or professional development and what the current priorities were in the school; Factors related to the community such as broad socio-economic status, or level of support for education, that might affect teaching and student achievement in the school.

This report presents the findings of an exploratory study conducted by the Center for Research and Evaluation during the summer and fall of 2009, which focuses on perceptions of the costs and benefits of the accreditation process for high schools in Maine. The study was commissioned by the Penquis Superintendents' Association, a professional organization in mid-Maine. The findings from this study indicate that while local schools continue to value the benefits of the accreditation process, there is growing concern about the ability of school districts to meet the financial cost for accreditation. Schools are also questioning whether their teachers can manage the workload of accreditation on top of other pressing state and federal mandates, and to what extent the accreditation work overlaps with or provides additional information that is not included in state or federal accountability efforts. Appendices include: (1) List of education stakeholder groups included in interviews; (2) List of colleges and universities included in interviews; (3) Interview protocol for high school interviews; (4) Interview protocol for education stakeholder groups ; and (5) Interview protocol for college admissions interviews. (Contains 3 tables.)

In June 2007 the Governor of Maine signed legislation that required the state's 290 school administrative units (SAUs) to seek partners for a reorganization that would reduce the number of SAUs to approximately 80 Regional School Units (RSUs) (Maine State Legislature, 2007). This policy initiative is the first reorganization of Maine's school districts since 1957. The planning for reorganization is to be done by Regional Planning Committees (RPCs) that consist of representatives of the municipalities, school boards, and the general public. This study examines the process and progress of school district reorganization, and perceptions of key actors about reorganization, drawing on data from case studies of five RPCs and RPCs statewide during the first year of the initiative. The findings to date from this study indicate that RPCs have been working diligently on the tasks required by legislation to reorganize Maine school districts. While there is compliance with the law and general agreement that more efficient ways of delivering high quality education programs can and should be found, the early findings of this study provide evidence that full support for reorganization is lacking in much of the education community. This lack of support appears to be related to several factors. Additional research is needed to validate these findings beyond the limited sample of this study. The findings from this research suggest that the lack of support for school district reorganization is rooted in four fundamental problems: (1) pressure from time constraints and mandates; (2) lack of confidence in the stability of the initiative; (3) lack of credibility of primary goals; and (4) threats to local values around governance and school choice. Overall, these factors describe threats to a policy initiative that, on the surface appears to be moving forward toward implementation. Seven appendices are included: (1) Table showing data sources for the study; (2) Tables showing the number of interviews and returned surveys from the first round of data collection (interviews from November 2007 to March 2008, surveys from November 2007 and January 2008); (3) Survey of RPC Members, Fall 2007; (4) Interview Protocol for RPC Members, Fall 2007; (5) Data tables showing the results of the RPC Member Survey; (6) Data table showing progress on elements of reorganization plans as of December 2007 and March 2008 for the five sample RPCs; and (7) Data tables showing some of the analyses of data from reorganization plans filed by districts statewide in December 2007. (Contains 25 tables.) [Funding for this research was provided by the Penquis Superintendents' Association.]