Search

The purpose of this evaluation was to examine the impact of i-Ready Personalized Instruction that met Curriculum Associates' recommended usage levels on mathematics achievement, as measured by the Massachusetts Comprehensive Assessment System (MCAS) mathematics assessment. This study compared mathematics achievement growth of students who used i-Ready Personalized Instruction with fidelity to comparison to that of students who only used the i-Ready Diagnostic Assessment suite, but did not have access to Personalized Instruction. Hierarchical linear modeling was used to compare mathematics achievement gains, and analyses were performed by grade level. Results showed that students in Grades 3, 5, 6, and 8 that used i-Ready Personalized Instruction with fidelity had significantly higher MCAS mathematics achievement gains in the 2020-21 school year than did comparison students who did not use i-Ready Personalized Instruction. These results help to illustrate the importance of students meeting Curriculum Associates' recommended usage levels of the Personalized Instruction suite in order to maximize the potential benefits of this instructional program.

The purpose of this evaluation was to examine the impact of i-Ready Personalized Instruction that met Curriculum Associates' recommended usage levels on ELA achievement, as measured by the Massachusetts Comprehensive Assessment System (MCAS) ELA assessment. This study compared the ELA achievement growth in the 2020-21 school year of students who used i-Ready Personalized Instruction with fidelity in ELA classrooms to that of students who used the i-Ready Diagnostic Assessment suite, but did not have access to Personalized Instruction. Hierarchical Linear Modeling (HLM) was used to compare ELA achievement gains, and analyses were performed by grade level. Results showed that students in Grades 4 and 8 that used i-Ready Personalized Instruction with fidelity had significantly larger MCAS ELA achievement gains than did comparison students who did not use Personalized Instruction. Additionally, ELA achievement gains for students in Grades 5 and 7 that used i-Ready Personalized Instruction, in relation to those of comparison students, approached statistical significance (p < 0.10). These results help to show the importance of students using the Personalized Instruction suite with fidelity in order to maximize potential benefits of this program.

The purpose of this study was to use a quasi-experimental design (QED) to examine the effectiveness of i-Ready Personalized Instruction on student achievement on a summative state assessment (Massachusetts Comprehensive Assessment System; MCAS) in a year of learning disruptions from the COVID-19 pandemic. The study sample consisted of about 11,000 grades 3-8 students from five school districts in Massachusetts. All schools used i-Ready Diagnostic assessments, but each district assigned some students to receive both the Diagnostic and Instruction products, while other students only received the Diagnostic product. Propensity score weighting (PSW) was used to create a comparison (Diagnostic-only) group that was as similar as possible to the treatment (Diagnostic and Instruction) group. Hierarchical linear modeling (HLM), using school as the clustering variable, was used to conduct quantitative analyses of available achievement data, and analyses were conducted by grade level. Results showed that i-Ready Personalized Instruction use was associated with significantly higher MCAS mathematics achievement gains. Students in grades 3, 5, and 6 that used i-Ready Personalized Instruction significantly outgained comparison students, by an average of 5-7 points. Further, grade-aggregated analyses showed that i-Ready Personalized Instruction students showed significantly higher mathematics achievement gains than did comparison students, with Instruction students averaging five points higher MCAS scores. These results give evidence supporting the efficacy of i-Ready's Personalized Instruction suite in a school year marked by near-constant learning disruptions. It is important to note that, for these analyses, the comparison group used the i-Ready Diagnostic assessment system, which itself has already been shown to be an effective research-based intervention. Thus, it is likely that efficacy estimates of the treatment (Diagnostic and Instruction) condition may be somewhat conservative, relative to a comparison condition where students received neither of the i-Ready components.

The purpose of this study was to compare student growth of SCSD-7 students on the MAP (Measures of Academic Progress) reading assessment to otherwise similar students who did not use Istation. This study followed two cohorts of elementary school students over three years (2016-17 through 2018-19) to assess the effect of Istation on reading achievement and growth. Over these two cohorts of students, Istation usage was associated with greater MAP reading gains, in relation to virtual comparison students identified by NWEA's Similar Schools Report. Across both cohorts, Istation students averaged a nearly one-point larger gain in MAP reading scores than did comparison students. This advantage was especially pronounced in Cohort 1, as Istation students averaged greater than a 2.5 point larger gain in MAP reading scores in relation to comparison students. In addition, a generally positive association between school-level average Istation usage and MAP reading gains in relation to comparison students was found across SCSD-7 elementary schools. As SCSD-7 students were matched with virtual comparison students on the basis of prior reading achievement and demographic variables, and a sufficiently large sample size was used, the results support the conclusion that Istation is related to larger reading achievement gains, in relation to non-Istation users.

Bet on Baltimore is a summer youth development program with a focus on creativity, making, and entrepreneurship created by Dent Education in 2017 and designed for Baltimore City public high school students. The goals are to engage high school students in Baltimore in meaningful work, leadership, and personal development activities and, according to the program's developers, to "catalyze student potential and prepare them to create impact and value in a 21st century economy." Dent Education's mission is to "promote equity by empowering under-resourced youth to discover and develop their innate creative potential to shape the world around them." Through its curriculum, Dent Education seeks to introduce students to the creative problem-solving process of design thinking, maker skills, and the experience of entrepreneurship and networking. Students explore community issues and business challenges, meet with community and business professionals, learn new skills such as 3D printing and music production, and work to create their own social or business enterprise. During the summer of 2020, 10 cohorts of students (n = 138) participated in Bet on Baltimore through a five-week paid summer job opportunity funded by YouthWorks. The cohorts focused in the following areas: Graphic Design, Recording Studio, Social Innovation, Start-up Garage, Web Design, and Made@Dent. The five-week session culminated in a virtual showcase, where students presented their ideas and shared lessons learned for an online audience of more than 300 people. During the summer of 2020, 20 coaches (2 per site) led students through the program with the support of four site supporters and four Fellows.

(Purpose) The purpose of this study was to understand if inclusion of computer activities and videos for pre-K and kinder students, with opportunities to view these videos again at home, providing a repeated learning experience with language concepts and vocabulary, improved young students' early reading skill. (Methods) The study followed one cohort of students over two years. Students who were in preschool in the 2014-15 school year and who progressed into kindergarten in the 2015-16 school year were included. The outcome measures for the impact study were tests that gauge students' language development and early reading skills. These included the Woodcock-Johnson III Tests of Achievement subtests: Letter-Word ID and Word Attack, the Peabody Picture Vocabulary Test, and the Test of Language Development-4, Sentence Imitation subtest. (Results) There were no statistically significant differences between ATC and control students in early reading skills, as measured by the confirmatory outcome measure. We also found no statistically significant differences between ATC and control students on additional post-tests analyzed for exploratory purposes. (Implications) The findings of the impact study suggest that the evaluation study may have been underpowered and that the effect of ATC on some measures of early reading skills compared with the business-as-usual condition may be directionally positive on average but less than 0.20 standard deviations. More research is needed with larger sample sizes and lower minimally detectable effect sizes. Another limitation of this study is that the business-as-usual condition consisted of programs that are, at least in some ways, similar to the ATC intervention. Thus, the effect size of ATC may be much greater in a context where ATC participation is compared with a lower quality preschool or kindergarten program. Supplemental tables are appended.

The present study sought to examine the impact of the Sanford Harmony and Inspire programs as implemented in 10 "showcase" elementary schools during the 2018-19 school year. This study is a continuation of Phase I of the Sanford Harmony and Inspire Demonstration Site program evaluation, which was conducted during the 2017-18 school year (Morrison et al., 2017). As part of this project, 10 elementary schools in San Diego, California implemented the Sanford Harmony and Inspire programs during the 2018-19 school year. Five schools began implementing Sanford Harmony during the 2017-18 school year (Cohort One); five additional schools began implementing during the 2018-19 school year (Cohort Two). This study used a mixed-methods design to focus on how principals, teachers, and students within these schools reacted to the program and to determine whether or not program use was associated with positive outcomes such as improved student behavior, school climate, and relationships among students. The study also explored the extent to which program use and experiences changed or evolved during the second year of implementation. Results suggest that showcase school educators continue to hold positive impressions of Harmony and feel that the program plays an important role in helping foster students' social-emotional development. Teachers frequently reported that the program has been effective in helping students improve their relationship skills, empathy for others, acceptance of diversity, and social awareness. They also noted that Harmony has equipped students with the necessary tools for understanding others, including providing students with vocabulary and shared language to express their feelings. Students highlighted many lessons they have taken from participating in the program, and shared examples of instances where they applied Harmony teachings to problem-solve in social situations, effectively manage being angry or frustrated, or tried to be kind to someone else.

The purpose of the present study was to gather formative and summative data related to the FCPSOn initiative during its third year of implementation in the 2018-19 school year within Fairfax County Public Schools (FCPS). The present study documents program implementation in 15 Phase One schools and stakeholder feedback for future FCPSOn schools. Key components of FCPSOn include the distribution of personal laptops to all students in Phase One schools, professional development (PD), and the resulting impact on intermediary outcomes relating to the goals of improving students' content area knowledge and Portrait of a Graduate skills. The present findings from schools now in their third year (2018-19) of implementing FCPSOn indicate that stakeholders are strongly invested in the initiative's success and are more comfortable overall with the 1:1 learning environment. Important differences in implementation are observed between the two groups that comprise Phase One that appear to be the result of a more cohesive and intentional approach to implementation in the Chantilly Pyramid than in eLearning Backpack schools. Nonetheless, SBTSs and principals in both types of schools commended their teachers for being more effective users of technology tools and, as a result, more flexible and effective teachers. Findings also suggest that students are more engaged and accountable learners. Students have gained important skills and have embraced an emerging emphasis on self-guided learning. Guaranteed access to a personal computer has succeeded in "leveling the playing field" among students of varying socio-economic backgrounds, learning interests, and needs.

The purpose of the present study was to gather summative and formative data related to the implementation of Zearn Math during the first year of a two-year study. The sample included students in a large urban school district comprised of predominantly African American (58%) and Hispanic (28%) students. Roughly 90% of students are economically disadvantaged. As of the 2016-17 school year, only 8% of the district's students in Grades 3-8 met mathematics proficiency levels as compared to 40% of all students in the state. The current research was conducted during the district's second year of implementation of Zearn Math. During the 2016-17 school year, the district piloted the curriculum in two schools and provided some implementation support. The following year, 15 elementary schools began learning with Zearn Math and implementing the rotational classroom model, while the remainder of the elementary schools in the district continued to learn with EngageNY. This study explores the treatment schools' implementation of Zearn Math and the impacts of the shift during the first year of implementation.

The purpose of the present study is to evaluate the ongoing implementation and outcomes of the Students and Teachers Accessing Tomorrow (S.T.A.T.) initiative in Baltimore County Public Schools (BCPS) that began in the 2014-15 school year. Key components of S.T.A.T., as reflected in the evaluation model (see Figure 1 presented in the main report), include professional development and the resulting impact on measurable outcomes relating to the goals of improving student achievement and preparing globally competitive students. The mid-year evaluation report focused on teacher survey results, classroom observations, and student focus group findings to examine professional development offered by S.T.A.T. teachers and the impact on measurable outcomes. For this study's fifth and final year, the participant group was composed exclusively of Lighthouse elementary, Lighthouse middle, and Lighthouse high schools at the request of the district. This sampling approach intended to better investigate the initiatives' long-range impact on those schools with the most extensive integration of the S.T.A.T. program. At the mid-point of Year 5, the evaluation results continue to suggest that the S.T.A.T. initiative is being implemented effectively across Lighthouse schools and is making valuable progress in helping improve the perceived quality of instruction in the district. Importantly, classroom teachers believe that their S.T.A.T. teachers have played an integral role in helping them move instruction in a more learner-centered direction, and students believe that learning has become both easier and more engaging. Findings from classroom observations this fall, though mostly similar with those from previous time points, demonstrated some modest evidence of instructional changes. Teachers continue to make more extensive use of coaching and facilitating types of instruction than they do of more teacher-centered forms of instruction (i.e., presentations). Furthermore, Cohort 1 and 2 teachers, who make up those most experienced with S.T.A.T., showed visibly more frequent use of collaborative learning activities, higher-level questioning, higher-order instructional feedback, and flexible grouping arrangements, than those from Cohorts 3 and 4. These findings potentially suggest that more widespread implementation of these strategies may occur as teachers gain experience in the initiative.

School leaders are increasingly being asked, whether by rhetoric or policy, to measurably improve student achievement. The resultant need to assist school leaders in their ability to improve teaching and learning for all students in their schools led to the establishment of the National Institute of School Leadership's (NISL's) Executive Development Program. The NISL program emphasizes the role of principals as strategic thinkers, instructional leaders, and creators of a just, fair, and caring culture in which all students meet high standards. The current national focus on the importance of effective, instructional leadership has, in turn, led to calls for principal evaluation to be tied directly to student achievement (Davis, Kearney, Sanders, Thomas, and Leon, 2011). Within this milieu, effective and proven principal leadership development programs are crucial. (Contains 3 tables and 2 figures.) [This report was produced by the Center for Educational Partnerships, Old Dominion University.]

The National Institute of School Leadership's (NISL's) Executive Development Program (EDP) was established to provide professional development to school leaders to drive their schools to high performance. The program emphasizes the role of principals as strategic thinkers, instructional leaders, and creators of a just, fair, and caring culture in which all students meet high standards. Its primary goal is to ensure that the participating school leaders have the knowledge, skills, and tools to effectively set direction for teachers, support their staffs, and design an efficient organization. The curriculum is organized into four courses: "World-Class Schooling" (Principal as a Strategic Thinker and School Designer, Standards-Based Instruction), "Teaching and Learning," "Developing Capacity and Commitment," and "Driving for Results." Training sessions are designed to be highly interactive through the use of simulations and assignment of "pre-work" and "homework" to participants. This study examined the impact of EDP on student achievement in Pennsylvania schools from 2006-2010. It updates and extends a prior evaluation (Nunnery, Ross, and Yen, 2010a) study of this same cohort from 2006-2009, which found that elementary, middle, and high schools served by EDP principals had significantly larger gains in the percentages of students achieving proficiency in reading and mathematics. (Contains 1 footnote, 9 figures, and 4 tables.)

The present quantitative study evaluated the effects of Supplemental Educational Services (SES), a federally mandated component of No Child Left Behind, on student achievement in reading and mathematics. SES provides free tutoring outside of school to disadvantaged students who attend Title I schools in their third year of failing to achieve Adequate Yearly Progress on state assessments. In the present study, data from the fifth year of SES implementation in a large urban school were analyzed to determine if the mostly small and nonsignificant effects obtained in prior years were stronger as the tutoring services acquired additional refinement and maturity. A matched treatment-control group design was employed, in which students who received SES tutoring in reading, mathematics, or both were matched to similar schoolmates who were eligible for SES services but did not participate. Results showed consistently positive but small, nonsignificant effect sizes for the outcome measures. Student achievement results and its implication for policy and practice are discussed. (Contains 6 tables.)

This study examined the impact of NISL's [National Institute for School Leadership's] Executive Development Program for principals on student achievement in Pennsylvania schools between 2006-2009. Roughly half of the NISL-trained principals started the program during the 2007 school year and completed it in the 2008 school year, whereas the other half started during the 2008 school year and completed it in the 2009 school year. Schools served by principals participating in the Executive Development Program were individually matched to comparison schools with similar school performance and demographic profiles in 2006, which served as the baseline year. For elementary schools, it was possible to make individual school matches within the same school district for 36 school pairs. An additional 32 elementary schools were included in an out-of-district matched comparison sample. For all middle and high schools, it was necessary to match outside the school district. The percentages of students achieving proficient or above in mathematics and reading or English/Language Arts (ELA) across all grade levels were used to create aggregate school performance indices for each year 2006-2009. Repeated-measures analyses were performed to determine whether there were differences in school performance trends between schools served by NISL-trained principals and matched comparison schools. At the elementary level, statistically significant differences in school performance trends were observed between the matched within-district NISL and comparison schools in both mathematics and reading/ELA. For both subjects, NISL schools had statistically significantly higher rates of improvement in school performance than did comparison schools. Further, a significant quadratic effect was observed for mathematics, indicating an acceleration in the rate of improvement for NISL schools. In terms of unadjusted results, NISL schools gained 3.9% versus 0.7% for comparison sites in mathematics, and 4.1% versus 3.7% in reading/ELA. For the matched out-of-district elementary schools, significant trends favoring NISL over comparison schools were also indicated in both mathematics and reading/ELA. Significant quadratic effects in mathematics further revealed acceleration in the rate of growth over time for NISL schools. As with the elementary school results, statistically significant positive effects of NISL status were observed for both mathematics and reading in the middle school sample. Unadjusted results showed that NISL students improved the percentage of students achieving proficiency in mathematics by 5% from 2006 to 2009, compared to a 1% improvement in comparison schools. In reading/ELA, NISL middle schools improved the percentage of students achieving proficiency by 3% versus 1% in comparison schools. As with elementary schools, a significant quadratic effect on mathematics school performance trends revealed that the rate of improvement was accelerating over time in NISL schools. Two appendixes present: (1) Supplementary Tables; and (2) Evaluation Rubric for Selecting Program Participants. (Contains 9 tables and 17 figures.)

This report summarizes the 2006-2007 evaluation that was focused toward investigating one primary question: What changes in tool-based, student-centered teaching happen as a result of the infusion of technology and professional development? The research methodology involved the use of trained external researchers from Florida EETT schools conducting multi-class and targeted classroom observations in each participating school during two time periods: baseline (fall 2006) and end of year one (spring 2007). A total of 381 hours of direct classroom observations were conducted in 845 FL EETT classrooms in 41 schools representing 11 districts. Observation data were collected with the School Observation Measure (SOM[C]) and the Survey of Computer Use (SCU[C]). The SOM was used to collect data regarding overall classroom activities and the SCU was used to assess student use of computers. Both descriptive and inferential analyses were conducted. The Mantel-Haentzel procedure was used to infer statistical differences between the fall and spring classroom observations. Both the SOM and SCU Multi-Class and Targeted observations revealed significant fall to spring increases in the use of teacher-centered practices. For the SOM, significant increases were found for both the Multi-Class and Targeted observations for student engagement in "Project-based learning", "Independent inquiry/research on the part of students", and student use of "Technology as a learning tool or resource." The SCU results from both the Multi-Class and Targeted observations yielded significant increases in students' overall use of newer and more up-to-date computers (laptops) and positive trends toward increased uses of production tools and Internet/research tools to support learning. A key finding that emerged from the results was the significant increase in the frequency with which teachers implemented meaningful computer activities that engaged students in higher-order thinking and problem solving through effective use of laptop-based technology tools. These first year results show promising trends in that the Florida EETT program seems to be serving as a catalyst for positive changes from traditional teaching environments to ones that are student-centered and engage learners in meaningful use of computers to enhance learning. However, the data also reveal room for continued growth due to the modest frequency with which most of these changed practices occurred. An additional consideration when reviewing the evaluation results is the possible bias that may occur due to observer involvement in the Florida EETT program. (Contains 9 tables and 4 figures.)

The purpose of the present evaluation study was to examine the progress made in program implementation, school climate, and student achievement by Tennessee charter schools. Six evaluation questions guided the methodology for this study. Student achievement results are addressed in a separate report. The following evaluation questions are addressed in this report: (1) What is the frequency of usage of various traditional and alternative instructional strategies in the charter schools and compared to national norms? (2) What is the school climate at the charter schools and how does the climate compare to national norms? (3) To what degree and levels of quality are the goals and strategies of the charter school being implemented? (4) What are teacher reactions to and experiences in the charter school and the adequacy and quality of professional development and resources? and (5) What are parent/caregiver reactions to and experiences with the charter school? The study found that across schools and cohorts, teacher-centered instruction remained the dominant orientation. The rates of teacher-centered instruction were comparable to national norms in many cases, but continue to reflect limited success by the schools to implement the more innovative pedagogy described in benchmarks and instructional plans. Overall, school climate remains a definite strength of charter schools. The most advanced levels of implementation were observed among first cohort schools in their third year of operation. Across schools, the strongest levels of implementation tended to be for benchmarks targeting support and organization; lower levels of implementation were more apparent in the areas of curriculum, instruction, and evaluation. Teachers' reactions to and experiences in the charter schools tended to be very positive: responses related to support of the educational program, understanding of the mission, and the likelihood for student success were particularly favorable; most charter schools were also rated as strong in the area of professional development; teacher perceptions of the availability and adequacy of resources were more varied with most schools rated as moderate. All 12 charter schools were rated as strong with respect to parental satisfaction, findings were similar to those obtained in previous years. Based on the overall findings, the following recommendations apply to the charter schools as a group: (1) Charter schools might adopt a wider array of instructional orientations or strategies shown to promote student achievement; (2) Benchmark documents need to be modified to better align with objective indicators and available data: (3) Continued efforts to develop and maintain supportive, collaborative relationship with the school district and external partners; (4) Continued efforts to increase active parent involvement or participation; and (5) Prioritization, coordination, and securing of more resources. School Observation Measure Summary is appended. (Contains 10 tables.) [For Second Year Evaluation, see ED491146.]

The purpose of this study was to compare the achievement of students in 14 Pascagoula schools and 9 Biloxi schools implementing School Renaissance (SR) to those of students in matched Control schools. The achievement measures to be examined were from the 2003 administration of the Mississippi Curriculum Test (MCT) for Reading, Language Arts, and Mathematics in grades 2-8 and the Mississippi Writing Assessment (WA) in grade 4 and 7. The Pascagoula schools began implementing SR in January, 2002, and the Biloxi schools began implementation in August, 2000. Thus, the present analyses of 2003 achievement data relates to the first and third full years of SR in Pascagoula and Biloxi, respectively. Given that positive school climate is often a precursor to enacting successful school reform, a supplementary part of the study examined school climate in SR relative to Control schools. A matched program-control school quasi-experimental design was employed for this study. Participants included 10,160 students in grades 3 through 8 attending one of 40 schools in 15 school districts in southern Mississippi. Results suggest that SR is having positive effects on the participating schools. The Fourth Grade Mississippi Writing Assessment Scoring Rubric is appended.

Edvantia, Inc. (formerly AEL) publishes "TransFormation" twice a year to provide policymakers with interpretive summaries of significant reports and rigorous studies about school transformation, giving special attention to research with possible implications for improving low-performing schools.This final print issue of "TransFormation" presents findings from recent research and evaluation studies from Edvantia. Most of these studies took place in schools in Kentucky, Tennessee, Virginia, and West Virginia--the region served by the Appalachia Educational Laboratory at Edvantia. These investigations offer insights and evidence to inform policy discussions and decisions. Topics covered in this issue include: (1) School Performance Coaches; (2) Culturally Responsive Instruction; (3) Online Professional Development; (4) Teacher Education; (5) Improvement Specialists/Districtwide Intervention; (6) Extended School Services; and (7) Rural Teacher Recruitment and Retention. [This document was produced by the Appalachia Educational Laboratory at Edvantia, formerly the Appalachian Educational Laboratory (AEL, Inc.).]

This report summarizes the results of the Year 3 evaluation study of the Students and Teachers Accessing Real-time Knowledge (STARK) Program in the West Orange Consolidated Independent School District (CSID). The overall purpose of the evaluation was twofold: (a) to provide formative evaluation data to the participant schools to serve as a basis for improvement planning and as documentation of their accomplishments to demonstrate progress; and (b) to provide cumulative evidence of the implementation progress and outcomes of the participant schools as well as identification of exemplary programs. The context for the evaluation consists of 8 schools receiving "Foundation" training developed by the Brazos-Sabine Connection and integration training provided by West Orange Collaborative. The Foundation training deals directly with developing skills in using technology in the classroom. The integration training builds on this and emphasizes using technology in the classroom to support student-centered teaching methods that promote higher-level learning outcomes. The present 2003-2004 (Year 3) evaluation data were collected from 8 schools over the academic year. The data were compared to 2002-2003 (Year 2) and the 2001-2002 (Year 1) baseline conditions to gauge program implementation progress.

Four fifth grade classrooms embarked on a modified ubiquitous computing initiative in Fall 2003. Two 15-computer wireless laptop carts were shared among the four classrooms in an effort to integrate technology across the curriculum and affect change in student learning and teacher pedagogy. This initiative?in contrast to other 1:1 programs and stationary labs?offers public schools alternatives to budget constraints and instructional space overhead. Results indicate positive teacher technology competence and confidence, as well as instructional strategies that were student centered made meaningful uses of technology. Teacher pedagogical knowledge, technological knowledge and a supportive culture seem to be strong indicators for impacting technology integration in this context.

This report presents results from a planned comprehensive evaluation of the implementation of Different Ways of Knowing (DWoK) for the Middle Grades in "Cohort I" schools. Cohort I consists of 15 schools in California, Kentucky, New York, and Texas that began implementation of DWoK for the Middle Grades in 2001-02, and thus have completed their second year at this time. The planned comprehensive evaluation is comprised of four sub-studies. The first study is an analysis of results of a teacher survey, developed and administered by WestEd in the four Cohort I states. The second study synthesizes interview results obtained from DWoK for Middle Grades instructional and artist coaches and the team leader in one of the four states (KY). For this second study, the main generic findings across the five schools are summarized. In order to protect the promised confidentiality of the individual schools, the initial report ? which contained very specific information about each school - was submitted to Galef as a separate formative evaluation report for internal use. The third study presents results from analyses of student achievement, attendance, and school climate comparing three urban DWoK schools to three matched control schools in the Jefferson County Public School district (KY). The fourth study was an analysis of achievement at a DWoK rural school compared to matched rural control school.

This report discusses the findings from an evaluation study of the Learning Without Limits program in the fifth grade at Riverdale Elementary in the Shelby County Schools district, Memphis, Tennessee. The overall purpose of the evaluation was twofold: (a) to provide formative evaluation data to Riverdale School to use as a basis for improvement planning and to document their accomplishments to demonstrate progress; and (b) to provide cumulative evidence of the implementation progress and outcomes of the participant classes as well as identification of exemplary practice. The Learning Without Limits program was a pilot project designed to determine the impact of changing the ways students learn and teachers instruct in a technology-rich environment. This evaluation consisted of four fifth grade classes in which two Apple iBook wireless laptop cars were shared among the four fifth grade teachers' classrooms. In addition, the four teachers each received a personal Apple PowerBook laptop to use during the initiative and individually focused their professional development opportunities on technology-related training offered through Shelby County School. The Learning Without Limits program evaluation was structured around five primary questions that focused on classroom practice, degree and type of technology use, academically focused time, student engagement, teacher technology skills, teacher attitudes toward technology, as well as student and teacher reactions to programs.

In the present study, 49 fifth-graders enrolled the KIPP D.I.A.M.O.N.D. Academy in the 2002-03 school year were individually matched to control student from five feeder schools on the basis of ethnicity, free-reduced lunch status, and fourth-grade achievement on the Reading and Mathematics subtests of the Tennessee Comprehensive Assessment Program: Achievement Test (TCAP:AT). Although the KIPP and control students had virtually identical means on all fourth-grade tests, the KIPP students outperformed the control students on five out of the six fifth-grade tests. The one exception was the Writing assessment, on which KIPP and control group means were virtually identical. In four of the TCAP:AT tests (NRT-Reading, NRT-Math, CRT-Reading/Language Arts, CRT-Math) the comparisons were statistically significant with effect sizes ranging from +0.31 to +0.63. Across all 6 tests, the median adjusted ES was +0.31, indicating a moderately strong effect. These positive outcomes compare favorably with those from the most successful whole-school reform models (Borman, Hewes, Overman, & Brown, 2002). Findings are discussed with regard to the KIPP components and conditions likely to have increased academic focus and learning in the first implementation year. (Contains 3 tables.)

This report presents the findings of a study conducted by the Center for Research in Educational Policy (CREP) on the second year of implementation of the Small Schools Initiative at the Manual High School Complex. The major goal of this research was to evaluate perceived progress and outcomes at all three high schools. The design and methodology was oriented around the following research questions: (1) What is the level of school climate at the three schools for the 2002-2003 school-year, and how does it compare to the first year of implementation (2001-2002)? (2) What are the most positive and negative areas this year, as compared to last year? (3) How did the three schools perform this year on the CSAP, relative to district and state scores, and as compared to last year? (4) What is the level of implementation as viewed by teachers, and to what extent have perceptions changed this year, as compared to last year? (5) What are the strengths and weaknesses of the Small Schools Initiative this year, as compared to last year? and (6) What are recommendations for strengthening the programs in the coming year? The report begins with a brief overview of the schools, a summary of findings for each of the three high schools, an overall synthesis of the data, and final recommendations, which include: (1) Schools should use the benchmarking process regularly to monitor progress in achieving implementation and outcome goals; (2) Continued development and clarification of school identity; (3) Identification of aspects of the Small Schools Initiative having greatest impact and investigation of additional funding for these areas or new areas of need; (4) Meeting with key stakeholders to determine needs and preferences, and identify structures, for sharing resources; (5) Continued efforts to improve student behavior, motivation, and involvement. Encourage consistent enforcement of rules for student behavior and appropriate discipline; and (6) Maintaining and strengthening strategies that have produced strong positive climate indicators. Assessment instruments included: the School Climate Inventory[C] (SCI[C]), the School Wide Program Teacher Questionnaire[C] (SWPTQ[C]), the SWPTQ[C] Open-Ended Comments and Benchmark Items, and the School Observation Measure[C] (SOM[C]). Four appendixes include charted data for each assessment instrument. (Contains 6 footnotes and 8 figures.)

This report summarizes the Year 2 evaluation study results of the Students and Teachers Accessing Real-time Knowledge (STARK) Program in the West Orange Consolidated Independent School District (CSID). The overall purpose of the evaluation was twofold: (a) to provide formative evaluation data to the participant schools to serve as a basis for improvement planning and as documentation of their accomplishments to demonstrate progress; and (b) to provide cumulative evidence of the implementation progress and outcomes of the participant schools as well as identification of exemplary programs. The context for the evaluation consists of 10 schools receiving "Foundation" training developed by the Brazos-Sabine Connection and integration training provided by West Orange Collaborative. The Integration training places emphasis on employing technology in the classroom to support student-centered teaching methods that promote higher-level learning outcomes. It builds on Foundation training that deals directly with developing skills in using technology in the classroom. The present 2002-2003 evaluation data were collected from 10 schools over the academic year. The data were compared to 2001-2002 baseline conditions to gauge program implementation progress. (Contains 8 figures and 15 tables.) [For the 2003-2004 edition of this report, see ED484273.]

The No Child Left Behind (NCLB) stimulates school-wide reform by covering virtually all aspects of school operations including instruction, assessment, classroom management, professional development, parental involvement, school management, and curriculum. Since whole-school reforms typically change practices in the classroom, educators need a reliable classroom observation instrument that measures a wide variety of classroom practices. Historically, classroom observation instruments were designed for individual students, teachers, or classrooms, not the whole school. Additionally, the reliability of those instruments was shown to be context specific (e.g. length of observation, number of observations, timing in academic year). The School Observation Measure (SOM) is designed to measure a variety of classroom practices at the school level and the current reliability study was designed to incorporate many of the contextual variables elucidated by previous research. The current research study provides evidence of reliability for the SOM in a range of contexts. On average, the phi coefficient across all SOM items was 0.74 for five observations and 0.82 for eight observations at a school. Additionally, the percentage of variance attributable to observations was less than 1% while the percentage of variance attributable to differences between schools was 37% and the error term was 62% of five SOMs be conducted at a school to maintain adequate levels of reliability. Additional SOMs can be conducted at a school and tables are provided for educators to balance the cost of conducting additional SOMs with the asymptotic rate of return in increased reliability. Disaggregating reliability coefficients by elementary/high schools and rural/urban locations also evidenced reliability across contexts. The phi coefficients for five SOMs conducted at urban elementary schools averaged 0.72, compared to 0.76 for rural elementary schools. At high schools, the average phi coefficient associated with conducting only five SOMs was 0.60. Although reliabilities for high schools were somewhat lower than were those from elementary schools, this may be a statistical artifact due to the lower number of high schools in the sample compared to the number of elementary schools. It is recommended that a minimum of five SOMs be conducted at a school to maintain adequate levels of reliability. Additional SOMs can be conducted at a school and tables are provided for educators to balance the cost of conducting additional SOMs with the asymptotic rate of return in increased reliability. The SOM Instrument and Norms for SOM Items are appended. (Contains 8 tables.)

The present research is a third-party study of the effects of the School Renaissance (SR) comprehensive school reform (CSR) model on student achievement in 11 elementary and middle schools in Texas. The primary measures used in the study were the Texas Learning Index (TLI) reading and mathematics scores obtained through administration of the Texas Assessment of Academic Skills (TAAS). The context for the study was the McKinney Independent School District (MISD). Implementation of Reading Renaissance in MISD elementary schools began in the spring, 2000 and was mandated in all district schools in the fall, 2000. The mathematics program was subsequently mandated in all schools in January 2002. Given the cooperation of MISD, the Texas Education Agency (TEA), and superintendents from 10 other Texas school districts, a matched treatment-control pretest-posttest design was employed to address the following primary research questions: (1) How do SR schools compare to matched Comparison schools (C) in student achievement on the Texas Assessment of Academic Skills? (2) Do SR effects relative to C schools vary for (a) reading and mathematics and (b) elementary and middle schools? (3) Do SR effects vary for different types of students? and (4) Do SR effects relative to C schools vary as a function of the degree of SR model implementation? Major conclusions from the study include: (1) In the elementary grades, SR schools showed a clear trend of improved performance; (2) In the middle school grades, SR schools had noticeably improved performance relative to C schools when viewing separate cohorts in both reading and mathematics, although performance was significantly better only in mathematics in the 2002 8th grade longitudinal cohort; (3) Descriptive data showed trends for higher SR than C gains in both reading and mathematics for both 2001 and 2002 cohorts in the free or reduced-price lunch subgroup and in the limited-English-proficiency subgroup but the reverse was true for the 2002 cohort in the low-achieving subgroup; and (4) Implementation intensity was positively related to SR program effects on both reading and mathematics scores. As a whole, these results are suggestive of its benefits for student achievement, and if consistently replicated in future studies would strongly imply proven effectiveness as CSR model. Results also imply that program impacts are significantly more positive when implementation intensity is high. In view of the overall positive results obtained in this study, it is recommended that future research examine school environment variables such as school climate and teacher buy-in to better explain varied program effectiveness across schools and identify the optimum conditions for schools to realize benefits from SR adoption. Summary of Student Sample Sizes is appended. (Contains 47 figures and 17 tables.)

This paper reviews historical events concerning Memphis City Schools' educational attainments and reports research on changes in schools and in this high poverty district over time. In 1992, the new superintendent focused on improving student achievement and initiated site-based management. Critical partnerships were formed, and in 1995, New American Schools selected Memphis City Schools as a jurisdiction committed to adopting innovative school reforms. District leadership selected eight reform models (including ATLAS, Co-NECT, and Paideia). Elements common to the models included high student performance standards, strong teacher involvement, increased site-based professional development/planning time, and increased use of performance assessments. Schools emphasized cooperative learning, thematic units, student-centered instruction, integrated curricula, technology, and community and family involvement. Data from classroom visits, observations, and teacher interviews indicated that teaching and learning became more active in schools that were undergoing restructuring. The expanded restructuring in 1998-99 spotlighted Memphis as the only New American Schools jurisdiction to implement full systemic reform. When the superintendent left in 1999, commitment to the reform diminished at many schools. Lessons learned from the experience include: teacher buy-in is critical to success, school districts should be active in readying schools for change, one size does not fit all, and schools must own the reform. (Contains 25 references.) (SM)

The overall purpose of this evaluation study was to determine the effectiveness of providing 5th and 6th grade students in Walled Lake Consolidated Schools (WLCS) (Tennessee) with access to laptop computers with regard to classroom learning activities, technology usage, and writing achievement. The Laptop Program classrooms were equipped with wireless access to the Internet and printers. The program also provided students and parents the opportunity to receive training on basic computer skills. The evaluation was structured around three primary research questions: (1) Is teaching different in a Laptop classroom? (2) Do students behave differently in a Laptop classroom? and (3) Do students achieve differently in a Laptop classroom? Data were gathered through classroom observations, student writing test scores, student surveys and focus groups, teacher surveys and interviews, and parent surveys and interviews. In this evaluation of the first year of the Laptop Program, the results were consistently supportive of beneficial impacts on students, teachers, and parents. Specifically, all three groups believed that the program was positively changing teaching and learning both at school and at home. (MES)

This paper analyzes data on elementary schools in Memphis, Tennessee, that had reorganized 4 years prior to the study. The Tennessee Value-Added Assessment System (TVAAS) was used to provide performance scores showing students' year-to-year gains, whereas data analyzed for this paper were derived from scores on the TerraNova, the state-mandated achievement test, for five subjects over a 5-year period for grades 4-5. The research questions in this report include the following: How do restructuring schools compare in student-achievement gains to nonrestructuring schools over a 4-year period? and do different restructuring designs differ in achievement outcomes across the time period studied? Results indicate that in years 2 and 4 of implementation, restructured schools significantly outperformed nonrestructured schools in mathematics. In language, mathematics, science, and social studies, the restructured schools demonstrated significantly greater gain from pre-reform to post-reform than did nonrestructuring schools. The results suggest that incremental positive gains occurred in association with reforms, though most of the schools still had a long way to go to achieve high levels of student learning. Early analysis indicates that success in improving achievement appeared less a result of uniform reforms than of different designs appearing to work at particular schools and with groups of schools in given years. (Contains 18 references) (RJM)

The present evaluation study sought to determine how LightSail is implemented within schools in three public school districts, Evergreen Public Schools (EPS), Denver Public Schools (DPS), and Oakland Unified School District (OUSD). Case studies conducted in a subsample of schools within the three districts included classroom observations, interviews, and focus groups. In addition, a survey was administered to teachers and students in two of the districts and usage data were collected from all three districts. Teachers, principals, coaches, and students found great value in the LightSail experience. Students responded enthusiastically to the opportunity to express themselves through self-selection of texts and the option to read without interruption for a period of time.

During the fall of the 2016-17 school year, first, second, and third grade students in six elementary schools -- two in St. Paul and Eagan, Minnesota, and four in Bedford County in the Piedmont region of Virginia -- were assigned to take part in randomized efficacy trials carried out for the purpose of rigorously evaluating the impact of "Tutoring with the Lightning Squad" on reading outcomes. The study was designed to follow procedures that could qualify for "meets standards without reservation" in the What Works Clearinghouse (Standards 3.0). "Tutoring with the Lightning Squad" showed a significant and substantial impact on Woodcock measures of Passage Comprehension and Word Attack for students who received at least 25 sessions of tutoring. No significant differences were seen for Letter-Word Identification -- a result which can be attributed, at least in part, to the fact that this skill is specifically addressed only in the earliest levels of the program. Results did not vary for students by grade, gender, or ethnicity. In summary, this efficacy study suggests that "Tutoring with the Lightning Squad" has the potential to provide schools with a cost-effective tool for increasing reading levels, as the impacts of our program among students in small groups supervised by a paraprofessional tutor compare favorably with the much more expensive one-to-one tutoring models considered the most effective intervention for struggling readers.

Early observations of 13 culturally and linguistically diverse elementary schools, each of which is implementing one of six internally-developed school restructuring designs, are reported here. The schools are all located in one demographically diverse county, and the restructuring process occurred over a 4-year period. The study examined a range of processes and outcomes at each site through a combination of qualitative and quantitative methods. The report focuses exclusively on implementation issues. It examines variability in fidelity of implementation across sites, citing possible explanations for the variance and describing some of the factors that may affect the successful implementation of the six designs over time in the multilingual, multicultural schools. To illuminate how the various reform designs can be modified to provide high quality educational services to students in diverse contexts, the successes and challenges the schools are experiencing in adapting the designs to suit the students' needs are discussed, and conditions that facilitate and hinder implementation are identified. (MSE)

As part of litigation challenging the equity and adequacy of school funding in Alabama, educational resources and school conditions were examined in the highest and lowest funded school districts in the state. Identification of these districts revealed that all eight of the highest funded districts were urban systems, while all eight of the lowest funded districts were rural or county systems. Site visits to 45 schools in 15 of the above districts, interviews with principals, and a teacher survey produced completely consistent findings of clear disparities between rural and urban schools. Compared to urban schools, rural schools had less attractive physical plants and grounds, fewer educational resources in virtually all areas, fewer instructional offerings, and staffs that were more dispirited about their abilities to provide effective education under existing conditions. Particular educational and environmental disadvantages of rural schools included restricted opportunities for participating in outdoor athletics; discomforts caused by inefficient heating and cooling systems, old and dark school interiors, and dirty rest rooms lacking in basic supplies; cramped classrooms lacking sufficient textbooks and maps; and old and inadequate libraries and gymnasiums. In addition, teachers and principals reported staff and student involvement in fund raising, and the lack of funds for enrichment programs, subjects such as drama and foreign languages, professional development, and teacher compensation for extra work. Includes tables of statistical data. (SV)