Search

This paper is a summary of the evaluation of the second complete year of the implementation of the Alabama Reading Initiative (ARI), which is designed to achieve 100% literacy by targeting reading performance of beginning reading and first-grade students, to expand the reading power for second- through twelfth-grade students and to intervene for struggling readers at all grade levels. In its second year of implementation, approximately 27,700 students, 2,354 teachers, 81 principals, 75 reading specialists, 64 higher education partners, and 221 preservice teacher education programs were directly involved in the ARI. The evaluation used information from student achievement data from surveys completed by teachers, reading specialists, principals, higher education partners, and higher education reading faculty. Both qualitative and quantitative data were analyzed. A pretest-posttest design was used to examine change over time for student outcomes. Results and recommendations regarding delivery of the ARI are: a large percentage of principals, teachers, reading specialists, and higher education partners reported the ARI had substantial positive impacts on student literacy; the principal should receive sufficient support, direction, and encouragement because their leadership role was related to gains in reading scores; the visibility and involvement of the reading specialist and higher education partner influenced teachers' attitudes toward the helpfulness of these individuals; higher education faculty in the program are often confronted with inadequate or competing reward structures; teachers involved in the first 2 years of the program felt that continued professional development would help sustain their enthusiasm; the Stanford Achievement Test, Level 9 is not the most appropriate measure for monitoring progress toward program goals; tests currently used to assess early literacy skills of kindergarten through second-grade students are inappropriate for use as tools for evaluating gains associated with ARI; and case studies of particularly effective schools should be conducted. (RS)

A study examined the effectiveness and the impact of preservice instruction of the second year of the Alabama Reading Initiative (ARI), which is designed to achieve 100% literacy by targeting reading performance of beginning reading and first-grade students, to expand the reading power for second- through twelfth-grade students, and to intervene for struggling readers at all grade levels. In its second year of implementation, approximately 27,700 students, 2,354 teachers, 81 principals, 75 reading specialists, 64 higher education partners, and 221 preservice teacher education programs were directly involved in the ARI. The evaluation used information from student achievement data from surveys completed by teachers, reading specialists, principals, higher education partners, and higher education reading faculty. Both qualitative and quantitative data were analyzed. A pretest-posttest design was used to examine change over time for student outcomes. Results and recommendations regarding delivery of the ARI are: a large percentage of principals, teachers, reading specialists, and higher education partners reported the ARI had substantial positive impacts on student literacy; the principal should receive sufficient support, direction, and encouragement because their leadership role was related to gains in reading scores; the visibility and involvement of the reading specialist and higher education partner influenced teachers' attitudes toward the helpfulness of these individuals; higher education faculty in the program are often confronted with inadequate or competing reward structures; teachers involved in the first 2 years of the program felt that continued professional development would help sustain their enthusiasm; the Stanford Achievement Test, Level 9 is not the most appropriate measure for monitoring progress toward program goals; tests currently used to assess early literacy skills of Kindergarten through second-grade students are inappropriate for use as tools for evaluating gains associated with ARI; and case studies of particularly effective schools should be conducted. (Contains 25 tables and a figure of data.) (RS)

This report summarizes research supports addressing three areas as targets for educational improvement in Alabama: beginning reading, expansion of reading power, and intensive intervention for those who are reading below grade level. The paper also describes Alabama's statewide initiative to implement best practices in these areas and summarizes the results of the reading improvement imitative for the year academic 1999-2000. The Alabama Reading Initiative (ARI), a collaborative model, was based on an extensive literature review and teacher training in summer institutes to help teachers implement research-based reading instruction. Implementation began at 16 schools named as Literacy Demonstration Sites for the 1998-1999 school year. In the second year of implementation, the year of this study, approximately 27,700 students, 2,354 teachers, 81 principals, 75 reading specialists, 64 higher education partners, and 21 preservice teacher education programs participated. The year-2 evaluation involved internal and external evaluation components that included surveys completed by 1,860 teachers, 77 principals, 74 reading specialists, 44 higher education partners, and 49 higher education faculty members. Overall findings include small but positive improvement on Stanford 9 reading test results for ARI students. Teachers at ARI schools reported increases in the use of research-based strategies and in student reading time, greater confidence and enthusiasm toward instruction, and improved student and teacher attitudes. Schools varied in the improvement noted on the Stanford 9 results, and findings suggested three factors linked to a greater degree of success: (1) the principal as a champion of ARI; (2) a hands-on and helpful reading specialist; and (3) a deeply involved higher education partner. (Contains 18 references.) (SLD)

This study compared effect sizes applied to raw, scaled, and normal curve equivalent (NCE) data. Recommendations for the interpretation of effect sizes vary. For example, some authors suggest that an effect size below 0.50 is small, between 0.50 and 1.00 is moderate, and above 1.00 is large. These are products of the criterion formally used by the U.S. Department of Education's Joint Dissemination Review Panel and the Program Effectiveness Panel. it is clear from the context of these articles that it is assumed that they were dealing with raw scores or scaled scores, not NCEs. NCE scores for individual students and, particularly, mean NCE scores for schools would not be expected to change from year to year without some type of intervention. This study computed effect sizes for the raw, scaled scores, and NCEs by school for grades 4, 6, and 8 on a national norm-referenced test for 749, 574, and 464 schools respectively representing 120,149 students. The results show that, as expected, the effect sizes for NCE scores were lower than those for raw and scaled scores. These results suggest that when rules-of-thumb for effect sizes are presented, they should take into account the type of metric on which it is being applied. (Contains 3 figures, 4 tables, and 23 references.) (SLD)

This paper describes the evaluation of the Renaissance Project for Improving Teacher Quality. The Renaissance Group is a national consortium of 16 colleges and universities committed to improving teacher education. Ten of these institutions, in collaboration with their schools of arts and sciences and their partner schools, have engaged in the 5-year project to reform their teacher education programs. The project includes a comprehensive evaluation component that responds to the unique character of each program. The study includes performance data from teacher candidates and partner school students; teacher candidate ratings by higher education and partner school faculty; classroom observations within teacher education and partner school classrooms; surveys and interviews with teacher candidates, higher education and partner school faculty, and business partners on project accomplishments; demographic data; document analysis; and site visits. Year 1 accomplishments include developing a system to track candidate performance at admissions, pre-student teaching, exit, and first year; developing a mentoring team model with adaptations for each site; identifying local business partners; and developing a dedicated project Web site. Three appendixes present: the overall 5-year project objectives and progress indicators; year 1 project activities and evaluation tasks; and a description of the project. (SM)

Locating and selecting an instrument that measures resilience is no simple task. This document provides information about several measures of resilience or hardiness that have been used in recent years. The discussion of each measure includes information about its origins, a description of the measure and its uses, and a discussion of the psychometric properties of the measure. The following are described: (1) Personal Views Survey III (Hardiness Institute, 1985); (2) Cognitive Hardiness Scale (K. Nowack 1989); (3) Psychological Hardiness Scale (S. Younkin and N. Betz, 1996); (4) Resilience Scale (G. Wagnild and H. Young, 1993); (5) Resiliency Scale (C. Jew, K. Green, and J. Kroger, 1999); (6) Personal Resilience Questionnaire and Organizational Resilience Questionnaire (D. Conner, 1993); and (7) Family Hardiness Index (H. McCubbin, A. Thompson, and M. McCubbin, 1996). (Contains 24 references.) (SLD)

Results obtained using three methods for gathering attitude data were compared. The methods are ranking, paired comparisons, and the Likert-type scale. Three attitude objects, each consisting of five items, were selected or developed for this study. One set had been used with graduate students previously. Participants were 392 students in education courses, most of whom were female, and most of whom were education majors. Correlations among the methods were very high for the set of items related to characteristics of good teachers. Also high were the correlations between the rankings and paired-comparisons methods for the other two sets of items. Correlations between the Likert-type items and the other two methods were not as dramatic or consistent for these two sets of items. Although the study was limited to three small sets of items, results provide preliminary evidence for agreement among methods under certain conditions. When items have been subjected to established conventions for item development and selection and they meet criteria for good items, high agreement among methods might be expected. The items are presented in an appendix. Eight tables are included. (Contains 9 references.) (SLD)

Six item bias detection techniques (distractor analysis, the Mantel-Haenszel procedure, Angoff's delta plot, Angoff's modified delta plot, Stricker's partial correlation index, and direct comparison of item difficulties) were compared for their ability to identify biased items at three stages of test development. The techniques were applied to items administered during the spring 1989 item pilot (n=about 1,000 for each of 36 forms), the fall 1989 form pilot (n=about 460 for each of three tests), and the spring 1990 final test version (n=about 46,000 each for the reading, mathematics, and language tests) of the Grade 9 Alabama Basic Competency Tests. Four subgroup pairs were analyzed for the reading, mathematics, and language tests; each administration; and each technique. The subgroup pairs included black versus white females, black versus white males, black females versus black males, and white females versus white males. The method with the greatest stability across administrations was the direct difficulty difference, followed closely by distractor analysis. The Mantel-Haenszel procedure and the partial correlation index both showed weak to moderate stability. The poorest results concerning stability were found for both Angoff techniques. The modified Angoff version is not a desirable technique, since it can identify inappropriate items. Results suggest the possible use of more than one index to examine a set of items. A 60-item list of references is included. (SLD)

Several analyses were performed as part of the process of validating a recently developed scale to measure the attitudes of students toward writing. The Florida Writing Project Student Survey is a 25-item measure of writing attitudes designed for use with students in grades six through twelve. It was developed to assess the attitudes of students who had participated in the Florida Writing Project. The evaluation consisted of six analyses for reliability, item analysis, correlation between the attitude scale and holistic writing scores, and the survey's sensitivity to change. Further analyses are recommended as a means of refining the survey and providing additional validation information. (DWH)

The role of educational research is examined, focusing on its relationship to the classroom teacher's efforts to promote student learning. After a discussion of the relationship between education and psychology, the individual roles of educational research are discussed; the latter include increasing and disseminating knowledge and assessing student performance. Seven criteria for avoiding failure to use research and evaluative information are discussed; these criteria are associated with historical relevance, the diagnosis of a specific problem, environmental support, organizational structure and individual attitudes, manipulability of the factor, political and economic feasibility, and effectiveness. In order to be more effective in impressing the importance of research upon policymakers and practitioners, researchers must make a better effort to understand cognition or the "action frames of mind" of such potential research users. (TJH)