Search

The gold standard in making causal inference on program effects is a randomized trial. Most randomization designs in education randomize classrooms or schools rather than individual students. Such "clustered randomization" designs have one principal drawback: They tend to have limited statistical power or precision. This study aims to provide empirical information needed to design adequately powered studies that randomize schools using data from Florida and North Carolina. The authors assess how different covariates contribute to improving the statistical power of a randomization design and examine differences between math and reading tests; differences between test types (curriculum-referenced tests versus norm-referenced tests); and differences between elementary school and secondary school, to see if the test subject, test type, or grade level makes a large difference in the crucial design parameters. Finally they assess bias in 2-level models that ignore the clustering of students in classrooms. (Contains 5 figures, 46 tables and 2 footnotes.)

This paper provides practical guidance for researchers who are designing and analyzing studies that randomize schools--which comprise three levels of clustering (students in classrooms in schools)--to measure intervention effects on student academic outcomes when information on the middle level (classrooms) is missing. This situation arises frequently in practice because many available data sets identify the schools that students attend but not the classrooms in which they are taught. Do studies conducted under these circumstances yield results that are substantially different from what they would have been if this information had been available? The paper first considers this problem in the context of planning a school-randomized study based on preexisting two-level information about how academic outcomes for students vary across schools and across students within schools (but not across classrooms in schools). The paper next considers this issue in the context of estimating intervention effects from school-randomized studies. Findings are based on empirical analyses of four multisite data sets using academic outcomes for students within classrooms within schools. The results indicate that in almost all situations one will obtain nearly identical results whether or not the classroom or middle level is omitted when designing or analyzing studies. Three-Level vs. Two-Level Model Comparisons: Estimated R-Squared for School- and Student-Level Covariates are appended. (Contains 9 tables and 2 footnotes.)

The purpose of this paper is to inform the community about teacher appraisal methods in the School District of Philadelphia, outline the difficulties of the current system, and suggest approaches that would strengthen the teacher appraisal process. The authors gathered their information over three months in mid-2009 from multiple sources: interviews with 7 principals and 13 teachers in the city, interviews with 5 education advocates, observations of community meetings; official documents on state, district, and charter school policies; news articles; and pertinent academic and policy publications. (Contains 2 tables and 7 footnotes.) [Funding for this report was also provided by the Samuel S. Fels Fund.]

Providing information on assessment and institutional effectiveness programs in South Carolina, this two-part document describes the use of performance indicators (PI's) and presents state guidelines for institutional effectiveness. The first part discusses assessment in South Carolina, indicating that institutions are required to develop measure to assess their effectiveness and prepare annual reports on specific PI's. This part also presents the following principles for using PI's: (1) the factor addressed and its relation to an appropriate goal must be clear; (2) a context should be provided for the PI; (3) diagnostic information should accompany the PI; (4) PI's are most useful when considered in groups; (5) while indicators are increasingly being used to determine institutional funding, the processes may not be sufficiently sophisticated or depoliticized to work; and (6) PI's will not lead to improvements if they are simply used for institutional ranking. The second part provides guidelines for institutional effectiveness assessments developed by the South Carolina Commission on Higher Education, indicating that assessment is required with respect to general education; student knowledge of their disciplines or majors; performance of graduates on licensing and certification exams; program changes due to external program evaluations; alumni follow-up studies; remedial programs; achievement of transfer students; undergraduate retention and attrition; minority student and faculty access and equity; academic performance of student athletes; library usage; administrative and financial performance; facilities usage; public service; and research. Appendixes to this section include an assessment schedule from 1988-1992, goals for quality assessment, and the state legislation authorizing the assessment programs. (KP)

In 1989, the Texas State Board of Education undertook the responsibility for developing a 4-year plan and for establishing a vision for education that extends into the next century. This document contains the goals for education established by the 71st Texas legislature. The mission of public education in Texas is for it to be characterized by quality, equity, and accountability. Policymakers envision a state whose first priority is children. To meet the challenges, the board established nine general goals. These fall under the headings of student learning, curriculum and programs, personnel, organization and management, finance, parent responsibility, community and business partnerships, research development and evaluation, and communications. A review of the social and economic conditions of Texas precedes a statistical analysis. For each general goal, specific legislatuve and state goals are listed. An outline describes the action steps to be taken to meet these goals and objectives at the state, regional, and local levels. Procedures for evaluation that form a critical yardstick of achievement in measuring performance are identified. Finally the national goals for education, how the plan was developed, and a compliance statement are given. Several bar graphs are included. (RR)

Considers the existence of halo effects in individuals' evaluations of target communicators across different dimensions. Notes that halo effects result from raters' inability to discriminate among conceptually distinct and theoretically independent aspects of a target's behavior. Discusses current conceptions of halo error and suggests several potential sources of halo error. (SG)

Discusses strategies designed to increase the instrumental use of program evaluation findings based on experiences in Australian schools. Highlights include an evaluation plan; awareness of the evaluation; timely reporting techniques; guidelines for implementation of findings; and the role of the evaluator as change agent. (Contains 21 references.) (LRW)