Search

Initiatives to increase leadership accountability coupled with efforts to promote data-driven leadership have led to widespread adoption of instruments to assess school leaders. In this paper we present a decision matrix that practitioners and researchers can use to facilitate instrument selection. Our decision matrix focuses on the psychometric properties of the instrument; the model of leadership used to construct the instrument; the feasibility of implementation; and the extent to which feedback lends itself to changes in behavior (actionability). We apply this decision matrix to two of the most prominent leadership feedback instruments, CALL and VAL-ED. We find substantial similarities between the instruments, yet each has unique strengths and drawbacks. We discuss the implications of the comparative strengths and drawbacks of the two instruments and how they might be most effectively deployed to improve leadership practices.

Researchers at AAAS and BSCS have developed a six-week unit that aims to help middle school students learn important chemistry ideas that can be used to explain growth and repair in animals and plants. By integrating core physical and life science ideas and engaging students in the science practices of modeling and constructing explanations, the unit is designed to address major recommendations in national standards documents, including the National Research Council's "A Framework for K-12 Science Education" (2012). In this paper, the authors focus on the iterative design process used to select and refine a set of learning goals for the unit that target the three dimensions of science learning identified in the "Framework"--science core ideas, science practices, and crosscutting concepts. The paper also describes the data on alignment, classroom implementation, and student and teacher learning that informed the revision of the learning goals through three iterations of the unit. Numerous examples are provided to illustrate the kinds of design issues that arose and how they were resolved to address the challenges inherent in taking a standards-based approach to curriculum design. (Contains 5 tables and 3 figures.)

The search for the Holy Grail to measure learning gains started in the US, but the Organisation for Economic Co-operation and Development (OECD) wants to take it global. Here we tell a bit of this story and raise serious questions regarding the validity of the Collegiate Learning Assessment test and suggest there are alternatives. The merit of the CLA as a true assessment of learning outcomes is, we dare say, debatable. In part, the arrival and success of the CLA is a story of markets. In essence, it is a successfully marketed product that is fulfilling a growing demand with few recognized competitors. As a result, the CLA is winning the "learning outcomes race," essentially becoming the "gold standard" in the US. We worry that the CLA's early market success is potentially thwarting the development of other valuable and more nuanced alternatives--whether it be other types of standardized tests that attest to measuring the learning curve of students, or other approaches such as student portfolios, contextually designed surveys on the student experience, and alumni feedback. In a new study published in the journal "Higher Education", we examine the relative merits of student experience surveys in gauging learning outcomes by analyzing results from the data from the Student Experience in the Research University (SERU) Survey. This essay discusses some of the main points from that article. There are real problems with student self-assessments. But as we argue here, universities can probably learn more about learning outcomes in a wide range of disciplines via properly designed census surveys than by standardized tests like the CLA. At present, we suggest there is tension between the accountability desires of governments and the needs of individual universities who must focus on institutional self-improvement. One might hope that they would be synonymous. But how to make ministries and other policymakers more fully understand the perils of a silver bullet test tool?

Since 1983, the California Postsecondary Education Commission has conducted six studies of the university eligibility of high school graduates to estimate the percentage of California public high school graduates who meet the admission requirements of the California State University and the University of California. The Commission's two most recent eligibility studies were for the high school graduating classes of 2001 and 2003. This working paper recommends that the Commission proceed with an eligibility study for the class of 2007 to give up-to-date estimates of the percentage of California public high school graduates eligible for the University of California and the California State University. The text of the Commission Resolution to Proceed with a 2007 Study appears in this working paper. Appendix A presents recent changes to the University of California's eligibility requirements. Appendix B presents electronic data collection in the 2003 eligibility study.

In state fiscal year (SFY) 2000, states expended at least $20 billion on child welfare services, 20 percent more than in SFY 1998. This increase occurred despite the fact that the number of children reported as abused and neglected and the number of children in foster care declined. This suggests that not only are child welfare agencies spending more money than they have in the past, they are spending more money per-child. Yet criticism of the performance of our nation's child welfare system has continued unabated. Many researchers and policy makers have argued that the existing structure of federal child welfare financing limits the ability of states to develop and implement innovative programs, services, and administrative reforms to address performance weaknesses. Over the past decade, a number of federal fiscal reform proposals have been introduced in Congress or floated around on Capitol Hill. Most recently, President Bush's FY 2004 budget proposed legislation that would offer states an alternative financing system that would provide for greater flexibility. This paper summarizes what is known about states' financing of child welfare services. The paper identifies commonly cited problems with the federal child welfare financing structure, describes and assesses recently proposed solutions, and questions how financing data can be used in assessments of state performance. (Contains 22 references.)(Author)

The No Child Left Behind Act requires standards-based accountability for school districts and schools receiving Title I funds. A major component of this policy is to report whether districts and schools are making "adequate yearly progress" (AYP) based on their performance goals. This paper raises questions for rural schools using the National Assessment of Educational Progress (NAEP) mathematics scores from 35 states and state student assessment results from Maine between 1992 and 1996. Assuming the nation's rural students will make the same amount of gain every 4 years as they did between 1992 and 1996, the number of rural students at or above proficient in mathematics will rise to 53 percent by 2014, indicating that the AYP goal is not feasible. Overall statewide academic improvement in Maine was approximately 2 times larger using the state assessment than with the NAEP assessment. That the assessment used can make such a large difference raises questions of validity. Because smaller sample sizes inherently produce unreliable scores, the successive cohort comparison is highly unreliable as a measure of academic progress in small rural schools. By setting a uniform AYP target for every school, the current formula does not consider the influence of schools' initial performance status on their chance to meet the target, which brings the fairness of the AYP into question. Recommendations include lowering the target achievement level or extending the timeline to reach the level for disadvantaged schools, allowing the use of multiple measures to demonstrate school progress, using rolling averages to stabilize performance variations, and allowing individualized AYP targets according to baseline performance levels. (Contains 19 references.) (TD)

This report examines the potential of adaptive testing, two?-stage testing in particular, for improving the data quality of the National Assessment of Educational Progress (NAEP). Following a discussion of the rationale for adaptive testing in assessment and a review of previous studies of two-?stage testing, this report describes a 1993 Ohio field trial of two-?stage assessment carried out, under the direction of the authors, by the National Opinion Research Center (NORC). The trial was part of a larger methodological study of science assessment at school-?leaving age supported by the National Science Foundation (NSF) and the Office of Educational Research and Improvement (OERI). This report summarizes the instrument design, procedures, and results of the field trial and discusses implications for the conduct of assessment generally, and for NAEP specifically. A technical appendix outlines the measurement justification for two design prototypes and describes procedures used in analyzing the data. [This paper was commissioned by the NAEP Validity Studies (NVS) Panel May 1998.]

In spring 1997, Oregon's Regional Workforce Committee commissioned a study to investigate the feasibility of creating a technical and industrial skills training center in Region 12 of the state. Interviews were conducted with 40 individuals from area companies, social service agencies, and training organizations regarding the need for such a center and its functions. Results from the survey indicated that all respondents were interested in a partnership approach, which would entail participation by local organizations as training providers or training customers. The center should act as a regional broker of technical skills training courses, focusing on local training and skill development needs. The main constituencies to be served will include secondary students wanting to complete training while in high school, post-secondary students, the unemployed, and currently employed individuals wishing to enhance their career opportunities. The duration of training will depend upon participants' personal and career goals, ranging from 3 to 20 weeks of 40-hour-per-week instructional modules. Program offerings might include courses related to career progression skills, technical communication, fabrication technology, or office information management, while the center is expected to foster linkages with training agencies, social service agencies, and employers. Based on the interest expressed in the center, it is recommended that a meeting be held among school officials and potential partners to determine the center's economic potential, organization, location, programs, staffing, and funding sources. (JDI)

National concern about U.S. competitiveness in the world market has focused attention on the need to improve the work force. The two major 1992 presidential candidates have included training programs as important planks in their campaigns. President Bush has issued two proposals. Initially, he has charged in the Job Training 2000 proposal that the federally supported training programs are ineffective. At issue is administrative overhauling, not funding. In "The New Century Work Force," President Bush calls for additional funds for training programs. Governor Clinton's proposals advocate full funding of the Women, Infants, and Children (WIC) and Head Start programs as well as achieving a world-class education and helping disadvantaged parents build an ethic of learning at home. The recommendations for a comprehensive training system proposed in this document are the following: full funding of prenatal care, WIC, and Head Start; parenting training for Aid to Families with Dependent Children (AFDC), WIC, and Head Start parents; reviving career education emphasis in public education; expanding cooperative education and tech prep programs; assessing the feasibility of national education standards and skill certification systems; expanding existing Job Training Partnership Act and displaced worker efforts; funding a work-based welfare system buttressed by public sector jobs for AFDC recipients and other hard to employ persons; and mandating employer provision of employee and youth training. (YLB)

As part of an effort to provide more economical methods for job-skill evaluation in the Army Enlisted Personnel Management System, a study investigated the use of television as a means of presenting test items for a sample of tests from the job field of tank crewman. The television scene provided the job setting for each item and then posed a question that required a real-time response from the examinee. All items were job-connected and therefore represented simulated skill items. The examinee responded directly to the face of the television screen by means of an electronic stylus or gun reticle. The test was compared with a paper-and-pencil test which covered the same items and a hands-on performance test which covered many of the same items. The results indicated that television testing is very acceptable to soldiers and feasible. The validity of the prototype test could not be determined precisely because the criterion hands-on test was not usable; most examinees made a perfect score on the hands-on test. (Author/GDC)

The preliminary study explores the feasibility of a voucher system for purchase of skill training by participants in the Work Incentive Program (WIN) and describes the Portland, Oregon voucher system. Time limits on vouchers, voucherable occupations, dollar limits on vouchers, eligibility of clients, eligibility of vendors, timing and financial assistance for voucher decisions, and client and counselor role content are discussed. Data on personal characteristics of 167 voucher clients (about one-third of those enrolled in the 1974 local WIN program) are compared to data on 161 clients who had taken regular WIN training in Portland in 1973. This section contains tables and discussion comparing the two groups with reference to occupations and the schools, the training occupations, the training institutions, length and costs of training, and counselor experience with the voucher programs. Conclusions deal with attractiveness of the program, clients' decisions and levels of occupational choice, preference of training schools, use of allowable time and money, and cost comparison. Conclusions point generally to administrative feasibility, and plans are outlined briefly for followup studies. (MDW)

This report is one of five submitted as products of a "Study Designed to Assist Planning of Research on Significant Instructional Features in Bilingual Education Programs." The reports are intended to assist the National Institute of Education in its plans for a major new research study in bilingual education. The present report summarizes the results of 123 open interviews conducted with bilingual practitioners who are administrators in: Los Angeles, California; Miami, Florida; New York, New York; Oakland, California; and Rough Rock, Arizona. The questions asked were designed to elicit responses about the working definitions of terms and the designs considered. The intent was to test the credibility and acceptability to consumers of alternative study approaches. Among important concerns to emerge from the interviews are respect for administrative protocol of each educational organization, the need for a clear prospectus of the study, financial burden and disruption of educational process, clarification of benefits of the study, and the need to secure parental cooperation. (Author/JB)

A study explored the feasibility of research on the impact of flexible work patterns on the educational and training opportunities available to women in low-status jobs. Intended to provide a basis for a more informed discussion of policy questions pertaining to flexible worktime, the study involved a review of related literature, a series of informal interviews, and the collection of preliminary data at selected worksites during 1981. These sources of data confirm the fact that concern over declining productivity, together with demographic shifts and workers' changing values, have made it more important than ever to develop more effective opportunities for education and training and to expand the use of flexible work patterns. During the study, an examination was made of recent attempts at adjusting working hours for education and training such as the use of part time, combined part time and tuition aid, 40-hour flexible schedules, and informal flexibility. This examination revealed positive, albeit tentative, connections between flexible hours and employer-provided learning programs. However, it also suggests that this linkage will affect women in low-status jobs on a significant scale only if other conditions also prevail. Therefore, more comprehensive research on the impact of flexibility is needed. (MN)

Pursuant to Education Code 66903, the California Postsecondary Education Commission is charged with reviewing and commenting on the need for new academic programs at California's public colleges and universities and with making recommendations to the Legislature and the Governor. In September 2006, the Commission considered a proposal to establish a School of Law at the University of California, Irvine (UCI). Following a preliminary finding of non-concurrence, UC Irvine and the UC Office of the President submitted supplemental information in response to concerns raised by staff. This report contains a final analysis of all proposal information and supporting documents.

Introduction: Effective classroom management is critical to creating a classroom environment in which social, emotional, and academic learning can take place. The present study investigated the association between early career, early elementary teachers' occupational health (job stress, burnout, and perceived teaching ability) and perceptions of program feasibility in relation to their implementation dosage and quality of two evidence-based classroom management programs implemented together: the PAX Good Behavior Game (GBG) and MyTeachingPartner (MTP) intervention. Methods: Teachers provided information on their occupational health at the start of the school year and were then randomized to the PAX GBG + MTP condition or control condition. Teachers' perceptions of the feasibility of the program, implementation dosage, and implementation quality of the intervention were measured at the end of the school year for the 94 intervention teachers. Results: Teachers participated in more MTP coaching cycles when they reported that the combined PAX GBG + MTP program was feasible. Although there were no main effects of occupational health on implementation, the associations between job stress and implementation quality were moderated by perceptions of feasibility. Discussion: Findings highlight the complexity of factors influencing the implementation of evidence-based programs in school settings. [This paper was published in "Frontiers in Psychology" v14 2023.]