Search

This study investigates, through a cluster-randomized trial, the effectiveness of two approaches to increasing middle school students' science learning using a traditional science curriculum. Ninety schools were randomly assigned into one of three arms: (a) a treatment arm in which the textbook curriculum was modified based on four principles of cognitive science coupled with teacher professional development (PD), (b) a second treatment arm in which teachers received PD designed to improve their knowledge of the science content, and (c) a business-as-usual control group. The PD was able to change teacher practice but barely improved teacher knowledge. No significant boost in student achievement was observed, except in a few instances, where there were some promising findings. Exploratory analyses were conducted to examine what makes the interventions more effective. Implications for future research were discussed.

Disrupted sleep has been linked to suicidal thoughts and behavior. Less is known, however, about the underlying mechanisms of this relationship. A more nuanced understanding of the link between sleep and suicide may help inform treatment decisions and the development of prevention and intervention strategies. The present study examined daily average sleepiness as a moderator to the relation between same-day passive and active suicide ideation (SI). Fifty-nine young adults (mean age = 21.04; SD = 2.22) endorsing SI at least twice in the two weeks prior to baseline completed 3–5 daily surveys of sleepiness and SI over 2 weeks as part of a broader study. Across several indicators of sleepiness (desire to stay awake, desire to fall asleep), passive SI (desire to die, desire to live), and active SI (occurrence, intensity, duration, and controllability), the overall findings demonstrated that daily average sleepiness magnified the relation between same-day passive SI and active SI severity. These findings indicate that being sleepier than usual may increase the likelihood that passive SI transitions to active SI. Future research is needed to test the causal influence of sleepiness on this transition. • Sleepiness is related to both passive and active suicide ideation. • Sleepiness moderates the relation between passive and active suicide ideation. • Results are consistent across various indicators of sleepiness and passive ideation. [ABSTRACT FROM AUTHOR]

The Vanderbilt Assessment for Leadership in Education (VAL-ED) is a 360-degree learning-centered behaviors principal evaluation tool that includes ratings from the principal, supervisors, and teachers. The current study assesses the test-retest reliability of the VAL-ED for a sample of seven school districts as part of multiple validity and reliability assessments based on various samples of real users of the VAL-ED. We administered the VAL-ED twice and examined the correlations and mean differences between time 1 and time 2. We find that the principal and teacher ratings from time 1 and time 2 have large, positive, and significant correlations. Additionally, for both time points, principals are rated as being at least satisfactorily effective. Principals rate themselves slightly higher at time 2, while teachers rate principals slightly higher at time 1.

Mentoring is a common form of support for beginning teachers. State and district mentoring policies vary along a number of dimensions, yet policymakers have little evidence to draw on in designing effective mentoring programs. We use quantitative and qualitative data from a study of beginning middle school mathematics teachers in 10 districts to investigate the relationship of mentor policy features with the quality of mentoring received by new teachers. We find a strong and significant association indicating that mentors who have time during the day to meet with their new teachers are associated with higher-quality mentoring interactions. These results are supported through teacher interviews, and interviews also suggest that teachers experience better outcomes when mentors are located in the same school and have an evaluative role. These results suggest that proximity and availability may be particularly important features if mentors are to provide high-quality support to new teachers.

Recent years have seen the convergence of two major policy streams in U.S. K-12 education: standards/accountability and teacher quality reforms. Work in these areas has led to the creation of multiple measures of teacher quality, including measures of their instructional alignment to standards/assessments, observational and student survey measures of pedagogical quality, and measures of teachers' contributions to student test scores. This article is the first to explore the extent to which teachers' instructional alignment is associated with their contributions to student learning and their effectiveness on new composite evaluation measures using data from the Bill & Melinda Gates Foundation's Measures of Effective Teaching study. Finding surprisingly weak associations, we discuss potential research and policy implications for both streams of policy.

The Vanderbilt Assessment of Leadership in Education (VAL-ED) provides educators with a tool for principal evaluation based on principal, teacher, and supervisor reports of principals' learning-centered leadership. In this study, we conduct a known group analysis as part of a larger argument for the validity of the VAL-ED in US elementary and secondary schools. We asked superintendents to select the principals in their district who they believe in performance of their duties are in the top 20% and the bottom 20%. We ask how accurately VAL-ED scores can identify membership of the two known groups. Using a discriminant analysis, the VAL-ED places principals in the superintendent groups, on average, 70% of the time for both elementary and secondary schools. Placement accuracy is greater for the top group than the bottom group.

Performance standards are a powerful way to communicate K-12 student achievement (e.g., proficiency) and are the cornerstone of standards-based reform. As education reform shifts the focus to college and career readiness, approaches for setting performance standards need to be revised. We argue that the focus on assessing student readiness can move performance standards toward an increasingly empirical grounding, leading to more meaningful and understandable standards for student achievement and better guideposts for instructional improvement. Specifically, we describe and illustrate the processes and practices associated with "evidence-based standard setting," taking as our example a specific application in the American Diploma Project Algebra II End-of-Course Exam, which was developed as an indicator of college readiness. (Contains 1 table, 1 figure, and 4 notes.)

Purpose: Performance evaluation informs professional development and helps school personnel improve student learning. Although psychometric literature indicates that a rational, sound, and coherent standard-setting process adds to the credibility of an assessment, few studies have empirically examined the decision-making process. This article sheds light on the inner workings of setting cut scores for leadership proficiency from the deliberative perspectives of expert-panel participants, including principals, teachers, district supervisors, and researchers. Research Design: Qualitative methods are used to observe and document a standards-setting process as it takes place and to analyze the results. Findings: The findings indicate that setting cut scores for principal leadership proficiency is a cognitively demanding but achievable process for effective implementation of carefully designed procedures. The study also provides insight regarding the influence of external factors, such as backgrounds of panelists, consideration of school contexts, and concerns about consequence during the standard-setting deliberation. (Contains 3 tables and 4 notes.)

Recent years have seen increased attention to the high school-college transition. Students are enrolling in college in record numbers, yet they are also taking increasing numbers of remedial courses. How to measure and report on academic readiness for college is an important policy issue receiving attention from the National Assessment Governing Board, Achieve, and several states. The focus of this article is how to create a measure of academic readiness for college, either by building and validating a new assessment or validating and repurposing an existing assessment. After first describing the disjuncture between high school and college and discussing definitions and indicators of academic readiness, the authors identify four strategies that might be used to create a readiness assessment. The pros and cons of each strategy are discussed. (Contains 1 note.)

Coherence is the core principle underlying standards-based educational reforms. Assessments aligned with content standards are designed to guide instruction and raise achievement. The authors investigate the coherence of standards-based reform's key instruments using the Surveys of Enacted Curriculum. Analyzing 138 standards-assessment pairs spread across grades and the three No Child Left Behind tested subjects, the authors find that roughly half of standards content is tested on the corresponding test and roughly half of test content corresponds to the standards. A moderate proportion of test content is at the wrong level of cognitive demand as compared to the corresponding standards, and vice versa. Between 17% and 27% of content on a typical test covers topics not mentioned in the corresponding standards. Policy and research implications are discussed. (Contains 5 tables and 3 figures.)

The Vanderbilt Assessment of Leadership in Education (VAL-ED) is a multirater assessment of principals' learning-centered leadership. The instrument was developed based on the Standards for Educational and Psychological Testing. In this article, we report on the validity and reliability evidence for the VAL-ED accumulated in a national field trial. Using data from more than 270 schools across a wide range of settings, we found that the instrument was reliable for measuring principals' learning-centered leadership. Furthermore, there was mostly positive evidence that the VAL-ED distinguished among subscales of principal performance. In conjunction with findings from the VAL-ED development phase, these results support the conclusion that the VAL-ED can be used by K-12 schools to assess learning-centered leadership.

Research has consistently shown that principal leadership matters for successful schools. Evaluating principals on the behaviors shown to improve student learning should be an important leverage point for raising leadership quality. Yet principals are often evaluated with the use of instruments with no theoretical background and little, if any, documented psychometric properties. To address this need, a team of researchers in principal leadership, assessment development, and psychometrics developed the Vanderbilt Assessment of Leadership in Education (VAL-ED). The purpose here is to report on iterative development work where the instrument was tested and revised across several cycles. Future work to investigate the instrument's psychometric properties is identified. After an extensive item writing and instrument development phase, the authors embarked on a series of studies designed to guide improvements to the instrument. These studies include a sorting study, two rounds of cognitive interviews, a bias review, and two rounds of small- scale pilot tests. Results and implications from each study are discussed. The iterative development process helped improve the clarity of instructions and items while building a growing collection of preliminary validity and reliability evidence. At the end of the development process, the VAL-ED represents a promising instrument for assessing principal instructional leadership.The VAL-ED also represents a tool for possible use by principal leadership researchers in measuring the effectiveness of school principals. (Contains 9 tables, 8 figures, and 1 note.)

The Vanderbilt Assessment of Leadership in Education is a 360-degree assessment of the effectiveness of principals' learning-centered leadership behaviors. In this report, we present results from a differential item functioning (DIF) study of the assessment. Using data from a national field trial, we searched for evidence of DIF on school level, geographic region, and urbanicity. We found evidence of intercept DIF for urbanicity on four items and slope DIF for urbanicity on one item, although all magnitudes were small to moderate. We conclude that the Vanderbilt Assessment of Leadership in Education's items are not biased on the basis of these school characteristics, bolstering its use in schools around the country. (Contains 3 notes, 2 figures and 4 tables.)

In this article we present results of a comprehensive review of principal leadership assessment practices in the United States. Our analyses of both the general content and the usage of 65 instruments, 56 at the district level and 9 at the state level, provided an in-depth look at what and how districts and states evaluate principals. Using the learning-centered leadership framework, we focused on identifying the congruence (or lack thereof) between documented assessment practices and the research-based criteria for effective leadership that are associated with improved school performance. Using an iterative and deductive process for instrument content analysis, we found that states and districts focused on a variety of performance areas (e.g., management, external environment, or personal traits) when evaluating their principals, with different formats at various levels of specificity. We also found very limited coverage of leadership behaviors that ensure rigorous curriculum and quality instruction, which are linked with schoolwide improvement for the ultimate purpose of enhanced student learning. In seeking information on how principals are evaluated, we found that in most cases, the practices of leadership assessment lacked justification and documentation in terms of the utility, psychometric properties, and accuracy of the instruments. (Contains 4 figures, 2 tables and 1 note.)

State content standards are the backbone of the standards-based reform movement. Content standards provide teachers with a set of guidelines for what students are expected to know and be able to do, defining the intended curriculum. And although the current 50-state system of education gives each state the task of setting content standards, there has been little empirical investigation of the similarities and differences among state content standards. This analysis uses the content analysis procedures of the Council of Chief State School Officers/State Collaboratives on Assessment and State Standards to consider whether there exists a de facto national curriculum as defined in state content standards. Data from English/language arts and reading (ELAR), science, and mathematics for Grades 4, 8, and K-8 are used. Results suggest considerable variability among states in the content of content standards, particularly in individual grades, but also for the aggregated standards. Further analysis suggests that state standards are no more well aligned to national professional standards (i.e., National Science Education, National Council of Teachers of Mathematics) than to other states' standards. Still, there exists a small "core curriculum" across states in each content area. The level of focus of the state standards varies substantially across states, and the redundancy of the standards is such that alignment within state, across grades is often as high as alignment within grade, across states. Policy implications are briefly discussed. (Contains 6 tables and 7 figures.)

This article examines the reliability of content analyses of state student achievement tests and state content standards. We use data from two states in three grades in mathematics and English language arts and reading to explore differences by state, content area, grade level, and document type. Using a generalizability framework, we find that reliabilities for four coders are generally greater than 0.80. For the two problematic reliabilities, they are partly explained by an odd rater out. We conclude that the content analysis procedures, when used with at least five raters, provide reliable information to researchers, policymakers, and practitioners about the content of assessments and standards.

With the exception of the procedures developed by Porter and colleagues (Porter, 2002), other methods of defining and measuring alignment are essentially limited to alignment between tests and standards. Porter's procedures have been generalized to investigating the alignment between content standards, tests, textbooks, and even classroom instruction as experienced by teachers and students. They have also led to the development of tools that hold promise for innovative uses in both research and the improvement of education practice. The purpose here is to define and illustrate innovative uses of two of the tools from Porter's work, content maps and a quantitative index of the degree of alignment. The intention is to put these tools in the hands of education researchers.

Formal mentoring is the cornerstone of most teacher induction programs, but teachers receive support from informal mentors as well. The authors report on findings from a study of 57 first-year mathematics teachers in 11 school districts to describe who teachers' formal and informal mentors are, how teachers work with them, and how the functions of formal and informal mentors serve for beginning teachers. They offer suggestions for how schools and districts can harness the power of these two forms of mentoring by embracing both.

The No Child Left Behind Act allows states to vary (a) the trajectories they select to move from the baseline percent proficient or above in 2002 to the 100% proficient goal in 2014, (b) the minimum number of students required for reporting of disaggregated subgroup results, and (c) whether or not they will use confidence intervals when determining whether or not an annual measurable objective has been met. We use data from Kentucky for the years 2003 and 2004 to explore the consequences of different design decisions. The effect of design decisions on number and percentage of schools meeting adequate yearly progress (AYP) is large, with important implications for education practice.

This chapter traces the development of, and evolving audiences for, a set of research tools developed at the University of Wisconsin-Madison by the authors and others between 1989 and 2003, with roots extending back to work Porter and his colleagues at Michigan State conducted during the 1980s. Originally designed for use in investigating questions concerning the effects of policies on classroom practice, these tools are now just as likely to be used by a state educational agency to examine alignment issues between state content standards and assessments, by school leadership teams to inform curriculum decision making, or by curriculum specialists to plan professional development at the district or school level. The central requirement for addressing all of these information needs is a systematic, rich, and multidimensional language for describing instructional content. The tools described in this chapter provide such a language, based on a two-dimensional model of instructional content consisting of topics and cognitive demand. (Contains 3 figures and 1 table.)

The purpose of this article is to put in the hands of researchers, practitioners, and policy makers a powerful framework for building and studying the effects of high-quality assessment and accountability programs. The framework is illustrated through a description and analysis of the assessment and accountability program in the School District of Philadelphia.

Examines extent to which all 50 states implemented performance assessment in making professional licensure decisions as of 2001-02; considers differences among various types of assessments and different uses thereof. Describes three challenges to low state implementation of performance-based licensure: costs; need to ensure validity, reliability, and fairness; and effects on teacher supply. (Contains 38 references.) (Authors/PKP)