Search

This paper proposes a teacher effect change model in the form of a latent variable regression 5-level hierarchical model (LVR-HM5). Using multiple years of student achievement data, the LVR-HM5 attempts to simultaneously estimate teacher effect as well as teacher initial status and the gap parameter to model the change of such latent parameters over time. The gap parameter, the latent variable regression coefficient (Choi and Seltzer 2010; Choi and Kim 2019), captures the relationship between initial status and rates of changes within each year’s classroom. Furthermore, the LVR-HM5 allows us to model the teacher effect over time as a function of both time-varying and time-invariant characteristics. Such studies that focus on finding key correlates of teacher effect may have policy implications on teacher education, teacher professional development, and teachers’ instructional strategies that are potentially associated with improving teacher effectiveness.

This article proposes a latent variable regression four-level hierarchical model (LVR-HM4) that uses a fully Bayesian approach. Using multisite multiple-cohort longitudinal data, for example, annual assessment scores over grades for students who are nested within cohorts within schools, the LVR-HM4 attempts to simultaneously model two types of change, arising from individual student over grades, and successive cohorts in the same grade over years. In addition, as an extension of Choi and Seltzer, the LVR coefficients, that is, gap-in-time parameter, capturing the relationships between initial status and rates of changes within each cohort and school, help bring to light the distribution of student growth and differences in the distribution over different cohorts within schools. Advantages associated with the LVR-HM4 can be highlighted in studies on monitoring school performance or evaluations of policies and practices that may target different aspects of student academic performance such as initial status, growth, or gap over time in schools.

This review introduces classical item response theory (IRT) models as well as more contemporary extensions to the case of multilevel, multidimensional, and mixtures of discrete and continuous latent variables through the lens of discrete multivariate analysis. A general modeling framework is discussed, and the applications of this framework in diverse contexts are presented, including large-scale educational surveys, randomized efficacy studies, and diagnostic measurement. Other topics covered include parameter estimation and model fit evaluation. Both classical (numerical integration based) and more modern (stochastic) parameter estimation approaches are discussed. Similarly, limited information goodness-of-fit testing and posterior predictive model checking are reviewed and contrasted. The review concludes with a discussion of some emerging strands in IRT research such as response time modeling, crossed random effects models, and non-standard models for response processes.

This paper describes results from field testing of middle-school math formative assessments alongside professional development and instructional resources. We employed a randomised, controlled design to address the question: Does using our formative assessment strategies improve student performance on assessments of key mathematical ideas relative to a comparison group? This study also provided data on the instructional sensitivity of the assessments, which is part of the validation needed for formative assessments. Teachers were recruited from two districts and seven middle schools. Nineteen treatment and 17 comparison group teachers and their students were included in study analyses. Scores on extended response and short-answer questions indicated that students in the treatment group performed better than students in the comparison group who received the formative assessments alone. These findings demonstrate both the feasibility and value of including performance task-types in a brief assessment context.

Value-added models and growth-based accountability aim to evaluate school’s performance based on student growth in learning. The current focus is on linking the results from value-added models to the ones from growth-based accountability systems including Adequate Yearly Progress decisions mandated by No Child Left Behind. We present a new statistical approach that extends the current value-added modeling possibilities and focuses on using latent longitudinal growth curves to estimate the probabilities of students reaching proficiency. The aim is to utilize time-series measures of student achievement scores to estimate latent growth curves and use them as predictors of a dichotomous outcome, such as proficiency or passing a high-stakes exam, within a single multilevel longitudinal model. We illustrated this method through analyzing a three-year data set of longitudinal achievement scores and California High School Exit Exam scores from a large urban school district. This latent variable growth logistic model is useful for (1) early identification of students at risk of failing or of those who are most in need; (2) a validation or/and adequacy of student growth over years with relation to distal outcome criteria; (3) evaluation of a longitudinal intervention study.

The authors describe results from a study of a middle school mathematics formative assessment strategy. They employed a randomized, controlled design to address the following question: Does using our strategy improve student performance on assessments of key mathematical ideas relative to a comparison group? Eighty-five teachers and 4,091 students were included. Students took a pretest and a transfer measure at the end of the year. Treatment students completed formative assessments. Treatment teachers had exposure to professional development and instructional resources. Results indicated students with higher pretest scores benefited more from the treatment compared to students with lower pretest scores. In addition treatment students significantly outperformed control students on distributive property items. This effect was larger as pretest scores increased. Results, limitations, and future directions are discussed.

This paper investigates whether inferences about school performance based on longitudinal models are consistent when different assessments and metrics are used as the basis for analysis. Using norm-referenced (NRT) and standards-based (SBT) assessment results from panel data of a large heterogeneous school district, we examine inferences based on vertically equated scale scores, normal curve equivalents (NCEs), and nonvertically equated scale scores. The results indicate that the effect of the metric depends upon the evaluation objective. NCEs significantly underestimate absolute individual growth, but NCEs and scale scores yield highly correlated (r >.90) school-level results based on mean initial status and growth estimates. SBT and NRT results are highly correlated for status but only moderately correlated for growth. We also find that as few as 30 students per school provide consistent results and that mobility tends to affect inferences based on status but not growth – irrespective of the assessment ...

In studies of change in education and numerous other fields, interest often centers on how differences in the status of individuals at the start of a period of substantive interest relate to differences in subsequent change. In this article, the authors present a fully Bayesian approach to estimating three-level Hierarchical Models in which latent variable regression (LVR) coefficients capturing the relationship between initial status and rates of change within each of J schools (Bw j, j = 1, …, J) are treated as varying across schools. Specifically, the authors treat within-group LVR coefficients as random coefficients in three-level models. Through analyses of data from the Longitudinal Study of American Youth, the authors show how modeling differences in Bwj as a function of school characteristics can broaden the kinds of questions they can address in school effects research. They also illustrate the possibility of conducting sensitivity analyses using t distributional assumptions at each level of such models (termed latent variable regression in a three-level hierarchical model [LVR-HM3s]), and present results from a small-scale simulation study that help provide some guidance concerning the specification of priors for variance components in LVR-HM3s. They outline extensions of LVR-HM3s to settings in which growth is nonlinear, and discuss the use of LVR-HM3s in other types of research including multisite evaluation studies in which time-series data are collected during a preintervention period, and cross-sectional studies in which within-cluster LVR slopes are treated as varying across clusters.

Effective schools should be superior in both enhancing students’ achievement levels and reducing the gap between high- and low-achieving students in the school. However, the focus has been placed mainly on schools’ achievement levels in most school effect studies. In this article, we focused our attention upon the school-specific achievement dispersion as well as achievement level in determining effective schools. The achievement dispersion in a particular school can be captured by within-school variance in achievement (σ2). Assuming heterogeneous within-school variance across schools in hierarchical modeling, it is possible to identify school factors related to high achievement levels and a small gap between high- and low-achieving students. By analyzing data from the TIMMS-R, we illustrated how to detect variance heterogeneity and how to find a systematic relationship between within-school variance and school practice. In terms of our results, we found that schools with a high achievement level tended to be more homogeneous in achievement dispersion, but even among schools with the same achievement level, schools varied in their achievement dispersion, depending on classroom practices.

The No Child Left Behind Act (NCLB, 2002) establishes ambitious goals for increasing student learning and attaining equity in the distribution of student performance. Schools must assure that all students, including all significant subgroups, show adequate yearly progress (AYP) toward the goal of 100% proficiency by the year 2014. In this paper, we illustrate an alternative way of evaluating AYP that both emphasizes individual student growth over time and focuses on the distribution of student growth between performance subgroups. We do so through analyses of a longitudinal data set from an urban school district in the state of Washington. We also examine what these patterns tell us about schools that have been designated as meeting their AYP targets and those that have not. This alternative way of measuring AYP helps bring to light potentially important aspects of school performance that might be masked if we limit our focus to classifying schools based only on current AYP criteria. In particular, we are able to identify some schools meeting Washington state's AYP criteria in which above-average students are making substantial progress but below-average students making little to no progress. In contrast, other schools making AYP have below-average students making adequate progress but above-average students showing little gains. These contrasts raise questions about the meaning of “adequate” progress and to whom the notion of progress refers. We believe that closely examining the distribution of student progress may provide an important supplementary or alternative measure of AYP.

Studying change in student achievement is of central importance in numerous areas of educational research, including efforts to monitor school performance, investigations of the effects of educational interventions over time, and school effects studies focusing on how differences in school policies and practices relate to differences in student progress. In this article, we argue that in studying patterns of change, it is often important to consider the relationship between where students start (i.e., their initial status) and how rapidly they progress (i.e., their rates of change). Drawing on recent advances in growth modeling methodology, we illustrate the potential value of such an approach in the context of monitoring school performance. In particular, we highlight the ways in which attending to initial status in analyses of student progress can help draw attention to possible concerns regarding the distribution of achievement within schools. To convey the logic of our approach and illustrate various analysis possibilities, we fit a series of growth models to the time series data for students in several schools in the Longitudinal Study of American Youth (LSAY) sample. In a final section, we discuss some of the possibilities that arise in employing a modeling approach of this kind in evaluating educational programs and in conducting school effects research.

Much work on sensitivity analysis for hierarchical models (HMs) has focused on level-2 outliers (e.g., in multisite evaluations, a site at which an intervention was unusually successful). However, efforts to draw sound conclusions concerning parameters of interest in HMs also require that we attend to extreme level-1 units (e.g., a person in the treatment group at a particular site whose post-test score [yij] is unusually small vis-á-vis the other members of that person’s group). One goal of this article is to examine the ways in which level-1 outliers can impact the estimation of fixed effects and random effects in HMs. A second goal is to outline and illustrate the use of Markov Chain Monte Carlo algorithms for conducting sensitivity analyses under t level-1 assumptions, including algorithms for settings in which the degrees of freedom at level 1 (v1) is treated as an unknown parameter.

This paper presents a strategy for specifying latent variable regressions in the hierarchical modeling framework (LVR-HM). This model takes advantage of the Structural Equation Modeling (SEM) approach in terms of modeling flexibility—regression among latent variables—and of the HM approach in terms of allowing for more general data structures. A fully Bayesian approach via Markov Chain Monte Carlo (MCMC) techniques is applied to the LVR-HM. Through analyzing the data from a longitudinal study of educational achievement, gender difference are explored in the growth of mathematical achievement across grade 7 through grade 10. Allowing for the fact that initial status effect to rates of change may differ for girls and boys, the LVR-HM is specified in a way that rates of change parameters are modeled as a function of initial status parameters and the interaction between initial status and gender.