Search

Intersectional approaches have become increasingly important for explaining educational inequalities because they help to improve our understanding of how individual experiences are shaped by simultaneous membership in multiple social categories that are associated with interconnected systems of power, privilege, and oppression. For years, there has been a call in psychological and educational research for quantitative approaches that can account for the intersection of multiple social categories. The present paper introduces the Multilevel Analysis of Individual Heterogeneity and Discriminatory Accuracy (MAIHDA) approach, a novel intersectional approach from epidemiology, to study educational inequalities. The MAIHDA approach uses a multilevel model as the statistical framework to define intersectional strata that represent individuals' membership in multiple social categories. By partitioning the variance within and between intersectional strata, the MAIHDA approach allows identifying intersectional effects at the strata level as well as obtaining information on the discriminatory accuracy of these strata for predicting individual educational outcomes. Compared to conventional quantitative intersectional approaches, MAIHDA analyses have several advantages, including better scalability for higher dimensions, model parsimony, and precision-weighted estimates of strata with small sample sizes. We provide a systematic review of its past application and illustrate its use by analyzing inequalities in reading achievement across 40 unique intersectional strata (combining the social categories of gender, immigrant background, parental education, and parental occupational status) using data from 15-year-old students in Germany (N = 5451). We conclude that the MAIHDA approach is a valuable intersectional tool to study inequalities in educational contexts.

There is no final consensus regarding which covariates should be used (in addition to prior achievement) when estimating value-added (VA) scores to evaluate a school's effectiveness. Therefore, we examined the sensitivity of evaluations of schools' effectiveness in math and language achievement to covariate selection in the applied VA model. Four covariate sets were systematically combined, including prior achievement from the same or different domain, sociodemographic and sociocultural background characteristics, and domain-specific achievement motivation. School VA scores were estimated using longitudinal data from the Luxembourg School Monitoring Programme with some 3600 students attending 153 primary schools in Grades 1 and 3. VA scores varied considerably, despite high correlations between VA scores based on the different sets of covariates (0.66 < r < 1.00). The explained variance and consistency of school VA scores substantially improved when including prior math and prior language achievement in VA models for math and prior language achievement with sociodemographic and sociocultural background characteristics in VA models for language. These findings suggest that prior achievement in the same subject, the most commonly used covariate to date, may be insufficient to control for between-school differences in student intake when estimating school VA scores. We thus recommend using VA models with caution and applying VA scores for informative purposes rather than as a mean to base accountability decisions upon.

Descriptive analyses of socially important or theoretically interesting phenomena and trends are a vital component of research in the behavioral, social, economic, and health sciences. Such analyses yield reliable results when using representative individual participant data (IPD) from studies with complex survey designs, including educational large-scale assessments (ELSAs) or social, health, and economic survey and panel studies. The meta-analytic integration of these results offers unique and novel research opportunities to provide strong empirical evidence of the consistency and generalizability of important phenomena and trends. Using ELSAs as an example, this tutorial offers methodological guidance on how to use the two-stage approach to IPD meta-analysis to account for the statistical challenges of complex survey designs (e.g., sampling weights, clustered and missing IPD), first, to conduct descriptive analyses (Stage 1), and second, to integrate results with three-level meta-analytic and meta-regression models to take into account dependencies among effect sizes (Stage 2). The two-stage approach is illustrated with IPD on reading achievement from the Programme for International Student Assessment (PISA). We demonstrate how to analyze and integrate standardized mean differences (e.g., gender differences), correlations (e.g., with students' socioeconomic status [SES]), and interactions between individual characteristics at the participant level (e.g., the interaction between gender and SES) across several PISA cycles. All the datafiles and R scripts we used are available online. Because complex social, health, or economic survey and panel studies share many methodological features with ELSAs, the guidance offered in this tutorial is also helpful for synthesizing research evidence from these studies.

The present integrative data analysis examined gender differences in achievement, achievement profiles, and achievement motivation in mathematics, reading, and science among 113,864 top-performing adolescent math students (top 5% in their respective countries). To do this, we applied the same analysis protocol to representative individual participant data from six cycles of the Programme for International Student Assessment (PISA 2000-2015; 82 countries) and integrated the results by using meta-analytical random coefficient models. We found that in the group of top-performing math students, male students were overrepresented (mean female-to-male ratio 1:1.50, 95% CI [1:1.58, 1:1:43]). Furthermore, female students possessed better reading skills (mean d = -0.23, 95% CI [-0.25, -0.21]) and more positive reading attitudes (-0.64, 95% CI [-0.69, -0.60] [less than or equal to] mean d [less than or equal to] -0.38, 95% CI [-0.46, -0.30]). Male students had stronger math self-efficacy (mean d = 0.32, 95% CI [0.28, 0.35]) and demonstrated mathematics-oriented achievement profiles, whereas female students' profiles were more balanced across domains. Moreover, female students were more interested in organic and medical fields (-0.44, 95% CI [-0.48, -0.40] [less than or equal to] mean d [less than or equal to] -0.30, 95% CI [-0.34, -0.25]), whereas male students showed greater interest in physics-related topics (0.39, 95% CI [0.36, 0.43] [less than or equal to] mean d [less than or equal to] 0.54, 95% CI [0.50, 0.58]). Gender equality indicators moderated the proportion of female students in the top 5% in mathematics and explained variability in achievement profiles across countries. Results are explained by social role theory and situated expectancy-value theory, and implications for women's underrepresentation in (specific) STEM fields are discussed.

It is well-documented that academic achievement is associated with students' self-perceptions of their academic abilities, that is, their academic self-concepts. However, low-achieving students may apply self-protective strategies to maintain a favorable academic self-concept when evaluating their academic abilities. Consequently, the relation between achievement and academic self-concept might not be linear across the entire achievement continuum. Capitalizing on representative data from three large-scale assessments (i.e., TIMSS, PIRLS, PISA; N = 470,804), we conducted an integrative data analysis to address nonlinear trends in the relations between achievement and the corresponding self-concepts in mathematics and the verbal domain across 13 countries and 2 age groups (i.e., elementary and secondary school students). Polynomial and interrupted regression analyses showed nonlinear relations in secondary school students, demonstrating that the relations between achievement and the corresponding self-concepts were weaker for lower achieving students than for higher achieving students. Nonlinear effects were also present in younger students, but the pattern of results was rather heterogeneous. We discuss implications for theory as well as for the assessment and interpretation of self-concept.

Differentiation hypotheses concern changes in the structural organization of cognitive abilities that depend on the level of general intelligence (ability differentiation) or age (developmental differentiation). Part 1 of this article presents a review of the literature on ability and developmental differentiation effects in children, revealing the need for studies that examine both effects simultaneously in this age group with appropriate statistical methods. Part 2 presents an empirical study in which nonlinear factor analytic models were applied to the standardization sample (N = 2,619 German elementary schoolchildren; 48% female; age: M = 8.8 years, SD = 1.2, range 6-12 years) of the THINK 1-4 intelligence test to investigate ability differentiation, developmental differentiation, and their interaction. The sample was nationally representative regarding age, gender, urbanization, and geographic location of residence but not regarding parents' education and migration background (overrepresentation of children with more educated parents, underrepresentation of children with migration background). The results showed no consistent evidence for the presence of differentiation effects or their interaction. Instead, different patterns were observed for figural, numerical, and verbal reasoning. Implications for the construction of intelligence tests, the assessment of intelligence in children, and for theories of cognitive development are discussed.

The structure of academic self-concept (ASC) is assumed to be multidimensional and hierarchical. This methodological review considers the most central models depicting the structure of ASC: a higher-order factor model, the Marsh/Shavelson model, the nested Marsh/Shavelson model, a bifactor representation based on exploratory structural equation modeling, and a first-order factor model. We elaborate on how these models represent the theoretical assumptions on the structure of ASC and outline their inherent psychometric properties. We analyzed these models using a data set of German 10th-grade students (N = 1,232) including a wide range of domain-specific ASCs as well as general ASC. The correlations among ASCs and between ASCs and academic achievement varied depending on the structural model used. We conclude with discussing recommendations for research purposes and advantages and limitations of each ASC model. Our approach may also guide research on other affective or motivational constructs (e.g., academic anxiety or interest).

To plan cluster-randomized trials with sufficient statistical power to detect intervention effects on student achievement, researchers need multilevel design parameters, including measures of between-classroom and between-school differences and the amounts of variance explained by covariates at the student, classroom, and school level. Previous research has mostly been conducted in the United States, focused on two-level designs, and limited to core achievement domains (i.e., mathematics, science, reading). Using representative data of students attending grades 1-12 from three German longitudinal large-scale assessments (3,963 [less than or equal to] N [less than or equal to] 14,640), we used three- and two-level latent (covariate) models to provide design parameters and corresponding standard errors for a broad array of domain-specific (e.g., mathematics, science, verbal skills) and domain-general (e.g., basic cognitive functions) achievement outcomes. Three covariate sets were applied comprising (a) pretest scores, (b) sociodemographic characteristics, and (c) their combination. Design parameters varied considerably as a function of the hierarchical level, achievement outcome, and grade level. Our findings demonstrate the need to strive for an optimal fit between design parameters and target research context. We illustrate the application of design parameters in power analyses.

Differentiation of intelligence refers to changes in the structure of intelligence that depend on individuals' level of general cognitive ability (ability differentiation hypothesis) or age (developmental differentiation hypothesis). The present article aimed to investigate ability differentiation, developmental differentiation, and their interaction with nonlinear factor analytic models in 2 studies. Study 1 was comprised of a nationally representative sample of 7,127 U.S. students (49.4% female; M[subscript age] = 14.51, SD = 1.42, range = 12.08-17.00) who completed the computerized adaptive version of the Armed Service Vocational Aptitude Battery. Study 2 analyzed the norming sample of the Berlin Intelligence Structure Test with 1,506 German students (44% female; M[subscript age] = 14.54, SD = 1.35, range = 10.00-18.42). Results of Study 1 supported the ability differentiation hypothesis but not the developmental differentiation hypothesis. Rather, the findings pointed to age-dedifferentiation (i.e., higher correlations between different abilities with increasing age). There was evidence for an interaction between age and ability differentiation, with greater ability differentiation found for older adolescents. Study 2 provided little evidence for ability differentiation but largely replicated the findings for age dedifferentiation and the interaction between age and ability differentiation. The present results provide insight into the complex dynamics underlying the development of intelligence structure during adolescence. Implications for the assessment of intelligence are discussed.

The aim of educational policy should be to provide a good education to all students. Thus, a key question arises regarding the extent to which key characteristics of school composition (proportion of students with migration background, socioeconomic status [SES], prior school achievement, and achievement heterogeneity), instructional quality, school quality, and later school achievement are interrelated. The present study addressed this research question by examining school inspection data, official school statistics, and large-scale achievement data from all primary schools in Berlin, Germany (N=343). The results of correlation and path analyses showed that school composition (average SES, average prior school achievement) predicted components of instructional quality (SES: classroom management, cognitive activation; achievement: cognitive activation, individual learning support). The relation between school composition characteristics and most components of school quality was close to zero. Contrary to our expectations, only the effect of school SES on later achievement was mediated by instructional quality.

Value-added (VA) modeling can be used to quantify teacher and school effectiveness by estimating the effect of pedagogical actions on students' achievement. It is gaining increasing importance in educational evaluation, teacher accountability, and high-stakes decisions. We analyzed 370 empirical studies on VA modeling, focusing on modeling and methodological issues to identify key factors for improvement. The studies stemmed from 26 countries (68% from the USA). Most studies applied linear regression or multilevel models. Most studies (i.e., 85%) included prior achievement as a covariate, but only 2% included noncognitive predictors of achievement (e.g., personality or affective student variables). Fifty-five percent of the studies did not apply statistical adjustments (e.g., shrinkage) to increase precision in effectiveness estimates, and 88% included no model diagnostics. We conclude that research on VA modeling can be significantly enhanced regarding the inclusion of covariates, model adjustment and diagnostics, and the clarity and transparency of reporting.

Background: Achievement in math and achievement in verbal school subjects are more strongly correlated than the respective academic self-concepts. The internal/external frame of reference model (I/E model; Marsh, 1986, "Am. Educ. Res. J.," 23, 129) explains this finding by social and dimensional comparison processes. We investigated a key assumption of the model that dimensional comparisons mainly depend on the difference in achievement between subjects. We compared correlations between subject-specific self-concepts of groups of elementary and secondary school students with or without achievement differences in the respective subjects. Aims: The main goals were (1) to show that effects of dimensional comparisons depend to a large degree on the existence of achievement differences between subjects, (2) to demonstrate the generalizability of findings over different grade levels and self-concept scales, and (3) to test a rarely used correlation comparison approach (CCA) for the investigation of I/E model assumptions. Samples: We analysed eight German elementary and secondary school student samples (grades 3-8) from three independent studies (Ns 326-878). Method: Correlations between math and German self-concepts of students with identical grades in the respective subjects were compared with the correlation of self-concepts of students having different grades using Fisher's Z test for independent samples. Results: In all samples, correlations between math self-concept and German self-concept were higher for students having identical grades than for students having different grades. Differences in median correlations had small effect sizes for elementary school students and moderate effect sizes for secondary school students. Conclusions: Findings generalized over grades and indicated a developmental aspect in self-concept formation. The CCA complements investigations within I/E-research.

In many countries, students are asked about their perceptions of teaching in order to make decisions about the further development of teaching practices on the basis of this feedback. The stability of this measurement of teaching quality is a prerequisite for the ability to generalize the results to other teaching situations. The present study aims to expand the extant empirical body of knowledge on the effects of situational factors on the stability of students' perceptions of teaching quality. Therefore, we investigate whether the degree of stability is moderated by three situational factors: time between assessments, subjects taught by teachers, and students' grade levels. To this end, we analyzed data from a web-based student feedback system. The study involved 497 teachers, each of whom conducted two student surveys. We examined the differential stability of student perceptions of 16 teaching constructs that were operationalized as latent correlations between aggregated student perceptions of the same teacher's teaching. Testing metric invariance indicated that student ratings provided measures of teaching constructs that were invariant across time, subjects, and grade levels. Stability was moderated to some extent by grade level but not by subjects taught nor time spacing between surveys. The results provide evidence of the extent to which situational factors may affect the stability of student perceptions of teaching constructs. The generalizability of the students' feedback results to other teaching situations is discussed.

Young children, ages 5-6 years, develop first beliefs about science and themselves as science learners, and these beliefs are considered important precursors of children's future motivation to pursue science. Yet, due to a lack of adequate measures, little is known about young children's motivational beliefs about learning science. The present two-part study explores the motivational beliefs of young children using a new measure--the Young Children's Science Motivation (Y-CSM) scale. Initial measurement development involved a thorough literature review of existing measures, and an extensive piloting phase until a final instrument was reached. To establish scale reliability, measurement invariance as well construct and criterion validity, the final instrument was administered to a new sample of 277 young children, age 5-6 years, in northern Germany. Results reveal that children's motivational beliefs can be empirically differentiated into their self-confidence and enjoyment in science at this young age. Older children were more motivated in science, but no significant gender differences were found. Importantly, children in preschools with a science focus reported significantly higher science motivation. This finding stresses the importance of early science education for the development of children's motivational beliefs science.

To plan group-randomized trials where treatment conditions are assigned to schools, researchers need design parameters that provide information about between-school differences in outcomes as well as the amount of variance that can be explained by covariates at the student (L1) and school (L2) levels. Most previous research has offered these parameters for U.S. samples and for achievement as the outcome. This paper and the online supplementary materials provide design parameters for 81 countries in three broad outcome categories (achievement, affect and motivation, and learning strategies) for domain-general and domain-specific (mathematics, reading, and science) measures. Sociodemographic characteristics were used as covariates. Data from representative samples of 15-year-old students stemmed from five cycles of the Programme for International Student Assessment (PISA; total number of students/schools: 1,905,147/70,098). Between-school differences as well as the amount of variance explained at L1 and L2 varied widely across countries and educational outcomes, demonstrating the limited generalizability of design parameters across these dimensions. The use of the design parameters to plan group-randomized trials is illustrated.

We present factor extension procedures for confirmatory factor analysis that provide estimates of the relations of common and unique factors with external variables that do not undergo factor analysis. We present identification strategies that build upon restrictions of the pattern of correlations between unique factors and external variables. The first restriction minimizes the sum of squared correlations between unique factors and external variables. This approach is similar to the traditional factor extension procedure. The second restriction minimizes the complexity of the pattern of external correlations of unique factors. This approach has similarities with the simple structure ideal imposed on most factor rotation strategies. The procedures are illustrated with a real data example that demonstrates their applicability to real-world research questions.

To examine developmental processes, intervention effects, or both, longitudinal studies often aim to include measurement intervals that are equally spaced for all participants. In reality, however, this goal is hardly ever met. Although different approaches have been proposed to deal with this issue, few studies have investigated the potential benefits of individual variation in time intervals. In the present paper, we examine how continuous time dynamic models can be used to study nonexperimental intervention effects in longitudinal studies where measurement intervals vary between and within participants. We empirically illustrate this method by using panel data (N = 2,877) to study the effect of the transition from primary to secondary school on students' motivation. Results of a simulation study also show that the precision and recovery of the estimate of the effect improves with individual variation in time intervals. [ABSTRACT FROM AUTHOR]

To assess the meaningfulness of an intervention effect on students' achievement, researchers may apply empirical benchmarks as standards for comparisons, involving normative expectations for students' academic growth as well as performance gaps between weak and average schools or policy-relevant groups (e.g., male and female students, students from socioeconomically advantaged or disadvantaged families, students with or without a migration background). Previous research made these empirical benchmarks available for students in the United States. We expand this line of research by providing novel meta-analytic evidence on these empirical benchmarks for students attending elementary and secondary schools in Germany for a broad variety of achievement outcomes. Drawing on the results obtained for large probability samples, we observed variations in each kind of benchmark across countries as well as across and within domains and student subpopulations within Germany. Thus, the assessment of the very same intervention effect may depend on the target population and outcome of the intervention. We offer guidelines and illustrations for applying empirical benchmarks to assess the magnitude of intervention effects. [ABSTRACT FROM AUTHOR]