Search

When planning a cluster randomized trial, evaluators often have access to an enumerated cohort representing the target population of clusters. Practicalities of conducting the trial, such as the need to oversample clusters with certain characteristics in order to improve trial economy or support inferences about subgroups of clusters, may preclude simple random sampling from the cohort into the trial, and thus interfere with the goal of producing generalizable inferences about the target population. We describe a nested trial design where the randomized clusters are embedded within a cohort of trial-eligible clusters from the target population and where clusters are selected for inclusion in the trial with known sampling probabilities that may depend on cluster characteristics (e.g., allowing clusters to be chosen to facilitate trial conduct or to examine hypotheses related to their characteristics). We develop and evaluate methods for analyzing data from this design to generalize causal inferences to the target population underlying the cohort. We present identification and estimation results for the expectation of the average potential outcome and for the average treatment effect, in the entire target population of clusters and in its non-randomized subset. In simulation studies, we show that all the estimators have low bias but markedly different precision. Cluster randomized trials where clusters are selected for inclusion with known sampling probabilities that depend on cluster characteristics, combined with efficient estimation methods, can precisely quantify treatment effects in the target population, while addressing objectives of trial conduct that require oversampling clusters on the basis of their characteristics.

Multilevel models are often used to account for the hierarchical structure of social data and the inherent dependencies to produce estimates of regression coefficients, variance components associated with each level, and accurate standard errors. Social network analysis is another important approach to analysing complex data that incorporate the social relationships between a number of individuals. Extended linear regression models, such as network autoregressive models, have been proposed that include the social network information to account for the dependencies between persons. In this article, we propose three types of models that account for both the multilevel structure and the social network structure together, leading to network autoregressive multilevel models. We investigate theoretically and empirically, using simulated data and a data set from the Dutch Social Behavior study, the effect of omitting the levels and the social network on the estimates of the regression coefficients, variance components, network autocorrelation parameter, and standard errors.

Academic literacies (AL) research has made significant contributions to understandings of student writing and literacy across higher education and particularly learning development. However, researchers and practitioners both within and external to the AL movement have struggled to clarify the relationship between AL and pedagogy. English for Academic Purposes researchers have highlighted the lack of a workable AL pedagogy, whilst AL researchers maintain that the model represents a design space or heuristic for thinking about practice in context, rather than a source of pedagogic prescriptions. This theoretical discussion elaborates concerns with the structural coherence of the AL model, its broadly social constructivist underpinnings and evidence base, and the impact of its ideological orientation on the pedagogy we derive from it. Underpinning these critiques is a suspicion that the interpretation of social constructivist epistemology on which AL relies to pinpoint weaknesses in the models of literacy/writing which it subsumes cannot generate a practical pedagogy. We argue that these structural and ideological tensions in the AL model help to explain confusion over its interpretation and implementation. We speculate that this singular focus on social constructivist-derived theory, though well-intentioned, does more to reinforce a particular ideological commitment than to enhance student learning.

This investigation examines the efficacy of multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) over fixed-effects models when performing intersectional studies. The research questions are as follows: (1) What are typical strata representation rates and outcomes on physics research-based assessments? (2) To what extent do MAIHDA models create more accurate predicted strata outcomes than fixed-effects models? and (3) To what extent do MAIHDA models allow the modeling of smaller strata sample sizes? We simulated 3,000 data sets based on real-world data from 5,955 students on the LASSO platform. We found that MAIHDA created more accurate and precise predictions than fixed-effects models. We also found that using MAIHDA could allow researchers to disaggregate their data further, creating smaller group sample sizes while maintaining more accurate findings than fixed-effects models. We recommend using MAIHDA over fixed-effects models for intersectional investigations.

Emergent technologies present platforms for educational researchers to conduct randomized controlled trials (RCTs) and collect rich data to study students' performance, behavior, learning processes, and outcomes in authentic learning environments. As educational research increasingly uses methods and data collection from such platforms, it is necessary to consider the most appropriate ways to analyze this data to draw causal inferences from RCTs. Here, we examine whether and how analysis results are impacted by accounting for multilevel variance in samples from RCTs with student-level randomization within one platform. We propose and demonstrate a method that leverages auxiliary non-experimental "remnant" data collected within a learning platform to inform analysis decisions. Specifically, we compare five commonly-applied analysis methods to estimate treatment effects while accounting for, or ignoring, class-level factors and observed measures of confidence and accuracy to identify best practices under real-world conditions. We find that methods that account for groups as either fixed effects or random effects consistently outperform those that ignore group-level factors, even though randomization was applied at the student level. However, we found no meaningful differences between the use of fixed or random effects as a means to account for groups. We conclude that analyses of online experiments should account for the naturally-nested structure of students within classes, despite the notion that student-level randomization may alleviate group-level differences. Further, we demonstrate how to use remnant data to identify appropriate methods for analyzing experiments. These findings provide practical guidelines for researchers conducting RCTs in similar educational technologies to make more informed decisions when approaching analyses.

Cluster randomized trials (CRTs) are commonly used to evaluate the causal effects of educational interventions, where the entire clusters (e.g., schools) are randomly assigned to treatment or control conditions. This study introduces statistical methods for designing and analyzing two-level (e.g., students nested within schools) and three-level (e.g., students nested within classrooms nested within schools) CRTs. Specifically, we utilize hierarchical linear models (HLMs) to account for the dependency of the intervention participants within the same clusters, estimating the average treatment effects (ATEs) of educational interventions and other effects of interest (e.g., moderator and mediator effects). We demonstrate methods and tools for sample size planning and statistical power analysis. Additionally, we discuss common challenges and potential solutions in the design and analysis phases, including the effects of omitting one level of clustering, non-compliance, heterogeneous variance, blocking, threats to external validity, and cost-effectiveness of the intervention. We conclude with some practical suggestions for CRT design and analysis, along with recommendations for further readings.

The present study investigated multilevel models posited to predict student approaches to homework. Participants were 1,072 middle school students in China. Results revealed that deep and surface approaches were positively associated with performance-approach. Furthermore, deep approach to homework was associated negatively with homework cost, yet positively with mastery-approach, homework expectancy, and prior knowledge. Surface approach to homework was associated positively with homework cost and parent education, yet negatively with homework expectancy and mastery-approach. Females were less likely to use surface approach to homework than males. At the class level, surface approach to homework was negatively associated with parent education. Implications for homework practice and future investigation are discussed in terms of these results.

For assessment scales applied to different groups (e.g., students from different states; patients in different countries), multigroup differential item functioning (MG-DIF) needs to be evaluated in order to ensure that respondents with the same trait level but from different groups have equal response probabilities on a particular item. The current study compares two approaches for DIF detection: a multiple-group item response theory (MG-IRT) model and a generalized linear mixed model (GLMM). In the MG-IRT model approach, item parameters are constrained to be equal across groups and DIF is evaluated for each item in each group. In the GLMM, groups are treated as random, and item difficulties are modeled as correlated random effects with a joint multivariate normal distribution. Its nested structure allows the estimation of item difficulty variances and covariances at the group level. We use an excerpt from the PISA 2015 reading domain as an exemplary empirical investigation, and conduct a simulation study to compare the performance of the two approaches. Results from the empirical investigation show that the detection of countries with DIF is similar in both approaches. Results from the simulation study confirm this finding and indicate slight advantages of the MG-IRT model approach.

The cross-classified data structure is ubiquitous in education, psychology, and health outcome sciences. In these areas, assessment instruments that are made up of multiple items are frequently used to measure latent constructs. The presence of both the cross-classified structure and multivariate categorical outcomes leads to the so-called item-level data with cross-classified structure. An example of such data structure is the routinely collected student evaluation of teaching (SET) data. Motivated by the lack of research on multilevel IRT modeling with crossed random effects and the need of an approach that can properly handle SET data, this study proposed a cross-classified IRT model, which takes into account both the cross-classified data structure and properties of multiple items in an assessment instrument. A new variant of the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm was introduced to address the computational complexities in estimating the proposed model. A preliminary simulation study was conducted to evaluate the performance of the algorithm for fitting the proposed model to data. The results indicated that model parameters were well recovered. The proposed model was also applied to SET data collected at a large public university to answer empirical research questions. Limitations and future research directions were discussed.

Identifying items with differential item functioning (DIF) in an assessment is a crucial step for achieving equitable measurement. One critical issue that has not been fully addressed with existing studies is how DIF items can be detected when data are multilevel. In the present study, we introduced a Lord's Wald X[superscript 2] test-based procedure for detecting both uniform and non-uniform DIF with polytomous items in the presence of the ubiquitous multilevel data structure. The proposed approach is a multilevel extension of a two-stage procedure, which identifies anchor items in its first stage and formally evaluates candidate items in the second stage. We applied the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm to estimate multilevel polytomous item response theory (IRT) models and to obtain accurate covariance matrices. To evaluate the performance of the proposed approach, we conducted a preliminary simulation study that considered various conditions to mimic real-world scenarios. The simulation results indicated that the proposed approach has great power for identifying DIF items and well controls the Type I error rate. Limitations and future research directions were also discussed.

This study examined the factors that determined the science achievement of fourth-grade students on the Trends in International Mathematics and Science Study (TIMSS) 2019 in the USA. The data were retrieved from the TIMSS international database and imported to the R program for manipulation. The EdSurvey package was used to conduct multilevel analysis, taking plausible values of outcome variables, and weighting into account. Level 1 predictors included gender, sense of school belonging, no student bullying, instructional clarity, and attitude toward science. Level 2 predictors included instruction being affected by science resource shortages, school emphasis on academic success, a lack of school discipline problems, and students' preexisting with literacy and numeracy skills. The findings include that the variance between schools equals 33% and boys outperformed girls in science. Students' attitudes toward science, their confidence, and the school's emphasis on academic success were the two strongest positive predictors of both individual student's achievement and the school's mean science achievement. The other predictive variables were found to have only trivially positive or no effect. The utilized models could explain the variance within and between schools by 6% and 39%, respectively. The implications for policymakers and educational practitioners include boosting up confidence in learning science and by actively engaging students in inquiry-based activities and providing instant and constructive feedback, encouraging, and treating all students regardless of gender identity equally in teaching and learning science and schools should emphasize academic success.

Past research suggests that a sense of belonging to a community is developmentally important for adolescents and affects their engagement in school, especially during the transition to high school. However, little research examines the teaching practices that simultaneously foster classroom belonging and behavioral engagement to create a classroom community of engaged learners. This study investigates the association between specific teaching practices (i.e., lesson organization and structure, academic support, and student-teacher trust) related to students perceiving they are in a classroom community of engaged learners. Hierarchical linear modeling was used to analyze survey responses of 16,137 ninth grade students in a large urban school district. Results show a positive relationship between the way a teacher organizes their classroom lessons and activities, the level of academic support, and student-teacher trust towards students perceiving they are in a community of engaged learners. These findings differ across student characteristics (e.g., race, sex, living in a high poverty neighborhood, special education status, grades). The findings suggest that teaching practices that are generally considered by educators within the profession as good instructional practices may also be key for creating a community of engaged learners.

Due to the increased use of scientific models and modelling in K-12 education, there is a need to uncover its effects on students over time. Prior research has shown that the use of scientific modelling in K-12 classes is associated with improved conceptual knowledge and problem-solving skills. However, few studies have explicitly tested the longitudinal benefits of using model-based instruction on students' scientific reasoning skills (SRS) and content knowledge. This paper studies the effects of the use of modelling-based pedagogy in a longitudinal comparative case study on students' SRS using hierarchical linear modeling. Our findings showed that initial exposure to modelling-based instruction increased scientific reasoning scores significantly. By the end of their first year of science instruction, the average high school freshman in our study achieved the scientific reasoning level of many undergraduate STEM majors. More importantly, students in the lowest quartile of scientific reasoning demonstrated increased scores over the three years of the modeling-based course sequence. In addition, reasoning scores in the modelling classes were a significant predictor of post-content knowledge in all subjects. Our results suggested that students should be exposed to model-based instruction early and consistently to achieve equity in science instruction.

The purpose of this study was to explore potential effects of a 12-week therapeutic mentoring program targeting social, emotional, and behavioral concerns in 52 children and adolescents between 11 and 17 years of age. Self-reported scores on a norm-referenced behavioral questionnaire were tracked across the span of a mentoring program, and then analyzed using multilevel modeling. Results showed that participant scores changed in a healthy direction across all domains measured (i.e., conduct, negative affect, cognitive/attention, and academic functioning). Predictors in the multilevel model included caregiver-reported sex assigned at birth, the semester that the intervention took place, and whether a participant had repeated the program. Findings lend further support to research-based mentoring programs as effective community interventions to address behavioral, emotional, social, and academic concerns in youth.

Research has shown that mathematical proficiency gaps are related to students' and schools' indicators of poverty, with fewer studies on neighborhood effects on achievement gaps. Although this literature has accounted for students' nesting within schools, so far, methodological constraints have not allowed researchers to formally account for multilevel and spatial effects. I contribute to this discussion by simultaneously considering test-takers' own socioeconomic standing and the impact of their nesting schools and neighborhood structures. Multilevel simultaneous autoregressive (MSAR) models and population-level data of 2.09 million test-takers, whose standardized performances were measured at Grades 3-8 in New York State, revealed the presence of geography of mathematical (dis)advantage. Because mathematical performance is spatially dependent across schools and neighborhoods, moving forward, applied researchers should rely on MSAR to account for sources of spatially driven bias that cannot be handled with multilevel models alone. Full replication code and data are provided at https://cutt.ly/N4zRstL.

Estimates show that, in 2019, only 41 per cent of students completed lower secondary and 28 per cent upper secondary education in the sub-Saharan Africa (SSA) region (UNESCO, 2021). One of the reasons for the low completion rates is the poor transition across secondary education due to the significant impact of factors at individual, household, and community levels shaping demand and supply constraints. This article employs a three-level logit to investigate the key determinants for transitions and their variability across countries and communities, and explores whether less wealth inequality is at odds with increasing countries' and communities' performances. It finds that variation on transition rates is 40-50 per cent larger between communities within countries than between countries themselves, and that heterogeneity is larger for upper secondary transition. Leading sources of inequality are wealth, gender, and communities' prevalence of early marriage. Further, the article finds that the equity-performance trade-off does not hold across countries, but it does at the community level where communities with stronger rates of transitions are more unequal. The analysis suggests policies to improve SSA youth chances to move up to the next level of secondary education, starting for narrowing heterogeneity across communities, boosting chances for the poorest groups and female youth living in communities with weak social norms, and measures to diminish the impact of community wealth on their transition performance.

Large-scale assessment survey (LSAS) data are collected via complex sampling designs with special features (e.g., clustering and unequal probability of selection). Multilevel models have been utilized to account for clustering effects whereas the probability weighting approach (PWA) has been used to deal with design informativeness derived from the unequal probability selection. However, the difficulty of applying PWA in multilevel models (MLM) has been generally underestimated and practical guidance is scarce. This study utilizes an empirical as well as a Monte Carlo simulation investigation to examine the performance of the multilevel pseudo maximum likelihood (MPML) estimation based on information derived from the Early Childhood Longitudinal Study Kindergarten cohort of 2010-2011 (ECLS-K:2011). Variance components and fixed effects estimators across four estimation methods including three MPML estimators (i.e., weighted without scaling, weighted size-scaled and weighted effective-scaled) and the unweighted estimator are provided. Practical guidance about the use of sampling weights in MLM analyses of LSAS data is also offered.

Applied educational researchers may be interested in exploring random slope effects in multilevel models, such as when examining individual growth trajectories with longitudinal data. Random slopes are effects for which the slope of an individual-level coefficient varies depending on group membership, however these effects can be difficult to interpret. The change in variance accounted for is often used as an effect size measure and could be appropriate for helping to interpret a random slope effect. Two methods for computing variance accounted for include likelihood-based methods and variance partition methods. It is unclear how results from these two methods compare to each other when used to compute a measure of change in variance accounted for with a random slope effect. The present study fills this gap through a simulation study comparing these two methods under various conditions. Results indicate that the value of variance accounted for may differ depending on the type of measure used, and that applied researchers should consider reporting values for both measures.

We conducted a pilot study using intentional teaching strategies with specially designed materials to improve accuracy and production of targeted English morphosyntax structures with six deaf and hard-of-hearing students (kindergarten to first grade). A multiple baseline single-case research design (SCRD) consisting of 20-minute sessions four times per week for the duration of a school year was implemented to determine the effect of the supplemental syntax curriculum. The data were inconsistent and highly variable. Visual analyses were problematic; therefore, hierarchical linear model (HLM) regression analyses were conducted with the time series SCRD data as an additional analysis. HLM regression analyses were used to interpret data that might otherwise be overlooked in SCRDs to provide specific values for the rate students were learning during the intervention phase of the study. This pilot study demonstrates that the syntax intervention produces promising results when data that are too messy for visual analysis are analyzed with HLM.

We have entered an era in which scientific evidence increasingly informs research practice and policy. As there is an exponential increase in the use of single-case experimental designs (SCEDs) to evaluate intervention effectiveness, there is accumulating evidence available for quantitative synthesis. Consequently, there is a growing interest in techniques suitable to meta-analyze SCED research. One technique that can be applied is individual patient data (IPD) meta-analysis. IPD is a flexible approach, allowing for a variety of modeling options such as modeling moderators to explain intervention heterogeneity. To date, no methodological research has been conducted to evaluate the statistical properties of effect estimates obtained by using IPD meta-analysis with the inclusion of moderators. This study is designed to address this by conducting a large-scale Monte Carlo study. Based on the results, specific recommendations are provided to indicate under which conditions the IPD meta-analysis including moderators is suitable.

The most critical parameter of audio and video information output is the playback speed, which affects many viewing or listening metrics, including when learning using tutoring systems. However, the availability of quantitative models for personalized playback speed control considering the learner's personal traits is still an open question. The work aims to develop a model to control the personalized playback speed of audio information for beginners and experienced learners for intelligent tutoring systems. Analysis of the data from the experimental study using traditional machine learning methods did not allow us to classify the preferred playback rate with accuracy higher than 60%. Therefore, we developed the three-level hierarchical logistic model that predicts the preferred playback speed of audio material on the scale from "very low speed" to "high speed" for beginners and experienced learners with 80% accuracy. The model uses a combination of cognitive and psychomotor traits of individual learners and aims to maximize audio listening convenience and satisfaction. We explained the influence of the learners' selected personal traits on the preferred speed of audio playback. We calculated the convenience of listening to the audio materials without and with the model. By using the model, the convenience of listening to audio materials increased by an average of 13% at a low speech speed and 37% at a high speech speed. The model extends the control theory of multimedia information in e-learning systems by describing the influence of selected psychophysiological traits of learners on the preferred playback speed of audio materials.

Comprehensive evaluation of treatment effects is aided by considerations for moderated effects. In educational research, the combination of natural hierarchical structures and prevalence of group-administered or shared facilitator treatments often produces three-level partially nested data structures. Literature details planning strategies for a variety of experimental designs when moderation effects are of interest but has yet to establish power formulas for detecting moderation effects in three-level partially nested designs. To address this gap, we derive and assess the accuracy of power formulas for detecting the different types of moderation effects possible in these designs. Using Monte Carlo simulation studies, we probe power rates and adequate sample sizes for detecting the different moderation effects while varying common influential factors including variance in the outcome explained by covariates, magnitude of the moderation effect, and sample sizes. The power formulas developed improve the planning of experimental studies with partial nesting and encourage the inclusion of moderator variables to capture for whom and under what conditions a treatment is effective. Educational researchers also have some initial guidance regarding adequate sample sizes and the factors that influence detecting moderation effects in three-level partially nested designs.

Traditionally, cluster randomized controlled trials are analyzed with the average intervention effect of interest. However, in populations that contain higher degrees of heterogeneity or variation may differ across different values of a covariate, which may not be optimal. Within education and social science contexts, exploring the variation in magnitude of treatment effect at different points in the population can indicate where the intervention is most effective rather than assuming an average effect. Data from [Owen, K.L., et al., 2021. Implementation support improves outcomes of a fluency-based mathematics strategy: A cluster-randomized controlled trial. Journal of research on educational effectiveness, 14 (3), 523-542.] were reanalyzed using three modeling approaches: conditional mean-modeling reporting the average treatment effect using linear mixed models, and two quantile regression-based methods. Quantile regressions report the quantile treatment effects at different percentiles: 10th, 25th, 50th, 75th and 90th. For the Quantile approaches, a significant intervention effect in the median to upper quantiles was found and linear quantile mixed model showed improved fit over the other approaches. An improved picture of intervention effects may be apparent using quantile regression methods when analyzing cluster randomized trials that have heterogeneous error variance. In particular, the linear quantile mixed model shows improved model fit allowing a multilevel framework to include random effects. There is considerable scope to extend this framework to incorporate more complex RCT designs.

The use of large-scale assessments (LSAs) in education has grown in the past decade though analysis of LSAs using multilevel models (MLMs) using R has been limited. A reason for its limited use may be due to the complexity of incorporating both plausible values and weighted analyses in the multilevel analyses of LSA data. We provide additional functions in R that extend the functionality of the WeMix (Bailey et al., 2023) package to allow for the automatic pooling of plausible values. In addition, functions for model comparisons using plausible values and the ability to export output to different formats (e.g., Word, html) are also provided.

We present a bias-adjusted three-step estimation approach for multilevel latent class models (LC) with covariates. The proposed approach involves (1) fitting a single-level measurement model while ignoring the multilevel structure, (2) assigning units to latent classes, and (3) fitting the multilevel model with the covariates while controlling for measurement error introduced in the second step. Simulation studies and an empirical example show that the three-step method is a legitimate modeling option compared to the existing one-step and two-step methods.

Several variants of the autoregressive structural equation model were suggested over the past years, including, for example, the random intercept autoregressive panel model, the latent curve model with structured residuals, and the STARTS model. The present work shows how to place these models into a mixed-effects model framework and how to estimate them in mixed-effects model software, namely the R package "nlme." We also show how "nlme" can be used to fit extensions of these models, for example, models that do not assume equally spaced time intervals between measurement occasions (i.e., continuous time models). Overall, our expositions show that autoregressive structural equations models and mixed-effects models are closely related. We think that this insight eases researchers to understand the differences between the variants of the autoregressive structural equation model and also allows them to profitably link the two different modeling perspectives.

A recent review found that 11% of published factor models are hierarchical models with second-order factors. However, dedicated recommendations for evaluating hierarchical model fit have yet to emerge. Traditional benchmarks like RMSEA <0.06 or CFI >0.95 are often consulted, but they were never intended to generalize to hierarchical models. Through simulation, we show that traditional benchmarks perform poorly at identifying misspecification in hierarchical models. This corroborates previous studies showing that traditional benchmarks do not maintain optimal sensitivity to misspecification as model characteristics deviate from those used to derive the benchmarks. Instead, we propose a hierarchical extension to the dynamic fit index (DFI) framework, which automates custom simulations to derive cutoffs with optimal sensitivity for specific model characteristics. In simulations to evaluate performance, results showed that the hierarchical DFI extension routinely exceeded 95% classification accuracy and 90% sensitivity to misspecification whereas traditional benchmarks applied to hierarchical models rarely exceeded 50% classification accuracy and 20% sensitivity.

The paper investigates the potential of using international large-scale assessment studies for conducting follow-up studies testing models of educational effectiveness. The impact of teacher factors coming from the "dynamic model of educational effectiveness" and the "dialogic education theory" on student literacy achievement is examined. Modern Greek teachers from secondary schools that participated in PISA 2018 (Accessed on https://www.oecd.org/pisa/data/pisa2018technicalreport/) in Cyprus were recruited. Two Grade 11 classes per teacher were selected, giving 392 students in total. Observation data from Greek lessons and student literacy achievement data were collected. Two multilevel regression analyses were conducted: one with the whole student sample and one with only students from whom PISA prior achievement measure was identified. When prior achievement (i.e., PISA measure) was considered, additional and larger effects of factors on student final achievement were detected. Both analyses reveal that when factors from both frameworks were considered, larger variance in student achievement was explained.

The recently proposed "continuous-time latent curve model with structured residuals" (CT-LCM-SR) addresses several challenges associated with longitudinal data analysis in the behavioral sciences. First, it provides information about process trends and dynamics. Second, using the continuous-time framework, the CT-LCM-SR can handle unequally spaced measurement occasions and describes processes independently of the length of the time intervals used in a given study. Third, it is a hierarchical model. Thus, multiple subjects can be analyzed simultaneously. However, subjects might also differ in dynamics and trends. Therefore, in the present paper, we extend the CT-LCM-SR to capture these differences as well. This newly proposed "random coefficients continuous-time latent curve model with structured residuals" (RC-CT-LCM-SR) is introduced theoretically and technically. Additionally, we provide an illustrative example with data from the Health and Retirement Study (HRS), and we show how the RC-CT-LCM-SR can be used to study multiple sources of between-subject differences over time.

The present study aimed to examine how to improve the effectiveness of moral exemplar-applied interventions based on the pillars of the self-determination theory framework, autonomy, competence, and relatedness. Past research has mainly focused on the relatedness and attainability of moral exemplars for predicting motivation outcomes. The data for this study consisted of synthesized data sets from previous studies examining the motivational impacts of distinct moral exemplars and intervention methods. The main syntheses for these data sets used multilevel modeling focusing on relatability, attainability, and intervention methods, corresponding to relatedness, competence, and autonomy in the self-determination theory, respectively, as predictors. In general, there was a significant interaction effect between the attainability or relatability, and the intervention method. Autonomous instruction methods, which support autonomy, were demonstrated to boost motivational outcomes. Implications from this study support the employment of self-determination theory to examine the use of moral exemplars in moral education and were consistent with previous exemplar studies.

Teachers' job satisfaction (TJS) can be defined as the emotional reactions of teachers to their jobs or teaching roles. In this study, it is aimed to investigate the determinants of teachers, principal and school-based factors on job satisfaction of teachers. In this study, which is based on relational survey model, secondary data obtained from TALIS-2018 evaluation were analyzed with Multilevel Structural Equation Modeling. 196 principals and 3952 teachers from Turkey who participated in TALIS-2018 survey constitute the sample of the research. According to the results of the study, teachers' age, gender, career preferences and participation in professional development activities, the locations of the schools they work in and the type of school (state/private) and the gender of the school principals were found to be determinants of job satisfaction. Teachers' work experience, having foreign students in their classes, school principal's age and work experience did not affect teachers' job satisfaction.

This study develops an Hierarchical Linear Model (HLM) model in order to identify the predictors that have an effect on teachers' achievement in curriculum 2013 training. It is a two-level model with teacher's characteristics as Level 1 variables and facilitators characteristics as Level 2 variables. The analysis in this study uses secondary data. The Indonesian Central Kalimantan Province Education Quality Assurance Institute has implemented education and training for the 2013 curriculum, and has a post test score document. The HLM analysis model is applied, with stages: zero model analysis, finalization of level-1, level-2, and final analysis model for teachers and facilitators. The results of the analysis for the teacher level, that the PRIA-B predictor has a direct positive effect on the results, C13ACH. Meanwhile, GENDER has a negative effect on C13ACH. The Facilitator, GENDER, and P_MEAN have a direct positive effect on C13ACH while GENDER is also found interacting with PRIAB that leads to the outcome variable, C13ACH. The interaction between GENDER and the slope of PRIAB is a positive one. The results of the model variance components indicate that the final full model explains about 46% of the total available variance.

Hierarchical linear modeling (HLM) has become an increasingly popular multilevel method of analyzing data among nested datasets, in particular, the effect of specialized academic programming within schools. The purpose of this methodological study is to demonstrate the use of HLM to determine the effectiveness of STEM programming in an Ohio middle school. This longitudinal study analyzes potential moderators of gender, socioeconomic status, student race, and attendance rate along with state test scores to quantify achievement. HLM determined integrated STEM education had a significant, positive effect on achievement in math and science combined (students scoring 31.8 points higher on average) and science achievement (students scoring 38.2 points higher on average) compared to traditional education students, respectively. There were little to no interaction effects determined between STEM participation and student factors. This demonstrates HLM as a powerful statistical tool for quantifying the impact of academic programming on student achievement.

School effectiveness is a topic of interest addressed by numerous research projects focused on clarifying which variables contribute to the explanation of educational performance. This research aims to find out to what extent social, cultural, and academic variables at the student and school levels, as perceived by families, influence performance, and to evaluate the relevance of high residual and gross score criteria in the selection of effective or ineffective schools. Census data from diagnostic evaluations of the Mathematical Reasoning and Linguistic Communication of students in a certain Spanish region, over five academic years, have been used. The multilevel hierarchical analyses carried out have enabled the detection of centers of high and low efficiency, as well as the identification of which factors, related to the idiosyncrasy of the students and the educational center they attend, significantly influence the performance of the students. It was concluded that the socioeconomic and cultural level of the families, the family expectations, the commitment to reading and the educational agreement were significant variables in the explanation of the students' educational performance, and that the residual score of the educational centers was a valid criterion to estimate their level of effectiveness once the socio-cultural factors have been controlled.

Level 3 characteristics are increasingly important for fingerprint identification, particularly when the fingerprints are problematic and lack level 2 details. In practice, no reliable visualization methods for level 3 features are transferred into a course for the general public, forensic students, or investigators. Our group has reported the membrane/water technique and wet-membrane method as simple, instantaneous, high-resolution, and reproducible methods for visualizing level 3 features of latent fingerprints (LFPs). Herein, we designed a simple-to-operate laboratory course on sweat pore visualization and analysis via the two methods mentioned. Three modules are included, and students participate in different modular combinations according to their interests and specialties. Module 1, consisting of a 15 min lecture on fingerprint theory followed by a brief demonstration of visualization procedures, is designed for broad audiences ranging from primary school to college laboratories. In module 2, students explored the deposition force impact on level 3 features (for students majoring in forensic science). Module 3 customized for forensic upper-division undergraduates and postgraduates invites them to conduct Gaussian fitting of the pore area and adjacent pore distance. After discussion, students form their own opinions on the identifiable strengths of the above parameters in fingerprint matching. In a way, this experiment can be evolved into the elective course and specialized curriculum, where elective course containing module 1 can stimulate public interest in chemistry and specialized curriculum including modules 1-2 (or modules 1-3) may guide students to extract level 3 features and understand the role of level 3 features at a quantitative level.

Hierarchical linear modeling (HLM) is often used to estimate the effects of socioeconomic status (SES) on academic achievement at different levels of an educational system. However, if a prior academic achievement measure is missing in a HLM model, biased estimates may occur on the effects of student SES and school SES. Phantom effects describe the phenomenon in which the effects of student SES and school SES disappear once prior academic achievement is added to the model. In the present analysis, partial simulation (i.e., simulated data are used together with real-world data) was employed to examine the phantom effects of student SES and school SES on science achievement, using the national sample of the United States from the 2015 Programme for International Student Assessment. The results showed that the phantom effects of student SES and school SES are rather real. The stronger the correlation between prior science achievement and (present) science achievement, the greater the chance that the phantom effects occur.

Student engagement is a multidimensional construct influenced by complex student, classroom, and school factors. Although prior research has examined various dimensions of student engagement, few studies have adopted a multilevel approach to simultaneously incorporate observational measures of classroom practices with student and school factors. The current study examined the association between student self-reported affective and behavioral engagement (n = 26,849 students) and observed instructional and behavioral classroom management practices within 401 classrooms in 54 high schools. We employed three-level hierarchical linear modeling and included student, classroom, and school variables. Results indicated that observed teacher use of positive behavior supports was positively and significantly associated with student reports of active engagement in the same classroom. Implications for improving teachers' classroom management practices to promote student engagement are considered. [For the Grantee Submission, see ED611739.]

The goal of the current study is to examine the degree to which measures of quality of implementation and student engagement moderate pretest--posttest changes in mediating variables that are targeted by DARE "keepin' it REAL." DARE officers (10 elementary school, five middle school) taught DARE "keepin' it REAL lessons to 1,017 elementary students (480 boys and 534 girls) and 435 middle school students (217 boys and 215 girls). We examined teachers' and students' ratings of elementary and middle schools in response to DARE officers' delivery of the program. HLM analyses revealed that students' engagement was a significant and meaningful predictor of changes in targeted mediators. Teachers' ratings of student responsiveness added little in terms of understanding these outcomes with main effects observed only for students' ability to respond to bulling and students' estimates of peer drug use. Teachers' ratings of the quality of officer implementation, on the other hand, did add to understanding students' outcomes. Effects were seen for three (peer norms about drug use, decision-making (DM) skills, intentions to avoid drug use) out of six outcome variables and suggest a stronger positive effect for elementary versus middle school students. At least for these three outcomes, understanding quality of implementation added to our ability to interpret results. Specifically, in addition to students' engagement, quality of implementation (which varied by grade) contributed to achieving positive changes in students' outcomes.

The use of new technologies is one of the themes at the centre of the educational debate, especially following Distance Education. The Avant-garde Educative Movement gathers schools from all over the country that share experiments based on a model in which the student is at the centre of his or her own educational path, a model capable of promoting meaningful learning. The Self Determination Theory (Deci & Ryan, 1985) highlights the extent to which the satisfaction of the needs for autonomy, competence and relationships contributes to student motivation. The purpose of the present study is to understand whether the rethinking of teaching promoted by widespread use of new technologies has had an impact in terms of student satisfaction. For the analysis, a Multilevel Model was used that assumes as the dependent variable students' satisfaction with their school life from the Clipper Test, a-Portfolio for the orientation of students aged 15-19 (Soresi & Nota, 2003). The scale captures satisfaction with the school experience in terms of preparation received, relationships established and congruence with expectations. Results show that satisfaction increases as the use and frequency of innovative teaching methodologies employed by the teacher increases and as the frequency of use of technological resources employed in the classroom and in homework assignments increases.