Search

When planning a cluster randomized trial, evaluators often have access to an enumerated cohort representing the target population of clusters. Practicalities of conducting the trial, such as the need to oversample clusters with certain characteristics in order to improve trial economy or support inferences about subgroups of clusters, may preclude simple random sampling from the cohort into the trial, and thus interfere with the goal of producing generalizable inferences about the target population. We describe a nested trial design where the randomized clusters are embedded within a cohort of trial-eligible clusters from the target population and where clusters are selected for inclusion in the trial with known sampling probabilities that may depend on cluster characteristics (e.g., allowing clusters to be chosen to facilitate trial conduct or to examine hypotheses related to their characteristics). We develop and evaluate methods for analyzing data from this design to generalize causal inferences to the target population underlying the cohort. We present identification and estimation results for the expectation of the average potential outcome and for the average treatment effect, in the entire target population of clusters and in its non-randomized subset. In simulation studies, we show that all the estimators have low bias but markedly different precision. Cluster randomized trials where clusters are selected for inclusion with known sampling probabilities that depend on cluster characteristics, combined with efficient estimation methods, can precisely quantify treatment effects in the target population, while addressing objectives of trial conduct that require oversampling clusters on the basis of their characteristics.

A critically important balance in educational measurement between practical concerns and matters of technique has atrophied in recent decades, and as a result, some important issues in the field have not been adequately addressed. I start with the work of E. F. Lindquist, who exemplified the balance that is now wanting. Lindquist was arguably the most prolific developer of achievement tests in the history of the field and an accomplished statistician, but he nonetheless focused extensively on the practical limitations of testing and their implications for test development, test use, and inference. I describe the withering of this balance and discuss two pressing issues that have not been adequately addressed as a result: the lack of robustness of performance standards and score inflation. I conclude by discussing steps toward reestablishing the needed balance.

Most current statistics courses include some instruction relevant to causal inference. Whether this instruction is incorporated as material on randomized experiments or as an interpretation of associations measured by correlation or regression coefficients, the way in which this material is presented may have important implications for understanding causal inference fundamentals. Although the connection between study design and the ability to infer causality is often described well, the link between the language used to describe study results and causal attribution typically is not well defined. The current study investigates this relationship experimentally using a sample of students in a statistics course at a large western university in the United States. It also provides (non-experimental) evidence about the association between statistics instruction and the ability to understand appropriate causal attribution. The results from our experimental vignette study suggest that the wording of study findings impacts causal attribution by the reader, and, perhaps more surprisingly, that this variation in level of causal attribution across different wording conditions seems to pale in comparison to the variation across study contexts. More research, however, is needed to better understand how to tailor statistics instruction to make students sufficiently wary of unwarranted causal interpretation.

Post-instrument covariates are often included as controls in instrumental variable (IV) analyses to address a violation of the exclusion restriction. However, we show that such analyses are subject to biases unless strong assumptions hold. Using linear constant-effects models, we present asymptotic bias formulas for three estimators (with and without measurement error): IV with post-instrument covariates, IV without post-instrument covariates, and ordinary least squares. In large samples and when the model provides a reasonable approximation, these formulas sometimes allow the analyst to bracket the parameter of interest with two estimators and allow the analyst to choose the estimator with the least asymptotic bias. We illustrate these points with a discussion of the settler mortality IV used by Acemoglu, Johnson, and Robinson.

This article uses the interventionist theory of causation, a counterfactual theory taken from philosophy of science, to strengthen causal analysis in process tracing research. Causal claims from process tracing are re-expressed in terms of so-called hypothetical interventions, and concrete evidential tests are proposed which are shown to corroborate process tracing claims. In particular, three steps are prescribed for an interventionist investigation, and each step in turn is shown to make the causal analysis more robust, amongst others by disambiguating causal claims and clarifying or strengthening the existing methodological advice on counterfactual analysis. The article's claims are then illustrated using a concrete example, Haggard and Kaufman's analysis of the Argentinian transition to democracy. It is shown that interventionism could have strengthened the authors' conclusions. The article concludes with a short Bayesian analysis of its key methodological proposals.

Building on previous work on the civic returns of adult learning, this article examines the association between adult education, personality traits, and demands for civic participation or volunteering. Based on National Education Panel Study data, the study finds openness to be a crucial personality trait for participating in further training, as well as for civic participation, as it relates to competence and self-confidence in dealing with new demands. Extraversion is required for social interaction and communication with other participants. The study also shows that participants in further private training engage more at the management level of volunteering, where demands such as leadership and organizational talent are emphasized. The findings consider sociological factors such as the role of peers.

Sampling distributions are fundamental for statistical inference, yet their abstract nature poses challenges for students. This research investigates the development of high school students' conceptions of sampling distribution through informal significance tests with the aid of digital technology. The study focuses on how technological tools contribute to forming these conceptions, guided by an emerging theory that describes this process. A workshop for high school students was organized, involving 36 participants working in pairs across four sessions, each with access to a computer. These sessions involved problem-solving activities, with the teacher introducing key concepts in the initial three sessions. The analysis, employing grounded theory, aimed to characterize the nature of students' conceptions of sampling distribution as evident in their responses. The findings reveal a transition from empirical to informal conceptions of sampling distribution among students, facilitated by computational mediation. This transition is marked by an abstraction process that includes mathematization, processing, uncertainty/randomness, and conditional reasoning. The study underscores the role of digital simulations in teaching statistical concepts, facilitating students' conceptual shift critical for grasping statistical inference.

Emergent technologies present platforms for educational researchers to conduct randomized controlled trials (RCTs) and collect rich data to study students' performance, behavior, learning processes, and outcomes in authentic learning environments. As educational research increasingly uses methods and data collection from such platforms, it is necessary to consider the most appropriate ways to analyze this data to draw causal inferences from RCTs. Here, we examine whether and how analysis results are impacted by accounting for multilevel variance in samples from RCTs with student-level randomization within one platform. We propose and demonstrate a method that leverages auxiliary non-experimental "remnant" data collected within a learning platform to inform analysis decisions. Specifically, we compare five commonly-applied analysis methods to estimate treatment effects while accounting for, or ignoring, class-level factors and observed measures of confidence and accuracy to identify best practices under real-world conditions. We find that methods that account for groups as either fixed effects or random effects consistently outperform those that ignore group-level factors, even though randomization was applied at the student level. However, we found no meaningful differences between the use of fixed or random effects as a means to account for groups. We conclude that analyses of online experiments should account for the naturally-nested structure of students within classes, despite the notion that student-level randomization may alleviate group-level differences. Further, we demonstrate how to use remnant data to identify appropriate methods for analyzing experiments. These findings provide practical guidelines for researchers conducting RCTs in similar educational technologies to make more informed decisions when approaching analyses.

Researchers tasked with understanding the effects of educational technology innovations face the challenge of providing evidence of causality. Given the complexities of studying learning in authentic contexts interwoven with technological affordances, conducting tightly-controlled randomized experiments is not always feasible nor desirable. Today, a set of tools is available that can help researchers reason about cause-and-effect, irrespective of the particular research design or approach. This theoretical paper introduces such a tool, a simple graphical formalism that can be used to reason about potential sources of bias. We further explain how causal graphs differ from structural equation models and highlight the value of explicit causal inference. The final section shows how causal graphs can be used in several stages of the research process, whether researchers plan to conduct observational or experimental research.

As evidence from evaluation and experimental studies continue to influence decision and policymaking, applied researchers and practitioners require tools to derive valid and credible inferences. Over the past several decades, research in causal inference has progressed with the development and application of propensity scores. Since their inception, propensity scores have made an important contribution to the improvement in estimation of causal impacts, particularly in the absence of randomization. When certain core assumptions hold, propensity score-based methods allow for bias-reduced estimation of average treatment effects. In addition to their important role in causal studies, propensity scores have also been integral in improving generalizations from causal studies, specifically when study samples are not randomly selected from a target population of inference. The current study provides an overview of propensity scores, a discussion of the assumptions needed to ensure their validity, and an illustration of the methods both for causal inference and generalization. We highlight the importance of propensity score methods and discuss current applications and directions for ongoing and future research.

Causal inference is a central topic in education research, although oftentimes it relies on observational studies, which makes causal identification methodologically challenging. This manuscript introduces causal graphs as a powerful language for elucidating causal theories and an effective tool for causal identification analysis. It discusses graphical criteria for causal identification, which provide principled approaches for removing bias and assessing causal identification given a causal theory. Through illustrative examples, this manuscript demonstrates the application of causal graphs and adjustment criterion for covariate selection in the context of education research, exemplifying their key advantages particularly in scenarios where randomized experiments are impractical. This manuscript aims to acquaint researchers with causal graphs as an effective tool for causal inference, thereby facilitating theory-based causal inquiries in applied education research.

Missing data are a pervasive problem in data analysis. Three common methods for addressing the problem are (a) complete-case analysis, where only units that are complete on the variables in an analysis are included; (b) weighting, where the complete cases are weighted by the inverse of an estimate of the probability of being complete; and (c) multiple imputation (MI), where missing values of the variables in the analysis are imputed as draws from their predictive distribution under an implicit or explicit statistical model, the imputation process is repeated to create multiple filled-in data sets, and analysis is carried out using simple MI combining rules. This article provides a non-technical discussion of the strengths and weakness of these approaches, and when each of the methods might be adopted over the others. The methods are illustrated on data from the Youth Cohort (Time) Series (YCS) for England, Wales and Scotland, 1984-2002.

Although parallel analysis has been found to be an accurate method for determining the number of factors in many conditions with complete data, its application under missing data is limited. The existing literature recommends that, after using an appropriate multiple imputation method, researchers either apply parallel analysis to every imputed data set and use the number of factors suggested by most of the data copies or average the correlation matrices across all data copies, followed by applying the parallel analysis to the average correlation matrix. Both approaches for pooling the results provide a single suggested number without reflecting the uncertainty introduced by missing values. The present study proposes the use of an alternative approach, which calculates the proportion of imputed data sets that result in k (k = 1, 2, 3 . . .) factors. This approach will inform applied researchers of the degree of uncertainty due to the missingness. Results from a simulation experiment show that the proposed method can more likely suggest the correct number of factors when missingness contributes to a large amount of uncertainty.

Students often believe that statistical significance is the only determinant of whether a quantitative result is "important." In this paper, I review traditional null hypothesis statistical testing to identify what questions inferential statistics can and cannot answer, including statistical significance, effect size and direction, causality, generalizability, and changeability of the independent variable. I illustrate these issues with examples from an empirical study of the association between how much time teenagers spent playing video games and time spent reading. I describe how study design and context determine each of those aspects of "importance," and close by summarizing how to provide a holistic view of importance when writing about a quantitative analysis. I also include exercises to guide students through applying these concepts to articles in newspapers and scholarly journals.

This paper analyzes the relationship between proportional reasoning and understanding fair games in Costa Rican students. We conducted a quantitative and qualitative analysis of the answers to six items on comparing ratios of increasing difficulty level and another item on prize estimation in a fair game. We describe the strategies employed and the semiotic conflicts detected in 292 Costa Rican students from Grades 6 to 10 (11-16-year-olds), comparing the findings with those established in previous research. The results show an increase in the level of proportional reasoning with the grade, although the age at which the higher levels are reached is lower than that assumed by Noelting. The percentage of students applying correct strategies in the fair game problem also increases with grade, and a relationship between the understanding of fair game and the level of proportional reasoning is observed.

We consider the statistical inference for noisy incomplete binary (or 1-bit) matrix. Despite the importance of uncertainty quantification to matrix completion, most of the categorical matrix completion literature focuses on point estimation and prediction. This paper moves one step further toward the statistical inference for binary matrix completion. Under a popular nonlinear factor analysis model, we obtain a point estimator and derive its asymptotic normality. Moreover, our analysis adopts a flexible missing-entry design that does not require a random sampling scheme as required by most of the existing asymptotic results for matrix completion. Under reasonable conditions, the proposed estimator is statistically efficient and optimal in the sense that the Cramer-Rao lower bound is achieved asymptotically for the model parameters. Two applications are considered, including (1) linking two forms of an educational test and (2) linking the roll call voting records from multiple years in the United States Senate. The first application enables the comparison between examinees who took different test forms, and the second application allows us to compare the liberal-conservativeness of senators who did not serve in the Senate at the same time.

Mixed Membership Models (MMMs) are a popular family of latent structure models for complex multivariate data. Instead of forcing each subject to belong to a single cluster, MMMs incorporate a vector of subject-specific weights characterizing partial membership across clusters. With this flexibility come challenges in uniquely identifying, estimating, and interpreting the parameters. In this article, we propose a new class of "Dimension-Grouped" MMMs (Gro-M[superscript 3]s) for multivariate categorical data, which improve parsimony and interpretability. In Gro-M[superscript 3]s, observed variables are partitioned into groups such that the latent membership is constant for variables within a group but can differ across groups. Traditional latent class models are obtained when all variables are in one group, while traditional MMMs are obtained when each variable is in its own group. The new model corresponds to a novel decomposition of probability tensors. Theoretically, we derive transparent identifiability conditions for both the unknown grouping structure and model parameters in general settings. Methodologically, we propose a Bayesian approach for Dirichlet Gro-M[superscript 3]s to inferring the variable grouping structure and estimating model parameters. Simulation results demonstrate good computational performance and empirically confirm the identifiability results. We illustrate the new methodology through applications to a functional disability survey dataset and a personality test dataset.

It was aimed to reveal the informal statistical inferences of primary school students on the concept of variability. A phenomenographic research approach was adopted. A task was adapted and used as a data collection tool. Clinical interviews were conducted with 20 primary school students from each grade level. Data were analyzed within the context of the analysis framework hierarchy of development for variability. The results showed that the highest number of students were at Level 1, followed by those who are at Level 0. It can also be seen that the numbers of the students who are at Level 3 and Level 4 are close to each other and smaller than the number of students at Level 1. In addition, it is seen that there is no 1st grade student at Level 4 and that the majority of the students at Level 3 and Level 4 are 3rd and 4th graders. One out of four students had difficulty in informal statistical inferences on the concept of variability. Informal statistical inferences on the concept of variability of three out of every four students, albeit at different levels, drew attention.

This paper presents a model of recurrent multinomial sequences. Though there exists a quite considerable literature on modeling autocorrelation in numerical data and sequences of categorical outcomes, there is currently no systematic method of modeling patterns of recurrence in categorical sequences. This paper develops a means of discovering recurrent patterns by employing a more restrictive Markov assumption. The resulting model, which I call the recurrent multinomial model, provides a parsimonious representation of recurrent sequences, enabling the investigation of recurrences on longer time scales than existing models. The utility of recurrent multinomial models is demonstrated by applying them to the case of conversational turn-taking in meetings of the Federal Open Market Committee (FOMC). Analyses are effectively able to discover norms around turn-reclaiming, participation, and suppression and to evaluate how these norms vary throughout the course of the meeting.

An online authentic learning environment (OnALE) is proposed in this study to facilitate students' learning of inferential statistics in a real-life context. The efficacy of the OnALE, in comparison to the conventional approach relative to the students' performance, was explored. Respondents from the experimental group were purposively selected to complete the Perception Questionnaire regarding the features of the OnALE. The Analysis of Covariance (ANCOVA) on the post-test scores using prior knowledge scores as covariate disclosed a significant variance in the post-test scores between control and experimental groups (F (1, 74) = 10.924, p < 0.05, partial [eta-squared] = 0.129), with the experimental group displaying a higher mean score. Outcomes from the Perception Questionnaire revealed that all respondents at least agreed that each authentic learning characteristic in the OnALE facilitated their learning. The highest and the lowest rated characteristics were "Collaboration" and "Multiple Roles and Perspectives," respectively. The framework of the OnALE characteristics for varying levels of students' performance unveiled the combinations of authentic learning characteristics beneficial to students from different performing groups. This framework functions as a guideline for statistics educators and learning designers to provide an effective online learning environment.

Sensitivity analysis, a statistical method crucial for validating inferences across disciplines, quantifies the conditions that could alter conclusions (Razavi et al., 2021). One line of work is rooted in linear models and foregrounds the sensitivity of inferences to the strength of omitted variables (Cinelli & Hazlett, 2019; Frank, 2000). A more recent approach is rooted in the potential outcomes framework for causal inference and foregrounds how hypothetical changes in a sample would alter an inference if such cases were otherwise observed (Frank et al., 2008, 2013; Frank & Min, 2007; Xu et al., 2019). One sensitivity measure is the "Impact Threshold of a Confounding Variable," or ITCV, which generates statements about the correlation of an omitted, confounding variable with both a predictor of interest and the outcome (Frank, 2000). The ITCV index can be calculated for any linear model. The "Robustness of an Inference to Replacement," RIR, assesses how replacing a certain percentage of cases with counterfactuals of zero treatment effect could nullify an inference (Frank et al., 2013). The RIR index is more general than the ITCV index.

There are over one million school-age children in Missouri, and we estimate 61,000 (6% of all school-age children) are homeschooled. Missouri is one of 29 states that does not require homeschooling to be reported. Using methods that can be replicated elsewhere with publicly available data, we test three approaches to estimating homeschool participation: using American Community Survey (ACS) data; subtracting public and estimated private school counts from ACS school-age totals; and polling parents. We comment on the usefulness and limitations of each approach and provide a model for researchers seeking to estimate homeschooling trends in states that lack administrative data.

This study contributes to the call for influencing practice by increasing attention to how learning environments can be designed to support learning in statistical inference. We report on a design experiment in secondary school (students 14-16 years old), that resulted in a set of lessons with the learning goal of teaching students how to apply concepts and principles of hypothesis testing for making an inference as to whether or not students in secondary school can taste the difference between two brands of cola soda. The design experiment resulted in four design principles for a simulation-based approach for learning hypothesis testing in secondary school. The design principles highlight the combination of practical and digital simulations of samplings. They stress the need for using random generators that allow for high reliability in collecting sample data and introduce a simulation-based method for determining p-values, i.e. to quantify how likely or surprising a sample result, or a result more extreme, is under a null hypothesis.

This paper aims to showcase the value of implementing a Bayesian framework to analyze and report results from international large-scale assessments and provide guidance to users who want to analyse ILSA data using this approach. The motivation for this paper stems from the recognition that Bayesian statistical inference is fast becoming a popular methodological framework for the analysis of educational data generally, and large-scale assessments more specifically. The paper argues that Bayesian statistical methods can provide a more nuanced analysis of results of policy relevance compared to standard frequentist approaches commonly found in large-scale assessment reports. The data utilized for this paper comes from the Teaching and Learning International Survey (TALIS). The paper provides steps in implementing a Bayesian analysis and proposes a workflow that can be applied not only to TALIS but to large-scale assessments in general. The paper closes with a discussion of other Bayesian approaches to international large-scale assessment data, in particularly for predictive modeling.

Parallel process latent growth curve mediation models (PP-LGCMMs) are frequently used to longitudinally investigate the mediation effects of treatment on the level and change of outcome through the level and change of mediator. An important but often violated assumption in empirical PP-LGCMM analysis is the absence of omitted confounders of the relationships among treatment, mediator, and outcome. In this study, we analytically examined how omitting pretreatment confounders impacts the inference of mediation from the PP-LGCMM. Using the analytical results, we developed three sensitivity analysis approaches for the PP-LGCMM, including the frequentist, Bayesian, and Monte Carlo approaches. The three approaches help investigate different questions regarding the robustness of mediation results from the PP-LGCMM, and handle the uncertainty in the sensitivity parameters differently. Applications of the three sensitivity analyses are illustrated using a real-data example. A user-friendly Shiny web application is developed to conduct the sensitivity analyses.

Mediation analysis is widely applied in various fields of science, such as psychology, epidemiology, and sociology. In practice, many psychological and behavioral phenomena are dynamic, and the corresponding mediation effects are expected to change over time. However, most existing mediation methods assume a static mediation effect over time, which overlooks the dynamic nature of mediation effect. To address this issue, we propose dynamic mediation models that can capture the dynamic nature of the mediation effect. Specifically, we model the path parameters of mediation models as auto-regressive (AR) processes of time that can vary over time. Additionally, we define the mediation effect under the potential outcome framework, and examine its identification and causal interpretation. Bayesian methods utilizing Gibbs sampling are adopted to estimate unknown parameters in the proposed dynamic mediation models. We further evaluate our proposed models and methods through extensive simulations and illustrate their application through a real data application.

Using software to teach statistical inference in introductory courses opens the door for methods and practices that are more conceptually appealing to students. With an increasing number of fields requiring competency in statistics including data science, natural and social sciences, public health and more, it is crucial that we as instructors deliver the basic concepts of statistics effectively. In line with guidelines presented in the GAISE College Report, this article demonstrates intuitive approaches to teaching proportion and mean inference that take advantage of statistical software and emphasize conceptual understanding. The article recommends putting aside asymptotic-based methods for proportion inference and using the exact binomial method. Regarding mean inference, we propose a more contextualized and simplified process that uses the distribution of the sample mean directly and avoids standardized statistics such as z or t. In both the proportion and mean inference contexts, we discuss the benefits of the proposed approaches and provide detailed examples that demonstrate the methods using the Rguroo statistical software.

Bayesian methods to infer model dimensionality in factor analysis generally assume a lower triangular structure for the factor loadings matrix. Consequently, the ordering of the outcomes influences the results. Therefore, we propose a method to infer model dimensionality without imposing any prior restriction on the loadings matrix. Our approach considers a relatively large number of factors and includes auxiliary multiplicative parameters, which may render null the unnecessary columns in the loadings matrix. The underlying dimensionality is then inferred based on the number of nonnull columns in the factor loadings matrix, and the model parameters are estimated with a postprocessing scheme. The advantages of the method in selecting the correct dimensionality are illustrated via simulations and using real data sets.

Machine learning (ML) methods for causal inference have gained popularity due to their flexibility to predict the outcome model and the propensity score. In this article, we provide a within-group approach for ML-based causal inference methods in order to robustly estimate average treatment effects in multilevel studies when there is cluster-level unmeasured confounding. We focus on one particular ML-based causal inference method based on the targeted maximum likelihood estimation (TMLE) with an ensemble learner called SuperLearner. Through our simulation studies, we observe that training TMLE within groups of similar clusters helps remove bias from cluster-level unmeasured confounders. Also, using within-group propensity scores estimated from fixed effects logistic regression increases the robustness of the proposed within-group TMLE method. Even if the propensity scores are partially misspecified, the within-group TMLE still produces robust ATE estimates due to double robustness with flexible modeling, unlike parametric-based inverse propensity weighting methods. We demonstrate our proposed methods and conduct sensitivity analyses against the number of groups and individual-level unmeasured confounding to evaluate the effect of taking an eighth-grade algebra course on math achievement in the Early Childhood Longitudinal Study.

Qualitative comparative analysis (QCA) is an empirical research method that has gained some popularity in the social sciences. At the same time, the literature has long been convinced that QCA is prone to committing causal fallacies when confronted with non-causal data. More specifically, beyond a certain case-to-factor ratio, the method is believed to fail in recognizing real data. To reduce that risk, some authors have proposed benchmark tables that put a limit on the number of exogenous factors given a certain number of cases. Many applied researchers looking for methodological guidance have since adhered to these tables. We argue that fears of inferential breakdown in QCA due to an "unfavorable" case-to-factor ratio are without foundation. What is more, we demonstrate that these benchmarks induce more fallacious inferences than they prevent. For valid causal inference, researchers are better off relying on the current state of knowledge in their respective fields.

Dynamic structural equation modeling (DSEM) is a useful technique for analyzing intensive longitudinal data. A challenge of applying DSEM is the missing data problem. The impact of missing data on DSEM, especially on widely applied DSEM such as the two-level vector autoregressive (VAR) cross-lagged models, however, is understudied. To fill the research gap, we evaluated how well the fixed effects and variance parameters in two-level bivariate VAR models are recovered under different missingness percentages, sample sizes, the number of time points, and heterogeneity in missingness distributions through two simulation studies. To facilitate the use of DSEM under customized data and model scenarios (different from those in our simulations), we provided illustrative examples of how to conduct Monte Carlo simulations in M"plus" to determine whether a data configuration is sufficient to obtain accurate and precise results from a specific DSEM.

A necessary step in applying bi-factor models is to evaluate the need for domain factors with a general factor in place. The conventional null hypothesis testing (NHT) was commonly used for such a purpose. However, the conventional NHT meets challenges when the domain loadings are weak or the sample size is insufficient. This article proposes using minimal-effect testing (MET) and equivalence testing (ET) to analyze bi-factor models. A key element in conducting MET and ET is the minimal size of factor loadings that can be regarded as noteworthy in practice, termed as "minimal noteworthy size." This article presents two approaches to formulating the minimal noteworthy size and compares the pros and cons of MET, ET, and the conventional NHT. Analysis shows that MET, ET, and the conventional NHT are complementary. Combining them to test the noteworthiness of domain loadings can help researchers make a comprehensive judgment. Real and simulated datasets illustrate the applications of the three methods. Monte Carlo results show that MET and ET can control type I errors reasonably well while maintaining good statistical power.

This report characterizes forms of dialogic support that a sixth-grade teacher generated during whole-class and small-group conversations to help students develop a practice of statistical modeling. During four weeks of instruction, students constructed and revised models to account for variability and uncertainty across a variety of random processes, many of which they experienced first-hand. Data sources for the research included field notes and video recordings of classroom conversations involving the teacher. Analyses identified and characterized teacher assistance during classroom episodes in which the teacher fostered the emergence and development of fundamental aspects of statistical modeling practice. These aspects included a) establishing mappings between models of chance and the situations or processes being modeled, b) imagining outcomes as constituents of a long-run random process, c) evaluating model fit, and d) making model-based inferences. Throughout instruction, the teacher systematically helped students coordinate these distinctive aspects of modeling practice. The findings suggest how teacher practices can promote the sustained development of statistical reasoning as a dialogue between data and model, and more generally, how teacher assistance attuned to student thinking about aspects of a disciplinary practice can foster a student-centered phenomenology of that practice.

This article introduces a collection of four datasets, similar to Anscombe's quartet, that aim to highlight the challenges involved when estimating causal effects. Each of the four datasets is generated based on a distinct causal mechanism: the first involves a collider, the second involves a confounder, the third involves a mediator, and the fourth involves the induction of M-Bias by an included factor. The article includes a mathematical summary of each dataset, as well as directed acyclic graphs that depict the relationships between the variables. Despite the fact that the statistical summaries and visualizations for each dataset are identical, the true causal effect differs, and estimating it correctly requires knowledge of the data-generating mechanism. These example datasets can help practitioners gain a better understanding of the assumptions underlying causal inference methods and emphasize the importance of gathering more information beyond what can be obtained from statistical tools alone. The article also includes R code for reproducing all figures and provides access to the datasets themselves through an R package named "quartets." Supplementary materials for this article are available online.

The present work focuses on the performance of two types of shrinkage priors--the horseshoe prior and the recently developed regularized horseshoe prior--in the context of inducing sparsity in path analysis and growth curve models. Prior research has shown that these horseshoe priors induce sparsity by at least as much as the "gold standard" spike-and-slab prior. The horseshoe priors are compared to the ridge prior and lasso prior, as well as default non-informative priors, in terms of the percent shrinkage in the model parameters and out-of-sample predictive performance. Empirical studies using data from two large-scale educational assessments reveal the clear advantages of the horseshoe priors in terms of both shrinkage and predictive performance. Simulation studies reveal clear advantages in terms of shrinkage, but less obvious advantages in terms of predictive performance, except in the small sample size condition where both horseshoe priors provide noticeably improved predictive performance.

Latent class analysis (LCA) is a widely used technique for detecting unobserved population heterogeneity in cross-sectional data. Despite its popularity, the performance of LCA is not well understood. In this study, we evaluate the performance of LCA with binary data by examining classification accuracy, parameter estimation accuracy, and coverage rates of confidence intervals (CIs) through Monte Carlo simulation studies. We address the issue of label switching with a distance-based relabeling approach and introduce an index to measure separation among latent classes. Our results show that classification accuracy, parameter estimation accuracy, and CI coverage rates are primarily influenced by class separation and the number of indicators used for LCA. We recommend using a large sample size to mitigate the effects of tiny class sizes. Additionally, the study finds that the parametric bootstrap CIs perform comparably well or better when compared with the CIs based on the standard maximum likelihood method.

In this article, we extend Liao's test for across-group comparisons of the fixed effects from the generalized linear model to the fixed and random effects of the generalized linear mixed model (GLMM). Using as our basis the Wald statistic, we developed an asymptotic test statistic for across-group comparisons of these effects. The test can be applied when the fixed and random effects are multivariate normally distributed, and it works well for any link function and conditional distribution of the dependent variable of the GLMM. We also derived the asymptotic properties of this test, and because power information does not exist for either our new test statistic or Liao's test, we implemented a power study to demonstrate the superiority of these tests over the alternatively proposed F test. Using an example, we show the application of the test and then discuss its possible restrictions with respect to the distribution of the random effects.

Structural equation modeling (SEM) is a statistical method widely used in educational research to investigate relationships between variables. SEM models are typically constructed based on theoretical foundations and assessed through fit indices. However, a well-fitting SEM model alone is not sufficient to verify the causal inferences underlying the proposed model, as there are statistically equivalent models with distinct causal structures that equally well fit the data. Therefore, it is crucial for researchers using SEM to consider statistically equivalent models and to clarify why the proposed model is more accurate than the equivalent ones. However, many SEM studies did not explicitly address this important step, and no prior study in physics education research has delved into potential methods for distinguishing statistically equivalent models with differing causal structures. In this study, we use a physics identity model as an example to discuss the importance of considering statistically equivalent models and how other data can help to distinguish them. Previous research has identified three dimensions of physics identity: perceived recognition, self-efficacy, and interest. However, the relationships between these dimensions have not been thoroughly understood. In this paper, we specify a model with perceived recognition predicting self-efficacy and interest, which is inspired by individual interviews with students in physics courses to make physics learning environments equitable and inclusive. We test our model with fit indices and discuss its statistically equivalent models with different causal inferences among perceived recognition, self-efficacy, and interest. We then discuss potential experiments that could further empirically test the causal inferences underlying the models, aiding the refinement to a more accurate causal model for guiding educational improvements.

Accounting for model misspecification in Bayesian structural equation models is an active area of research. We present a uniquely Bayesian approach to misspecification that models the degree of misspecification as a parameter--a parameter akin to the correlation root mean squared residual. The misspecification parameter can be interpreted on its own terms as a measure of absolute model fit and allows for comparing different models fit to the same data. By estimating the degree of misspecification simultaneously with structural parameters, the uncertainty about structural parameters reflects the degree of model misspecification. This results in a model that produces more reliable inference than extant Bayesian structural equation modeling. In addition, the approach estimates the residual covariance matrix that can be the basis for diagnosing misspecifications and updating a hypothesized model. These features are confirmed using simulation studies. Demonstrations with a variety of real-world examples show additional properties of the approach.