Search

For multilevel models (MLMs) with fixed slopes, it has been widely recognized that a level-1 variable can have distinct between-cluster and within-cluster fixed effects, and that failing to disaggregate these effects yields a conflated, uninterpretable fixed effect. For MLMs with random slopes, however, we clarify that two different types of slope conflation can occur: that of the fixed component (termed fixed conflation) and that of the random component (termed random conflation). The latter is rarely recognized and not well understood. Here we explain that a model commonly used to disaggregate the fixed component—the contextual effect model with random slopes—troublingly still yields a conflated random component. Negative consequences of such random conflation have not been demonstrated. Here we show that they include erroneous interpretation and inferences about the substantively important extent of between-cluster differences in slopes, including either underestimating or overestimating such slope heterogeneity. Furthermore, we show that this random conflation can yield inappropriate standard errors for fixed effects. To aid researchers in practice, we delineate which types of random slope specifications yield an unconflated random component. We demonstrate the advantages of these unconflated models in terms of estimating and testing random slope variance (i.e., improved power, Type I error, and bias) and in terms of standard error estimation for fixed effects (i.e., more accurate standard errors), and make recommendations for which specifications to use for particular research purposes.

Applications of multilevel models (MLMs) with three or more levels have increased alongside expanding software capability and dataset availability. Though researchers often express interest in R-squared measures as effect sizes for MLMs, R-squareds previously proposed for MLMs with three or more levels cover a limited subset of choices for how to quantify explained variance in these models. Additionally, analytic relationships between total and level-specific versions of MLM R-squared measures have not been clarified, despite such relationships becoming increasingly important to understand when there are more levels. Furthermore, the impact of predictor centering strategy on R-squared computation and interpretation has not been explicated for MLMs with any number of levels. To fill these gaps, we extend the Rights and Sterba two-level MLM R-squared framework to three or more levels, providing a general set of measures that includes preexisting three-level measures as special cases and yields additional results not obtainable from existing measures. We mathematically and pedagogically relate total and level-specific R-squareds, and show how all total and level-specific R-squared measures in our framework can be computed under any centering strategy. Finally, we provide and empirically demonstrate software (available in the r2mlm R package) to compute measures and graphically depict results.

Methodologists have often acknowledged that, in multilevel contexts, level-1 variables may have distinct within-cluster and between-cluster effects. However, a prevailing notion in the literature is that separately estimating these effects is primarily important when there is specific interest in doing so. Consequently, in practice, researchers uninterested in disaggregating these effects (or unaware of their difference) routinely fit models that conflate them. Furthermore, even researchers who properly disaggregate the fixed components in a model (avoid

Experiencing stressors related to the COVID-19 pandemic such as health-related concern, social isolation, occupational disruption, financial insecurity, and resource scarcity can adversely impact mental health; however, the extent of the impact varies greatly between individuals. In this study, we examined the role of neuroticism as an individual-level risk factor that exacerbates the association between pandemic stressors and depressive symptoms. With repeated assessments of pandemic stressors and depressive symptoms collected from 3181 participants over the course of the pandemic, we used multilevel modeling to test if neuroticism moderated the association between pandemic stressors and depressive symptoms at both between- and within-person levels. At the between-person level, we found that participants who reported more pandemic stressors on average had higher levels of depressive symptoms and that this association was stronger among those high in neuroticism. At the within-person level, reporting more pandemic stressors relative to one's average on any given occasion was also associated with heightened depressive symptoms and this effect was similarly exacerbated by neuroticism. The findings point to pandemic stressor exposure and neuroticism as risk factors for depressive symptoms and, in demonstrating their synergistic impact, may help identify individuals at greatest risk for adverse psychological responses to the COVID-19 pandemic.

Developmental researchers commonly utilize multilevel models (MLMs) to describe and predict individual differences in change over time. In such growth model applications, researchers have been widely encouraged to supplement reporting of statistical significance with measures of effect size, such as R-squareds (R2 ) that convey variance explained by terms in the model. An integrative framework for computing R-squareds in MLMs with random intercepts and/or slopes was recently introduced by Rights and Sterba and it subsumed pre-existing MLM R-squareds as special cases. However, this work focused on cross-sectional applications, and hence did not address how the computation and interpretation of MLM R-squareds are affected by modeling considerations typically arising in longitudinal settings: (a) alternative centering choices for time (e.g., centering-at-a-constant vs. person-mean-centering), (b) nonlinear effects of predictors such as time, (c) heteroscedastic level-1 errors and/or (d) autocorrelated level-1 errors. This paper addresses these gaps by extending the Rights and Sterba R-squared framework to longitudinal contexts. We: (a) provide a full framework of total and level-specific R-squared measures for MLMs that utilize any type of centering, and contrast these with Rights and Sterba's measures assuming cluster-mean-centering, (b) explain and derive which measures are applicable for MLMs with nonlinear terms, and extend the R-squared computation to accommodate (c) heteroscedastic and/or (d) autocorrelated errors. Additionally, we show how to use differences in R-squared (ΔR2 ) measures between growth models (adding, for instance, time-varying covariates as level-1 predictors or time-invariant covariates as level-2 predictors) to obtain effects sizes for individual terms. We provide R software (r2MLMlong) and a running pedagogical example analyzing growth in adolescent self-efficacy to illustrate these methodological developments. With these developments, researchers will have greater ability to consider effect size when analyzing and predicting change using MLMs.

Questionnaire and mood-induction measures of emotional and cognitive reactivity (ER and CR) have noteworthy strengths as well as potential liabilities (as do virtually all measures of psychological constructs). Toward a solution to the fallible measurement problem, methodologists have long advocated utilizing qualitatively diverse assessment methods to converge upon the constructs of interests. The current article introduces and provides initial validation of the Behind Your Back measure that (unlike any other measure) simultaneously assesses both ER and CR in college students via methods that avoid some of the problems associated with conventional measures. For both ER and CR, 3 dimensions are hypothesized, representing (a) person-driven, (b) event-driven, and (c) person/event-driven aspects of ER and CR. Results support this tripartite structure and provide evidence of convergent and construct validity of the method, including significant relations to self-reported symptoms of depression. Implications of these BYB-derived dimensions for theory and enhancing practice are discussed. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

BackgroundThe incidence of depression in human females rises steadily throughout adolescence, a critical period of pubertal maturation marked by increasing levels of gonadal hormones including estrogens and progesterone. These gonadal hormones play a central role in social and emotional development and may also contribute to the increased occurrence of depression in females that begins in early adolescence. In this study, we examine whether and how introducing synthetic estrogen and progestin derivatives through the use of combined hormonal contraceptives (CHC), affects adolescent females’ risk for developing depression. We further assess potential links between CHC use and alterations in stress responses and social-emotional functioning.MethodsUsing a longitudinal cohort design, we will follow a sample of adolescent females over the span of three years. Participants will be assessed at three time points: once when they are between 13 and 15 years of age, and at approximately 18 and 36 months after their initial assessment. Each time point will consist of two online sessions during which participants will complete a clinical interview that screens for key symptoms of mental health disorders, along with a series of questionnaires assessing their level of depressive symptoms and history of contraceptive use. They will also complete a standardized social-evaluative stress test and an emotion recognition task, as well as provide saliva samples to allow for assessment of their circulating free cortisol levels.DiscussionIn this study we will assess the effect of CHC use during adolescence on development of Major Depressive Disorder (MDD). We will control for variables previously found to or proposed to partially account for the observed relationship between CHC use and MDD, including socioeconomic status, age of sexual debut, and CHC-related variables including age of first use, reasons for use, and its duration. In particular, we will discover whether CHC use increases depressive symptoms and/or MDD, whether elevated depressive symptoms and/or MDD predict a higher likelihood of starting CHC, or both. Furthermore, this study will allow us to clarify whether alterations in stress reactivity and social-emotional functioning serve as pathways through which CHC use may result in increased risk of depressive symptoms and/or MDD.

In the multilevel modelling literature, methodologists widely acknowledge that a level-1 variable can have distinct within-cluster and between-cluster effects, and that failing to disaggregate these can yield a slope estimate that is an uninterpretable, conflated blend of the two. Methodologists have stated, however, that including conflated slopes of level-1 variables in a model is not problematic if substantive interest lies only in effects of level-2 predictors. Researchers commonly follow this advice and use methods that do not disaggregate effects of level-1 control variables (e.g., grand mean centering) when examining effects of level-2 predictors. The primary purpose of this paper is to show that this is a dangerous practice. When level-specific effects of level-1 variables differ, failing to disaggregate them can severely bias estimation of level-2 predictor slopes. We show mathematically why this is the case and highlight factors that can exacerbate such bias. We corroborate these findings with simulations and present an empirical example, showing how such distortions can severely alter substantive conclusions. We ultimately recommend that simply including the cluster mean of the level-1 variable as a control will alleviate the problem.

When comparing multilevel models (MLMs) differing in fixed and/or random effects, researchers have had continuing interest in using R-squared differences to communicate effect size and importance of included terms. However, there has been longstanding confusion regarding which R-squared difference measures should be used for which kind of MLM comparisons. Furthermore, several limitations of recent studies on R-squared differences in MLM have led to misleading or incomplete recommendations for practice. These limitations include computing measures that are by definition incapable of detecting a particular type of added term, considering only a subset of the broader class of available R-squared difference measures, and incorrectly defining what a given R-squared difference measure quantifies. The purpose of this paper is to elucidate and resolve these issues. To do so, we define a more general set of total, within-cluster, and between-cluster R-squared difference measures than previously considered in MLM comparisons and give researchers concrete step-by-step procedures for identifying which measure is relevant to which model comparison. We supply simulated and analytic demonstrations of limitations of previous MLM studies on R-squared differences and show how application of our step-by-step procedures and general set of measures overcomes each. Additionally, we provide and illustrate graphical tools and software allowing researchers to automatically compute and visualize our set of measures in an integrated manner. We conclude with recommendations, as well as extensions involving (a) how our framework relates to and can be used to obtain pseudo-R-squareds, and (b) how our framework can accommodate both simultaneous and hierarchical model-building approaches.

Multilevel models are used ubiquitously in the social and behavioural sciences and effect sizes are critical for contextualizing results. A general framework of R-squared effect size measures for multilevel models has only recently been developed. Rights and Sterba (2019) distinguished each source of explained variance for each possible kind of outcome variance. Though researchers have long desired a comprehensive and coherent approach to computing R-squared measures for multilevel models, the use of this framework has a steep learning curve. The purpose of this tutorial is to introduce and demonstrate using a new R package – r2mlm – that automates the intensive computations involved in implementing the framework and provides accompanying graphics to visualize all multilevel R-squared measures together. We use accessible illustrations with open data and code to demonstrate how to use and interpret the R package output.

An increasing number of studies are applying multilevel modeling (MLM) to daily diary assessments of emotional and cognitive reactivity (ER and CR). Despite their generation of promising results, these methods have yet to be validated. The current study, consisting of 449 participants from over 90 different colleges and universities, had 2 goals: (a) to assess the convergent validity of these methods in relation to more conventional measures, and (b) to assess the construct validity of these methods in relation to depressive symptoms. Results support the extraction of within- versus between-person aspects of both constructs from daily diary data. Evidence of convergent validity derives from the association of MLM-based estimates of ER and CR with established self-report questionnaire methods. Evidence of construct validity derives from the relation of these estimates to symptoms of depression. The value of distinguishing within- from between-person aspects of ER and CR is discussed. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Understanding parenting from both parent and child perspectives is critical to child clinical and developmental research. Similarities and differences between parents' and children's reports can be highly informative, but only if they derive from psychometrically sound measures that assess the same parenting constructs. We examined the psychometric properties of the child and parent forms of the Parenting Perception Inventory (Bruce et al., 2006), which measures perceptions of two higher-order dimensions: positive, warm, supportive parenting; and negative, harsh, critical parenting. Data from a four-wave, longitudinal study of community children and adolescents (n = 876, Mage = 9.5 at the beginning), and data from a study of children (n = 131, Mage = 9.35) of depressed and nondepressed mothers provided psychometric support for both measures. Factor analyses revealed the existence of two factors in both the child and parent forms, and showed strong congruence across the two forms. Other analyses examined longitudinal structure, item difficulty, item discriminations, and scale coverage of the child form. Parents' and children's perceptions of parenting were related to children's affect, emotionality, and depressive symptoms. Parents' perceptions of parenting were related to parents' depressive symptoms and to parenting self-efficacy. (PsycINFO Database Record

Objective This study aimed to characterize mothers' communication with their children in a sample of families with a new or newly relapsed pediatric cancer diagnosis, first using factor analysis and second using structural equation modeling to examine relations between self-reported maternal distress (anxiety, depression, and posttraumatic stress) and maternal communication in prospective analyses. A hierarchical model of communication was proposed, based on a theoretical framework of warmth and control. Methods The sample included 115 children (age 5-17 years) with new or newly relapsed cancer (41% leukemia, 18% lymphoma, 6% brain tumor, and 35% other) and their mothers. Mothers reported distress (Beck Anxiety Inventory, Beck Depression Inventory-II, and Impact of Events Scale-Revised) 2 months after diagnosis (Time 1). Three months later (Time 2), mother-child dyads were video-recorded discussing cancer. Maternal communication was coded with the Iowa Family Interaction Ratings Scales. Results Confirmatory factor analysis demonstrated poor fit. Exploratory factor analysis suggested a six-factor model (root mean square error of approximation = .04) with one factor reflecting Positive Communication, four factors reflecting Negative Communication (Hostile/Intrusive, Lecturing, Withdrawn, and Inconsistent), and one factor reflecting Expression of Negative Affect. Maternal distress symptoms at Time 1 were all significantly, negatively related to Positive Communication and differentially related to Negative Communication factors at Time 2. Maternal posttraumatic stress and depressive symptoms each predicted Expression of Negative Affect. Conclusions Findings provide a nuanced understanding of maternal communication in pediatric cancer and identify prospective pathways of risk between maternal distress and communication that can be targeted in intervention.

Item banks are often created in large-scale research and testing settings in the social sciences to predict individuals’ latent trait scores. A common procedure is to fit multiple candidate item response theory (IRT) models to a calibration sample and select a single best-fitting IRT model. The parameter estimates from this model are then used to obtain trait scores for subsequent respondents. However, this model selection procedure ignores model uncertainty stemming from the fact that the model ranking in the calibration phase is subject to sampling variability. Consequently, the standard errors of trait scores obtained from subsequent respondents do not reflect such uncertainty. Ignoring such sources of uncertainty contributes to the current replication crisis in the social sciences. In this article, we propose and demonstrate an alternative procedure to account for model uncertainty in this context—model averaging of IRT trait scores and their standard errors. We outline the general procedure step-by-step and provide software to aid researchers in implementation, both for large-scale research settings with item banks and for smaller research settings involving IRT scoring. We then demonstrate the procedure with a simulated item-banking illustration, comparing model selection and model averaging within sample in terms of predictive coverage. We conclude by discussing ways that model averaging and IRT scoring can be used and investigated in future research.

The longitudinal structure of depression in children and adolescents was examined by applying a Trait-State-Occasion structural equation model to 4 waves of self, teacher, peer, and parent reports in 2 age groups (9 to 13 and 13 to 16 years old). Analyses revealed that the depression latent variable consisted of 2 longitudinal factors: a time-invariant dimension that was completely stable over time and a time-varying dimension that was not perfectly stable over time. Different sources of information were differentially sensitive to these 2 dimensions. Among adolescents, self- and parent reports better reflected the time-invariant aspects. For children and adolescents, peer and teacher reports better reflected the time-varying aspects. Relatively high cross-informant agreement emerged for the time-invariant dimension in both children and adolescents. Cross-informant agreement for the time-varying dimension was high for adolescents but very low for children. Implications emerge for theoretical models of depression and for its measurement, especially when attempting to predict changes in depression in the context of longitudinal studies. (PsycINFO Database Record

Prior theory and research have linked negative appraisals (NA), emotion reactivity (ER), and cognitive reactivity (CR) to depression; however, few studies have examined whether even two of these constructs simultaneously, but none have done so in child or adolescent populations. A total of 571 youths (ages 9–13) completed a novel procedure in which all three constructs were assessed in response to the same personally relevant, hypothetical, peer victimization events. Multilevel modeling enabled the extraction of dynamic, within-person, latent-variable measures of NA, ER, and CR. All three constructs were related to children's depressive symptoms in ways that were commensurate with most (but not all) theoretical frameworks. Gender and age differences also emerged. Support for an NA-predicts-ER-predicts-CR model suggests ways that these constructs can be integrated into a more complete, transtheoretical understanding of the cognitive-emotional substrate of depression in children.

This study aims to 1) examine the temporal influence of peer victimization on mood, sleep quality, pain, and activity limitations in clinical and community samples of youth, and 2) test mood and sleep as mediators of peer victimization-pain pathways. One hundred fifty-six adolescents (n = 74 chronic pain group) completed a week of online diary monitoring assessing their daily peer victimization experiences, negative mood, sleep quality, pain intensity, and pain-related activity limitations. In multilevel models controlling for group status, person-mean peer victimization (averaged across days) significantly predicted worse mood, pain, and activity limitations (all Ps.01) while daily victimization predicted worse mood (P.05). Results from within-person mediation indicated a significant indirect effect of daily peer victimization on next-day activity limitations, through daily negative mood. Results from between-person mediation indicated that negative mood significantly mediated the relation between peer victimization and pain and the relation between peer victimization and activity limitations. Peer victimization is associated with negative health indicators in clinical and community samples of youth and may exert its influence on pain and pain-related activity limitations through negative mood. PERSPECTIVE: This article examines the temporal influence of peer victimization on pain in adolescents with and without chronic pain, and examines mood and sleep quality as mechanisms linking victimization to pain. This information may be useful for pain prevention researchers as well as providers who assess and treat pain in childhood.

Researchers often mention the utility and need for R-squared measures of explained variance for multilevel models (MLMs). Although this topic has been addressed by methodologists, the MLM R-squared literature suffers from several shortcomings: (a) analytic relationships among existing measures have not been established so measures equivalent in the population have been redeveloped 2 or 3 times; (b) a completely full partitioning of variance has not been used to create measures, leading to gaps in the availability of measures to address key substantive questions; (c) a unifying approach to interpreting and choosing among measures has not been provided, leading to researchers' difficulty with implementation; and (d) software has inconsistently and infrequently incorporated available measures. We address these issues with the following contributions. We develop an integrative framework of R-squared measures for MLMs with random intercepts and/or slopes based on a completely full decomposition of variance. We analytically relate 10 existing measures from different disciplines as special cases of 5 measures from our framework. We show how our framework fills gaps by supplying additional total and level-specific measures that answer new substantive research questions. To facilitate interpretation, we provide a novel and integrative graphical representation of all the measures in the framework; we use it to demonstrate limitations of current reporting practices for MLM R-squareds, as well as benefits of considering multiple measures from the framework in juxtaposition. We supply and empirically illustrate an R function, r2MLM, that computes all measures in our framework to help researchers in considering effect size and conveying practical significance. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Psychologists commonly apply regression mixture models in single-level (i.e., unclustered) and multilevel (i.e., clustered) data analysis contexts. Though researchers applying nonmixture regression models typically report R-squared measures of explained variance, there has been no general treatment of R-squared measures for single-level and multilevel regression mixtures. Consequently, it is common for researchers to summarize results of a fitted regression mixture by simply reporting class-specific regression coefficients and their associated p values, rather than considering measures of effect size. In this article, we fill this gap by providing an integrative framework of R-squared measures for single-level regression mixture models and multilevel regression mixture models (with classes at Level-2 or both levels). Specifically, we describe 11 R-squared measures that are distinguished based on what the researcher chooses to consider as outcome variance and what sources the researcher chooses to contribute to predicted variance. We relate these measures analytically and through simulated graphical illustrations. Further, we demonstrate how these R-squared measures can be decomposed in novel ways into substantively meaningful sources of explained variance. We describe and demonstrate new software tools to allow researchers to compute these R-squared measures and decompositions in practice. Using 2 empirical examples, we show how researchers can answer distinct substantive questions with each measure and can gain insights by interpreting the set of measures in juxtaposition to each other. (PsycINFO Database Record

Item parceling remains widely used under conditions that can lead to parcel-allocation variability in results. Hence, researchers may be interested in quantifying and accounting for parcel-allocation variability within sample. To do so in practice, three key issues need to be addressed. First, how can we combine sources of uncertainty arising from sampling variability and parcel-allocation variability when drawing inferences about parameters in structural equation models? Second, on what basis can we choose the number of repeated item-to-parcel allocations within sample? Third, how can we diagnose and report proportions of total variability per estimate arising due to parcel-allocation variability versus sampling variability? This article addresses these three methodological issues. Developments are illustrated using simulated and empirical examples, and software for implementing them is provided.

Research interest often lies in comparing structural model specifications implying different relationships among latent factors. In this context parceling is commonly accepted, assuming the item-level measurement structure is well known and, conservatively, assuming items are unidimensional in the population. Under these assumptions, researchers compare competing structural models, each specified using the same parcel-level measurement model. However, little is known about consequences of parceling for model selection in this context-including whether and when model ranking could vary across alternative item-to-parcel allocations within-sample. This article first provides a theoretical framework that predicts the occurrence of parcel-allocation variability (PAV) in model selection index values and its consequences for PAV in ranking of competing structural models. These predictions are then investigated via simulation. We show that conditions known to manifest PAV in absolute fit of a single model may or may not manifest PAV in model ranking. Thus, one cannot assume that low PAV in absolute fit implies a lack of PAV in ranking, and vice versa. PAV in ranking is shown to occur under a variety of conditions, including large samples. To provide an empirically supported strategy for selecting a model when PAV in ranking exists, we draw on relationships between structural model rankings in parcel- versus item-solutions. This strategy employs the across-allocation modal ranking. We developed software tools for implementing this strategy in practice, and illustrate them with an example. Even if a researcher has substantive reason to prefer one particular allocation, investigating PAV in ranking within-sample still provides an informative sensitivity analysis. (PsycINFO Database Record