Search

Network meta-analysis (NMA) allows the combination of evidence on the effectiveness of several interventions. NMA has mainly been applied in the medical science field, whereas in the domain of psychology and educational sciences its use is less frequent. Consequently, systematic reviews that describe the characteristics of published NMAs are limited to the field of medicine, and nothing is known about the characteristics of NMAs published in the psychology and educational sciences field. However, this information is still relevant for the design of future simulation studies and for detecting good and bad research practices. Thus, this study describes the features of the meta-analytic datasets of NMAs published in the field of psychology and educational sciences, as well as their methodological characteristics, and compares them to those observed in the medical domain. Results show that the number of studies included is larger in NMAs from psychology and educational sciences, the most commonly used effect size is the standardized mean difference (unlike the odds ratio in medicine), the sample size is smaller, more intervention groups are included, and inconsistent effects are observed more often. These results can be used in future simulation studies to generate realistic datasets. Finally, we warn about the poor quality of reporting of some technical aspects of the NMA, such as the statistical model used. ispartof: BEHAVIOR RESEARCH METHODS vol:55 issue:4 pages:2093-2108 ispartof: location:United States status: published

Some previous research works have reported potential benefits of screen media use in children aged under 6 years, and others have evidenced that screen media use might be particularly detrimental. To our knowledge, this is the first study to provide the first meta-analytic synthesis of existing research on the relation between overall screen time use and EFs. To address this issue, the current meta-analysis aimed to review the relations between overall screen time and EF in infants, toddlers and preschoolers. A systematic search was done on Web of Science and EBSCO to identify the eligible studies published until January 2023. Fifteen manuscripts with 6922 participants aged 0–6 years were included, and yielded 44 effect sizes. Three-level models were carried out, and the following study characteristics were tested as potential moderators: mean age, percentage of females, EF type, and whether exposure was active or passive. There was no statistically significant association in the relation between overall time use and EF or in the chosen moderators. Nevertheless, the study highlights the need to consider other contextual-related and development-related factors to determine the overall screen time use effect on EF in children.

To conduct a multilevel meta-analysis of multiple single-case experimental design (SCED) studies, the individual participant data (IPD) can be analyzed in one or two stages. In the one-stage approach, a multilevel model is estimated based on the raw data. In the two-stage approach, an effect size is calculated for each participant and these effect sizes and their sampling variances are subsequently combined to estimate a meta-analytic multilevel model. The multilevel model in the two-stage approach has fewer parameters to estimate, in exchange for the reduction of information of the raw data to effect sizes. In this paper we explore how the one-stage and two-stage IPD approaches can be applied in the context of meta-analysis of single-case designs. Both approaches are compared for several single-case designs of increasing complexity. Through a simulation study we show that the two-stage approach obtains better convergence rates for more complex models, but that model estimation does not necessarily converge at a faster speed. The point estimates of the fixed effects are unbiased for both approaches across all models, as such confirming results from methodological research on IPD meta-analysis of group-comparison designs. In light of these results, we discuss the implementation of both methods in R. ispartof: MULTIVARIATE BEHAVIORAL RESEARCH vol:57 issue:2-3 ispartof: location:United States status: published

In meta-analysis, primary studies often include multiple, dependent effect sizes. Several methods address this dependency, such as the multivariate approach, three-level models, and the robust variance estimation (RVE) method. As for today, most simulation studies that explore the performance of these methods have focused on the estimation of the overall effect size. However, researchers are sometimes interested in obtaining separate effect size estimates for different types of outcomes. A recent simulation study (Park & Beretvas, 2019) has compared the performance of the three-level approach and the RVE method in estimating outcome-specific effects when several effect sizes are reported for different types of outcomes within studies. The goal of this paper is to extend that study by incorporating additional simulation conditions and by exploring the performance of additional models, such as the multivariate model, a three-level model that specifies different study-effects for each type of outcome, a three-level model that specifies a common study-effect for all outcomes, and separate three-level models for each type of outcome. Additionally, we also tested whether the a posteriori application of the RV correction improves the standard error estimates and the 95% confidence intervals. Results show that the application of separate three-level models for each type of outcome is the only approach that consistently gives adequate standard error estimates. Also, the a posteriori application of the RV correction results in correct 95% confidence intervals in all models, even if they are misspecified, meaning that Type I error rate is adequate when the RV correction is implemented. ispartof: BEHAVIOR RESEARCH METHODS vol:53 issue:2 pages:702-717 ispartof: location:United States status: published

Background.The implications of cannabis use in the onset of early psychosis and the severity of psychotic symptoms have resulted in a proliferation of studies on this issue. However, few have examined the effects of cannabis use on the cognitive symptoms of psychosis (i.e., neurocognitive functioning) in patients with first-episode psychosis (FEP). This systematic review and meta-analysis aim to assess the neurocognitive functioning of cannabis users (CU) and nonusers (NU) with FEP.Methods.Of the 110 studies identified through the systematic review of 6 databases, 7 met the inclusion criteria, resulting in 14 independent samples and 78 effect sizes. The total sample included 304 CU with FEP and 369 NU with FEP. The moderator variables were age at first use, duration of use, percentage of males, and age.Results.Effect sizes were not significantly different from zero in any neurocognitive domain when users and NU were compared. Part of the variability in effect sizes was explained by the inclusion of the following moderator variables: (1) frequency of cannabis use (β= 0.013,F= 7.56,p= 0.017); (2) first-generation antipsychotics (β= 0.019,F= 34.46,p≤ 0.001); and (3) country where the study was carried out (β= 0.266,t= 2.06,p= 0.043).Conclusions.This meta-analysis indicates that cannabis use is not generally associated with neurocognitive functioning in patients with FEP. However, it highlights the deleterious effect of low doses of cannabis in some patients. It also stresses the importance of the type of antipsychotic prescription and cannabis dose as moderator variables in the neurocognitive functioning of CU with FEP.

This study attempts to clarify the effects of varying testing conditions on creative performance in divergent thinking and product-based tasks. Two meta-analyses, one for time limits (short vs. long) and one for instructions (standard vs. explicit), were conducted. Moderator analyses were performed to examine whether the effects of time limits and instructions differ by measurement method, domain of creative performance, gender, study’s country of origin, educational level, study quality, and scoring method of originality. It was also investigated whether the effects of time limits vary depending on whether long time condition is timed or untimed, and whether the effects of instructions vary under timed versus untimed conditions. A total of 57 effect sizes (12 studies) for time limits and 165 effect sizes (38 studies) for instructions were analyzed using a meta-analytic three-level model. The time limits meta-analysis showed that long time limits (vs. short time limits) significantly enhanced creative performance, with smaller effects in studies from the USA (vs. non-USA). Analyses on divergent thinking indicators showed that long time limits (vs. short time limits) yielded higher originality scores, with no significant differences for fluency or flexibility. Long time limits showed greater effects on fluency and flexibility in timed (vs. untimed) long conditions. The instructions meta-analysis revealed that explicit (vs. standard) instructions significantly enhanced creative performance. An examination of divergent thinking indicators showed that explicit (vs. standard) instructions resulted in higher originality scores, with no significant differences for fluency or flexibility. Explicit instructions demonstrated larger effects on originality in non-college (vs. college) subjects, as well as in untimed (vs. timed) conditions. ispartof: Psychology of Aesthetics, Creativity, and the Arts vol:14 issue:1 pages:1-24 status: Published online

The development of divergent thinking (DT) in school-age children and adolescents has received considerable attention in the educational psychology literature since the 1970s. A body of research has outlined the existence of slumps (i.e., temporary declines) in this development with, however, conflicting findings concerning the magnitude and timing of these slumps. This study is the first to meta-analyze prior research findings regarding DT development from Grades 1 to 12, with a particular emphasis on the widely controversial fourth-grade slump. A total of 2139 standardized means from 41 studies involving 40,918 subjects were analyzed using a meta-analytic three-level model. The findings showed an overall upward developmental trend of DT across grade levels, with some discontinuities. Specifically, there was no evidence of a general fourth-grade slump; rather, evidences for a seventh-grade slump were found. Moderator analyses indicated that a fourth-grade slump might be observed depending on DT test, task content domain, intellectual giftedness, and country of study. The existence of the seventh-grade slump was also moderated by DT test, task content domain, and gender. Together, this study deciphers a longstanding debate regarding DT development, a prerequisite knowledge to support age-appropriate educational strategies that encourage creativity development. Implications of these findings for creativity research and practice are discussed.

Previous research found bias in the estimate of the overall fixed effects and variance components using multilevel meta-analyses of standardized single-case data. Therefore, we evaluate two...

© 2018 Psychonomic Society, Inc. It is common for the primary studies in meta-analyses to report multiple effect sizes, generating dependence among them. Hierarchical three-level models have been proposed as a means to deal with this dependency. Sometimes, however, dependency may be due to multiple random factors, and random factors are not necessarily nested, but rather may be crossed. For instance, effect sizes may belong to different studies, and, at the same time, effect sizes might represent the effects on different outcomes. Cross-classified random-effects models (CCREMs) can be used to model this nonhierarchical dependent structure. In this article, we explore by means of a simulation study the performance of CCREMs in comparison with the use of other meta-analytic models and estimation procedures, including the use of three- and two-level models and robust variance estimation. We also evaluated the performance of CCREMs when the underlying data were generated using a multivariate model. The results indicated that, whereas the quality of fixed-effect estimates is unaffected by any misspecification in the model, the standard error estimates of the mean effect size and of the moderator variables’ effects, as well as the variance component estimates, are biased under some conditions. Applying CCREMs led to unbiased fixed-effect and variance component estimates, outperforming the other models. Even when a CCREM was not used to generate the data, applying the CCREM yielded sound parameter estimates and inferences. ispartof: Behavior Research Methods vol:51 issue:3 pages:1286-1304 ispartof: location:United States status: published

© 2017, Springer Science+Business Media Dordrecht. Previous research on children’s and adolescents’ well-being at school has been focused on the possible determinants. However, no previous research has analysed children’s and adolescents’ lay-beliefs or conceptualizations of happiness at school. In the present work, we studied children’s (N = 104, 9–10-year-olds) and adolescents’ (N = 113, 15–16-year-olds) conceptualizations of happiness at school and its link with self-reported happiness (assessed 3 months later) and academic achievement (assessed 7 months later). For both samples, seven conceptualizations emerged: happiness as ‘being with friends’, ‘being praised’, ‘getting good grades’, ‘learning’, ‘leisure’, ‘enjoyment’, and ‘helping’. Age differences appeared for the conceptualizations of ‘being friends’ and ‘helping’, as children mentioned significantly more the former and adolescents the latter. No gender differences emerged. For adolescents, the conceptualizations of happiness at school as ‘being with friends’, ‘being praised’, ‘helping’, and not ‘having leisure time’ were positively related to self-reported happiness, which was positively related to academic achievement. For children, none of the conceptualizations were positively related to self-reported happiness. The conceptualization of happiness as ‘learning’ was positively related to academic achievement. The results are discussed in regards to their implications for children’s and adolescents’ well-being at school. ispartof: JOURNAL OF HAPPINESS STUDIES vol:19 issue:6 pages:1811-1830 status: published

Hyperspectral remote sensing is increasingly being recognized as a powerful tool to map ecosystem properties and functions through time and space. However, general information on the accuracy of this technology to assess the vegetation's biophysical and -chemical trait composition, and on the variables which are mediating this accuracy, is often lacking so far. Here, we addressed this knowledge gap for grass- and shrubland ecosystems and applied novel three-level meta-analytical regression equations to 77 studies that validated hyperspectral remote sensing data with field observations. Our results showed that the accuracy of hyperspectral sensors is generally high, but strongly depends on the trait being studied (leaf area index: R² = 0.79 and nRMSE = 0.19, chlorophyll: R² = 0.77 and nRMSE = 0.21, carotenoids: R² = 0.80 and nRMSE = 0.29, phosphorus: R² = 0.75 and nRMSE = 0.14, nitrogen: R² = 0.74 and nRMSE = 0.09, water: R² = 0.69 and nRMSE = 0.13, and lignin content: R² = 0.64 and nRMSE = 0.26). Moreover, they indicated that the use of multivariate signal processing techniques could improve these estimation accuracies (adjusted p

Background Patients with first episode psychosis (FEP) frequently report cannabis use although its effects on cognitive functioning are still unclear. Several studies suggest a decrease in the executive function, verbal memory and working memory of FEP cannabis users (González-Pinto et al., 2016; Mata et al. 2008) while other studies show improvements in the neurocognitive function of this group (Setién-Suero et al., 2017, Cuhna et al., 2013, Leeson et al., 2012, Yücel et al., 2012; Rodríguez-Sánchez et al., 2010) or even absence of neurocognitive differences between FEP cannabis users and non-users (Burgra et al., 2013). This meta-analysis aims to explore the magnitude of effect of cannabis use on neurocognition in patients with FEP. Methods Articles for consideration were identified through extensive literature searches using online databases, which included PubMed, Medline and PsychInfo. The search was limited to English language articles. The used keywords were: “first episode psychosis” OR, “neurocognition and cannabis”, in combination with a number of neuropsychology-related terms including “neurocog*” and “neuropsycholog*”. Given that other substances including alcohol, cocaine, and stimulants are associated with altered cognitive performance, studies in which participants met for poly-substance use disorders, even if there was preferential use towards cannabis, were excluded. Eight studies from 2008 to 2017 met inclusion criteria from a total sample of 16 initial studies. Five hundred and eighteen of these participants were cannabis users with FEP, and 639 were patients with no cannabis use. A total of 58 effect sizes of neuropsychological test variables were categorized into 4 cognitive domains (premorbid IQ, executive functioning, working memory and verbal memory and learning). Age of first cannabis use, duration of cannabis use, percentage of males and age were abstracted and assembled as moderator variables. Standardized mean differences were computed for each cognitive domain between cannabis-using patients and patients with no history of cannabis use. Negative effect sizes would display better cognitive functioning of non-cannabis users. We employed a meta-analytic three level model to combine effect sizes across studies. Results Effect sizes were not significantly different from zero in any of the neurocognitive domains when FEP cannabis users and non-users patients were compared [working memory (d= -0.03, SE=0.15, CI = -0.33–0.26, p=0.83), executive function (d= 0,14, SE=0.16, CI = -0.17–0.45, p=0.37), verbal memory and learning (d= 0.04, SE=0.15, CI = -0.25–0.33, p=0.27) and premorbid IQ (d= 0.06, SE=0.09, CI = -0.24–0.12, p=0.50)]. Only one moderator variable resulted significant in the executive function denoting superior performance in FEP cannabis-using patients as they were older. Discussion Cannabis use is not related to an ameliorated or improved neurocognitive functioning in patients with a first episode psychosis. This is consistent with previous studies which showed absence of differences in the neurocognitive functioning between FEP cannabis users and nonusers (Burgra et al., 2013). However, it has been demonstrated that continued cannabis intake worsens cognitive performance although some of the FEP patients had better premorbid capacities (González-Pinto, 2016). Moreover, the doses and the different types of cannabis preparations may interfere the present results. Meta-analysis on longitudinal studies which include these potential moderator variables may be performed in the future.

IntroductionReproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. Scientific claims should not gain credence because of the status or authority of their originator but by the replicability of their supporting evidence. Even research of exemplary quality may have irreproducible empirical findings because of random or systematic error.RationaleThere is concern about the rate and predictors of reproducibility, but limited evidence. Potentially problematic practices include selective reporting, selective analysis, and insufficient specification of the conditions necessary or sufficient to obtain the results. Direct replication is the attempt to recreate the conditions believed sufficient for obtaining a previously observed finding and is the means of establishing reproducibility of a finding with new data. We conducted a large-scale, collaborative effort to obtain an initial estimate of the reproducibility of psychological science.ResultsWe conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. There is no single standard for evaluating replication success. Here, we evaluated reproducibility using significance and P values, effect sizes, subjective assessments of replication teams, and meta-analysis of effect sizes. The mean effect size (r) of the replication effects (Mr = 0.197, SD = 0.257) was half the magnitude of the mean effect size of the original effects (Mr = 0.403, SD = 0.188), representing a substantial decline. Ninety-seven percent of original studies had significant results (P < .05). Thirty-six percent of replications had significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.ConclusionNo single indicator sufficiently describes replication success, and the five indicators examined here are not the only ways to evaluate reproducibility. Nonetheless, collectively these results offer a clear conclusion: A large portion of replications produced weaker evidence for the original findings despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes. Moreover, correlational evidence is consistent with the conclusion that variation in the strength of initial evidence (such as original P value) was more predictive of replication success than variation in the characteristics of the teams conducting the research (such as experience and expertise). The latter factors certainly can influence replication success, but they did not appear to do so here. Reproducibility is not well understood because the incentives for individual scientists prioritize novelty over replication. Innovation is the engine of discovery and is vital for a productive, effective scientific enterprise. However, innovative ideas become old news fast. Journal reviewers and editors may dismiss a new test of a published idea as unoriginal. The claim that “we already know this” belies the uncertainty of scientific evidence. Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not. This project provides accumulating evidence for many findings in psychological research and suggests that there is still more work to do to verify whether we know what we think we know.