Search

Multiple baseline (MB) designs are becoming more prevalent in educational and behavioral research, and as they do, there is growing interest in combining effect size estimates across studies. To further refine the meta-analytic methods of estimating the effect, this study developed and compared eight alternative methods of estimating intervention effects from a set of MB studies. The methods differed in the assumptions made and varied in whether they relied on within- or between-series comparisons, modeled raw data or effect sizes, and did or did not standardize. Small sample functioning was examined through two simulation studies, which showed that when data were consistent with assumptions the bias was consistently less than 5% of the effect size for each method, whereas root mean squared error varied substantially across methods. When assumptions were violated, substantial biases were found. Implications and limitations are discussed.

Reading is a fundamental skill to acquire during children's school career. The present meta-analysis examined research on the effectiveness of digital technologies to foster early reading skills during Tier-1 interventions (ie, high-quality core reading instruction which is intended to promote learning for all children). Unlike previous meta-analyses, this meta-analysis investigated the effectiveness in a broad way, taking into account cognitive versus non-cognitive learning outcomes, near versus far transfer outcomes and immediate versus delayed outcomes. Furthermore, different study characteristics were taken into account including participant characteristics, the targeted reading subskills, duration of intervention, type of technology and the level of integration. A total of 568 effect sizes from 72 studies encompassing 60,890 participants were analysed using a meta-analytic three-level model. A Hedges' g effect size of 0.37 was obtained, suggesting that using digital technologies generally have a positive, albeit small, effect compared to traditional teaching methods. Moderator analyses indicated that this effect was robust to cognitive and non-cognitive outcomes, near and far transfer outcomes, and immediate and delayed outcomes, but differed by participants' age and study quality. Recommendations are formulated to push forward research on how digital interventions can be effectively implemented in the classroom.

Recent studies demonstrated that the adult-preschooler interaction during shared book reading (SBR) contributes to its effectiveness (Mol et al., 2008). The level of abstraction, or complexity, of the mathematical questions adults formulate during SBR serves as an indicator of the interaction quality. We aimed to investigate the chance of spontaneously formulating a mathematical question and the level of abstraction of the mathematical questions future preschool teachers propose to formulate during SBR, and their association with teachers' professional knowledge and beliefs, and type of picture book. Participants were 111 future preschool teachers. We investigated their chance of formulating a mathematical question and the level of abstraction of their mathematical questions using a video-based instrument and distinguished between two types of picture books, namely, mathematical and non-mathematical picture books. We additionally assessed their (1) mathematical content knowledge, (2) mathematical pedagogical content knowledge, and (3) beliefs about mathematics in general and about the teaching and learning of mathematics, with three online questionnaires. Data were analyzed using multilevel analyses. Results revealed that mathematical picture books increase the likelihood of formulating a mathematical question and provoked more abstract mathematical questions compared to non-mathematical picture books. There were no significant associations between teachers' professional knowledge and beliefs and the dependent variables. Our findings point to the importance of adequately selecting picture books to stimulate mathematical preschoolers' development via SBR and also call for further investigations on the learning-supportive picture book characteristics and teacher characteristics.

This article presents an updated meta-analysis of survey experiments comparing the performance of the item count technique (ICT) and the direct questioning method. After synthesizing 246 effect sizes from 54 studies, we find that the probability that a sensitive item will be selected is 0.089 higher when using ICT compared to direct questioning. In recognition of the heterogeneity across studies, we seek to explain this variation by means of moderator analyses. We find that the relative effectiveness of ICT is moderated by cultural orientation in the context in which ICT is conducted (collectivism vs. individualism), the valence of topics involved in the applications (socially desirable vs. socially undesirable), and the number of nonkey items. In the Discussion section, we elaborate on the methodological implications of the main findings.

In cognitive diagnosis assessment (CDA), the impact of misspecified item-attribute relations (or "Q-matrix") designed by subject-matter experts has been a great challenge to real-world applications. This study examined parameter estimation of the CDA with the expert-designed Q-matrix and two refined Q-matrices for international large-scale data. Specifically, the G-DINA model was used to analyze TIMSS data for Grade 8 for five selected countries separately; and the need of a refined Q-matrix specific to the country was investigated. The results suggested that the two refined Q-matrices fitted the data better than the expert-designed Q-matrix, and the stepwise validation method performed better than the nonparametric classification method, resulting in a substantively different classification of students in attribute mastery patterns and different item parameter estimates. The results confirmed that the use of country-specific Q-matrices based on the G-DINA model led to a better fit compared to a universal expert-designed Q-matrix.

This study examined the temporal relationship between attitude toward mathematics and mathematics achievement among first-year secondary (i.e. middle) school students (grade 7, about 14-15 years) with a focus on sex differences in Central Uganda. Random-intercept cross-lagged panel models, based on structural equation modeling, were used to analyze data which were collected through students' questionnaires and mathematics tests at three measurement points. The proportion of variance at the class and school level was larger for the achievement measures than for the attitude measures. At these two levels, also the correlation coefficients between the two constructs were higher for boys than for girls. The study found evidence in support of the reciprocal-effects model as the best-fitting structural model within and across both sexes. Between the first and second measurement points, there was evidence for the self-enhancement model (attitude influencing achievement), while between the second and third points, the skill-development model (achievement influencing attitude) was supported.

During the last decade, many governments and ed-tech companies have demonstrated an increased interest in digital personalised learning, which resulted in a variety of often game-like adaptive learning environments. However, there has been limited attention for the impact of these personalised learning technologies on children's learning efficiency. Does digital personalised learning, like popular claims insist, foster learning in young children? This study attempts to empirically validate the beneficial impact of adaptive learning technology by analysing log-data from the Number Sense Game (NSG), an educational game that trains early numerical skills. In total, 81 children were randomly assigned to use either an adaptive or a non-adaptive version of the NSG in six sessions in a three-week period. Using a longitudinal item response model children's progress within and across sessions was modelled and compared between the two versions of the game. Regardless of the version of the NSG, children demonstrated progress within and across sessions. However, compared to the non-adaptive NSG, the progress across sessions was stronger in the adaptive NSG. These results provide empirical evidence that adaptive learning environments can improve learning efficiency in young children.

The development of divergent thinking (DT) in school-age children and adolescents has received considerable attention in the educational psychology literature since the 1970s. A body of research has outlined the existence of slumps (i.e., temporary declines) in this development with, however, conflicting findings concerning the magnitude and timing of these slumps. This study is the first to meta-analyze prior research findings regarding DT development from Grades 1 to 12, with a particular emphasis on the widely controversial fourth-grade slump. A total of 2139 standardized means from 41 studies involving 40,918 subjects were analyzed using a meta-analytic three-level model. The findings showed an overall upward developmental trend of DT across grade levels, with some discontinuities. Specifically, there was no evidence of a general fourth-grade slump; rather, evidences for a seventh-grade slump were found. Moderator analyses indicated that a fourth-grade slump might be observed depending on DT test, task content domain, intellectual giftedness, and country of study. The existence of the seventh-grade slump was also moderated by DT test, task content domain, and gender. Together, this study deciphers a longstanding debate regarding DT development, a prerequisite knowledge to support age-appropriate educational strategies that encourage creativity development. Implications of these findings for creativity research and practice are discussed.

This study explores the performance of classical methods for detecting publication bias--namely, Egger's regression test, Funnel Plot test, Begg's Rank Correlation and Trim and Fill method--in meta-analysis of studies that report multiple effects. Publication bias, outcome reporting bias, and a combination of these were generated. Egger's regression test and the Funnel Plot test were extended to three-level models, and possible cutoffs for the estimator of the Trim and Fill method were explored. Furthermore, we checked whether the combination of results of several methods yielded a better control of Type I error rates. Results show that no method works well across all conditions and that performance depends mainly on the population effect size value and the total variance.

This study reports on the effects of a 1-year parental involvement intervention on reading development of primary school children. The intervention included a teacher and parent training, focusing on teacher-parent communication and parental involvement in homework and reading at home. We used a cluster randomized controlled trial design with pre-, post- and follow-up measurements to evaluate the intervention. Participants were 600 second grade children (n = 264 in the intervention group, n = 336 in the control group), with their parents and teachers from 24 schools in Dar Es Salaam, Tanzania. A hierarchical linear model, which assessed children's reading growth across three time points, demonstrated that compared to children in the control condition, children in the intervention condition made significantly more progress in decoding skills, reading fluency and reading comprehension from pre- to post-intervention, and from pre-intervention to follow-up (8-9 months after the intervention). We discuss the usefulness of this intervention in helping children who are at risk for reading problems in primary school.

Previous research found bias in the estimate of the overall fixed effects and variance components using multilevel meta-analyses of standardized single-case data. Therefore, we evaluate two adjustments in an attempt to reduce the bias and improve the statistical properties of the parameter estimates. The results confirm the existence of bias when estimating an overall effect based on standardized data and bias when estimating variance components. The relative bias decreases as the number of measurement occasions increases. The results indicate that the between-case variance and, to a smaller degree, the between-study variance is overestimated, especially when measurement occasions per phase is only 10. In general, Hedges's bias correction procedure led to better estimations of both fixed effects and variance components.

Single‐case experimental designs (SCEDs) may offer a reliable and internally valid way to evaluate technology‐enhanced learning (TEL). A systematic review was conducted to provide an overview of what, why and how SCEDs are used to evaluate TEL. Accordingly, 136 studies from nine databases fulfilling the inclusion criteria were included. The results showed that most of the studies were conducted in the field of special education focusing on evaluating the effectiveness of computer‐assisted instructions, video prompts and mobile devices to improve language and communication, socio‐emotional, skills and mental health. The research objective of most studies was to evaluate the effects of the intervention; often no specific justification for using SCED was provided. Additionally, multiple baseline and phase designs were the most common SCED types, with most measurements in the intervention phase. Frequent data collection methods were observation, tests, questionnaires and task analysis, whereas, visual and descriptive analysis were common methods for data analysis. Nearly half of the studies did not acknowledge any limitations, while a few mentioned generalization and small sample size as limitations. The review provides valuable insights into utilizing SCEDs to advance TEL evaluation methodology and concludes with a reflection on further opportunities that SCEDs can offer for evaluating TEL.Practitioner notesWhat is already known about this topicSCEDs use multiple measurements to study a single participant over multiple conditions, in the absence and presence of an interventionSCEDs can be rigorous designs for evaluating behaviour change caused by any intervention, including for testing technology‐based interventions.What this paper addsReveals patterns, trends and gaps in the use of SCED for TEL.Identifies the study disciplines, EdTech tools and outcome variables studied using SCEDs.Provides a comprehensive understanding of how SCEDs are used to evaluate TEL by shedding light on methodological techniques.Enriches insights about justifications and limitations of using SCEDs for TEL.Implications for practice and/or policyInforms about the use of the rigorous method, SCED, for evaluation of technology‐driven interventions across various disciplines.Contributes therefore to the quality of an evidence base, which provides policymakers, and different stakeholders a consolidated resource to design, implement and decide about TEL. [ABSTRACT FROM AUTHOR]

Prior research suggests that secure base script knowledge is categorically distributed in middle childhood but becomes dimensionally distributed from late adolescence onward, potentially indicating a developmental shift in the nature of secure base script knowledge. Secure base script knowledge may initially be sparse, giving rise to categorical individual differences, while increased relational experiences later in development might contribute to more elaborated secure base script knowledge and dimensional individual differences. However, the cross-sectional nature of prior research limits inferences about developmental changes. To address this, we conducted a three-year, three-wave longitudinal study with a Western European sample transitioning from middle childhood to adolescence. At Wave 1 (n = 599, Mage = 10.30), secure base script knowledge was categorically distributed. By Wave 2 (n = 435, Mage = 11.30), distribution was ambiguous, and by Wave 3 (n = 370, Mage = 12.09), individual differences were dimensional. These results suggest a developmental shift in secure base script knowledge during the transition into adolescence. [ABSTRACT FROM AUTHOR]

Formative assessments are an important component of massive open online courses (MOOCs), online courses with open access and unlimited student participation. Accurate conclusions on students' proficiency via formative, however, face several challenges: (a) students are typically allowed to make several attempts; and (b) student performance might be affected by other variables, such as interest. Thus, neglecting the effects of attempts and interest in proficiency evaluation might result in biased conclusions. In this study, we try to solve this limitation and propose two extensions of the common psychometric model, the Rasch model, by including the effects of attempts and interest. We illustrate these extensions using real MOOC data and evaluate them using cross-validation. We found that (a) the effects of attempts and interest on the performance are positive on average but both vary among students; (b) a part of the variance in proficiency parameters is due to variation between students in the effect of interest; and (c) the overall accuracy of prediction of student's item responses using the extensions is 4.3% higher than using the Rasch model.

In many educational systems, textbooks are a leading factor for daily instructional practices, and consequently may affect educational outcomes. Whereas in many countries textbook assessment is a common practice in light of evaluating and improving educational effectiveness, in other countries these practices are absent, mainly under the realm of educational freedom. In this study, we attempt to highlight the potential contribution of textbook assessment to educational freedom, effectiveness, and improvement, by means of mathematics textbook assessment primary education (Grade 4) in Flanders. Multilevel (multivariate) modelling was used, studying the relation of textbooks with both mean achievement (overall and in different content and cognitive subdomains) and (social and ethnic) equity. Furthermore, multilevel quantile regression analyses were used to investigate the effect of textbooks in different quantiles of the achievement distribution. Implications of the results of textbook assessment are discussed in light of its contribution to educational freedom, effectiveness, and improvement.

Recently, researchers have used multilevel models for estimating intervention effects in single-case experiments that include replications across participants (e.g., multiple baseline designs) or for combining results across multiple single-case studies. Researchers estimating these multilevel models have primarily relied on restricted maximum likelihood (REML) techniques, but Bayesian approaches have also been suggested. The purpose of this Monte Carlo simulation study was to examine the impact of estimation method (REML versus Bayesian with noninformative priors) on the estimation of treatment effects (relative bias, root mean square error) and on the inferences about those effects (interval coverage) for autocorrelated multiple-baseline data. Simulated conditions varied with regard to the number of participants, series length, and distribution of the variance within and across participants. REML and Bayesian estimation led to estimates of the fixed effects that showed little to no bias but that differentially impacted the inferences about the fixed effects and the estimates of the variances. Implications for applied researchers and methodologists are discussed.

This study explores the sources of variability in Mathematics achievement of Ugandan students at the student, classroom and school level. The Mathematics score and questionnaire responses of 4,819 first-year secondary school students (Grade Seven, about 14-15 years old) from 78 classrooms of 49 schools were analysed. A three-level linear model was used. The results indicate that out of the total variance in Mathematics achievement 68.8%, 14.2% and 17.0% are situated at student, classroom and school level, respectively. Of all the considered explanatory variables at the three levels, i.e. socio-economic status, gender, prior Mathematics achievement, parental support, peer influence, class mean of prior Mathematics achievement and of students' perception of good classroom assessment, school mean of class climate (class mean of attitude toward mathematics) and of parental support were significant predictors of Mathematics achievement. The relevant factors could explain 7.6%, 73.1% and 84.3%, respectively, of student-, classroom- and school-level differences. Implications of our study are considered.

Adolescents differ in their degree of Environmental Sensitivity, that is, the ability to perceive and process information about their environment. The present study aimed to investigate the psychometric properties of the Highly Sensitive Child scale (HSC), a self-report measure of Environmental Sensitivity, in two Belgian and UK samples with a total of 3056 adolescents. First, the factor structure, internal consistency, dimensionality, and construct validity of the HSC scale were examined. Second, measurement invariance of the HSC scale across developmental stage, gender, and country was tested. Results supported a bifactor model with a general sensitivity factor and three group factors: Ease of Excitation (EOE), Low Sensory Threshold (LST), and Aesthetic Sensitivity (AES). Cronbach's alpha and McDonalds's (hierarchical) omega indicated that the HSC scale is a reliable measure of Environmental Sensitivity, except for AES. Furthermore, AES was associated with different personality traits than EOE and LST. Second, the HSC scale was partially measurement invariant across developmental stage, gender, and country. The results provide important insights in the psychometrics of a first measurement of Environmental Sensitivity in early to late adolescents. Implications for further research are discussed., Author(s): Sofie Weyn [sup.1] , Karla Van Leeuwen [sup.1] , Michael Pluess [sup.2] , Francesca Lionetti [sup.2] , Corina U. Greven [sup.3] [sup.4] [sup.5] , Luc Goossens [sup.1] , Hilde [...]

Social relationships are of vital importance for children's and adolescents' development, and disruptions in these relationships can have serious implications. Such disruptions play a central role in both loneliness and social anxiety. Although both phenomena are closely related, they have largely been studied separately, and important questions have remained unanswered concerning how both go together within and across time. Multilevel meta-analyses were performed on 102 cross-sectional studies, published between 1981 and 2016, including 41,776 participants (39% males) with a mean age of 15.59 years. Longitudinal associations were examined in 10 studies, including 3,995 participants (46% males), using a novel technique that enables the examination of such associations even when these were not reported in the original empirical studies. Results indicated a strong, positive cross-sectional association between loneliness and social anxiety symptoms. This associations did not systematically differ in strength across childhood and adolescence. Moreover, results showed that loneliness and social anxiety symptoms were reciprocally associated over time. To conclude, loneliness and social anxiety symptoms are positively associated both within and across time, and across childhood and adolescence. Breaking this vicious cycle is of great importance, as both phenomena may be associated with profound problems in multiple domains of youth development. Moreover, failing to pay attention to both loneliness and social anxiety symptoms might substantially reduce the effectiveness of intervention programs focusing on either of the two.

Previous studies often assumed that parenting practices are similar across families. This assumption is difficult to hold, especially throughout adolescence, a period of major change for both adolescents and their parents. By combining a person-centered and a variable-centered approach, the present study adds to the literature by identifying trajectory classes in parenting behaviors and assessing their associations with externalizing problem behavior. The study aimed (a) to examine the existence of subgroups with different trajectories for five parenting dimensions (i.e., support, proactive control, punitive control, harsh punitive control, psychological control) in mothers and fathers separately, and (b) to assess whether membership of a subgroup is associated with the development of rule-breaking and aggressive behavior, respectively. The current study used four waves of data, with adolescents' age ranging from 12 to 18 years. Mothers (N = 747) and fathers (N = 645) reported on their own parenting behavior, whereas adolescent (N = 1,116) reported on externalizing problem behavior. Latent class growth analyses per parenting dimension showed that trajectory classes could be distinguished for support, proactive, punitive, and psychological control, but not harsh punitive control, and this for both mother and father. Conditional growth models per parenting dimension and per parent did not show different trajectories for aggressive and rule-breaking behavior across adolescence for the distinct parenting trajectories. However, analyses indicated that depending on the parenting trajectory, there was a difference in initial (Age 12) levels of problem behavior. Suggestions for additional research on longitudinal heterogeneity of parenting among mothers and fathers of adolescents are outlined.

Multilevel modeling has been utilized for combining single-case experimental design (SCED) data assuming simple level-1 error structures. The purpose of this study is to compare various multilevel analysis approaches for handling potential complexity in the level-1 error structure within SCED data, including approaches assuming simple and complex error structures (heterogeneous, autocorrelation, and both) and those using fit indices to select between alternative error structures. A Monte Carlo study was conducted to empirically validate the suggested multilevel modeling approaches. Results indicate that each approach leads to fixed effect estimates with little to no bias and that inferences for fixed effects were frequently accurate, particularly when a simple homogeneous level-1 error structure or a first-order autoregressive structure was assumed and the inferences were based on the Kenward-Roger method. Practical implications and recommendations are discussed.

In the Trends in Mathematics and Science Study (TIMSS) 2015, the educational systems from the Dutch speaking part of Belgium (Flanders), Germany, and Norway included scales capturing three dimensions of instructional quality (INQUA): classroom management, supportive climate, and cognitive activation. With the inclusion of these extra scales, a unique opportunity was created to investigate the various dimensions of INQUA and their relation to educational outcomes. In this study, multilevel structural equational modelling analyses are conducted to answer three research questions: (a) 'Do the items reliably measure the three dimensions of INQUA as classroom constructs? And if not, can we build reliable scales with the items, capturing the dimensions of INQUA as classroom constructs?' (b) 'To what extent can INQUA contribute to achievement?', and (c) 'To what extent can INQUA contribute to (social and ethnic) equity?'. Results indicate that INQUA might serve as a catalyst in increasing achievement in education systems. Furthermore, results indicate that INQUA does not relate to equity and consequently that all students benefit from the same educational practices. However, issues in the cross-cultural conceptualization and measurement of INQUA are raised, questioning the extent to which the three well-known dimensions of INQUA (a) are well-defined and might be sufficiently differentiated from each other, (b) sufficiently capture the diverse set of educational practices relating to students' educational outcomes, and (c) can be established across countries in a unified manner. The results of this study give direction to educational practitioners and policy makers in creating and providing effective learning environments.

The current study explored longitudinally whether oxytocin receptor gene methylation (OXTRm) changes moderated the association between parental sensitivity changes and children's attachment changes over three waves. Six hundred six Flemish children (10–12 years, 42.8%–44.8% boys) completed attachment measures and provided salivary OXTRm data on seven CpG sites. Their parents reported their sensitive parenting. Results suggest that OXTRm changes hardly link to attachment (in)security changes after the age of 10. Some support was found for interaction effects between parental sensitivity changes and OXTRm changes on attachment changes over time. Effects suggest that for children with increased OXTRm in the promotor region and decreased methylation in the inhibitor region over time, increased parental sensitivity was associated with increased secure attachment and decreased insecure attachment over time. [ABSTRACT FROM AUTHOR]

Instructional support can be implemented in learning environments to pseudo-modify the difficulty or time intensity of items presented to persons. This support can affect both the response accuracy of persons towards items as well as the time persons require to complete items. This study proposes a framework to model response time in learning environments as a function of instructional support. Moreover, it explores the effect of instructional support on response time in assembly tasks training using Virtual Reality. Three models are fitted with real-life data collected by a project that involves both industry and academic partners from Belgium. A Bayesian approach is followed to implement the models, where the Bayes factor is used to select the best fitting model. [ABSTRACT FROM AUTHOR]