Search

This special issue of Perspective on Behavior Science is a productive contribution to current advances in the use and documentation of single-case research designs. We focus in this article on major themes emphasized by the articles in this issue and suggest directions for improving professional standards focused on the design, analysis, and dissemination of single-case research. more...

Researchers conducting small-scale cluster randomized controlled trials (RCTs) during the pilot testing of an intervention often look for evidence of promise to justify an efficacy trial. We developed a method to test for intervention effects that is adaptive (i.e., responsive to data exploration), requires few assumptions, and is statistically valid (i.e., controls the type I error rate), by adapting masked visual analysis techniques to cluster RCTs. We illustrate the creation of masked graphs and their analysis using data from a pilot study in which 15 high school programs were randomly assigned to either business as usual or an intervention developed to promote psychological and academic well-being in 9th grade students in accelerated coursework. We conclude that in small-scale cluster RCTs there can be benefits of testing for effects without a priori specification of a statistical model or test statistic. more...

The Repeated Acquisition Design (RAD) is a type of single-case research design (SCRD) that involves repeated and rapid measurement of irreversible discrete skills or behaviors through pre-and postintervention probes across different sets of stimuli. Researchers interested in the study of learning in animals and humans have used the RAD because of its sensitivity to detect immediate changes in rate or accuracy. Despite its strengths, critics of the RAD have cautioned against its use due to reasonable threats to internal validity like pretest effects, history, and maturation. Furthermore, many methodologists and researchers have neglected the RAD in their SCRD standards (e.g., What Works Clearinghouse [WWC], 2020; Horner et al., 2005). Unless given guidance to address threats to internal validity, researchers may avoid the design altogether or continue to use a weak version of the RAD. Therefore, we propose a set of 15 quality RAD indicators, comprising foundational elements that should be present in all RAD studies and additional features that enhance causal inference and external validity. We review contemporary RAD use and describe how the additional features strengthen the rigor of RAD studies. We end the article with suggested guidelines for interpreting effects and the strength of the evidence generated by RAD studies. We invite researchers to use these initial guidelines as a jumping off point for a more RAD future. more...

Results from research indicate writing is a critical skill linked to several academic outcomes. To promote improvements in writing quantity and quality, intervention might target increasing students' academic engagement during time designated to practice writing. The purpose of this study was to implement an evidence-based classwide behavioral intervention, the Good Behavior Game (GBG), during daily writing practice time in two classrooms. Participants (n = 45) included students in a Grade 1 and Grade 2 class enrolled in an elementary school in a large suburb in the northeast U.S. Findings based on visual analysis and multilevel modeling indicate that students, on average, wrote more words (quantity) and more correct writing sequences (quality) when the GBG was played versus when it was not. Implications include the need for replication studies to extend findings and explore how school psychologists might consider the use of behavioral interventions to promote improved engagement and academic output in the classroom. (PsycInfo Database Record (c) 2021 APA, all rights reserved). more...

The limited research available on writing in Grade 1 led to the development and implementation of an intervention for students who were performing below expectations. Ten students participated in a... more...

The purpose of this study was to contribute information to service delivery gaps within the intervention literature related to the expense and inaccessibility of evidence-informed interventions for... more...

Objective: There is a growing interest in the potential benefits of applying Bayesian estimation for multilevel models of SCED data. Methodological studies have shown that Bayesian estimation resolves convergence issues, can be adequate for the small sample, and can improve the accuracy of the variance components. Despite the potential benefits, the lack of accessibility to software codes makes it difficult for applied researchers to implement Bayesian estimation in their studies. The purpose of this article is to illustrate a feasible way to implement Bayesian estimation using OpenBUGS software to analyze a complex SCED model where within-participants variability and autocorrelation may differ across cases. Method: By using extracted data from a published study, step-by-step guidance in analyzing the data using OpenBUGS software is provided, including (1) model specification, (2) prior distributions, (3) data entering, (4) model estimation, (5) convergence criteria, and (6) posterior inferences and interpretations. Result: Full codes for the analysis are provided. more...

The COVID-19 outbreak emphasizes the need for alternative methods for data gathering and collaboration among researchers in a virtual research environment One experimental design that is well suited in a social distancing research context is the single-case experimental design (SCD) SCDs can handle disruptions as (a) they do not require large groups gathering for data collection or intervention administration, (b) interventions are administered individually and in some cases remotely, (c) no comparison group is needed, and (d) they are adaptive and flexible designs The purpose of this article is to introduce the mobile application, SCD-MVA (2019), developed to assists in the design of an SCD, data gathering, data analysis, and remote collaboration The application allows data management and data sharing among researchers, provides an in real time visualization of the gathered data, stimulates interaction between researchers in terms of designing the SCD, gathering the data, and analyzing the gathered data, and does all these things with no need for in-person meetings of the research team © 2020 Wiley Periodicals LLC more...

Previous simulation studies of randomization tests applied in single-case educational intervention research contexts have typically focused on A-to-B phase changes in means/levels. In the present simulation study, we report the results of two multiple-baseline investigations, one targeting between-phase changes in slopes/trends and the other targeting between-phase changes in variability. For each of these measures, we examine the comparative type I errors and powers of several randomization test procedures that have previously appeared in the literature. In so doing, we propose an alternative measure of variability that is more sensitive to detecting between-phase change than is the variance itself. We conclude by providing a summary table of recommended randomization test procedures for assessing different types of intervention-based effects associated with level, trend, and variability. more...

Factor mixture modeling (FMM) has been increasingly used to investigate unobserved population heterogeneity. This study examined the issue of covariate effects with FMM in the context of measurement invariance testing. Specifically, the impact of excluding and misspecifying covariate effects on measurement invariance testing and class enumeration was investigated via Monte Carlo simulations. Data were generated based on FMM models with (1) a zero covariate effect, (2) a covariate effect on the latent class variable, and (3) covariate effects on both the latent class variable and the factor. For each population model, different analysis models that excluded or misspecified covariate effects were fitted. Results highlighted the importance of including proper covariates in measurement invariance testing and evidenced the utility of a model comparison approach in searching for the correct specification of covariate effects and the level of measurement invariance. This approach was demonstrated using an empirical data set. Implications for methodological and applied research are discussed. more...

Detailed is a 20-year arduous journey to develop a statistically viable two-phase (AB) single-case two independent-samples randomization test procedure. The test is designed to compare the effectiveness of two different interventions that are randomly assigned to cases. In contrast to the unsatisfactory simulation results produced by an earlier proposed randomization test, the present test consistently exhibited acceptable Type I error control under various design and effect-type configurations, while at the same time possessing adequate power to detect moderately sized intervention-difference effects. Selected issues, applications, and a multiple-baseline extension of the two-sample test are discussed. more...

The methodology of single-case experimental designs (SCED) has been expanding its efforts toward rigorous design tactics to address a variety of research questions related to intervention effective... more...

Western health research’s approach to knowledge translation (KT) has been criticized by Indigenous scholars, leaders, and communities for its misalignment with Indigenous ways of knowing and relational approaches to sharing knowledge. Conversely, Indigenous KT is understood as ‘sharing what we know about living a good life’ (Kaplan-Myrth & Smylie, 2006). Whereas KT in Euro-Western science contexts focuses on closing the know-do gap implying a separation of knowledge and action, knowledge in the Indigenous context is inherently practical and based on centuries old practices including oral traditions, experiential knowledge, and cross-cultural sharing. This article describes the development of a decolonized KT strategy. This community-engaged KT initiative was developed at the suggestion of Indigenous mothers who participated in a research study in Hamilton, Canada, which examined their experiences using health care to meet the health needs of their infants. Indigenous mothers participated in three main roles related to the KT activities: sharing their story in video, participating as team members on an Advisory Board, and directing the creation of a video series and website educational resource (KT strategy). Five Indigenous mothers participated as members of the Advisory Board. The process of participating had positive impacts on the mothers, namely, empowerment, strength, ‘I am not alone’, and healing. These unexpected findings, which go beyond the original project purpose to create an educational resource, show the significant and important benefits for research participants, particularly those from Indigenous communities, to be involved in decolonized KT strategies. more...

The focus of the current study is on handling the dependence among multiple regression coefficients representing the treatment effects when meta-analyzing data from single-case experimental studies. We compare the results when applying three different multilevel meta-analytic models (i.e., a univariate multilevel model avoiding the dependence, a multivariate multilevel model ignoring covariance at higher levels, and a multivariate multilevel model modeling the existing covariance) to deal with the dependent effect sizes. The results indicate better estimates of the overall treatment effects and variance components when a multivariate multilevel model is applied, independent of modeling or ignoring the existing covariance. These findings confirm the robustness of multilevel modeling to misspecifying the existing covariance at the case and study level in terms of estimating the overall treatment effects and variance components. The results also show that the overall treatment effect estimates are unbiased regardless of the underlying model, but the between-case and between-study variance components are biased in certain conditions. In addition, the between-study variance estimates are particularly biased when the number of studies is smaller than 40 (i.e., 10 or 20) and the true value of the between-case variance is relatively large (i.e., 8). The observed bias is larger for the between-case variance estimates compared to the between-study variance estimates when the true between-case variance is relatively small (i.e., 0.5). ispartof: BEHAVIOR RESEARCH METHODS vol:52 issue:5 pages:2008-2019 ispartof: location:United States status: published more...

Single-case experimental designs are often used to examine the efficacy and effectiveness of interventions that are designed to change behaviors so that they are less problematic and more similar t... more...

High school students in accelerated classes face heightened school-related stressors but have access to few specialized supports. This article describes the development and initial application of the Motivation, Assessment, and Planning (MAP) intervention, developed to meet the needs of freshmen in Advanced Placement (AP) and International Baccalaureate (IB) curricula. MAP is grounded in motivational interviewing and designed to evoke behavior change following one to two individual meetings with a coach. This study used a multimethod, multisource approach to evaluate the applicability and acceptability of MAP as a Tier 2 support for 9th-grade students completing AP/IB coursework. Quantitative and qualitative acceptability data from 49 AP/IB students, 7 coaches, and 3 potential end-users of MAP were examined. Results indicate MAP was perceived as an acceptable intervention for addressing the social–emotional needs of high-achieving students taking AP/IB classes. Findings and directions for further research of MAP are discussed. more...

There is evidence to suggest the level of caregiver strain experienced by parents and other caregivers may have implications for engagement in their child’s mental health and education services. This is particularly important for caregivers of youth with emotional and behavioral disorders in special education, as these caregivers tend to be less involved than other caregivers. The purpose of this study was to understand caregiver strain and theoretically-related constructs in caregivers of youth with emotional and behavioral disorders. The sample included 112 caregivers of youth receiving special education services for Emotional Disturbance (ED). Using structural equation modeling and the Modified Double ABCX Model as a theoretical framework, the relationships among caregiver strain, stressors, resources, and perceptions were examined. Consistent with findings from previous studies, results indicated that more problems with youth functioning were related to higher levels of caregiver strain. Resources were found to be positively related to both perceptions and caregiver strain. The strongest relationship was observed between caregiver perceptions and caregiver strain, with more negative perceptions associated with higher levels of caregiver strain. Findings reveal the nature of the relationships among caregiver strain and related constructs, as well as the potential importance of perceptions as a target for interventions for families and youth. An understanding of caregiver strain as it relates to stressors, resources, and perceptions provides insight into how we might leverage these factors in order to increase caregiver engagement in services, ultimately improving outcomes for youth and their families. more...

Following up on articles recently published in this journal, the present contribution tells (some of) "the rest of the story" about the value of randomization in single-case intervention research investigations. Invoking principles of internal, statistical-conclusion, and external validity, we begin by emphasizing the critical distinction between design randomization and analysis randomization, along with the necessary correspondence between the two. Four different types of single-case design-and-analysis randomization are then discussed. The persistent negative influence of serially dependent single-case outcome observations is highlighted, accompanied by examples of inappropriate applications of parametric and nonparametric tests that have appeared in the literature. We conclude by presenting valid applications of single-case randomization procedures in various single-case intervention contexts, with specific reference to a freely available Excel-based software package that can be accessed to incorporate the present randomization schemes into a wide variety of single-case intervention designs and analyses. more...

The impact of the Good Behavior Game (GBG) on students’ classroom behavior has been studied for 50 years. What is less established is the impact of the GBG on students’ academic progress. With emerging research in curriculum-based measurement for written expression (WE-CBM), it may be possible to observe changes in students’ writing output while playing the GBG versus when the game is not played. The purpose of the current study was to systematically introduce the GBG during writing practice time in a Grade 1 and Grade 2 classroom, and observe any changes to all students’ academic engagement, disruptive behavior, as well as target students’ writing output using WE-CBM. Results indicated large increases in all students’ academic engagement and decreases in disruptive behavior when the GBG was played. For writing output, target students demonstrated modest improvement in the amount of words written and accuracy of writing when the game was played, especially students identified as having emerging writing skills. Future studies might continue to empirically explore the connection between behavioral intervention and academic output by replicating study procedures in different contexts and/or with alternative WE-CBM indices. more...

Training in trauma-informed early intervention programs is greatly needed for Part C early intervention providers (EIs). This study evaluated the feasibility, acceptability, and initial outcomes of Trauma-Informed Behavioral Parenting (TIBP; subsequently renamed Smart Start), an early intervention program designed to meet the social–emotional needs of toddlers enrolled in Part C services. This pilot study consisted of training and supervising 5 EIs to implement TIBP with 8 parent–child dyads as part of their Part C services. Preliminary changes in child behavior problems, trauma exposures, parenting stress, and parenting skills were evaluated quantitively. EIs completed treatment integrity measures. Focus groups provided data regarding the feasibility of implementation and caregiver attitudes about TIBP. Significant reductions in child trauma symptoms on both the YCPC-PTSD (S = 11.5, p = 0.0586, rMPRB = 0.64) and YCPC-FI (S = 11.5, p = 0.0586, rMPRB = 0.53) were indicated. No significant changes in externalizing behaviors were observed. Caregivers demonstrated improved use of positive verbal reinforcement and verbal reflections, and reported improvements on parent–child dysfunctional interactions on the PSI-SF (S = 16, p = 0.0117, rMPRB = 0.89). All focus group participants reported benefits from participating in TIBP. EIs demonstrated high adherence to treatment protocol and identified TIBP as a feasible program to implement within Part C services. This study showed promising results as a means to disseminate evidence-based practices into a community-level program to address the social–emotional needs of toddlers. The need for additional, large sample research was highlighted. more...

This study examined the impact of omitting covariates interaction effect on parameter estimates in multilevel multiple-indicator multiple-cause models as well as the sensitivity of fit indices to model misspecification when the between-level, within-level, or cross-level interaction effect was left out in the models. The parameter estimates produced in the correct and the misspecified models were compared under varying conditions of cluster number, cluster size, intraclass correlation, and the magnitude of the interaction effect in the population model. Results showed that the two main effects were overestimated by approximately half of the size of the interaction effect, and the between-level factor mean was underestimated. None of comparative fit index, Tucker–Lewis index, root mean square error of approximation, and standardized root mean square residual was sensitive to the omission of the interaction effect. The sensitivity of information criteria varied depending majorly on the magnitude of the omitted interaction, as well as the location of the interaction (i.e., at the between level, within level, or cross level). Implications and recommendations based on the findings were discussed. more...

Multiple baseline (MB) designs are becoming more prevalent in educational and behavioral research, and as they do, there is growing interest in combining effect size estimates across studies. To further refine the meta-analytic methods of estimating the effect, this study developed and compared eight alternative methods of estimating intervention effects from a set of MB studies. The methods differed in the assumptions made and varied in whether they relied on within- or between-series comparisons, modeled raw data or effect sizes, and did or did not standardize. Small sample functioning was examined through two simulation studies, which showed that when data were consistent with assumptions the bias was consistently less than 5% of the effect size for each method, whereas root mean squared error varied substantially across methods. When assumptions were violated, substantial biases were found. Implications and limitations are discussed. more...

The paper describes the applicability and acceptability of a selective intervention-Motivation, Assessment, and Planning (MAP)-for high school students that was developed based on the principles of motivational interviewing (MI) and tailored to the unique needs and strengths of students taking accelerated coursework, specifically Advanced Placement (AP) and International Baccalaureate (IB) classes. In addition to detailing the intervention in terms of MI spirit, processes, and relational and technical skills, we report applicability and acceptability data from a second iteration of MAP implementation in eight AP/IB programs in a Southeastern state during spring 2018. We analyzed quantitative and qualitative acceptability data from 121 high school freshmen (97 from AP and 24 from IB courses), as well as the seven MAP coaches who were trained using the Motivational Interview Training and Assessment System (Frey et al. 2017). To gain perspectives from the intended end users of the refined MAP, 12 school counselors and school psychologists who were not trained in MAP evaluated the intervention and provided qualitative and quantitative data on applicability and acceptability. All three stakeholder groups (students, coaches, and school mental health staff) rated and described the intervention as highly acceptable and appropriate for addressing the social-emotional needs of adolescents in AP/IB classes. more...

Purpose This experiment investigated the effects of a book-sharing intervention implemented in coparenting homes on the conversations of preschoolers with their parents. Method A multiple baseline design across behaviors was used to evaluate the effects of embedding decontextualized language utterances during book-sharing delivered by four families. A visual analysis, a two-level mixed-effects model, and a social validity evaluation were used to examine the varying effects of the program on mothers and fathers' storybook conversations. Results Embedding decontextualized language prompts in books not only increased parental decontextualized language utterances, but most parents were able to maintain use of strategies without prompts in the books. The intervention effects were consistently higher for parents than for their children. Social validity results demonstrated parental satisfaction with program delivery and content. Conclusion This study adds to the limited literature on father–child and mother–child decontextualized conversations during book-sharing and illustrates a feasible and adaptable way of promoting language experiences in the home that yield engaging decontextualized conversations in meaningful book-reading contexts. Supplemental Material https://doi.org/10.23641/asha.13210799 more...

In this paper, we provide a critique focused on the What Works Clearinghouse (WWC) Standards for Single-Case Research Design (Standards 4.1). Specifically, we (a) recommend the use of visual-analysis to verify a single-case intervention study's design standards and to examine the study's operational issues, (b) identify limitations of the design-comparable effect-size measure and discuss related statistical matters, (c) review the applicability and practicality of Standards 4.1 to single-case designs (SCDs), and (d) recommend inclusion of content pertaining to diversity, equity, and inclusion in future standards. Within the historical context of the WWC Pilot Standards for Single-Case Design (1.0), we suggest that Standards 4.1 may best serve as standards for meta-analyses of SCDs but will need to make clear distinctions among the various types of SCD studies that are included in any research synthesis. In this regard, we argue for transparency in SCD studies that meet design standards and those that do not meet design standards in any meta-analysis emanating from the WWC. The intent of these recommendations is to advance the science of SCD research both in research synthesis and in promoting evidence-based practices. more...

To conduct a multilevel meta-analysis of multiple single-case experimental design (SCED) studies, the individual participant data (IPD) can be analyzed in one or two stages. In the one-stage approach, a multilevel model is estimated based on the raw data. In the two-stage approach, an effect size is calculated for each participant and these effect sizes and their sampling variances are subsequently combined to estimate a meta-analytic multilevel model. The multilevel model in the two-stage approach has fewer parameters to estimate, in exchange for the reduction of information of the raw data to effect sizes. In this paper we explore how the one-stage and two-stage IPD approaches can be applied in the context of meta-analysis of single-case designs. Both approaches are compared for several single-case designs of increasing complexity. Through a simulation study we show that the two-stage approach obtains better convergence rates for more complex models, but that model estimation does not necessarily converge at a faster speed. The point estimates of the fixed effects are unbiased for both approaches across all models, as such confirming results from methodological research on IPD meta-analysis of group-comparison designs. In light of these results, we discuss the implementation of both methods in R. ispartof: MULTIVARIATE BEHAVIORAL RESEARCH vol:57 issue:2-3 ispartof: location:United States status: published more...

Single-case researchers often implement multiple-baseline designs as their preferred methodology for intervention evaluations. Recent writings and empirical investigations have argued in favor of incorporating various forms of randomization into such designs for the purpose of elevating the intervention study's internal validity and scientific credibility. In this article, we consider a variety of randomized multiple-baseline designs and associated randomization statistical tests, along with their potential strengths and limitations. In what amounts to a practical guide, we refer school psychology researchers to these versatile randomization procedures for planning and executing their intervention studies. more...

© 2019, © 2019 Taylor & Francis Group, LLC. Recently, researchers have used multilevel models for estimating intervention effects in single-case experiments that include replications across participants (e.g., multiple baseline designs) or for combining results across multiple single-case studies. Researchers estimating these multilevel models have primarily relied on restricted maximum likelihood (REML) techniques, but Bayesian approaches have also been suggested. The purpose of this Monte Carlo simulation study was to examine the impact of estimation method (REML versus Bayesian with noninformative priors) on the estimation of treatment effects (relative bias, root mean square error) and on the inferences about those effects (interval coverage) for autocorrelated multiple-baseline data. Simulated conditions varied with regard to the number of participants, series length, and distribution of the variance within and across participants. REML and Bayesian estimation led to estimates of the fixed effects that showed little to no bias but that differentially impacted the inferences about the fixed effects and the estimates of the variances. Implications for applied researchers and methodologists are discussed. ispartof: JOURNAL OF EXPERIMENTAL EDUCATION vol:88 issue:4 pages:698-710 status: published more...

In the context of single-case experimental designs, replication is crucial. On the one hand, the replication of the basic effect within a study is necessary for demonstrating experimental control. On the other hand, replication across studies is required for establishing the generality of the intervention effect. Moreover, the "replicability crisis" presents a more general context further emphasizing the need for assessing consistency in replications. In the current text, we focus on replication of effects within a study and we specifically discuss the consistency of effects. Our proposal for assessing the consistency of effects refers to one of the promising data analytical techniques: multilevel models, also known as hierarchical linear models or mixed effects models. One option is to check, for each case in a multiple-baseline design, whether the confidence interval for the individual treatment effect excludes zero. This is relevant for assessing whether the effect is replicated as being non-null. However, we consider that it is more relevant and informative to assess, for each case, whether the confidence interval for the random effects includes zero (i.e., whether the fixed effect estimate is a plausible value for each individual effect). This is relevant for assessing whether the effect is consistent in size, with the additional requirement that the fixed effect itself is different from zero. The proposal for assessing consistency is illustrated with real data and it is implemented in free user-friendly software. more...

In multilevel multiple-indicator multiple-cause (MIMIC) models, covariates can interact at the within level, at the between level, or across levels. This study examines the performance of multilevel MIMIC models in estimating and detecting the interaction effect of two covariates through a simulation and provides an empirical demonstration of modeling the interaction in multilevel MIMIC models. The design factors include the location of the interaction effect (i.e., between, within, or across levels), cluster number, cluster size, intraclass correlation (ICC) level, magnitude of the interaction effect, and cross-level measurement invariance status. Type I error, power, relative bias, and root mean square of error of the interaction effects are examined. The results showed that multilevel MIMIC models performed well in detecting the interaction effect at the within or across levels. However, when the interaction effect was at the between level, the performance of multilevel MIMIC models depended on the magnitude of the interaction effect, ICC, and sample size, especially cluster number. Overall, cross-level measurement noninvariance did not make a notable impact on the estimation of interaction in the structural part of multilevel MIMIC models when factor loadings were allowed to be different across levels. more...

BACKGROUND: Methodological rigor is a fundamental factor in the validity and credibility of the results of a meta-analysis. AIM: Following an increasing interest in single-case experimental design (SCED) meta-analyses, the current study investigates the methodological quality of SCED meta-analyses. METHODS AND PROCEDURES: We assessed the methodological quality of 178 SCED meta-analyses published between 1985 and 2015 through the modified Revised-Assessment of Multiple Systematic Reviews (R-AMSTAR) checklist. OUTCOMES AND RESULTS: The main finding of the current review is that the methodological quality of the SCED meta-analyses has increased over time, but is still low according to the R-AMSTAR checklist. A remarkable percentage of the studies (93.80% of the included SCED meta-analyses) did not even reach the midpoint score (22, on a scale of 0-44). The mean and median methodological quality scores were 15.57 and 16, respectively. Relatively high scores were observed for "providing the characteristics of the included studies" and "doing comprehensive literature search". The key areas of deficiency were "reporting an assessment of the likelihood of publication bias" and "using the methods appropriately to combine the findings of studies". CONCLUSIONS AND IMPLICATIONS: Although the results of the current review reveal that the methodological quality of the SCED meta-analyses has increased over time, still more efforts are needed to improve their methodological quality. ispartof: Res Dev Disabil ispartof: location:United States more...

BACKGROUND: When developmental disabilities researchers use multiple-baseline designs they are encouraged to delay the start of an intervention until the baseline stabilizes or until preceding cases have responded to intervention. Using ongoing visual analyses to guide the timing of the start of the intervention can help to resolve potential ambiguities in the graphical display; however, these forms of response-guided experimentation have been criticized as a potential source of bias in treatment effect estimation and inference. AIMS AND METHODS: Monte Carlo simulations were used to examine the bias and precision of average treatment effect estimates obtained from multilevel models of four-case multiple-baseline studies with series lengths that varied from 19 to 49 observations per case. We varied the size of the average treatment effect, the factors used to guide intervention decisions (baseline stability, response to intervention, both, or neither), and whether the ongoing analysis was masked or not. RESULTS: None of the methods of responding to the data led to appreciable bias in the treatment effect estimates. Furthermore, as timing-of-intervention decisions became responsive to more factors, baselines became longer and treatment effect estimates became more precise. CONCLUSIONS: Although the study was conducted under limited conditions, the response-guided practices did not lead to substantial bias. By extending baseline phases they reduced estimation error and thus improved the treatment effect estimates obtained from multilevel models. ispartof: RESEARCH IN DEVELOPMENTAL DISABILITIES vol:79 pages:77-87 ispartof: location:United States status: published more...

The increasing diversity in classrooms throughout the U.S. calls for teachers who are culturally competent and have quality professional development in working with dual language learners (DLLs) in order to ensure children from diverse groups are receiving equitable and high-quality education. This study focuses on teacher characteristics that play a significant role in the development of young children’s bilingual abilities with regard to language, literacy and math. We examined the school readiness skills of 217 Latino DLLs attending Head Start taking into account specific teacher characteristics that may impact the development of readiness skills. Different teacher characteristics have different effects on the developmental outcomes. Amount of training had a positive significant effect on several children’s outcomes in English and in Spanish. The years of experience a teacher had working in classrooms with DLLs had a positive effect on children’s English spelling skills. Teachers’ cultural competency rating had a positive effect on receptive language and literacy in Spanish. Findings and implications are discussed. more...

Research Findings: Naturalistic instruction is the intentional use of strategies embedded within typical activities of the environment to create learning opportunities for children. We used multilevel modeling to analyze data from this single-case design study that focused on improving teachers’ use of naturalistic language strategies and children’s communication through professional development with bug-in-ear coaching. Our results showed that professional development was positively associated with children’s communication during and after professional development. During generalization sessions in different types of activities, teachers’ use of the naturalistic language strategies was positively associated with children’s functional communication. We also found that children were more likely to communicate during child-led activities. Practice or Policy: Associations among professional development, teachers’ use of evidence-based strategies, and children’s outcomes have yet to be fully understoo... more...

Research has shown that students in Advanced Placement (AP) classes and International Baccalaureate (IB) programs experience higher levels of stress compared to students in general education classes. Elevated stress can serve as a risk factor for students’ academic and mental health problems. Given the documented stress of these students, additional investigations are needed to more fully understand how students experience these curricula and the factors associated with positive student outcomes. Thus, we set out to identify factors associated with success among AP/IB students, with an emphasis on exploring potentially malleable factors that could be targeted with existing or newly developed interventions. Data were collected via self-report measures and school records from 2,379 students (Grades 9-12) enrolled in AP or IB in 20 school programs in one state. We examined the relationships among 34 predictors (e.g., stressors, coping styles, student engagement, family factors, school factors, and demographic features) of success. Success was represented by five outcomes in two domains: mental health (life satisfaction, psychopathology, school burnout) and academic (GPA, AP/IB exam scores). Better outcomes in both domains were associated with higher levels of achievement motivation and cognitive engagement, as well as lower levels of parent–child conflict, stress from major life events, and use of avoidance coping strategies. Higher levels of affective engagement, use of approach coping, and authoritative parenting were robust predictors of positive mental health outcomes and unrelated (in multivariate analyses) to academic outcomes. Findings have implications for subsequent development of intervention efforts targeting factors associated with student success. more...

Wording effects associated with positively and negatively worded items have been found in many scales. Such effects may threaten construct validity and introduce systematic bias in the interpretation of results. A variety of models have been applied to address wording effects, such as the correlated uniqueness model and the correlated traits and correlated methods model. This study presents the multilevel bifactor approach to handling wording effects of mixed-format scales used in a multilevel context. The Students Confident in Mathematics scale is used to illustrate this approach. Results from comparing a series of models showed that positive and negative wording effects were present at both the within and the between levels. When the wording effects were ignored, the within-level predictive validity of the Students Confident in Mathematics scale was close to that under the multilevel bifactor model. However, at the between level, a lower validity coefficient was observed when ignoring the wording effects. Implications for applied researchers are discussed. more...

We developed masked visual analysis (MVA) as a structured complement to traditional visual analysis. The purpose of the present investigation was to compare the effects of computer-simulated MVA of a four-case multiple-baseline (MB) design in which the phase lengths are determined by an ongoing visual analysis (i.e., response-guided) versus those in which the phase lengths are established a priori (i.e., fixed criteria). We observed an acceptably low probability (less than .05) of false detection of treatment effects. The probability of correctly detecting a true effect frequently exceeded .80 and was higher when: (a) the masked visual analyst extended phases based on an ongoing visual analysis, (b) the effects were larger, (c) the effects were more immediate and abrupt, and (d) the effects of random and extraneous error factors were simpler. Our findings indicate that MVA is a valuable combined methodological and data-analysis tool for single-case intervention researchers. more...

A number of randomization statistical procedures have been developed to analyze the results from single-case multiple-baseline intervention investigations. In a previous simulation study, comparisons of the various procedures revealed distinct differences among them in their ability to detect immediate abrupt intervention effects of moderate size, with some procedures (typically those with randomized intervention start points) exhibiting power that was both respectable and superior to other procedures (typically those with single fixed intervention start points). In Investigation 1 of the present follow-up simulation study, we found that when the same randomization-test procedures were applied to either delayed abrupt or immediate gradual intervention effects: (1) the powers of all of the procedures were severely diminished; and (2) in contrast to the previous study's results, the single fixed intervention start-point procedures generally outperformed those with randomized intervention start points. In Investigation 2 we additionally demonstrated that if researchers are able to successfully anticipate the specific alternative effect types, it is possible for them to formulate adjusted versions of the original randomization-test procedures that can recapture substantial proportions of the lost powers. more...

Purpose Single-case experimental designs are widely used to study interventions for communication disorders. Traditionally, single-case experiments follow a response-guided approach, where design decisions during the study are based on participants' observed patterns of behavior. However, this approach has been criticized for its high rate of Type I error. In masked visual analysis (MVA), response-guided decisions are made by a researcher who is blinded to participants' identities and treatment assignments. MVA also makes it possible to conduct a hypothesis test assessing the significance of treatment effects. Method This tutorial describes the principles of MVA, including both how experiments can be set up and how results can be used for hypothesis testing. We then report a case study showing how MVA was deployed in a multiple-baseline across-subjects study investigating treatment for residual errors affecting rhotics. Strengths and weaknesses of MVA are discussed. Conclusions Given their important role in the evidence base that informs clinical decision making, it is critical for single-case experimental studies to be conducted in a way that allows researchers to draw valid inferences. As a method that can increase the rigor of single-case studies while preserving the benefits of a response-guided approach, MVA warrants expanded attention from researchers in communication disorders. more...

The focus of the current study is on handling the dependence among multiple regression coefficients representing the treatment effects when meta-analyzing data from single-case experimental studies. We compare the results when applying three different multilevel meta-analytic models (i.e., a univariate multilevel model avoiding the dependence, a multivariate multilevel model ignoring covariance at higher levels, and a multivariate multilevel model modeling the existing covariance) to deal with the dependent effect sizes. The results indicate better estimates of the overall treatment effects and variance components when a multivariate multilevel model is applied, independent of modeling or ignoring the existing covariance. These findings confirm the robustness of multilevel modeling to misspecifying the existing covariance at the case and study level in terms of estimating the overall treatment effects and variance components. The results also show that the overall treatment effect estimates are unbiased regardless of the underlying model, but the between-case and between-study variance components are biased in certain conditions. In addition, the between-study variance estimates are particularly biased when the number of studies is smaller than 40 (i.e., 10 or 20) and the true value of the between-case variance is relatively large (i.e., 8). The observed bias is larger for the between-case variance estimates compared to the between-study variance estimates when the true between-case variance is relatively small (i.e., 0.5). more...

The MultiSCED web application has been developed to assist applied researchers in behavioral sciences to apply multilevel modeling to quantitatively summarize single-case experimental design (SCED) studies through a user-friendly point-and-click interface embedded within R. In this paper, we offer a brief introduction to the application, explaining how to define and estimate the relevant multilevel models and how to interpret the results numerically and graphically. The use of the application is illustrated through a re-analysis of an existing meta-analytic dataset. By guiding applied researchers through MultiSCED, we aim to make use of the multilevel modeling technique for combining SCED data across cases and across studies more comprehensible and accessible. ispartof: Behavior Research Methods vol:52 issue:1 pages:1-16 ispartof: location:United States status: Published online more...

PurposeMany preschoolers, especially those from low-income households, would benefit from instruction to enrich their vocabulary and language repertoires. Yet, explicit instruction of vocabulary and language skills generally occurs infrequently in early childhood education settings. This study investigated the additive effects of teacher-led, classwide review strategies to a previously studied small-group intervention on children's learning of academic vocabulary.MethodParticipants included 23 children with limited oral language skills at risk for reading difficulties enrolled in single-case experimental designs. Effects of the classroom strategies alone also were examined in 10 children with above-average language abilities from 2 classrooms.ResultsVisual analyses of the adapted alternating treatments designs showed consistent learning improvements when vocabulary instruction was extended into the classroom for 12 children, ceiling effects were evident for 3 participants regardless of condition, and inconsistent or minimal effects were demonstrated by 8 participants. Multilevel modeling used to evaluate the effects statistically revealed strong treatment effects. In addition, the 10 children with above-average language showed impressive learning of vocabulary words from books subject to teacher review strategies in comparison to words from books to which they were not exposed. Teachers varied in the extent to which they implemented review strategies in their classrooms. Nevertheless, their responses to social validity assessments were positive, supporting the feasibility of this intervention.ConclusionsThe addition of classwide review and practice opportunities is an effective means of enhancing the effects of an easy-to-implement small-group intervention that teaches challenging vocabulary words within prerecorded stories. This approach holds promise as a way to shrink the pervasive word gap that typically exists when children in high-poverty communities enter school. more...