Search

In science in general and in the context of single-case experimental designs, replication of the effects of the intervention within and/or across participants or experiments is crucial for establishing causality and for assessing the generality of the intervention effect. Specific developments and proposals for assessing whether an effect has been replicated or not (or to what extent) are scarce, in the general context of behavioral sciences, and practically null in the single-case experimental designs context. We propose an extension of the modified Brinley plot for assessing how many of the effects replicate. To make this assessment possible, a definition of replication is suggested, on the basis of expert judgment, rather than on statistical criteria. The definition of replication and its graphical representation are justified, presenting their strengths and limitations, and illustrated with real data. A user-friendly software is made available for obtaining automatically the graphical representation.

AbstractOne of the commonly used single-case experimental designs (SCEDs) in education research is the ABAB design, commonly referred to as reversal design. This design is applicable when the studied behavior is reversible (i.e., reverses to baseline levels when the intervention is withdrawn) and it allows for a within-unit replication of the “basic effect”, that is, a comparison between two adjacent conditions (with and without an active intervention). Methodologically, one of its strengths is to provide three phase change moments to demonstrate experimental control. However, the withdrawal of the intervention (i.e., the comparison between the first intervention phase and the second baseline phase) has not received sufficient attention in the SCED literature. The aim of the current text is to propose quantifications that enable using the information from the withdrawal. Specifically, the quantifications allow for assessing the degree to which withdrawing the intervention leads to initial baseline levels as well as incorporating the reversal in an overall effect size. The proposals are illustrated with several real educational science data sets, including both within-unit analysis and replication of the ABAB design across units, and representing different degrees of reversal of the target behavior when the intervention is withdrawn. [ABSTRACT FROM AUTHOR]

Changing criterion designs (CCD) are single-case experimental designs that entail a step-by-step approximation of the final level desired for a target behavior. Following a recent review on the desirable methodological features of CCDs, the current text focuses on an analytical challenge: the definition of an objective rule for assessing the correspondence between criterion levels and behavior. We review extensively applied research and methodological recommendations for establishing experimental control, such as the range-bound CCD, and integrate them into a proposal for a formal expression of an objective rule. We illustrate how this proposal can lead to establishing the acceptable level of behavior prior to gathering the intervention phase data and how it leaves space for professional judgment. A discussion of software implementing the suggested proposal is included.

One of the data features that are expected to be assessed when analyzing single-case experimental designs (SCED) data is trend. The current text deals with four different questions that applied researchers can ask themselves when assessing trend and especially when dealing with improving baseline trend: (a) What options exist for assessing the presence of trend?; (b) Once assessed, what criterion can be followed for deciding whether it is necessary to control for baseline trend?; (c) What strategy can be followed for controlling for baseline trend?; and (d) How to proceed in case there is baseline trend only in some A-B comparisons? Several options are reviewed for each of these questions in the context of real data, and tentative recommendations are provided. A new user-friendly website is developed to implement the options for fitting a trend line and a criterion for selecting a specific technique for that purpose. Trend-related and more general data analytical recommendations are provided for applied researchers. Trial registration: ClinicalTrials.gov identifier: NCT04560777 [ABSTRACT FROM AUTHOR]

Aim: To determine the effectiveness of the Cognitive Orientation to daily Occupational Performance (CO‐OP) approach in improving the occupational performance goals of children and young people with executive function deficits after acquired brain injury (ABI) (e.g. etiologies such as stroke, encephalitis, brain tumor, and traumatic brain injury). Method: A replicated single‐case experimental study using a randomized multiple baseline design across participants and goals was used. Three clusters of four participants (12 participants, nine males and three females, aged 8–16 years) were included. The intervention consisted of 14 individual CO‐OP sessions. Each participant chose four goals; three goals were trained during the intervention sessions and a fourth goal served as the control. The Goal Attainment Scale (GAS) was used as a repeated measure to determine goal achievement while the Canadian Occupational Performance Measure (COPM) was used to identify the perceived goal achievement of children, young people, and their parents. Results: For 26 of the 35 trained goals, the intervention led to statistically significant improvements in the GAS. Perceived occupational performance and satisfaction improved significantly for the trained goals (30 out of 35 goals for the COPM performance and satisfaction of participants; 26 out 31 goals for the COPM performance of parents; 24 out of 31 goals for the COPM satisfaction of parents) and were maintained at the follow‐up. Almost all COPM control goal results were significant, but these changes were not supported by the GAS measures or the statistical analysis. Interpretation: The generally positive results of this study provide evidence of the benefits of using the CO‐OP approach with this population. [ABSTRACT FROM AUTHOR]

Group functioning and performance in different contexts is related to the extent to which group members are complementary or supplementary in terms of psychological attributes. This paper describes a procedure for quantifying the degree of dissimilarity at group level. Unlike most existing techniques the one described here is normalized and is both location and scale invariant, thereby making it suitable for comparing dissimilarity on interval and ratio scales with different ranges and in groups of different sizes. Dissimilarity is measured in relative terms regardless of the exact place on the scale at which individuals are located. When a combination of several scales is not theoretically justified, the dissimilarity for each scale can be quantified. Additionally, dyadic and individual contributions to either the global or scale index can be obtained. The descriptive measures are complemented by statistical significance values in order to compare the results obtained with several discrete distributions of reference, both symmetrical and skewed, which can be specified using the expressions developed. The information that can be provided by the indices and the "p" values -- both obtainable through an R package -- is illustrated using data from an empirical study.

In the first part of the study, nine estimators of the first-order autoregressive parameter are reviewed and a new estimator is proposed. The relationships and discrepancies between the estimators are discussed in order to achieve a clear differentiation. In the second part of the study, the precision in the estimation of autocorrelation is studied. The performance of the ten lag-one autocorrelation estimators is compared in terms of Mean Square Error (combining bias and variance) using data series generated by Monte Carlo simulation. The results show that there is not a single optimal estimator for all conditions, suggesting that the estimator ought to be chosen according to sample size and to the information available on the possible direction of the serial dependence. Additionally, the probability of labelling an actually existing autocorrelation as statistically significant is explored using Monte Carlo sampling. The power estimates obtained are quite similar among the tests associated with the different estimators. These estimates evidence the small probability of detecting autocorrelation in series with less than 20 measurement times. (Contains 2 figures and 7 tables.)

N = 1 designs imply repeated registrations of the behaviour of the same experimental unit and the measurements obtained are often few due to time limitations, while they are also likely to be sequentially dependent. The analytical techniques needed to enhance statistical and clinical decision making have to deal with these problems. Different procedures for analysing data from single-case AB designs are discussed, presenting their main features and revising the results reported by previous studies. Randomization tests represent one of the statistical methods that seemed to perform well in terms of controlling false alarm rates. In the experimental part of the study a new simulation approach is used to test the performance of randomization tests and the results suggest that the technique is not always robust against the violation of the independence assumption. Moreover, sensitivity proved to be generally unacceptably low for series lengths equal to 30 and 40. Considering the evidence available, there does not seem to be an optimal technique for single-case data analysis. (Contains 4 figures and 1 table.)

The current text comments on three systematic reviews published in the special section "Issues and Advances in the Systematic Review of Single-Case Research: An Update and Exemplars." The commentary is provided in relation to the need to combine the assessment of the methodological quality of the studies included in systematic reviews, the assessment of the presence of functional relations via visual analysis following objective rules, and the quantification of the magnitudes of effect, providing meaningful information. Although it was not required that the exemplars follow specific guidelines for conduct and reporting, we applied an existing methodological quality checklist for systematic reviews and meta-analyses. Finally, we point at specific signs of advance in the field of performing systematic reviews of single-case design studies, as identified in the three exemplars, and we also suggest some issues requiring further research and discussion. [For the reviews discussed in this commentary, see EJ1160913, EJ1160914, and EJ1160915.]

Visual analysis of single-case research is commonly described as a gold standard, but it is often unreliable. Thus, an objective tool for applying visual analysis is necessary, as an alternative to the Conservative Dual Criterion, which presents some drawbacks. The proposed free web-based tool enables assessing change in trend and level between two adjacent phases, while taking data variability into account. The application of the tool results in (a) a dichotomous decision regarding the presence or absence of an immediate effect, a progressive or delayed effect, or an overall effect and (b) a quantification of overlap. The proposal is evaluated by applying it to both real and simulated data, obtaining favorable results. The visual aid and the objective rules are expected to make visual analysis more consistent, but they are not intended as a substitute for the analysts' judgment, as a formal test of statistical significance, or as a tool for assessing social validity. [ABSTRACT FROM AUTHOR]

We developed a reporting guideline to provide authors with guidance about what should be reported when writing a paper for publication in a scientific journal using a particular type of research design: the single-case experimental design. This report describes the methods used to develop the Single-Case Reporting guideline In BEhavioural interventions (SCRIBE) 2016. As a result of 2 online surveys and a 2-day meeting of experts, the SCRIBE 2016 checklist was developed, which is a set of 26 items that authors need to address when writing about single-case research. This article complements the more detailed SCRIBE 2016 Explanation and Elaboration article (Tate et al., 2016) that provides a rationale for each of the items and examples of adequate reporting from the literature. Both these resources will assist authors to prepare reports of single-case research with clarity, completeness, accuracy, and transparency. They will also provide journal reviewers and editors with a practical checklist against which such reports may be critically evaluated. We recommend that the SCRIBE 2016 is used by authors preparing manuscripts describing single-case research for publication, as well as journal reviewers and editors who are evaluating such manuscripts.

The present study focuses on single-case data analysis specifically on two procedures for quantifying differences between baseline and treatment measurements. The first technique tested is based on generalized least square regression analysis and is compared to a proposed non-regression technique, which allows obtaining similar information. The comparison is carried out in the context of generated data representing a variety of patterns including both independent and serially related measurements arising from different underlying processes. Heterogeneity in autocorrelation and data variability was also included, as well as different types of trend, and slope and level changes. The results suggest that the two techniques perform adequately for a wide range of conditions and that researchers can use both of them with certain guarantees. The regression-based procedure offers more efficient estimates, whereas the proposed non-regression procedure is more sensitive to intervention effects. Considering current and previous findings, some tentative recommendations are offered to applied researchers in order to help choosing among the plurality of single-case data analysis techniques. (Contains 6 tables and 3 figures.)

There is currently a considerable diversity of quantitative measures available for summarizing the results in single-case studies. Given that the interpretation of some of them is difficult due to the lack of established benchmarks, the current article proposes an approach for obtaining further numerical evidence on the importance of the results, complementing the substantive criteria, visual analysis, and primary summary measures. This additional evidence consists of obtaining the statistical significance of the outcome when referred to the corresponding sampling distribution. This sampling distribution is formed by the values of the outcomes (expressed as data nonoverlap, R[superscript 2], etc.) in case the intervention is ineffective. The approach proposed here is intended to offer the outcome's probability of being as extreme when there is no treatment effect without the need for some assumptions that cannot be checked with guarantees. Following this approach, researchers would compare their outcomes to reference values rather than constructing the sampling distributions themselves. The integration of single-case studies is problematic when different metrics are used across primary studies and not all raw data are available. Via the approach for assigning "p" values it is possible to combine the results of similar studies regardless of the primary effect size indicator. The alternatives for combining probabilities are discussed in the context of single-case studies, pointing out 2 potentially useful methods--one based on a weighted average and the other on the binomial test. (Contains 4 tables and 2 figures.)

Single-case experimental designs (SCEDs) are a class of experimental designs suited for answering research questions at an individual level. The main designs available in SCED research are phase designs, multiple baseline designs, alternation designs, and changing criterion designs. Embedded designs, also referred to as combination or hybrid designs, consist of one of these basic designs forms embedded in another design (e.g., a changing criterion design embedded in a multiple baseline design). Systematic reviews of SCEDs have repeatedly indicated that embedded designs are frequently used in applied SCED research. In spite of their popularity, specific recommendations on the conduct and analysis of embedded SCED designs are lacking to date. The purpose of the present article is therefore to provide guidance to applied researchers wishing to conduct embedded SCED designs in terms of design options, design requirements, randomization, and data analysis. [ABSTRACT FROM AUTHOR]

If single-case experimental designs are to be used to establish guidelines for evidence-based interventions in clinical and educational settings, numerical values that reflect treatment effect sizes are required. The present study compares four recently developed procedures for quantifying the magnitude of intervention effect using data with known characteristics. Monte Carlo methods were used to generate AB designs data with potential confounding variables (serial dependence, linear and curvilinear trend, and heteroscedasticity between phases) and two types of treatment effect (level and slope change). The results suggest that data features are important for choosing the appropriate procedure and, thus, inspecting the graphed data visually is a necessary initial stage. In the presence of serial dependence or a change in data variability, the nonoverlap of all pairs (NAP) and the slope and level change (SLC) were the only techniques of the four examined that performed adequately. Introducing a data correction step in NAP renders it unaffected by linear trend, as is also the case for the percentage of nonoverlapping corrected data and SLC. The performance of these techniques indicates that professionals' judgments concerning treatment effectiveness can be readily complemented by both visual and statistical analyses. A flowchart to guide selection of techniques according to the data characteristics identified by visual inspection is provided. (Contains 9 tables and 2 figures.)

The current study proposes a new procedure for separately estimating slope change and level change between two adjacent phases in single-case designs. The procedure eliminates baseline trend from the whole data series before assessing treatment effectiveness. The steps necessary to obtain the estimates are presented in detail, explained, and illustrated. A simulation study is carried out to explore the bias and precision of the estimators and compare them to an analytical procedure matching the data simulation model. The experimental conditions include 2 data generation models, several degrees of serial dependence, trend, and level and/or slope change. The results suggest that the level and slope change estimates provided by the procedure are unbiased for all levels of serial dependence tested and trend is effectively controlled for. The efficiency of the slope change estimator is acceptable, whereas the variance of the level change estimator may be problematic for highly negatively autocorrelated data series. (Contains 4 tables and 3 figures.)

This study deals with the statistical properties of a randomization test applied to an ABAB design in cases where the desirable random assignment of the points of change in phase is not possible. To obtain information about each possible data division, the authors carried out a conditional Monte Carlo simulation with 100,000 samples for each systematically chosen triplet. The authors studied robustness and power under several experimental conditions--different autocorrelation levels and different effect sizes as well as different phase lengths determined by the points of change. Type I error rates were distorted by the presence of autocorrelation for the majority of data divisions. The authors obtained satisfactory Type II error rates only for large treatment effects. The relation between the lengths of the four phases appeared to be an important factor for the robustness and power of the randomization test. (Contains 9 tables.)

Generalization from single-case designs can be achieved by replicating individual studies across different experimental units and settings. When replications are available, their findings can be summarized using effect size measurements and integrated through meta-analyses. Several procedures are available for quantifying the magnitude of treatment effect in N = 1 designs, and some of them are studied in this article. Monte Carlo simulations were used to generate different data patterns (trend, level change, and slope change). The experimental conditions simulated were defined by the degrees of serial dependence and phase length. Out of all the effect size indices studied, the percentage of nonoverlapping data and standardized mean difference proved to be less affected by autocorrelation and to perform better for shorter data series. The regression-based procedures proposed specifically for single-case designs did not differentiate between data patterns as well as did simpler indices. (Contains 4 figures.)

This text focuses on the quantitative integration of studies using alternating treatments designs (ATDs) and changing criterion designs (CCDs) with other types of single-case experimental designs (SCEDs), such as multiple-baseline and withdrawal, which have received more attention in terms of statistical developments. First, a review of how published meta-analyses have dealt with ATDs and CCDs suggests a variety of analytical strategies and insufficient transparency in reporting the exact comparisons performed. Second, we review data-analytical techniques for ATDs and CCDs, looking for alternatives that can be useful for meta-analysis. Third, without losing sight of the underlying logic of the ATDs and CCDs, we propose using multilevel models, comparing data paths in ATDs and the baseline to the last intervention subphase in CCDs. Furthermore, we suggest additional evaluations of the data pattern, beyond the quantification of the magnitude of effect. We also advocate for transparent reporting of how exactly the conditions are being compared for the different SCEDs. This is possible by specifying the design matrices for the multilevel models used. Fourth, in the context of one of the meta-analyses reviewed, we comment on a series of analytical decisions that need to be made before carrying out a meta-analysis using multilevel models. [ABSTRACT FROM AUTHOR]