Back to Search Start Over

Examining the normality assumption of a design-comparable effect size in single-case designs.

Authors :
Chen, Li-Ting
Chen, Yi-Kai
Yang, Tong-Rong
Chiang, Yu-Shan
Hsieh, Cheng-Yu
Cheng, Che
Ding, Qi-Wen
Wu, Po-Ju
Peng, Chao-Ying Joanne
Source :
Behavior Research Methods. Jan2024, Vol. 56 Issue 1, p379-405. 27p.
Publication Year :
2024

Abstract

What Works Clearinghouse (WWC, 2022) recommends a design-comparable effect size (D-CES; i.e., gAB) to gauge an intervention in single-case experimental design (SCED) studies, or to synthesize findings in meta-analysis. So far, no research has examined gAB's performance under non-normal distributions. This study expanded Pustejovsky et al. (2014) to investigate the impact of data distributions, number of cases (m), number of measurements (N), within-case reliability or intra-class correlation (ρ), ratio of variance components (λ), and autocorrelation (ϕ) on gAB in multiple-baseline (MB) design. The performance of gAB was assessed by relative bias (RB), relative bias of variance (RBV), MSE, and coverage rate of 95% CIs (CR). Findings revealed that gAB was unbiased even under non-normal distributions. gAB's variance was generally overestimated, and its 95% CI was over-covered, especially when distributions were normal or nearly normal combined with small m and N. Large imprecision of gAB occurred when m was small and ρ was large. According to the ANOVA results, data distributions contributed to approximately 49% of variance in RB and 25% of variance in both RBV and CR. m and ρ each contributed to 34% of variance in MSE. We recommend gAB for MB studies and meta-analysis with N ≥ 16 and when either (1) data distributions are normal or nearly normal, m = 6, and ρ = 0.6 or 0.8, or (2) data distributions are mildly or moderately non-normal, m ≥ 4, and ρ = 0.2, 0.4, or 0.6. The paper concludes with a discussion of gAB's applicability and design-comparability, and sound reporting practices of ES indices. [ABSTRACT FROM AUTHOR]

Details

Language :
English
ISSN :
1554351X
Volume :
56
Issue :
1
Database :
Academic Search Index
Journal :
Behavior Research Methods
Publication Type :
Academic Journal
Accession number :
174839850
Full Text :
https://doi.org/10.3758/s13428-022-02035-8