1. Sample Size Planning for Statistical Power and Accurate Estimation
- Author
-
Samantha F. Anderson and Sophia J. Lamp
- Abstract
At the most general level, sample size refers to the number of participants (typically human or animal subjects) who provide data for a scientific experiment or study. Selecting an appropriate sample size is a fundamental feature of a well-designed study, and, as the title suggests, sample size can be carefully planned ahead of time. Sample size planning is important because the sample size of a study has implications for statistical power as well as accuracy. In conventional terms, statistical power reflects the probability of rejecting the null hypothesis, assuming that the purported effect is non-null (non-zero) in reality. Although power can be treated as a function of unknown parameter values, power is often defined conditionally on a specific value of population effect size. (On a technical level, power is conditional upon the population noncentrality parameter, which is a combination of sample size and effect size, but relying on effect size as a proxy is generally acceptable.) Power is calculated from the statistical significance threshold (otherwise known as the nominal Type I error rate or alpha-level), the population effect size, and the sample size. Because of the connection between sample size and power, sample size planning is sometimes called a priori power analysis. For example, if a new therapy developed to treat depression is truly beneficial, the sample size of a study assessing this therapy should be large enough to demonstrate the beneficial outcome as statistically significant. Accuracy reflects how close an estimate is to the parameter it aims to estimate. Accurate estimates are expressed with narrow confidence intervals (small margins of error). Sample size also has a connection to accuracy. For example, if the investigators in the previous example want to estimate how much the therapy reduces depressive symptoms, the sample size should be large enough so that the effect size estimate derived from the study is an accurate estimate of the true magnitude of the treatment effect. An underlying challenge to sample size planning for both goals is that the population effect size is unknown. An important corollary is that whether a particular sample size provides appropriate power (and in some cases, accuracy) depends on the population effect size, and as such, sample size cannot completely be judged without context. Although there are sometimes limitations to sample size, such as when working with a limited budget or specialized populations, and although sample size is often selected based on convenience or historical precedent, when possible it is desirable to directly plan the study sample size at the study design phase. Explicitly planning sample size helps to ensure that the sample size will be effective in meeting the investigators’ goals and answering the scientific questions of interest.
- Published
- 2022
- Full Text
- View/download PDF