Search

Whereas the theory of confirmatory adaptive designs is well understood for uncensored data, implementation of adaptive designs in the context of survival trials remains challenging. Commonly used adaptive survival tests are based on the independent increments structure of the log-rank statistic. This implies some relevant limitations: On the one hand, essentially only the interim log-rank statistic may be used for design modifications (such as data-dependent sample size recalculation). Furthermore, the treatment arm allocation ratio in these classical methods is assumed to be constant throughout the trial period. Here, we propose an extension of the independent increments approach to adaptive survival tests that addresses some of these limitations. We present a confirmatory adaptive two-sample log-rank test that allows rejection regions and sample size recalculation rules to be based not only on the interim log-rank statistic, but also on point-wise survival rate estimates, simultaneously. In addition, the possibility is opened to adapt the treatment arm allocation ratio after each interim analysis in a data-dependent way. The ability to include point-wise survival rate estimators in the rejection region of a test for comparing survival curves might be attractive, e.g., for seamless phase II/III designs. Data-dependent adaptation of the allocation ratio could be helpful in multi-arm trials in order to successively steer recruitment into the study arms with the greatest chances of success. The methodology is motivated by the LOGGIC Europe Trial from pediatric oncology. Distributional properties are derived using martingale techniques in the large sample limit. Small sample properties are studied by simulation.

Recent minimum sample size formula (Riley et al.) for developing clinical prediction models help ensure that development datasets are of sufficient size to minimise overfitting. While these criteria are known to avoid excessive overfitting on average, the extent of variability in overfitting at recommended sample sizes is unknown. We investigated this through a simulation study and empirical example to develop logistic regression clinical prediction models using unpenalised maximum likelihood estimation, and various post-estimation shrinkage or penalisation methods. While the mean calibration slope was close to the ideal value of one for all methods, penalisation further reduced the level of overfitting, on average, compared to unpenalised methods. This came at the cost of higher variability in predictive performance for penalisation methods in external data. We recommend that penalisation methods are used in data that meet, or surpass, minimum sample size requirements to further mitigate overfitting, and that the variability in predictive performance and any tuning parameters should always be examined as part of the model development process, since this provides additional information over average (optimism-adjusted) performance alone. Lower variability would give reassurance that the developed clinical prediction model will perform well in new individuals from the same population as was used for model development.

Ultrahigh-dimensional gene features are often collected in modern cancer studies in which the number of gene features p is extremely larger than sample size n. While gene expression patterns have been shown to be related to patients' survival in microarray-based gene expression studies, one has to deal with the challenges of ultrahigh-dimensional genetic predictors for survival predicting and genetic understanding of the disease in precision medicine. The problem becomes more complicated when two types of survival endpoints, distant metastasis-free survival and overall survival, are of interest in the study and outcome data can be subject to semi-competing risks due to the fact that distant metastasis-free survival is possibly censored by overall survival but not vice versa. Our focus in this paper is to extract important features, which have great impacts on both distant metastasis-free survival and overall survival jointly, from massive gene expression data in the semi-competing risks setting. We propose a model-free screening method based on the ranking of the correlation between gene features and the joint survival function of two endpoints. The method accounts for the relationship between two endpoints in a simply defined utility measure that is easy to understand and calculate. We show its favorable theoretical properties such as the sure screening and ranking consistency, and evaluate its finite sample performance through extensive simulation studies. Finally, an application to classifying breast cancer data clearly demonstrates the utility of the proposed method in practice. Ministry of Education (MOE) Submitted/Accepted version Xiang’s research was supported by the Singapore Ministry of Education Academic Research Fund Tier 1 grant RG98/20 and Peng’s research was supported by National Natural Science Foundation of China (NSFC Grant No. 92046005).

Sample size calculations for cluster-randomised trials require inclusion of an inflation factor taking into account the intra-cluster correlation coefficient. Often, estimates of the intra-cluster correlation coefficient are taken from pilot trials, which are known to have uncertainty about their estimation. Given that the value of the intra-cluster correlation coefficient has a considerable influence on the calculated sample size for a main trial, the uncertainty in the estimate can have a large impact on the ultimate sample size and consequently, the power of a main trial. As such, it is important to account for the uncertainty in the estimate of the intra-cluster correlation coefficient. While a commonly adopted approach is to utilise the upper confidence limit in the sample size calculation, this is a largely inefficient method which can result in overpowered main trials. In this paper, we present a method of estimating the sample size for a main cluster-randomised trial with a continuous outcome, using numerical methods to account for the uncertainty in the intra-cluster correlation coefficient estimate. Despite limitations with this initial study, the findings and recommendations in this paper can help to improve sample size estimations for cluster randomised controlled trials by accounting for uncertainty in the estimate of the intra-cluster correlation coefficient. We recommend this approach be applied to all trials where there is uncertainty in the intra-cluster correlation coefficient estimate, in conjunction with additional sources of information to guide the estimation of the intra-cluster correlation coefficient.

By combining data across multiple studies, researchers increase sample size, statistical power, and precision for pooled analyses of biomarker–disease associations. However, researchers must adjust for between-study variability in biomarker measurements. Previous research often treats the biomarker measurements from a reference laboratory as a gold standard, even though those measurements are certainly not equal to their true values. This paper addresses measurement error and bias arising from both the reference and study-specific laboratories. We develop two calibration methods, the exact calibration method and approximate calibration method, for pooling biomarker data drawn from nested or matched case–control studies, where the calibration subset is obtained by randomly selecting controls from each contributing study. Simulation studies are conducted to evaluate the empirical performance of the proposed methods. We apply the proposed methods to a pooling project of nested case–control studies to evaluate the association between circulating 25-hydroxyvitamin D (25(OH)D) and colorectal cancer risk.

The modified Poisson regression coupled with a robust sandwich variance has become a viable alternative to log-binomial regression for estimating the marginal relative risk in cluster randomized trials. However, a corresponding sample size formula for relative risk regression via the modified Poisson model is currently not available for cluster randomized trials. Through analytical derivations, we show that there is no loss of asymptotic efficiency for estimating the marginal relative risk via the modified Poisson regression relative to the log-binomial regression. This finding holds both under the independence working correlation and under the exchangeable working correlation provided a simple modification is used to obtain the consistent intraclass correlation coefficient estimate. Therefore, the sample size formulas developed for log-binomial regression naturally apply to the modified Poisson regression in cluster randomized trials. We further extend the sample size formulas to accommodate variable cluster sizes. An extensive Monte Carlo simulation study is carried out to validate the proposed formulas. We find that the proposed formulas have satisfactory performance across a range of cluster size variability, as long as suitable finite-sample corrections are applied to the sandwich variance estimator and the number of clusters is at least 10. Our findings also suggest that the sample size estimate under the exchangeable working correlation is more robust to cluster size variability, and recommend the use of an exchangeable working correlation over an independence working correlation for both design and analysis. The proposed sample size formulas are illustrated using the Stop Colorectal Cancer (STOP CRC) trial.

Simulation offers a simple and flexible way to estimate the power of a clinical trial when analytic formulae are not available. The computational burden of using simulation has, however, restricted its application to only the simplest of sample size determination problems, minimising a single parameter (the overall sample size) subject to power being above a target level. We describe a general framework for solving simulation-based sample size determination problems with several design parameters over which to optimise and several conflicting criteria to be minimised. The method is based on an established global optimisation algorithm widely used in the design and analysis of computer experiments, using a non-parametric regression model as an approximation of the true underlying power function. The method is flexible, can be used for almost any problem for which power can be estimated using simulation, and can be implemented using existing statistical software packages. We illustrate its application to three increasingly complicated sample size determination problems involving complex clustering structures, co-primary endpoints, and small sample considerations.

In many experiments and especially in translational and preclinical research, sample sizes are (very) small. In addition, data designs are often high dimensional, i.e. more dependent than independent replications of the trial are observed. The present paper discusses the applicability of max t-test-type statistics (multiple contrast tests) in high-dimensional designs (repeated measures or multivariate) with small sample sizes. A randomization-based approach is developed to approximate the distribution of the maximum statistic. Extensive simulation studies confirm that the new method is particularly suitable for analyzing data sets with small sample sizes. A real data set illustrates the application of the methods.

Growth reference centile charts are widely used in child health to assess weight, height and other age-varying measurements. The centiles are easy to construct from reference data, using the LMS method or GAMLSS (Generalised Additive Models for Location Scale and Shape). However, there is as yet no clear guidance on how to design such studies, and in particular how many reference data to collect, and this has led to study sizes varying widely. The paper aims to provide a theoretical framework for optimally designing growth reference studies based on cross-sectional data. Centiles for weight, height, body mass index and head circumference, in 6878 boys aged 0–21 years from the Fourth Dutch Growth Study, were fitted using GAMLSS. The effect on precision of varying the sample size and the distribution of measurement ages (sample composition) was explored by fitting a series of GAMLSS models to simulated data. Sample composition was defined as uniform on the age λ scale, where λ was chosen to give constant precision across the age range. Precision was measured on the z-score scale, and was the same for all four measurements, with a standard error of 0.041 z-score units for the median and 0.066 for the 2nd and 98th centiles. Compared to a naïve calculation, the process of smoothing the centiles increased the notional sample size two- to threefold by ‘borrowing strength’. The sample composition for estimating the median curve was optimal for λ=0.4, reflecting considerable over-sampling of infants compared to children. However, for the 2nd and 98th centiles, λ=0.75 was optimal, with less infant over-sampling. The conclusion is that both sample size and sample composition need to be optimised. The paper provides practical advice on design, and concludes that optimally designed studies need 7000–25,000 subjects per sex.

To estimate the mean of a quantitative variable in a hierarchical population, it is logistically convenient to sample in two stages (two-stage sampling), i.e. selecting first clusters, and then individuals from the sampled clusters. Allowing cluster size to vary in the population and to be related to the mean of the outcome variable of interest (informative cluster size), the following competing sampling designs are considered: sampling clusters with probability proportional to cluster size, and then the same number of individuals per cluster; drawing clusters with equal probability, and then the same percentage of individuals per cluster; and selecting clusters with equal probability, and then the same number of individuals per cluster. For each design, optimal sample sizes are derived under a budget constraint. The three optimal two-stage sampling designs are compared, in terms of efficiency, with each other and with simple random sampling of individuals. Sampling clusters with probability proportional to size is recommended. To overcome the dependency of the optimal design on unknown nuisance parameters, maximin designs are derived. The results are illustrated, assuming probability proportional to size sampling of clusters, with the planning of a hypothetical survey to compare adolescent alcohol consumption between France and Italy.

The stepped wedge cluster randomized design has received increasing attention in pragmatic clinical trials and implementation science research. The key feature of the design is the unidirectional crossover of clusters from the control to intervention conditions on a staggered schedule, which induces confounding of the intervention effect by time. The stepped wedge design first appeared in the Gambia hepatitis study in the 1980s. However, the statistical model used for the design and analysis was not formally introduced until 2007 in an article by Hussey and Hughes. Since then, a variety of mixed-effects model extensions have been proposed for the design and analysis of these trials. In this article, we explore these extensions under a unified perspective. We provide a general model representation and regard various model extensions as alternative ways to characterize the secular trend, intervention effect, as well as sources of heterogeneity. We review the key model ingredients and clarify their implications for the design and analysis. The article serves as an entry point to the evolving statistical literatures on stepped wedge designs.

Introduction In a confirmatory diagnostic accuracy study, sensitivity and specificity are considered as co-primary endpoints. For the sample size calculation, the prevalence of the target population must be taken into account to obtain a representative sample. In this context, a general problem arises. With a low or high prevalence, the study may be overpowered in one subpopulation. One further issue is the correct pre-specification of the true prevalence. With an incorrect assumption about the prevalence, an over- or underestimated sample size will result. Methods To obtain the desired power independent of the prevalence, a method for an optimal sample size calculation for the comparison of a diagnostic experimental test with a prespecified minimum sensitivity and specificity is proposed. To face the problem of an incorrectly pre-specified prevalence, a blinded one-time re-estimation design of the sample size based on the prevalence and a blinded repeated re-estimation design of the sample size based on the prevalence are evaluated by a simulation study. Both designs are compared to a fixed design and additionally among each other. Results The type I error rates of both blinded re-estimation designs are not inflated. Their empirical overall power equals the desired theoretical power and both designs offer unbiased estimates of the prevalence. The repeated re-estimation design reveals no advantages concerning the mean squared error of the re-estimated prevalence or sample size compared to the one-time re-estimation design. The appropriate size of the internal pilot study in the one-time re-estimation design is 50% of the initially calculated sample size. Conclusions A one-time re-estimation design of the prevalence based on the optimal sample size calculation is recommended in single-arm diagnostic accuracy studies.

Unbiased estimation of causal effects with observational data requires adjustment for confounding variables that are related to both the outcome and treatment assignment. Standard variable selection techniques aim to maximize predictive ability of the outcome model, but they ignore covariate associations with treatment and may not adjust for important confounders weakly associated to outcome. We propose a novel method for estimating causal effects that simultaneously considers models for both outcome and treatment, which we call the bilevel spike and slab causal estimator (BSSCE). By using a Bayesian formulation, BSSCE estimates the posterior distribution of all model parameters and provides straightforward and reliable inference. Spike and slab priors are used on each covariate coefficient which aim to minimize the mean squared error of the treatment effect estimator. Theoretical properties of the treatment effect estimator are derived justifying the prior used in BSSCE. Simulations show that BSSCE can substantially reduce mean squared error over numerous methods and performs especially well with large numbers of covariates, including situations where the number of covariates is greater than the sample size. We illustrate BSSCE by estimating the causal effect of vasoactive therapy vs. fluid resuscitation on hypotensive episode length for patients in the Multiparameter Intelligent Monitoring in Intensive Care III critical care database.

Clinicians and researchers alike are increasingly interested in how best to personalize interventions. A dynamic treatment regimen (DTR) is a sequence of pre-specified decision rules which can be used to guide the delivery of a sequence of treatments or interventions that are tailored to the changing needs of the individual. The sequential multiple-assignment randomized trial (SMART) is a research tool which allows for the construction of effective DTRs. We derive easy-to-use formulae for computing the total sample size for three common two-stage SMART designs in which the primary aim is to compare mean end-of-study outcomes for two embedded DTRs which recommend different first-stage treatments. The formulae are derived in the context of a regression model which leverages information from a longitudinal outcome collected over the entire study. We show that the sample size formula for a SMART can be written as the product of the sample size formula for a standard two-arm randomized trial, a deflation factor that accounts for the increased statistical efficiency resulting from a longitudinal analysis, and an inflation factor that accounts for the design of a SMART. The SMART design inflation factor is typically a function of the anticipated probability of response to first-stage treatment. We review modeling and estimation for DTR effect analyses using a longitudinal outcome from a SMART, as well as the estimation of standard errors. We also present estimators for the covariance matrix for a variety of common working correlation structures. Methods are motivated using the ENGAGE study, a SMART aimed at developing a DTR for increasing motivation to attend treatments among alcohol- and cocaine-dependent patients., Supplementary material at https://osf.io/q7zv8/ This version updates funding information not previously included. Text remains the final accepted version of the manuscript in Statistical Methods in Medical Research

An important goal of personalized medicine is to identify heterogeneity in treatment effects and then use that heterogeneity to target the intervention to those most likely to benefit. Heterogeneity is assessed using the predicted individual treatment effects framework, and a permutation test is proposed to establish if significant heterogeneity is present given the covariates and predictive model or algorithm used for predicted individual treatment effects. We first show evidence for heterogeneity in the effects of treatment across an illustrative example data set. We then use simulations with two different predictive methods (linear regression model and Random Forests) to show that the permutation test has adequate type-I error control. Next, we use an example dataset as the basis for simulations to demonstrate the ability of the permutation test to find heterogeneity in treatment effects for a predicted individual treatment effects estimate as a function of both effect size and sample size. We find that the proposed test has good power for detecting heterogeneity in treatment effects when the heterogeneity was due primarily to a single predictor, or when it was spread across the predictors. Power was found to be greater for predictions from a linear model than from random forests. This non-parametric permutation test can be used to test for significant differences across individuals in predicted individual treatment effects obtained with a given set of covariates using any predictive method with no additional assumptions.

Non-adherence to assigned treatment is a common issue in cluster randomised trials (CRTs). In these settings, the efficacy estimand may be also of interest. Many methodological contributions in recent years have advocated using instrumental variables to identify and estimate the local average treatment effect (LATE). However, the clustered nature of randomisation in CRTs adds to the complexity of such analyses. In this paper, we show that under certain assumptions, the LATE can be estimated via two-stage least squares (TSLS) using cluster-level summaries of outcomes and treatment received. Implementation needs to account for this, as well as the possible heteroscedasticity, to obtain valid inferences. We use simulations to assess the performance of TSLS of cluster-level summaries under cluster-level or individual-level non-adherence, with and without weighting and robust standard errors. We also explore the impact of adjusting for cluster-level covariates and of appropriate degrees of freedom correction for inference. We find that TSLS estimation using cluster-level summaries provides estimates with small to negligible bias and coverage close to nominal level, provided small sample degrees of freedom correction is used for inference, with appropriate use of robust standard errors. We illustrate the methods by re-analysing a CRT in UK primary health settings., 21 pages, 6 Figures

Background Among various design aspects, the choice of randomization procedure have to be agreed on, when planning a clinical trial stratified by center. The aim of the paper is to present a methodological approach to evaluate whether a randomization procedure mitigates the impact of bias on the test decision in clinical trial stratified by center. Methods We use the weighted t test to analyze the data from a clinical trial stratified by center with a two-arm parallel group design, an intended 1:1 allocation ratio, aiming to prove a superiority hypothesis with a continuous normal endpoint without interim analysis and no adaptation in the randomization process. The derivation is based on the weighted t test under misclassification, i.e. ignoring bias. An additive bias model combing selection bias and time-trend bias is linked to different stratified randomization procedures. Results Various aspects to formulate stratified versions of randomization procedures are discussed. A formula for sample size calculation of the weighted t test is derived and used to specify the tolerated imbalance allowed by some randomization procedures. The distribution of the weighted t test under misclassification is deduced, taking the sequence of patient allocation to treatment, i.e. the randomization sequence into account. An additive bias model combining selection bias and time-trend bias at strata level linked to the applied randomization sequence is proposed. With these before mentioned components, the potential impact of bias on the type one error probability depending on the selected randomization sequence and thus the randomization procedure is formally derived and exemplarily calculated within a numerical evaluation study. Conclusion The proposed biasing policy and test distribution are necessary to conduct an evaluation of the comparative performance of (stratified) randomization procedure in multi-center clinical trials with a two-arm parallel group design. It enables the choice of the best practice procedure. The evaluation stimulates the discussion about the level of evidence resulting in those kind of clinical trials.

The case-cohort design is an economical approach to estimate the effect of risk factors on the survival outcome when collecting exposure information or covariates on all patients is expensive in a large cohort study. Variables often have group structure such as categorical variables and highly correlated continuous variables. The existing literature for case-cohort data is limited to identifying non-zero variables at individual level only. In this article, we propose a bi-level variable selection method to select non-zero group and within-group variables for case-cohort data when variables have group structure. The proposed method allows the number of variables to diverge as the sample size increases. The asymptotic properties of the estimator including bi-level variable selection consistency and the asymptotic normality are shown. We also conduct simulations to compare our proposed method with some existing method and apply them to the Busselton Health data.

In oncology trials, control group patients often switch onto the experimental treatment during follow-up, usually after disease progression. In this case, an intention-to-treat analysis will not address the policy question of interest – that of whether the new treatment represents an effective and cost-effective use of health care resources, compared to the standard treatment. Rank preserving structural failure time models (RPSFTM), inverse probability of censoring weights (IPCW) and two-stage estimation (TSE) have often been used to adjust for switching to inform treatment reimbursement policy decisions. TSE has been applied using a simple approach (TSEsimp), assuming no time-dependent confounding between the time of disease progression and the time of switch. This is problematic if there is a delay between progression and switch. In this paper we introduce TSEgest, which uses structural nested models and g-estimation to account for time-dependent confounding, and compare it to TSEsimp, RPSFTM and IPCW. We simulated scenarios where control group patients could switch onto the experimental treatment with and without time-dependent confounding being present. We varied switching proportions, treatment effects and censoring proportions. We assessed adjustment methods according to their estimation of control group restricted mean survival times that would have been observed in the absence of switching. All methods performed well in scenarios with no time-dependent confounding. TSEgest and RPSFTM continued to perform well in scenarios with time-dependent confounding, but TSEsimp resulted in substantial bias. IPCW also performed well in scenarios with time-dependent confounding, except when inverse probability weights were high in relation to the size of the group being subjected to weighting, which occurred when there was a combination of modest sample size and high switching proportions. TSEgest represents a useful addition to the collection of methods that may be used to adjust for treatment switching in trials in order to address policy-relevant questions.

Several modeling approaches have been developed to quantify differences in multiple sclerosis lesion evolution on magnetic resonance imaging to identify the effect of treatment on disease progression. These studies have limited clinical applicability due to onerous scan frequency and lengthy study duration. Efficient methods are needed to reduce the required sample size, study duration, and sampling frequency in longitudinal magnetic resonance imaging studies. We develop a data-driven approach to identify parameters of study design for evaluation of longitudinal magnetic resonance imaging biomarkers of multiple sclerosis lesion evolution. Our design strategies are considerably shorter than those described in previous studies, thus having the potential to lower costs of clinical trials. From a dataset of 36 multiple sclerosis patients with at least six monthly magnetic resonance imagings, we extracted new lesions and performed principal component analysis to estimate a biomarker that recapitulated lesion recovery. We tested the effect of multiple sclerosis disease modifying therapy on the lesion evolution index in three experimental designs and calculated sample sizes needed to appropriately power studies. Our proposed methods can be used to calculate required sample size and scan frequency in observational studies of multiple sclerosis disease progression as well as in designing clinical trials to find effects of treatment on multiple sclerosis lesion evolution.

Contains fulltext : 215531.pdf (Publisher’s version ) (Open Access) Goal Attainment Scaling is an assessment instrument to evaluate interventions on the basis of individual, patient-specific goals. The attainment of these goals is mapped in a pre-specified way to attainment levels on an ordinal scale, which is common to all goals. This approach is patient-centred and allows one to integrate the outcomes of patients with very heterogeneous symptoms. The latter is of particular importance in clinical trials in rare diseases because it enables larger sample sizes by including a broader patient population. In this paper, we focus on the statistical analysis of Goal Attainment Scaling outcomes for the comparison of two treatments in randomised clinical trials. Building on a general statistical model, we investigate the properties of different hypothesis testing approaches. Additionally, we propose a latent variable approach to generate Goal Attainment Scaling data in a simulation study, to assess the impact of model parameters such as the number of goals per patient and their correlation, the choice of discretisation thresholds and the type of design (parallel group or cross-over). Based on our findings, we give recommendations for the design of clinical trials with a Goal Attainment Scaling endpoint. Furthermore, we discuss an application of Goal Attainment Scaling in a clinical trial in mastocytosis.

The GH-2000 score has been developed as a powerful and unique technique for the detection of growth hormone misuse by sportsmen and women. The score depends upon the measurement of two growth hormone sensitive markers, insulin-like growth factor-I and the amino-terminal pro-peptide of type III collagen. It also includes a term to adjust for the age of the athlete. Decision limits for the GH-2000 score have been developed and are incorporated into the guidelines of the World Anti-Doping Agency. These decision limits are derived by setting a 1 in 10,000 false-positive rate rule. As these decision limits are estimated from samples of GH-2000 scores, they carry uncertainty. In previous work, this uncertainty has been addressed by establishing an upper 95% confidence interval for the true decision limits based on a normal approximation which has been shown to be appropriate if sample sizes are large (such as 1000 and above). Here, we show that these approximations, whether reasonable or not, can be entirely avoided by developing an upper 95% confidence interval for the true decision limits using an approach based upon the t-distribution. While there are considerable differences for smaller sample sizes, these become negligible when the sample size is large such as 1000 and above.

Motivated by a study exploring differences in glycemic control between non-Hispanic black and non-Hispanic white veterans with type 2 diabetes, we aim to address a type of confounding that arises in spatially referenced observational studies. Specifically, we develop a spatial doubly robust propensity score estimator to reduce bias associated with geographic confounding, which occurs when measured or unmeasured confounding factors vary by geographic location, leading to imbalanced group comparisons. We augment the doubly robust estimator with spatial random effects, which are assigned conditionally autoregressive priors to improve inferences by borrowing information across neighboring geographic regions. Through a series of simulations, we show that ignoring spatial variation results in increased absolute bias and mean squared error, while the spatial doubly robust estimator performs well under various levels of spatial heterogeneity and moderate sample sizes. In the motivating application, we construct three global estimates of the risk difference between race groups: an unadjusted estimate, a doubly robust estimate that adjusts only for patient-level information, and a hierarchical spatial doubly robust estimate. Results indicate a gradual reduction in the risk difference at each stage, with the inclusion of spatial random effects providing a 20% reduction compared to an estimate that ignores spatial heterogeneity. Smoothed maps indicate poor glycemic control across Alabama and southern Georgia, areas comprising the so-called “stroke belt.” These results suggest the need for community-specific interventions to target diabetes in geographic areas of greatest need.

The random effects model in meta-analysis is a standard statistical tool often used to analyze the effect sizes of the quantity of interest if there is heterogeneity between studies. In the special case considered here, meta-analytic data contain only the sample means in two treatment arms and the sample sizes, but no sample standard deviation. The statistical comparison between two arms for this case is not possible within the existing meta-analytic inference framework. Therefore, the main objective of this paper is to estimate the overall mean difference and associated variances, the between-study variance and the within-study variance, as specified as the important elements in the random effects model. These estimators are obtained using maximum likelihood estimation. The standard errors of the estimators and a quantification of the degree of heterogeneity are also investigated. A measure of heterogeneity is suggested which adjusts the original suggested measure of Higgins’ I2 for within study sample size. The performance of the proposed estimators is evaluated using simulations. It can be concluded that all estimated means converged to their associated true parameter values, and its standard errors tended to be small if the number of the studies involved in the meta-analysis was large. The proposed estimators could be favorably applied in a meta-analysis on comparing two surgeries for asymptomatic congenital lung malformations in young children.

We consider modelling and inference as well as sample size estimation and reestimation for clinical trials with longitudinal count data as outcomes. Our approach is general but is rooted in design and analysis of multiple sclerosis trials where lesion counts obtained by magnetic resonance imaging are important endpoints. We adopt a binomial thinning model that allows for correlated counts with marginal Poisson or negative binomial distributions. Methods for sample size planning and blinded sample size reestimation for randomised controlled clinical trials with such outcomes are developed. The models and approaches are applicable to data with incomplete observations. A simulation study is conducted to assess the effectiveness of sample size estimation and blinded sample size reestimation methods. Sample sizes attained through these procedures are shown to maintain the desired study power without inflating the type I error. Data from a recent trial in patients with secondary progressive multiple sclerosis illustrate the modelling approach.

Cluster-level dynamic treatment regimens can be used to guide sequential treatment decision-making at the cluster level in order to improve outcomes at the individual or patient-level. In a cluster-level dynamic treatment regimen, the treatment is potentially adapted and re-adapted over time based on changes in the cluster that could be impacted by prior intervention, including aggregate measures of the individuals or patients that compose it. Cluster-randomized sequential multiple assignment randomized trials can be used to answer multiple open questions preventing scientists from developing high-quality cluster-level dynamic treatment regimens. In a cluster-randomized sequential multiple assignment randomized trial, sequential randomizations occur at the cluster level and outcomes are observed at the individual level. This manuscript makes two contributions to the design and analysis of cluster-randomized sequential multiple assignment randomized trials. First, a weighted least squares regression approach is proposed for comparing the mean of a patient-level outcome between the cluster-level dynamic treatment regimens embedded in a sequential multiple assignment randomized trial. The regression approach facilitates the use of baseline covariates which is often critical in the analysis of cluster-level trials. Second, sample size calculators are derived for two common cluster-randomized sequential multiple assignment randomized trial designs for use when the primary aim is a between-dynamic treatment regimen comparison of the mean of a continuous patient-level outcome. The methods are motivated by the Adaptive Implementation of Effective Programs Trial which is, to our knowledge, the first-ever cluster-randomized sequential multiple assignment randomized trial in psychiatry.

Just over half of publicly funded trials recruit their target sample size within the planned study duration. When recruitment targets are missed, the funder of a trial is faced with the decision of either committing further resources to the study or risk that a worthwhile treatment effect may be missed by an underpowered final analysis. To avoid this challenging situation, when there is insufficient prior evidence to support predicted recruitment rates, funders now require feasibility assessments to be performed in the early stages of trials. Progression criteria are usually specified and agreed with the funder ahead of time. To date, however, the progression rules used are typically ad hoc. In addition, rules routinely permit adaptations to recruitment strategies but do not stipulate criteria for evaluating their effectiveness. In this paper, we develop a framework for planning and designing internal pilot studies which permit a trial to be stopped early if recruitment is disappointing or to continue to full recruitment if enrolment during the feasibility phase is adequate. This framework enables a progression rule to be pre-specified and agreed upon prior to starting a trial. The novel two-stage designs stipulate that if neither of these situations arises, adaptations to recruitment should be made and subsequently evaluated to establish whether they have been successful. We derive optimal progression rules for internal pilot studies which minimise the expected trial overrun and maintain a high probability of completing the study when the recruitment rate is adequate. The advantages of this procedure are illustrated using a real trial example.

This paper compares the power of the parallel group design, the matched-pairs design, and several options for the stepped wedge and delayed start designs for testing a possible effect of intranasal insulin with respect to placebo on developmental growth of children with a rare disorder like Phelan-McDermid syndrome. A subject-specific linear mixed effects model for the primary outcome developmental age in a longitudinal setting with five time points was assumed. Monte Carlo simulation studies with small sample sizes were applied since the rare disorder prohibits large trials. The stepped wedge designs, which were initially preferred for ethical reasons, appear to be competitive in power to other designs and were in some settings even the best. The assumed statistical model also demonstrates that all of the designs can be viewed as a stepped wedge or delayed treatment design. Our results show that the stepped wedge design is an appropriate alternative for randomized controlled trials on developmental growth with small numbers of participants under the formulated statistical conditions.

We conducted an extensive set of empirical analyses to examine the effect of the number of events per variable (EPV) on the relative performance of three different methods for assessing the predictive accuracy of a logistic regression model: apparent performance in the analysis sample, split-sample validation, and optimism correction using bootstrap methods. Using a single dataset of patients hospitalized with heart failure, we compared the estimates of discriminatory performance from these methods to those for a very large independent validation sample arising from the same population. As anticipated, the apparent performance was optimistically biased, with the degree of optimism diminishing as the number of events per variable increased. Differences between the bootstrap-corrected approach and the use of an independent validation sample were minimal once the number of events per variable was at least 20. Split-sample assessment resulted in too pessimistic and highly uncertain estimates of model performance. Apparent performance estimates had lower mean squared error compared to split-sample estimates, but the lowest mean squared error was obtained by bootstrap-corrected optimism estimates. For bias, variance, and mean squared error of the performance estimates, the penalty incurred by using split-sample validation was equivalent to reducing the sample size by a proportion equivalent to the proportion of the sample that was withheld for model validation. In conclusion, split-sample validation is inefficient and apparent performance is too optimistic for internal validation of regression-based prediction models. Modern validation methods, such as bootstrap-based optimism correction, are preferable. While these findings may be unsurprising to many statisticians, the results of the current study reinforce what should be considered good statistical practice in the development and validation of clinical prediction models.

Population attributable fraction is a widely used measure for quantifying the disease burden associated with a modifiable exposure of interest at the population level. It has been extended to a time-varying measure, population attributable hazard function, to provide additional information on when and how the exposure's impact varies over time. However, like the classic population attributable fraction, the population attributable hazard is generally biased if confounders are present. In this article, we provide a natural definition of adjusted population attributable hazard to take into account the effects of confounders, and its alternative that is identifiable from case–control studies under the rare disease assumption. We propose a novel estimator, which combines the odds ratio estimator from logistic regression model, and the conditional density function estimator of the exposure and confounding variables distribution given the failure times of cases or the current times of controls from a kernel smoother. We show that the proposed estimators are consistent and asymptotically normal with variance that can be estimated empirically from the data. Simulation studies demonstrate that the proposed estimators perform well in finite sample sizes. Finally, we illustrate the method by an analysis of a case–control study of colorectal cancer. Supplementary materials for this article are available online.

Confidence intervals for intraclass correlation coefficients in agreement studies with continuous outcomes are model-specific and no generic approach exists. This paper provides two generic approaches for intraclass correlation coefficients of the form [Formula: see text]. The first approach uses Satterthwaite’s approximation and an F-distribution. The second approach uses the first and second moments of the intraclass correlation coefficient estimate in combination with a Beta distribution. Both approaches are based on the restricted maximum likelihood estimates for the variance components involved. Simulation studies are conducted to examine the coverage probabilities of the confidence intervals for agreement studies with a mix of small sample sizes. Two different three-way variance components models and balanced and unbalanced one-way random effects models are investigated. The proposed approaches are compared with other approaches developed for these specific models. The approach based on the F-distribution provides acceptable coverage probabilities, but the approach based on the Beta distribution results in accurate coverages for most settings in both balanced and unbalanced designs. A real agreement study is provided to illustrate the approaches.

Many medical (and ecological) processes involve the change of shape, whereby one trajectory changes into another trajectory at a specific time point. There has been little investigation into the study design needed to investigate these models. We consider the class of fixed effect change-point models with an underlying shape comprised two joined linear segments, also known as broken-stick models. We extend this model to include two sub-groups with different trajectories at the change-point, a change and no change class, and also include a missingness model to account for individuals with incomplete follow-up. Through a simulation study, we consider the relationship of sample size to the estimates of the underlying shape, the existence of a change-point, and the classification-error of sub-group labels. We use a Bayesian framework to account for the missing labels, and the analysis of each simulation is performed using standard Markov chain Monte Carlo techniques. Our simulation study is inspired by cognitive decline as measured by the Mini-Mental State Examination, where our extended model is appropriate due to the commonly observed mixture of individuals within studies who do or do not exhibit accelerated decline. We find that even for studies of modest size ( n = 500, with 50 individuals observed past the change-point) in the fixed effect setting, a change-point can be detected and reliably estimated across a range of observation-errors., This work was supported by the Medical Research Council (Unit Programme number U105292687).

Cross-sectional prevalent cohort design has drawn considerable interests in the studies of association between risk factors and time-to-event outcome. The sampling scheme in such design gives rise to length-biased data that require specialized analysis strategy but can improve study efficiency. The power and sample size calculation methods are however lacking for studies with prevalent cohort design, and using the formula developed for traditional survival data may overestimate sample size. We derive the sample size formulas that are appropriate for the design of cross-sectional prevalent cohort studies, under the assumptions of exponentially distributed event time and uniform follow-up for cross-sectional prevalent cohort design. We perform numerical and simulation studies to compare the sample size requirements for achieving the same power between prevalent cohort and incident cohort designs. We also use a large prospective prevalent cohort study to demonstrate the procedure. Using rigorous designs and proper analysis tools, the prospective prevalent cohort design can be more efficient than the incident cohort design with the same total sample sizes and study durations.

We derived sample size formulae for detecting main effects in group-based randomized clinical trials with different levels of data hierarchy between experimental and control arms. Such designs are necessary when experimental interventions need to be administered to groups of subjects whereas control conditions need to be administered to individual subjects. This type of trial, often referred to as a partially nested or partially clustered design, has been implemented for management of chronic diseases such as diabetes and is beginning to emerge more commonly in wider clinical settings. Depending on the research setting, the level of hierarchy of data structure for the experimental arm can be three or two, whereas that for the control arm is two or one. Such different levels of data hierarchy assume correlation structures of outcomes that are different between arms, regardless of whether research settings require two or three level data structure for the experimental arm. Therefore, the different correlations should be taken into account for statistical modeling and for sample size determinations. To this end, we considered mixed-effects linear models with different correlation structures between experimental and control arms to theoretically derive and empirically validate the sample size formulae with simulation studies.

Simon’s two-stage design has been widely used in early phase clinical trials to assess the activity of a new investigated treatment. In practice, the actual sample sizes do not always follow the study design precisely, especially in the second stage. When over- or under-enrollment occurs in a study, the original critical values for the study design are no longer valid for making proper statistical inference in a clinical trial. The hypothesis for such studies is always one-sided, and the null hypothesis is rejected when only a few responses are observed. Therefore, a one-sided lower interval is suitable to test the hypothesis. The commonly used approaches for confidence interval construction are based on asymptotic approaches. These approaches generally do not guarantee the coverage probability. For this reason, Clopper-Pearson approach can be used to compute exact confidence intervals. This approach has to be used in conjunction with a method to order the sample space. The frequently used method is based on point estimates for the response rate, but this ordering has too many ties which lead to conservativeness of the exact intervals. We propose developing exact one-sided intervals based on the p-value to order the sample space. The proposed approach outperforms the existing asymptotic and exact approaches. Therefore, it is recommended for use in practice.

We propose a semiparametric multi-state frailty model to analyze clustered event-history data subject to interval censoring. The proposed model is motivated by an attempt to study the life course of dental caries at the tooth level, taking into account the multiplicity of caries states and the intra-oral clustering of observations made at periodic time points. Of particular interest is the study of the intra-oral distribution of processes leading to carious lesions, and whether this distribution varies with gender. The model assumes, in view of the covariate profile, a proportionality of the transition intensities conditional on subject-level frailties, coupled with a linear spline approximation of the log baseline intensities. The model estimation is conducted using a penalized likelihood where the smoothing parameters are estimated as reciprocal variance components under a mixed-model representation. A Bayesian method is proposed to predict tooth-level caries transition probabilities, which can be used for tailoring tooth-level caries treatment and prevention plans. Intensive simulation studies indicate that the model fitting and prediction perform reasonably well under realistic sample sizes. The practical utility of the methods is illustrated using data from a longitudinal study on oral health among children from low-income families residing in the city of Detroit, Michigan.

Phase II clinical trials are concerned with making decision of whether a treatment is sufficiently efficacious to be worth further investigations in late large scale Phase III trials. In oncology Phase II trials, frequentist single-arm two-stage group-sequential designs with a binary endpoint are commonly used. To allow for more flexibility, adaptive versions of these designs have been proposed. In this paper, we propose point and interval estimation for adaptive designs in which the second stage sample size is a pre-specified function of first stage’s number of responses. Our approach is based on sample space orderings, from which we derivep-values, and point and interval estimates. Simulation studies show that our proposed methods perform better, in terms of bias and root mean square error, than the fixed-sample maximum likelihood estimator.

Sample size calculations are needed to design and assess the feasibility of case-control studies. Although such calculations are readily available for simple case-control designs and univariate analyses, there is limited theory and software for multivariate unconditional logistic analysis of case-control data. Here we outline the theory needed to detect scalar exposure effects or scalar interactions while controlling for other covariates in logistic regression. Both analytical and simulation methods are presented, together with links to the corresponding software.

Stepped wedge and cluster randomised crossover trials are examples of cluster randomised designs conducted over multiple time periods that are being used with increasing frequency in health research. Recent systematic reviews of both of these designs indicate that the within-cluster correlation is typically taken account of in the analysis of data using a random intercept mixed model, implying a constant correlation between any two individuals in the same cluster no matter how far apart in time they are measured: within-period and between-period intra-cluster correlations are assumed to be identical. Recently proposed extensions allow the within- and between-period intra-cluster correlations to differ, although these methods require that all between-period intra-cluster correlations are identical, which may not be appropriate in all situations. Motivated by a proposed intensive care cluster randomised trial, we propose an alternative correlation structure for repeated cross-sectional multiple-period cluster randomised trials in which the between-period intra-cluster correlation is allowed to decay depending on the distance between measurements. We present results for the variance of treatment effect estimators for varying amounts of decay, investigating the consequences of the variation in decay on sample size planning for stepped wedge, cluster crossover and multiple-period parallel-arm cluster randomised trials. We also investigate the impact of assuming constant between-period intra-cluster correlations instead of decaying between-period intra-cluster correlations. Our results indicate that in certain design configurations, including the one corresponding to the proposed trial, a correlation decay can have an important impact on variances of treatment effect estimators, and hence on sample size and power. An R Shiny app allows readers to interactively explore the impact of correlation decay.

Split-mouth designs are frequently used in dental clinical research, where a mouth is divided into two or more experimental segments that are randomly assigned to different treatments. It has the distinct advantage of removing a lot of inter-subject variability from the estimated treatment effect. Methods of statistical analyses for split-mouth design have been well developed. However, little work is available on sample size consideration at the design phase of a split-mouth trial, although many researchers pointed out that the split-mouth design can only be more efficient than a parallel-group design when within-subject correlation coefficient is substantial. In this paper, we propose to use the generalized estimating equation (GEE) approach to assess treatment effect in split-mouth trials, accounting for correlations among observations. Closed-form sample size formulas are introduced for the split-mouth design with continuous and binary outcomes, assuming exchangeable and “nested exchangeable” correlation structures for outcomes from the same subject. The statistical inference is based on the large sample approximation under the GEE approach. Simulation studies are conducted to investigate the finite-sample performance of the GEE sample size formulas. A dental clinical trial example is presented for illustration.

Objectives Assessing high-sensitivity tests for mortal illness is crucial in emergency and critical care medicine. Estimating the 95% confidence interval (CI) of the likelihood ratio (LR) can be challenging when sample sensitivity is 100%. We aimed to develop, compare, and automate a bootstrapping method to estimate the negative LR CI when sample sensitivity is 100%. Methods The lowest population sensitivity that is most likely to yield sample sensitivity 100% is located using the binomial distribution. Random binomial samples generated using this population sensitivity are then used in the LR bootstrap. A free R program, “bootLR,” automates the process. Extensive simulations were performed to determine how often the LR bootstrap and comparator method 95% CIs cover the true population negative LR value. Finally, the 95% CI was compared for theoretical sample sizes and sensitivities approaching and including 100% using: (1) a technique of individual extremes, (2) SAS software based on the technique of Gart and Nam, (3) the Score CI (as implemented in the StatXact, SAS, and R PropCI package), and (4) the bootstrapping technique. Results The bootstrapping approach demonstrates appropriate coverage of the nominal 95% CI over a spectrum of populations and sample sizes. Considering a study of sample size 200 with 100 patients with disease, and specificity 60%, the lowest population sensitivity with median sample sensitivity 100% is 99.31%. When all 100 patients with disease test positive, the negative LR 95% CIs are: individual extremes technique (0,0.073), StatXact (0,0.064), SAS Score method (0,0.057), R PropCI (0,0.062), and bootstrap (0,0.048). Similar trends were observed for other sample sizes. Conclusions When study samples demonstrate 100% sensitivity, available methods may yield inappropriately wide negative LR CIs. An alternative bootstrapping approach and accompanying free open-source R package were developed to yield realistic estimates easily. This methodology and implementation are applicable to other binomial proportions with homogeneous responses.

This paper deals with the optimal sample sizes for a multicentre trial in which the cost-effectiveness of two treatments in terms of net monetary benefit is studied. A bivariate random-effects model, with the treatment-by-centre interaction effect being random and the main effect of centres fixed or random, is assumed to describe both costs and effects. The optimal sample sizes concern the number of centres and the number of individuals per centre in each of the treatment conditions. These numbers maximize the efficiency or power for given research costs or minimize the research costs at a desired level of efficiency or power. Information on model parameters and sampling costs are required to calculate these optimal sample sizes. In case of limited information on relevant model parameters, sample size formulas are derived for so-called maximin sample sizes which guarantee a power level at the lowest study costs. Four different maximin sample sizes are derived based on the signs of the lower bounds of two model parameters, with one case being worst compared to others. We numerically evaluate the efficiency of the worst case instead of using others. Finally, an expression is derived for calculating optimal and maximin sample sizes that yield sufficient power to test the cost-effectiveness of two treatments.

Sample size calculation for cluster randomized trials (CRTs) with a [Formula: see text] factorial design is complicated due to the combination of nesting (of individuals within clusters) with crossing (of two treatments). Typically, clusters and individuals are allocated across treatment conditions in a balanced fashion, which is optimal under homogeneity of variance. However, the variance is likely to be heterogeneous if there is a treatment effect. An unbalanced allocation is then more efficient, but impractical because the optimal allocation depends on the unknown variances. Focusing on CRTs with a [Formula: see text] design, this paper addresses two questions: How much efficiency is lost by having a balanced design when the outcome variance is heterogeneous? How large must the sample size be for a balanced allocation to have sufficient power under heterogeneity of variance? We consider different scenarios of heterogeneous variance. Within each scenario, we determine the relative efficiency of a balanced design, as a function of the level (cluster, individual, both) and amount of heterogeneity of the variance. We then provide a simple correction of the sample size for the loss of power due to heterogeneity of variance when a balanced allocation is used. The theory is illustrated with an example of a published [Formula: see text] CRT.

Composite endpoints are frequently used in clinical outcome trials to provide more endpoints, thereby increasing statistical power. A key requirement for a composite endpoint to be meaningful is the absence of the so-called qualitative heterogeneity to ensure a valid overall interpretation of any treatment effect identified. Qualitative heterogeneity occurs when individual components of a composite endpoint exhibit differences in the direction of a treatment effect. In this paper, we develop a general statistical method to test for qualitative heterogeneity, that is to test whether a given set of parameters share the same sign. This method is based on the intersection–union principle and, provided that the sample size is large, is valid whatever the model used for parameters estimation. We propose two versions of our testing procedure, one based on a random sampling from a Gaussian distribution and another version based on bootstrapping. Our work covers both the case of completely observed data and the case where some observations are censored which is an important issue in many clinical trials. We evaluated the size and power of our proposed tests by carrying out some extensive Monte Carlo simulations in the case of multivariate time to event data. The simulations were designed under a variety of conditions on dimensionality, censoring rate, sample size and correlation structure. Our testing procedure showed very good performances in terms of statistical power and type I error. The proposed test was applied to a data set from a single-center, randomized, double-blind controlled trial in the area of Alzheimer’s disease.

This special issue on ‘Efficient design and sample size calculation for trials with clustered data’ is a post-hoc issue. Over the last years, six papers on this topic from our department have been accepted by Statistical Methods in Medical Research and have become available online. Some months ago, it occurred to us that they might be more easily found by combining them into a single issue. We are grateful to the editor, Professor Brian Everitt, for his positive response to this idea. We apologise to all statisticians who are, or once were, active in the same area as we, and whose work is not included into this issue for the reason given above: Anthony Atkinson, Valerii Fedorov, Stephen Raudenbush, Xiaofeng Liu, Allan Donner, Neil Klar, David Murray, Larry Hedges, Sonja McKinlay, Henry Feldman, Don Hedeker, Robert Gibbons, Scott Maxwell, John Overall, Weng Kee Wong, Andrew Forbes, Chul Ahn, Moonseong Heo, Kaifeng Lu, Richard Hayes, Lawrence Moulton, Michael Campbell, Chris and Stephen Roberts, Rebecca Turner, Donna Spiegelman, Xavier Basagana, Sandra Eldridge, Karla Hemming, Bruno Giraudeau, Peter Goos, Mirjam Moerbeek, Steven Teerenstra, Joop Hox, Cora Maas, Tom Snijders, Peter Van De Ven, and our own colleagues Hubert Schouten, Bjorn Winkens and Valeria Limapassos. There are many more and the whole editorial might perhaps be filled by their names alone. We are sorry not to be able to do so. Of the six papers in this issue, five are about cluster randomised and multicentre trials, and one is about longitudinal studies with a repeatedly measured binary outcome. We give a summary of each paper in turn. The first paper by Lemme et al. discusses the optimal treatment allocation in cluster randomised and multicentre trials with a two-by-two factorial design and a quantitative outcome, to be analysed with mixed linear regression, using as fixed part either the classical ANOVA model or Helmert contrasts. The optimality criterion is maximum precision of treatment effect estimation, as measured by the determinant of the covariance matrix of the fixed effect estimators (D-criterion), or its trace (A-criterion), or the variance of one specific fixed effect estimator. For each criterion and model, the authors derive the optimal allocation of the clusters to the four treatment arms in a cluster randomised trial and compare it with the popular balanced allocation. The theory is extended to the optimal allocation of individuals in a multicentre trial and is illustrated on a cluster randomised trial of smoking prevention and a multicentre trial on lifestyle.

Doubly censored data often arise in medical studies of disease progression involving two related events for which both an originating and a terminating event are interval-censored. Although regression modeling for such doubly censored data may be complicated, we propose a simple semiparametric regression modeling strategy based on jackknife pseudo-observations obtained using nonparametric estimators of the survival function. Inference is carried out via generalized estimating equations. Simulations studies show that the proposed method produces virtually unbiased covariate effect estimates, even for moderate sample sizes. A prostate cancer study example illustrates the practical advantages of the proposed approach.

Current methods of power and sample size calculations for the design of longitudinal studies to evaluate mediation effects are mostly based on simulation studies and do not provide closed-form formulae. A further challenge due to the longitudinal study design is the consideration of missing data, which almost always occur in longitudinal studies due to staggered entry or drop out. In this article, we consider the product of coefficients as a measure for the longitudinal mediation effect and evaluate three methods for testing the hypothesis on the longitudinal mediation effect: the joint significant test, the normal approximation and the test of b methods. Formulae for power and sample size calculations are provided under each method while taking into account missing data. Performance of the three methods under limited sample size are examined using simulation studies. An example from the Einstein aging study is provided to illustrate the methods.

Bland and Altman described approximate methods in 1986 and 1999 for calculating confidence limits for their 95% limits of agreement, approximations which assume large subject numbers. In this paper, these approximations are compared with exact confidence intervals calculated using two-sided tolerance intervals for a normal distribution. The approximations are compared in terms of the tolerance factors themselves but also in terms of the exact confidence limits and the exact limits of agreement coverage corresponding to the approximate confidence interval methods. Using similar methods the 50th percentile of the tolerance interval are compared with the k values of 1.96 and 2, which Bland and Altman used to define limits of agreements (i.e. [Formula: see text]+/− 1.96Sd and [Formula: see text]+/− 2Sd). For limits of agreement outer confidence intervals, Bland and Altman’s approximations are too permissive for sample sizes

Pilot studies are small‐scale studies conducted to gather information and provide a foundation for the design of a definitive trial. They do not seek to estimate treatment efficacy or effectiveness themselves but may be used to assess whether a definitive trial is feasible and how it can be carried out. The objectives of a study can be met only if a sufficient number of participants are recruited. A formal power calculation is not usually necessary for a pilot study where the focus is on estimation – for example, of feasibility parameters or the population variance – and the precision of these estimates. However, the sample size should still be justified as appropriate to meet the aims of the trial. The appropriate analysis of a pilot study would usually be a descriptive summary of the parameters of interest, addressing the original trial objectives. The analysis should not contain hypothesis testing for which the trial is not powered. The interpretation of the results should be in line with the feasibility of the planned main trial. The decision‐making process and changes made to the trial design based on the pilot study should be documented, and results presented in accordance with the CONSORT extension for pilot trials. Every effort should be made to publish the trial for transparency and to allow learning to be shared within the research community.

Stepped-wedge (SW) designs have been steadily implemented in a variety of trials. A SW design typically assumes a three-level hierarchical data structure where participants are nested within times or periods which are in turn nested within clusters. Therefore, statistical models for analysis of SW trial data need to consider two correlations, the first and second level correlations. Existing power functions and sample size determination formulas had been derived based on statistical models for two-level data structures. Consequently, the second-level correlation has not been incorporated in conventional power analyses. In this paper, we derived a closed-form explicit power function based on a statistical model for three-level continuous outcome data. The power function is based on a pooled overall estimate of stratified cluster-specific estimates of an intervention effect. The sampling distribution of the pooled estimate is derived by applying a fixed-effect meta-analytic approach. Simulation studies verified that the derived power function is unbiased and can be applicable to varying number of participants per period per cluster. In addition, when data structures are assumed to have two levels, we compare three types of power functions by conducting additional simulation studies under a two-level statistical model. In this case, the power function based on a sampling distribution of a marginal, as opposed to pooled, estimate of the intervention effect performed the best. Extensions of power functions to binary outcomes are also suggested.