Search

A statistical methodology named “capture recapture”, a Kaplan–Meier Summary Statistic, and an urn model framework are presented to describe the elicitation, then estimate both the number of interviews and the total number of items (“codes”) that will be elicited during patient interviews, and present a summary graphical statistic that “saturation” has occurred. This methodology is developed to address a gap in the FDA 2009 PRO and 2012 PFDD guidance for determining the number of interviews (sample size). This estimate of the number of interviews (sample size) uses a two-step procedure. The estimate of the total number of items is then used to estimate the number of interviews to elicit all items. A framework called an urn model is a framework for describing the elicitation and demonstrate the algorithm for declaring saturation “first interview with zero new codes”. A caveat emptor is that due to independence assumptions, the urn model is not used as a method for estimating probabilities. The URN model provides a framework to demonstrate that an algorithm such as “first interview with zero new codes” may establish that all codes have been elicited. The limitations of the Urn model, capture recapture, and Kaplan–Meier are summarized. The statistical methods and the estimates supplement but do not replace expert judgement and declaration of “saturation.” A graphical summary statistic is presented to summarize “saturation,” after expert declaration for two algorithms. An example of a capture-recapture estimate, using simulated data is provided. The example suggests that the estimate of total number of codes may be accurate when prepared as early as the second interview. A second simulation is presented with an URN model, under a strong assumption of independence that an algorithm such as ‘first interview with zero new codes” may fail to identify all codes. Potential errors in declaration of saturation are presented. Recommendations are presented for additional research and the use of the algorithm “first interview with zero new codes.” [ABSTRACT FROM AUTHOR]

Inter‐rater reliability (IRR) is one of the commonly used tools for assessing the quality of ratings from multiple raters. However, applicant selection procedures based on ratings from multiple raters usually result in a binary outcome; the applicant is either selected or not. This final outcome is not considered in IRR, which instead focuses on the ratings of the individual subjects or objects. We outline the connection between the ratings' measurement model (used for IRR) and a binary classification framework. We develop a simple way of approximating the probability of correctly selecting the best applicants which allows us to compute error probabilities of the selection procedure (i.e., false positive and false negative rate) or their lower bounds. We draw connections between the IRR and the binary classification metrics, showing that binary classification metrics depend solely on the IRR coefficient and proportion of selected applicants. We assess the performance of the approximation in a simulation study and apply it in an example comparing the reliability of multiple grant peer review selection procedures. We also discuss other possible uses of the explored connections in other contexts, such as educational testing, psychological assessment, and health‐related measurement, and implement the computations in the R package IRR2FPR. [ABSTRACT FROM AUTHOR]

Linear mixed-effects models (LMEMs) are widely used in medical, engineering, and social applications. The accurate specification of the covariance matrix structure within the error term is known to impact the estimation and inference procedures. Thus, it is crucial to detect the source of errors in LMEMs specifications. In this study, we propose combining a user-friendly computational test with an analytical method to visualize the source of errors. Through statistical simulations under different scenarios, we evaluate the performance of the proposed test in terms of the Power and Type I error rate. Our findings indicate that as the sample size n increases, the proposed test effectively detects misspecification in the systematic component, the number of random effects, the within-subject covariance structure, and the covariance structure of the error term in the LMEM with high Power while maintaining the nominal Type I error rate. Finally, we show the practical usefulness of our proposed test with a real-world application. [ABSTRACT FROM AUTHOR]

Background: Outcome measures that are count variables with excessive zeros are common in health behaviors research. Examples include the number of standard drinks consumed or alcohol‐related problems experienced over time. There is a lack of empirical data about the relative performance of prevailing statistical models for assessing the efficacy of interventions when outcomes are zero‐inflated, particularly compared with recently developed marginalized count regression approaches for such data. Methods: The current simulation study examined five commonly used approaches for analyzing count outcomes, including two linear models (with outcomes on raw and log‐transformed scales, respectively) and three prevailing count distribution‐based models (ie, Poisson, negative binomial, and zero‐inflated Poisson (ZIP) models). We also considered the marginalized zero‐inflated Poisson (MZIP) model, a novel alternative that estimates the overall effects on the population mean while adjusting for zero‐inflation. Motivated by alcohol misuse prevention trials, extensive simulations were conducted to evaluate and compare the statistical power and Type I error rate of the statistical models and approaches across data conditions that varied in sample size (N=100$$ N=100 $$ to 500), zero rate (0.2 to 0.8), and intervention effect sizes. Results: Under zero‐inflation, the Poisson model failed to control the Type I error rate, resulting in higher than expected false positive results. When the intervention effects on the zero (vs. non‐zero) and count parts were in the same direction, the MZIP model had the highest statistical power, followed by the linear model with outcomes on the raw scale, negative binomial model, and ZIP model. The performance of the linear model with a log‐transformed outcome variable was unsatisfactory. Conclusions: The MZIP model demonstrated better statistical properties in detecting true intervention effects and controlling false positive results for zero‐inflated count outcomes. This MZIP model may serve as an appealing analytical approach to evaluating overall intervention effects in studies with count outcomes marked by excessive zeros. [ABSTRACT FROM AUTHOR]

In the linear regression model, the multicollinearity problem arises when there is a linear relationship between independent variables. This situation causes the variance of the estimations of the model parameters obtained by the Least Squares Estimator method to increase and move away from the true value, resulting in unstable and incorrect results. Biased Estimator methods are developed to eliminate the adverse effects caused by multicollinearity. In this study, a test statistic is obtained to test the significance of the model coefficients for the Liu-Type Estimator using the test statistic method suggested in the study of Halawa and El-Bassiouni (2000). With a simulation study, the significance of the model coefficients of the Ridge, Liu, and Liu type biased estimators in different situations is tested; the type I errors and power values of the estimators are calculated; the results are compared. In addition, a real data application is performed to better understand the test procedure. [ABSTRACT FROM AUTHOR]

Randomization procedures that enforce balance in prognostic factors, most commonly stratified randomization, are often employed in clinical trials. When the number of factors or factor levels is large, dynamic allocation procedures, such as the Pocock and Simon's covariate-adaptive randomization (minimization) are preferred. In their ground-breaking work Ye and Shao identified two classes of covariate-adaptive randomization procedures. They have demonstrated theoretically that for these classes, when the model is misspecified, the robust score test (Lin and Wei) as well as the unstratified log-rank test used for analysis of time-to-event endpoints, are valid or conservative (Ye and Shao). This fact, however, was not established for minimization other than through simulations of survival endpoints. In this article, we point out that the results of Ye and Shao can be expanded to a more general class of randomization procedures. We show, in part theoretically, in part through simulations of the within-strata imbalances, that minimization belongs to this class. Along the way we describe the asymptotic correlation matrix of the normalized within-stratum imbalances following minimization with equal prevalence of all strata. We expand the robust tests proposed by Ye and Shao for stratified randomization to minimization and examine their performance through simulations. [ABSTRACT FROM AUTHOR]

Recently, Asparouhov and Muthén Structural Equation Modeling: A Multidisciplinary Journal, 28, 1–14, (2021a, 2021b) proposed a variant of the Wald test that uses Markov chain Monte Carlo machinery to generate a chi-square test statistic for frequentist inference. Because the test's composition does not rely on analytic expressions for sampling variation and covariation, it potentially provides a way to get honest significance tests in cases where the likelihood-based test statistic's assumptions break down (e.g., in small samples). The goal of this study is to use simulation to compare the new MCM Wald test to its maximum likelihood counterparts, with respect to both their type I error rate and power. Our simulation examined the test statistics across different levels of sample size, effect size, and degrees of freedom (test complexity). An additional goal was to assess the robustness of the MCMC Wald test with nonnormal data. The simulation results uniformly demonstrated that the MCMC Wald test was superior to the maximum likelihood test statistic, especially with small samples (e.g., sample sizes less than 150) and complex models (e.g., models with five or more predictors). This conclusion held for nonnormal data as well. Lastly, we provide a brief application to a real data example. [ABSTRACT FROM AUTHOR]

Exclusive hypothesis testing is a new and special class of hypothesis testing. This kind of testing can be applied in survival analysis to understand the association between genomics information and clinical information about the survival time. Besides, it is well known that Cox's proportional hazards model is the most commonly used model for regression analysis of failure time. In this paper, the authors consider doing the exclusive hypothesis testing for Cox's proportional hazards model with right-censored data. The authors propose the comprehensive test statistics to make decision, and show that the corresponding decision rule can control the asymptotic Type I errors and have good powers in theory. The numerical studies indicate that the proposed approach works well for practical situations and it is applied to a set of real data arising from Rotterdam Breast Cancer Data study that motivated this study. [ABSTRACT FROM AUTHOR]

Linear mixed-effects models (LMEMs) are widely used in medical, engineering, and social applications. The accurate specification of the covariance matrix structure within the error term is known to impact the estimation and inference procedures. Thus, it is crucial to detect the source of errors in LMEMs specifications. In this study, we propose combining a user-friendly computational test with an analytical method to visualize the source of errors. Through statistical simulations under different scenarios, we evaluate the performance of the proposed test in terms of the Power and Type I error rate. Our findings indicate that as the sample size n increases, the proposed test effectively detects misspecification in the systematic component, the number of random effects, the within-subject covariance structure, and the covariance structure of the error term in the LMEM with high Power while maintaining the nominal Type I error rate. Finally, we show the practical usefulness of our proposed test with a real-world application.

A necessary step in applying bi-factor models is to evaluate the need for domain factors with a general factor in place. The conventional null hypothesis testing (NHT) was commonly used for such a purpose. However, the conventional NHT meets challenges when the domain loadings are weak or the sample size is insufficient. This article proposes using minimal-effect testing (MET) and equivalence testing (ET) to analyze bi-factor models. A key element in conducting MET and ET is the minimal size of factor loadings that can be regarded as noteworthy in practice, termed as minimal noteworthy size. This article presents two approaches to formulating the minimal noteworthy size and compares the pros and cons of MET, ET, and the conventional NHT. Analysis shows that MET, ET, and the conventional NHT are complementary. Combining them to test the noteworthiness of domain loadings can help researchers make a comprehensive judgment. Real and simulated datasets illustrate the applications of the three methods. Monte Carlo results show that MET and ET can control type I errors reasonably well while maintaining good statistical power. [ABSTRACT FROM AUTHOR]

AbstractThe ratio of two Poisson means is commonly used in biological, epidemiological, and medical. In this article, we consider the problem of testing the ratio of two Poisson means and propose a valid and efficient test based on the inference model (IM). The simulation studies show that the type I error and power of the IM-based test are competitive or even better than the six recommended tests: The likelihood ratio, mid-p, logarithmic transformation, and three tests based on the method of variance estimates recovery (MOVER). For R<1, the Rao-MOVER and fiducial-MOVER tests are slightly liberal, the mid-p test is somewhat conservative. For R>1, the Rao-MOVER and fiducial-MOVER tests are slightly conservative. The likelihood ratio and logarithmic transformation tests cannot control the type I error well. Therefore, the IM test could be recommended for practical applications. A real numerical example is presented to illustrate the flexibility of the proposed test. [ABSTRACT FROM AUTHOR]

AbstractIn clinical trials for chronic heart failure (CHF), time to a composite of first hospitalization for worsening heart failure or death is a widely accepted primary efficacy measure. Motivated by lower event rates in recent CHF trials, there are proposals to use endpoints accounting for recurrent heart failure hospitalization (HFH) events (e.g., recurrent HFH events plus death as an additional event) as the primary endpoint to better quantify disease burden and to improve trial efficiency. However, analysis and interpretation of recurrent event endpoints may be complicated by the terminal event death. In a previous simulation study it was shown that the Type I error rate was well-controlled for the analysis of the composite recurrent event endpoint using the negative binomial (NB) model and the Lin-Wei-Yang-Ying (LWYY) model. As these results were based on limited scenarios, we extended the previous simulation study and identified situations where the Type I error rate is no longer controlled for the analysis of the composite recurrent event endpoint using the LWYY model, despite a neutral effect on HFH and a detrimental effect on death of the experimental treatment. Therefore, depending on clinical settings, the LWYY model should be applied and interpreted with more caution. [ABSTRACT FROM AUTHOR]

Platform trials offer a framework to study multiple interventions in one trial with the opportunity of opening and closing arms. The use of common controls can increase efficiency as compared to individual controls. The need for multiplicity adjustment because of common controls is currently a debate among researchers, pharmaceutical companies, and regulators. The impact of common controls on the type one error in a fixed platform trial, i.e. when all treatments start and end recruitment at the same time, has been discussed in the literature before. We complement these findings by investigating the impact of a common control on the type one error and power in a flexible platform trial, i.e. when one arm joins the platform later. We derived the correlation of test statistics to assess the impact of the overlap and compared the results to a trial with individual controls. Furthermore, we evaluate the power, and the impact of multiplicity adjustment on the power in fixed and flexible platform trials. These methodological considerations are complemented by a regulatory guideline review. With multiple arms, the FWER is inflated when no multiplicity adjustment is applied. However, the FWER inflation is smaller with common controls than with individual controls. Even after multiplicity adjustment, a trial with common controls is often beneficial in terms of sample size and power. However, in some cases, the trial with common controls loses the efficiency gain and it might be advisable to run a separate trial rather than joining a platform trial. [ABSTRACT FROM AUTHOR]

AbstractIn this article, we propose equivalence tests for the difference of two survival functions in the presence of cure fraction. We consider both the non mixture cure model (NMCM) and mixture cure model (MCM). We first demonstrate that the existing tests can be used to test equivalence of two improper survival functions under the class of log transformation (LT) and Box-Cox transformation (BCT) NMCM. For the MCM, a logistic regression model is assumed for the probability of being uncured and the class of LT or BCT models are used to describe the distribution of event time for the uncured subjects. Under the MCM, we consider testing separately the null hypothesis of identical cure rate and that of equivalence of two survival functions of the uncured subjects. We also develop a Wald statistic for simultaneously testing whether or not both the maximal difference between cure rates and that between two short-term survival functions fall within some equivalence margins. Simulation results indicate that the proposed tests have a satisfactory size and an adequate power in finite sample. [ABSTRACT FROM AUTHOR]

For interrupted time series with small samples and outliers, the standard least squares regression for estimating intervention effects can be biased when the assumption of normality is violated. This article proposes a rank-based regression method combined with bootstrapping to address outliers and non-normality. The proposed method minimizes the Jaeckel dispersion function based on ranked residuals, offering robustness against outliers, skewed error distributions, and correlated errors. Simulation studies and a real-world example compare the performance of the proposed method with the least squares method under various conditions. Both produce unbiased estimates for the intervention effects under normality. However, with non-normal errors, outliers, and correlated errors, the rank-based bootstrapping method outperforms least squares producing narrower confidence intervals, higher efficiency, lower type I error rates, and greater power. Application to the Istanbul declaration on illegal organ activities confirms the improved robustness and precision of the rank-based method with bootstrapping over least squares. [ABSTRACT FROM AUTHOR]

AbstractClinical trials are an essential component of the drug development process, providing crucial data on the efficacy and safety of new treatments. However, traditional clinical trial designs can be inefficient and ineffective, leading to increased costs and a higher risk of failure. The FDA Oncology Center of Excellence (OCE) has recently initiated Project Optimus to reform the dose selection paradigm for oncology treatments. We propose the adaptive seamless phase II/III clinical trial designs (ASD) with the sequential estimation-adjusted urn (SEU) model for randomization to achieve efficient and ethical objectives. However, the combination of ASD and SEU poses a challenge in controlling the type I error rate: ASD exerts a dual influence of multiplicity and selection; all the responses and treatment assignments are not independent due to SEU. In this paper, we investigated how to overcome these difficulties, utilize the two adaptive approaches’ advantages, and control the type I error rate. We provide a theoretical foundation for this procedure, and numerical studies demonstrate that our methods can assign more people to better treatments, leading to fewer failures while still controlling the type I error rate and preserving power. [ABSTRACT FROM AUTHOR]

In research, outcomes are often categorized as primary and secondary. The primary outcome is the most important one; it determines whether the study is considered ‘successful’ or not. Secondary outcomes are chosen because they provide supporting evidence for the results of the primary outcome or additional information about the subject being studied. For reasons that are explained in this paper, secondary outcomes should be cautiously interpreted. There are varying practices regarding publishing secondary outcomes. Some authors publish these separately, while others include them in the main publication. In some contexts, the former can lead to concerns about the quality and relevance of the data being published. In this article, we discuss primary and secondary outcomes, the importance and interpretation of secondary outcomes, and considerations for publishing multiple outcomes in separate papers. We also discuss the special case of secondary analyses and post hoc analyses and provide guidance on good publishing practices. Throughout the article, we use relevant examples to make these concepts easier to understand. While the article is primarily aimed at early career researchers, it offers insights that may be helpful to researchers, reviewers, and editors across all levels of expertise.

Background Variables such as dietary intake are measured with error yet frequently used in observational epidemiology. Although this limitation is sometimes noted, these variables are still often modeled as covariates without formal correction or sincere dialogue about measurement unreliability potentially weakening the validity of statistical conclusions. Further, larger sample sizes increase power (bias) to detect spurious correlations. Counterintuitively, recent work suggested a non-monotonic relationship between confounder unreliability and how much controlling for the confounder reduces (or induces) bias when testing for an exposure-outcome association. If true, such non-monotonicity would be especially concerning for applications such as nutrition, where measurement reliability varies substantially, and large sample sizes are common. Methods We offer a detailed derivations of the square partial correlation between the outcome and exposure, controlling for the confounder. In our derivation, the measurement reliabilities of exposures and confounders are not arbitrarily constrained to be equal. Further, our theoretical results are investigated using simulations. Results Reassuringly, these derivations and simulations show that the counterintuitive non-monotonicity relationship between confounder unreliability and how much controlling for the confounder reduces (or induces) bias when testing for an exposure-outcome association is an artifact of the arbitrary constraint which forces the measurement reliabilities of exposures and confounders to be equal, which that does not always hold. Conclusions The profound and manifold effects of measurement error on estimation and statistical conclusion validity in realistic scenarios indicate that merely mentioning measurement error as a limitation and then dispensing with it is not an adequate response. We also explore questions for optimal study design subject to resource constraints when considering reliability of exposures, covariates, and outcomes.

The analysis of multiple bivariate correlations is often carried out by conducting simple tests to check whether each of them is significantly different from zero. In addition, pairwise differences are often judged by eye or by comparing the p‐values of the individual tests of significance despite the existence of statistical tests for differences between correlations. This paper uses simulation methods to assess the accuracy (empirical Type I error rate), power, and robustness of 10 tests designed to check the significance of the difference between two dependent correlations with overlapping variables (i.e., the correlation between X1 and Y and the correlation between X2 and Y). Five of the tests turned out to be inadvisable because their empirical Type I error rates under normality differ greatly from the nominal alpha level of .05 either across the board or within certain sub‐ranges of the parameter space. The remaining five tests were acceptable and their merits were similar in terms of all comparison criteria, although none of them was robust across all forms of non‐normality explored in the study. Practical recommendations are given for the choice of a statistical test to compare dependent correlations with overlapping variables. [ABSTRACT FROM AUTHOR]

In research, outcomes are often categorized as primary and secondary. The primary outcome is the most important one; it determines whether the study is considered 'successful' or not. Secondary outcomes are chosen because they provide supporting evidence for the results of the primary outcome or additional information about the subject being studied. For reasons that are explained in this paper, secondary outcomes should be cautiously interpreted. There are varying practices regarding publishing secondary outcomes. Some authors publish these separately, while others include them in the main publication. In some contexts, the former can lead to concerns about the quality and relevance of the data being published. In this article, we discuss primary and secondary outcomes, the importance and interpretation of secondary outcomes, and considerations for publishing multiple outcomes in separate papers. We also discuss the special case of secondary analyses and post hoc analyses and provide guidance on good publishing practices. Throughout the article, we use relevant examples to make these concepts easier to understand. While the article is primarily aimed at early career researchers, it offers insights that may be helpful to researchers, reviewers, and editors across all levels of expertise. [ABSTRACT FROM AUTHOR]

In many scientific experiments, multiarmed bandits are used as an adaptive data collection method. However, this adaptive process can lead to a dependence that renders many commonly used statistical inference methods invalid. An example of this is the sample mean, which is a natural estimator of the mean parameter but can be biased. This can cause test statistics based on this estimator to have an inflated type I error rate, and the resulting confidence intervals may have significantly lower coverage probabilities than their nominal values. To address this issue, we propose an alternative approach called randomized multiarm bandits (rMAB). This combines a randomization step with a chosen MAB algorithm, and by selecting the randomization probability appropriately, optimal regret can be achieved asymptotically. Numerical evidence shows that the bias of the sample mean based on the rMAB is much smaller than that of other methods. The test statistic and confidence interval produced by this method also perform much better than its competitors. [ABSTRACT FROM AUTHOR]

Due to highly unmet medical needs in rare disease areas, there is great desire to speed up the drug development process. A two-stage adaptive design option for a placebo-controlled registration study is being evaluated. Stage 1 consists of participants in an ongoing phase 2 study with 1-year double blinded (DB) treatment and Stage 2 includes newly enrolled participants with 2-year DB treatment period. The primary endpoint is the annualized rate of change (slope) for a continuous longitudinal measurement, which will be evaluated through a random coefficient linear mixed model. An unblinded interim analysis will be performed using Stage 1 data to re-estimate the sample size for Stage 2, followed by another interim analysis for potential early efficacy stopping when all participants completed the 1-year DB treatment. To control the overall Type 1 error rate, rather than using a conservative approach, the actual correlation between the interim and final test statistics will be taken into account to determine the final significance level given the pre-specified significance level for the interim efficacy analysis. Multiplicity adjustments for secondary endpoints including considerations for order switching between interim and final analysis in this specific case study will also be discussed. [ABSTRACT FROM AUTHOR]

Careful consideration of the tradeoff between Type I and Type II error rates when designing experiments is critical for maximizing statistical decision accuracy. Typically, Type I error rates (e.g., .05) are significantly lower than Type II error rates (e.g., .20 for .80 power) in psychological science. Further, positive findings (true effects and Type I errors) are more likely to be the focus of replication. This conventional approach leads to very high rates of Type II error. Analyses show that increasing the Type I error rate to .10, thereby increasing power and decreasing the Type II error rate for each test, leads to higher overall rates of correct statistical decisions. This increase of Type I error rate is consistent with, and most beneficial in the context of, the replication and "New Statistics" movements in psychology. Making decisions under conditions of uncertainty (e.g., science) always involves a tradeoff between two types of errors. The decision maker can conclude something is present when it is not (false positive or Type I error) or conclude no difference is present, when in reality there is a difference (a miss or Type II error). We address the need to carefully consider the balance of these error types in scientific decision making. In psychology, it is typical to more stringently control the Type I error rate at .05, while the Type II error rate is .20 (i.e., when power is .80). We demonstrate that the overall percentage correct decisions in psychological science can be increased by increasing the Type I error rate to .10 to increase power, and decrease the Type II error rate. This is particularly relevant given the recent methodological and statistical changes in the field, particularly the long-needed emphasis on replication. [ABSTRACT FROM AUTHOR]

Intraclass correlation in bilateral data has been investigated in recent decades with various statistical methods. In practice, stratifying bilateral data by some control variables will provide more sophisticated statistical results to satisfy different research proposed in randomized clinical trials. In this article, we propose three test statistics (the likelihood ratio test, score test, and Wald-type test statistics) to evaluate the homogeneity of proportion ratios for stratified bilateral correlated data under an equal correlation assumption. Monte Carlo simulations of Type I error and power are performed, and the score test yields a robust outcome based on empirical Type I error and power. Lastly, two real data examples are conducted to illustrate the proposed three tests. [ABSTRACT FROM AUTHOR]

Classical tests for a difference in means control the Type I error rate when the groups are defined a priori. However, when the groups are instead defined via clustering, then applying a classical test yields an extremely inflated Type I error rate. Notably, this problem persists even if two separate and independent datasets are used to define the groups and to test for a difference in their means. To address this problem, in this article, we propose a selective inference approach to test for a difference in means between two clusters. Our procedure controls the selective Type I error rate by accounting for the fact that the choice of null hypothesis was made based on the data. We describe how to efficiently compute exact p-values for clusters obtained using agglomerative hierarchical clustering with many commonly used linkages. We apply our method to simulated data and to single-cell RNA-sequencing data. for this article are available online. [ABSTRACT FROM AUTHOR]

When comparing the performance of two or more competing tests, simulation studies commonly focus on statistical power. However, if the size of the tests being compared are either different from one another or from the nominal size, comparing tests based on power alone may be misleading. By analogy with diagnostic accuracy studies, we introduce relative positive and negative likelihood ratios to factor in both power and size in the comparison of multiple tests. We derive sample size formulas for a comparative simulation study. As an example, we compared the performance of six statistical tests for small‐study effects in meta‐analyses of randomized controlled trials: Begg's rank correlation, Egger's regression, Schwarzer's method for sparse data, the trim‐and‐fill method, the arcsine‐Thompson test, and Lin and Chu's combined test. We illustrate that comparing power alone, or power adjusted or penalized for size, can be misleading, and how the proposed likelihood ratio approach enables accurate comparison of the trade‐off between power and size between competing tests. [ABSTRACT FROM AUTHOR]

The assumption of homoscedasticity is one of the most crucial assumptions for many parametric tests used in the biological sciences. The aim of this paper is to compare the empirical probability of type I error and the power of ten parametric and two non-parametric tests for homoscedasticity with simulations under different types of distributions, number of groups, number of samples per group, variance ratio and significance levels, as well as through empirical data from an agricultural experiment. According to the findings of the simulation study, when there is no violation of the assumption of normality and the groups have equal variances and equal number of samples, the Bhandary-Dai, Cochran’s C, Hartley’s Fmax, Levene (trimmed mean) and Bartlett tests are considered robust. The Levene (absolute and square deviations) tests show a high probability of type I error in a small number of samples, which increases as the number of groups rises. When data groups display a nonnormal distribution, researchers should utilize the Levene (trimmed mean), O’Brien and Brown-Forsythe tests. On the other hand, if the assumption of normality is not violated but diagnostic plots indicate unequal variances between groups, researchers are advised to use the Bartlett, Z-variance, Bhandary-Dai and Levene (trimmed mean) tests. Assessing the tests being considered, the test that stands out as the most well-rounded choice is the Levene’s test (trimmed mean), which provides satisfactory type I error control and relatively high power. According to the findings of the study and for the scenarios considered, the two non-parametric tests are not recommended. In conclusion, it is suggested to initially check for normality and consider the number of samples per group before choosing the most appropriate test for homoscedasticity. [ABSTRACT FROM AUTHOR]

Sample size estimation remains as a cornerstone in the meticulous planning and execution of clinical trials, pivotal for ensuring studies possess the requisite statistical power to discern meaningful treatment effects. Insufficient sample sizes compromise the robustness of findings, whereas excessively large samples inflate costs and compromise data integrity. This article meticulously explains the multifaceted factors that outline sample size determination, encompassing various factors such as research design, types of hypotheses, error thresholds, effect size considerations, validity and precision. It investigates into the scope of methodologies available for sample size computation, spanning from intricate statistical formulas to pragmatic tabular approaches. Moreover, it underscores the significance of post-hoc power analysis in retrospectively evaluating completed studies, shedding light on their statistical robustness. This literature review furnishes a nuanced understanding of the sample size estimation landscape in clinical trials, delineating their strengths, limitations, and real-world applications. Anticipating participant attrition assumes paramount importance for proactively adjusting sample sizes, ensuring studies remain methodologically sound. Equipped with a profound grasp of these principles, researchers are empowered to conduct scientifically rigorous and impactful clinical trials, furnishing compelling evidence to inform judicious decision-making in healthcare interventions.

Intraclass correlation in bilateral data has been investigated in recent decades with various statistical methods. In practice, stratifying bilateral data by some control variables will provide more sophisticated statistical results to satisfy different research proposed in randomized clinical trials. In this article, we propose three test statistics (the likelihood ratio test, score test, and Wald-type test statistics) to evaluate the homogeneity of proportion ratios for stratified bilateral correlated data under an equal correlation assumption. Monte Carlo simulations of Type I error and power are performed, and the score test yields a robust outcome based on empirical Type I error and power. Lastly, two real data examples are conducted to illustrate the proposed three tests.

Monitoring the conduct of phase III randomized controlled trials is driven by ethical reasons to protect the study integrity and the safety of trial participants. We propose a group sequential, pragmatic approach for monitoring the accumulating efficacy information in randomized controlled trials. The "Population Health Research Institute boundary" is simple to implement and sensible, as it considers the reduction in uncertainty with increasing information as the study progresses. It is also pragmatic, since it takes into consideration the typical monitoring behavior of monitoring committees of large multicenter trials and is relatively easily implemented. It not only controls the overall Lan-DeMets type I error probability (alpha) spent, but performs better than other group sequential boundaries for the total nominal study alpha. We illustrate the use of our monitoring approach in the early termination of two past completed trials., Competing Interests: Declaration of conflicting interestsThe author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Heating Ventilation and Air Conditioning (HVAC) systems are omnipresent in modern-day industrial plants, office spaces, and residential complexes. Chillers and Air Handling Units (AHU) are the critical HVAC sub-systems that perform vital operations and consume maximum energy. Any fault in these sub-systems can thus adversely affect any organization's operations, making fault detection pivotal for effective maintenance. This study proposes a machine learning-based chiller and AHU fault detection framework that aims to reduce the risk of Type I Error and detect the fault severity. We first discuss detecting specific chiller and AHU faults using the Neyman-Pearson principle, controlling the Type I Error rate under 5%, with high probability. We then propose an ensemble-based feature selection method, which yields a set of essential features that can accurately estimate the severity level of specific faults. Further, Random Forests and k -Nearest Neighbors-based classifiers are employed to detect fault severity. We evaluate the performance of the proposed framework on two benchmark data sets, ASHRAE-RP-1043(Chiller) and ASHRAE-RP-1312(AHU), respectively, which show promising results. [ABSTRACT FROM AUTHOR]

• Hierarchical bootstrap as alternative to parametric Type I/II error simulations. • Identify inflated Type I & II errors of two most common approaches to RT analyses. • Identify substantial over-optimism of RT analyses in the literature. • Identify issues with past parametric validations of analysis approaches. • Provide recommendations for new standards in RT analyses. We investigate the statistical power and Type I error rate of the two most common approaches to reading time (RT) analyses: assuming normality of residuals and homogeneity of variance in raw or log-transformed RTs. We first show that the assumptions of such analyses—such as t -tests, ANOVAs, and linear mixed-effects models—are neither consistently met by raw RTs, nor by log-transformed RTs (or any other common power transforms, incl. inverse-transformed RTs). Only a non-power transform (log-shift) provides a decent fit for all data sets and data preparation steps we consider. We then compare the statistical power and Type I error rate for linear mixed-effects models over raw or log-transformed RTs. Previous studies on this matter relied on parametrically generated data. We show why this is problematic, and introduce as an alternative a hierarchical bootstrap approach over naturally distributed reading times. This approach yields substantially different—and arguably more informative—results than the parametric simulation approaches we compare it to. Our results suggests that it is time to heed the advice others have provided for reading research: for any but the simplest designs, we find both the rate of spurious significances and the rate of undetected true effects can strongly depend on the scale (e.g., raw or log-RTs) in which effects are assumed to be linear. Researchers should thus clearly motivate the choice of analysis based on theoretical grounds, assess the robustness of findings under different analysis approaches, and discuss potential mismatches between analyses. The R scripts and libraries shared in the accompanying OSF repo allow researchers to assess the reliability of their analyses via hierarchical bootstrap over their own data. [ABSTRACT FROM AUTHOR]

Background: Randomized controlled trials have been the gold standard for evaluating medical treatments for many decades but they are often criticized for requiring large sample sizes. Given the urgent need for better therapies for glioblastoma, it has been argued that data collected from patients treated with the standard regimen can provide high-quality external control data to supplement or replace concurrent control arm in future glioblastoma trials., Methods: In this article, we provide an in-depth appraisal of the use of external control data in the context of neuro-oncology trials. We describe several clinical trial designs with particular attention to how external information is utilized and address common fallacies that may lead to inappropriate adoptions of external control data., Results: Using 2 completed glioblastoma trials, we illustrate the use of an assessment tool that lays out a blueprint for assembling a high-quality external control data set. Using statistical simulations, we draw caution from scenarios where these approaches can fall short on controlling the type I error rate., Conclusions: While this approach may hold promise in generating informative data in certain settings, this sense of optimism should be tampered with a healthy dose of skepticism due to a myriad of design and analysis challenges articulated in this review. Importantly, careful planning is key to its successful implementation., (© The Author(s) 2024. Published by Oxford University Press on behalf of the Society for Neuro-Oncology. All rights reserved. For commercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com.)