Search

A statistical methodology named “capture recapture”, a Kaplan–Meier Summary Statistic, and an urn model framework are presented to describe the elicitation, then estimate both the number of interviews and the total number of items (“codes”) that will be elicited during patient interviews, and present a summary graphical statistic that “saturation” has occurred. This methodology is developed to address a gap in the FDA 2009 PRO and 2012 PFDD guidance for determining the number of interviews (sample size). This estimate of the number of interviews (sample size) uses a two-step procedure. The estimate of the total number of items is then used to estimate the number of interviews to elicit all items. A framework called an urn model is a framework for describing the elicitation and demonstrate the algorithm for declaring saturation “first interview with zero new codes”. A caveat emptor is that due to independence assumptions, the urn model is not used as a method for estimating probabilities. The URN model provides a framework to demonstrate that an algorithm such as “first interview with zero new codes” may establish that all codes have been elicited. The limitations of the Urn model, capture recapture, and Kaplan–Meier are summarized. The statistical methods and the estimates supplement but do not replace expert judgement and declaration of “saturation.” A graphical summary statistic is presented to summarize “saturation,” after expert declaration for two algorithms. An example of a capture-recapture estimate, using simulated data is provided. The example suggests that the estimate of total number of codes may be accurate when prepared as early as the second interview. A second simulation is presented with an URN model, under a strong assumption of independence that an algorithm such as ‘first interview with zero new codes” may fail to identify all codes. Potential errors in declaration of saturation are presented. Recommendations are presented for additional research and the use of the algorithm “first interview with zero new codes.” [ABSTRACT FROM AUTHOR]

Inter‐rater reliability (IRR) is one of the commonly used tools for assessing the quality of ratings from multiple raters. However, applicant selection procedures based on ratings from multiple raters usually result in a binary outcome; the applicant is either selected or not. This final outcome is not considered in IRR, which instead focuses on the ratings of the individual subjects or objects. We outline the connection between the ratings' measurement model (used for IRR) and a binary classification framework. We develop a simple way of approximating the probability of correctly selecting the best applicants which allows us to compute error probabilities of the selection procedure (i.e., false positive and false negative rate) or their lower bounds. We draw connections between the IRR and the binary classification metrics, showing that binary classification metrics depend solely on the IRR coefficient and proportion of selected applicants. We assess the performance of the approximation in a simulation study and apply it in an example comparing the reliability of multiple grant peer review selection procedures. We also discuss other possible uses of the explored connections in other contexts, such as educational testing, psychological assessment, and health‐related measurement, and implement the computations in the R package IRR2FPR. [ABSTRACT FROM AUTHOR]

Linear mixed-effects models (LMEMs) are widely used in medical, engineering, and social applications. The accurate specification of the covariance matrix structure within the error term is known to impact the estimation and inference procedures. Thus, it is crucial to detect the source of errors in LMEMs specifications. In this study, we propose combining a user-friendly computational test with an analytical method to visualize the source of errors. Through statistical simulations under different scenarios, we evaluate the performance of the proposed test in terms of the Power and Type I error rate. Our findings indicate that as the sample size n increases, the proposed test effectively detects misspecification in the systematic component, the number of random effects, the within-subject covariance structure, and the covariance structure of the error term in the LMEM with high Power while maintaining the nominal Type I error rate. Finally, we show the practical usefulness of our proposed test with a real-world application. [ABSTRACT FROM AUTHOR]

Background: Outcome measures that are count variables with excessive zeros are common in health behaviors research. Examples include the number of standard drinks consumed or alcohol‐related problems experienced over time. There is a lack of empirical data about the relative performance of prevailing statistical models for assessing the efficacy of interventions when outcomes are zero‐inflated, particularly compared with recently developed marginalized count regression approaches for such data. Methods: The current simulation study examined five commonly used approaches for analyzing count outcomes, including two linear models (with outcomes on raw and log‐transformed scales, respectively) and three prevailing count distribution‐based models (ie, Poisson, negative binomial, and zero‐inflated Poisson (ZIP) models). We also considered the marginalized zero‐inflated Poisson (MZIP) model, a novel alternative that estimates the overall effects on the population mean while adjusting for zero‐inflation. Motivated by alcohol misuse prevention trials, extensive simulations were conducted to evaluate and compare the statistical power and Type I error rate of the statistical models and approaches across data conditions that varied in sample size (N=100$$ N=100 $$ to 500), zero rate (0.2 to 0.8), and intervention effect sizes. Results: Under zero‐inflation, the Poisson model failed to control the Type I error rate, resulting in higher than expected false positive results. When the intervention effects on the zero (vs. non‐zero) and count parts were in the same direction, the MZIP model had the highest statistical power, followed by the linear model with outcomes on the raw scale, negative binomial model, and ZIP model. The performance of the linear model with a log‐transformed outcome variable was unsatisfactory. Conclusions: The MZIP model demonstrated better statistical properties in detecting true intervention effects and controlling false positive results for zero‐inflated count outcomes. This MZIP model may serve as an appealing analytical approach to evaluating overall intervention effects in studies with count outcomes marked by excessive zeros. [ABSTRACT FROM AUTHOR]

In the linear regression model, the multicollinearity problem arises when there is a linear relationship between independent variables. This situation causes the variance of the estimations of the model parameters obtained by the Least Squares Estimator method to increase and move away from the true value, resulting in unstable and incorrect results. Biased Estimator methods are developed to eliminate the adverse effects caused by multicollinearity. In this study, a test statistic is obtained to test the significance of the model coefficients for the Liu-Type Estimator using the test statistic method suggested in the study of Halawa and El-Bassiouni (2000). With a simulation study, the significance of the model coefficients of the Ridge, Liu, and Liu type biased estimators in different situations is tested; the type I errors and power values of the estimators are calculated; the results are compared. In addition, a real data application is performed to better understand the test procedure. [ABSTRACT FROM AUTHOR]

Randomization procedures that enforce balance in prognostic factors, most commonly stratified randomization, are often employed in clinical trials. When the number of factors or factor levels is large, dynamic allocation procedures, such as the Pocock and Simon's covariate-adaptive randomization (minimization) are preferred. In their ground-breaking work Ye and Shao identified two classes of covariate-adaptive randomization procedures. They have demonstrated theoretically that for these classes, when the model is misspecified, the robust score test (Lin and Wei) as well as the unstratified log-rank test used for analysis of time-to-event endpoints, are valid or conservative (Ye and Shao). This fact, however, was not established for minimization other than through simulations of survival endpoints. In this article, we point out that the results of Ye and Shao can be expanded to a more general class of randomization procedures. We show, in part theoretically, in part through simulations of the within-strata imbalances, that minimization belongs to this class. Along the way we describe the asymptotic correlation matrix of the normalized within-stratum imbalances following minimization with equal prevalence of all strata. We expand the robust tests proposed by Ye and Shao for stratified randomization to minimization and examine their performance through simulations. [ABSTRACT FROM AUTHOR]

Recently, Asparouhov and Muthén Structural Equation Modeling: A Multidisciplinary Journal, 28, 1–14, (2021a, 2021b) proposed a variant of the Wald test that uses Markov chain Monte Carlo machinery to generate a chi-square test statistic for frequentist inference. Because the test's composition does not rely on analytic expressions for sampling variation and covariation, it potentially provides a way to get honest significance tests in cases where the likelihood-based test statistic's assumptions break down (e.g., in small samples). The goal of this study is to use simulation to compare the new MCM Wald test to its maximum likelihood counterparts, with respect to both their type I error rate and power. Our simulation examined the test statistics across different levels of sample size, effect size, and degrees of freedom (test complexity). An additional goal was to assess the robustness of the MCMC Wald test with nonnormal data. The simulation results uniformly demonstrated that the MCMC Wald test was superior to the maximum likelihood test statistic, especially with small samples (e.g., sample sizes less than 150) and complex models (e.g., models with five or more predictors). This conclusion held for nonnormal data as well. Lastly, we provide a brief application to a real data example. [ABSTRACT FROM AUTHOR]

Exclusive hypothesis testing is a new and special class of hypothesis testing. This kind of testing can be applied in survival analysis to understand the association between genomics information and clinical information about the survival time. Besides, it is well known that Cox's proportional hazards model is the most commonly used model for regression analysis of failure time. In this paper, the authors consider doing the exclusive hypothesis testing for Cox's proportional hazards model with right-censored data. The authors propose the comprehensive test statistics to make decision, and show that the corresponding decision rule can control the asymptotic Type I errors and have good powers in theory. The numerical studies indicate that the proposed approach works well for practical situations and it is applied to a set of real data arising from Rotterdam Breast Cancer Data study that motivated this study. [ABSTRACT FROM AUTHOR]

Linear mixed-effects models (LMEMs) are widely used in medical, engineering, and social applications. The accurate specification of the covariance matrix structure within the error term is known to impact the estimation and inference procedures. Thus, it is crucial to detect the source of errors in LMEMs specifications. In this study, we propose combining a user-friendly computational test with an analytical method to visualize the source of errors. Through statistical simulations under different scenarios, we evaluate the performance of the proposed test in terms of the Power and Type I error rate. Our findings indicate that as the sample size n increases, the proposed test effectively detects misspecification in the systematic component, the number of random effects, the within-subject covariance structure, and the covariance structure of the error term in the LMEM with high Power while maintaining the nominal Type I error rate. Finally, we show the practical usefulness of our proposed test with a real-world application.

To test pleiotropy based on longitudinal data, two main difficulties are the construction of test statistics and the formulation of decision-making criteria. In particular, the commonly used log-likelihood ratio test is not available since there does not exist a likelihood function and the estimation is based on estimating equations in general. To address this and construct a statistic similar to the -2 log-likelihood ratio test statistic, we propose a new concept, a complex log-likelihood function, such that its first derivative is equal to a given estimating function. In consequence, the -2 complex log-likelihood ratio test is constructed for testing genetic pleiotropy, and the validity of the test procedure is established. An extensive simulation study is performed, which suggests that the proposed method works well for practical situations. Furthermore, it is applied to a set of longitudinal data on tropical storms. [ABSTRACT FROM AUTHOR]

A necessary step in applying bi-factor models is to evaluate the need for domain factors with a general factor in place. The conventional null hypothesis testing (NHT) was commonly used for such a purpose. However, the conventional NHT meets challenges when the domain loadings are weak or the sample size is insufficient. This article proposes using minimal-effect testing (MET) and equivalence testing (ET) to analyze bi-factor models. A key element in conducting MET and ET is the minimal size of factor loadings that can be regarded as noteworthy in practice, termed as minimal noteworthy size. This article presents two approaches to formulating the minimal noteworthy size and compares the pros and cons of MET, ET, and the conventional NHT. Analysis shows that MET, ET, and the conventional NHT are complementary. Combining them to test the noteworthiness of domain loadings can help researchers make a comprehensive judgment. Real and simulated datasets illustrate the applications of the three methods. Monte Carlo results show that MET and ET can control type I errors reasonably well while maintaining good statistical power. [ABSTRACT FROM AUTHOR]

AbstractThe ratio of two Poisson means is commonly used in biological, epidemiological, and medical. In this article, we consider the problem of testing the ratio of two Poisson means and propose a valid and efficient test based on the inference model (IM). The simulation studies show that the type I error and power of the IM-based test are competitive or even better than the six recommended tests: The likelihood ratio, mid-p, logarithmic transformation, and three tests based on the method of variance estimates recovery (MOVER). For R<1, the Rao-MOVER and fiducial-MOVER tests are slightly liberal, the mid-p test is somewhat conservative. For R>1, the Rao-MOVER and fiducial-MOVER tests are slightly conservative. The likelihood ratio and logarithmic transformation tests cannot control the type I error well. Therefore, the IM test could be recommended for practical applications. A real numerical example is presented to illustrate the flexibility of the proposed test. [ABSTRACT FROM AUTHOR]

AbstractIn clinical trials for chronic heart failure (CHF), time to a composite of first hospitalization for worsening heart failure or death is a widely accepted primary efficacy measure. Motivated by lower event rates in recent CHF trials, there are proposals to use endpoints accounting for recurrent heart failure hospitalization (HFH) events (e.g., recurrent HFH events plus death as an additional event) as the primary endpoint to better quantify disease burden and to improve trial efficiency. However, analysis and interpretation of recurrent event endpoints may be complicated by the terminal event death. In a previous simulation study it was shown that the Type I error rate was well-controlled for the analysis of the composite recurrent event endpoint using the negative binomial (NB) model and the Lin-Wei-Yang-Ying (LWYY) model. As these results were based on limited scenarios, we extended the previous simulation study and identified situations where the Type I error rate is no longer controlled for the analysis of the composite recurrent event endpoint using the LWYY model, despite a neutral effect on HFH and a detrimental effect on death of the experimental treatment. Therefore, depending on clinical settings, the LWYY model should be applied and interpreted with more caution. [ABSTRACT FROM AUTHOR]

Platform trials offer a framework to study multiple interventions in one trial with the opportunity of opening and closing arms. The use of common controls can increase efficiency as compared to individual controls. The need for multiplicity adjustment because of common controls is currently a debate among researchers, pharmaceutical companies, and regulators. The impact of common controls on the type one error in a fixed platform trial, i.e. when all treatments start and end recruitment at the same time, has been discussed in the literature before. We complement these findings by investigating the impact of a common control on the type one error and power in a flexible platform trial, i.e. when one arm joins the platform later. We derived the correlation of test statistics to assess the impact of the overlap and compared the results to a trial with individual controls. Furthermore, we evaluate the power, and the impact of multiplicity adjustment on the power in fixed and flexible platform trials. These methodological considerations are complemented by a regulatory guideline review. With multiple arms, the FWER is inflated when no multiplicity adjustment is applied. However, the FWER inflation is smaller with common controls than with individual controls. Even after multiplicity adjustment, a trial with common controls is often beneficial in terms of sample size and power. However, in some cases, the trial with common controls loses the efficiency gain and it might be advisable to run a separate trial rather than joining a platform trial. [ABSTRACT FROM AUTHOR]

AbstractIn this article, we propose equivalence tests for the difference of two survival functions in the presence of cure fraction. We consider both the non mixture cure model (NMCM) and mixture cure model (MCM). We first demonstrate that the existing tests can be used to test equivalence of two improper survival functions under the class of log transformation (LT) and Box-Cox transformation (BCT) NMCM. For the MCM, a logistic regression model is assumed for the probability of being uncured and the class of LT or BCT models are used to describe the distribution of event time for the uncured subjects. Under the MCM, we consider testing separately the null hypothesis of identical cure rate and that of equivalence of two survival functions of the uncured subjects. We also develop a Wald statistic for simultaneously testing whether or not both the maximal difference between cure rates and that between two short-term survival functions fall within some equivalence margins. Simulation results indicate that the proposed tests have a satisfactory size and an adequate power in finite sample. [ABSTRACT FROM AUTHOR]

For interrupted time series with small samples and outliers, the standard least squares regression for estimating intervention effects can be biased when the assumption of normality is violated. This article proposes a rank-based regression method combined with bootstrapping to address outliers and non-normality. The proposed method minimizes the Jaeckel dispersion function based on ranked residuals, offering robustness against outliers, skewed error distributions, and correlated errors. Simulation studies and a real-world example compare the performance of the proposed method with the least squares method under various conditions. Both produce unbiased estimates for the intervention effects under normality. However, with non-normal errors, outliers, and correlated errors, the rank-based bootstrapping method outperforms least squares producing narrower confidence intervals, higher efficiency, lower type I error rates, and greater power. Application to the Istanbul declaration on illegal organ activities confirms the improved robustness and precision of the rank-based method with bootstrapping over least squares. [ABSTRACT FROM AUTHOR]

AbstractClinical trials are an essential component of the drug development process, providing crucial data on the efficacy and safety of new treatments. However, traditional clinical trial designs can be inefficient and ineffective, leading to increased costs and a higher risk of failure. The FDA Oncology Center of Excellence (OCE) has recently initiated Project Optimus to reform the dose selection paradigm for oncology treatments. We propose the adaptive seamless phase II/III clinical trial designs (ASD) with the sequential estimation-adjusted urn (SEU) model for randomization to achieve efficient and ethical objectives. However, the combination of ASD and SEU poses a challenge in controlling the type I error rate: ASD exerts a dual influence of multiplicity and selection; all the responses and treatment assignments are not independent due to SEU. In this paper, we investigated how to overcome these difficulties, utilize the two adaptive approaches’ advantages, and control the type I error rate. We provide a theoretical foundation for this procedure, and numerical studies demonstrate that our methods can assign more people to better treatments, leading to fewer failures while still controlling the type I error rate and preserving power. [ABSTRACT FROM AUTHOR]

Researchers with mediation hypotheses must consider which design to use: within-subject or between-subject? In this paper, I argue that three factors should influence design choice: validity, causality, and statistical power. Threats to validity include carry-over effects, participant awareness, measurement, and more. Causality is a core element of mediation, and the assumptions required for causal inference differ between the two designs. Between-subject designs require more restrictive no-confounder assumptions, but within-subject designs require the assumption of no carry-over effects. Statistical power should be higher in within-subject designs, but the degree and conditions of this advantage are unknown for mediation analysis. A Monte Carlo simulation compares designs under a broad range of sample sizes, effect sizes, and correlations among repeated measurements. The results show within-subject designs require about half the sample size of between-subject designs to detect indirect effects of the same size, but this difference can vary with population parameters. I provide an empirical example and R script for conducting power analysis for within-subject mediation analysis. Researchers interested in conducting mediation analysis should not select within-subject designs merely because of higher power, but they should also consider validity and causality in their decision, both of which can favor between-subject designs.

In research, outcomes are often categorized as primary and secondary. The primary outcome is the most important one; it determines whether the study is considered ‘successful’ or not. Secondary outcomes are chosen because they provide supporting evidence for the results of the primary outcome or additional information about the subject being studied. For reasons that are explained in this paper, secondary outcomes should be cautiously interpreted. There are varying practices regarding publishing secondary outcomes. Some authors publish these separately, while others include them in the main publication. In some contexts, the former can lead to concerns about the quality and relevance of the data being published. In this article, we discuss primary and secondary outcomes, the importance and interpretation of secondary outcomes, and considerations for publishing multiple outcomes in separate papers. We also discuss the special case of secondary analyses and post hoc analyses and provide guidance on good publishing practices. Throughout the article, we use relevant examples to make these concepts easier to understand. While the article is primarily aimed at early career researchers, it offers insights that may be helpful to researchers, reviewers, and editors across all levels of expertise.

Background Variables such as dietary intake are measured with error yet frequently used in observational epidemiology. Although this limitation is sometimes noted, these variables are still often modeled as covariates without formal correction or sincere dialogue about measurement unreliability potentially weakening the validity of statistical conclusions. Further, larger sample sizes increase power (bias) to detect spurious correlations. Counterintuitively, recent work suggested a non-monotonic relationship between confounder unreliability and how much controlling for the confounder reduces (or induces) bias when testing for an exposure-outcome association. If true, such non-monotonicity would be especially concerning for applications such as nutrition, where measurement reliability varies substantially, and large sample sizes are common. Methods We offer a detailed derivations of the square partial correlation between the outcome and exposure, controlling for the confounder. In our derivation, the measurement reliabilities of exposures and confounders are not arbitrarily constrained to be equal. Further, our theoretical results are investigated using simulations. Results Reassuringly, these derivations and simulations show that the counterintuitive non-monotonicity relationship between confounder unreliability and how much controlling for the confounder reduces (or induces) bias when testing for an exposure-outcome association is an artifact of the arbitrary constraint which forces the measurement reliabilities of exposures and confounders to be equal, which that does not always hold. Conclusions The profound and manifold effects of measurement error on estimation and statistical conclusion validity in realistic scenarios indicate that merely mentioning measurement error as a limitation and then dispensing with it is not an adequate response. We also explore questions for optimal study design subject to resource constraints when considering reliability of exposures, covariates, and outcomes.

The analysis of multiple bivariate correlations is often carried out by conducting simple tests to check whether each of them is significantly different from zero. In addition, pairwise differences are often judged by eye or by comparing the p‐values of the individual tests of significance despite the existence of statistical tests for differences between correlations. This paper uses simulation methods to assess the accuracy (empirical Type I error rate), power, and robustness of 10 tests designed to check the significance of the difference between two dependent correlations with overlapping variables (i.e., the correlation between X1 and Y and the correlation between X2 and Y). Five of the tests turned out to be inadvisable because their empirical Type I error rates under normality differ greatly from the nominal alpha level of .05 either across the board or within certain sub‐ranges of the parameter space. The remaining five tests were acceptable and their merits were similar in terms of all comparison criteria, although none of them was robust across all forms of non‐normality explored in the study. Practical recommendations are given for the choice of a statistical test to compare dependent correlations with overlapping variables. [ABSTRACT FROM AUTHOR]

In research, outcomes are often categorized as primary and secondary. The primary outcome is the most important one; it determines whether the study is considered 'successful' or not. Secondary outcomes are chosen because they provide supporting evidence for the results of the primary outcome or additional information about the subject being studied. For reasons that are explained in this paper, secondary outcomes should be cautiously interpreted. There are varying practices regarding publishing secondary outcomes. Some authors publish these separately, while others include them in the main publication. In some contexts, the former can lead to concerns about the quality and relevance of the data being published. In this article, we discuss primary and secondary outcomes, the importance and interpretation of secondary outcomes, and considerations for publishing multiple outcomes in separate papers. We also discuss the special case of secondary analyses and post hoc analyses and provide guidance on good publishing practices. Throughout the article, we use relevant examples to make these concepts easier to understand. While the article is primarily aimed at early career researchers, it offers insights that may be helpful to researchers, reviewers, and editors across all levels of expertise. [ABSTRACT FROM AUTHOR]

In many scientific experiments, multiarmed bandits are used as an adaptive data collection method. However, this adaptive process can lead to a dependence that renders many commonly used statistical inference methods invalid. An example of this is the sample mean, which is a natural estimator of the mean parameter but can be biased. This can cause test statistics based on this estimator to have an inflated type I error rate, and the resulting confidence intervals may have significantly lower coverage probabilities than their nominal values. To address this issue, we propose an alternative approach called randomized multiarm bandits (rMAB). This combines a randomization step with a chosen MAB algorithm, and by selecting the randomization probability appropriately, optimal regret can be achieved asymptotically. Numerical evidence shows that the bias of the sample mean based on the rMAB is much smaller than that of other methods. The test statistic and confidence interval produced by this method also perform much better than its competitors. [ABSTRACT FROM AUTHOR]

Due to highly unmet medical needs in rare disease areas, there is great desire to speed up the drug development process. A two-stage adaptive design option for a placebo-controlled registration study is being evaluated. Stage 1 consists of participants in an ongoing phase 2 study with 1-year double blinded (DB) treatment and Stage 2 includes newly enrolled participants with 2-year DB treatment period. The primary endpoint is the annualized rate of change (slope) for a continuous longitudinal measurement, which will be evaluated through a random coefficient linear mixed model. An unblinded interim analysis will be performed using Stage 1 data to re-estimate the sample size for Stage 2, followed by another interim analysis for potential early efficacy stopping when all participants completed the 1-year DB treatment. To control the overall Type 1 error rate, rather than using a conservative approach, the actual correlation between the interim and final test statistics will be taken into account to determine the final significance level given the pre-specified significance level for the interim efficacy analysis. Multiplicity adjustments for secondary endpoints including considerations for order switching between interim and final analysis in this specific case study will also be discussed. [ABSTRACT FROM AUTHOR]

Careful consideration of the tradeoff between Type I and Type II error rates when designing experiments is critical for maximizing statistical decision accuracy. Typically, Type I error rates (e.g., .05) are significantly lower than Type II error rates (e.g., .20 for .80 power) in psychological science. Further, positive findings (true effects and Type I errors) are more likely to be the focus of replication. This conventional approach leads to very high rates of Type II error. Analyses show that increasing the Type I error rate to .10, thereby increasing power and decreasing the Type II error rate for each test, leads to higher overall rates of correct statistical decisions. This increase of Type I error rate is consistent with, and most beneficial in the context of, the replication and "New Statistics" movements in psychology. Making decisions under conditions of uncertainty (e.g., science) always involves a tradeoff between two types of errors. The decision maker can conclude something is present when it is not (false positive or Type I error) or conclude no difference is present, when in reality there is a difference (a miss or Type II error). We address the need to carefully consider the balance of these error types in scientific decision making. In psychology, it is typical to more stringently control the Type I error rate at .05, while the Type II error rate is .20 (i.e., when power is .80). We demonstrate that the overall percentage correct decisions in psychological science can be increased by increasing the Type I error rate to .10 to increase power, and decrease the Type II error rate. This is particularly relevant given the recent methodological and statistical changes in the field, particularly the long-needed emphasis on replication. [ABSTRACT FROM AUTHOR]

Intraclass correlation in bilateral data has been investigated in recent decades with various statistical methods. In practice, stratifying bilateral data by some control variables will provide more sophisticated statistical results to satisfy different research proposed in randomized clinical trials. In this article, we propose three test statistics (the likelihood ratio test, score test, and Wald-type test statistics) to evaluate the homogeneity of proportion ratios for stratified bilateral correlated data under an equal correlation assumption. Monte Carlo simulations of Type I error and power are performed, and the score test yields a robust outcome based on empirical Type I error and power. Lastly, two real data examples are conducted to illustrate the proposed three tests. [ABSTRACT FROM AUTHOR]

Classical tests for a difference in means control the Type I error rate when the groups are defined a priori. However, when the groups are instead defined via clustering, then applying a classical test yields an extremely inflated Type I error rate. Notably, this problem persists even if two separate and independent datasets are used to define the groups and to test for a difference in their means. To address this problem, in this article, we propose a selective inference approach to test for a difference in means between two clusters. Our procedure controls the selective Type I error rate by accounting for the fact that the choice of null hypothesis was made based on the data. We describe how to efficiently compute exact p-values for clusters obtained using agglomerative hierarchical clustering with many commonly used linkages. We apply our method to simulated data and to single-cell RNA-sequencing data. for this article are available online. [ABSTRACT FROM AUTHOR]

When comparing the performance of two or more competing tests, simulation studies commonly focus on statistical power. However, if the size of the tests being compared are either different from one another or from the nominal size, comparing tests based on power alone may be misleading. By analogy with diagnostic accuracy studies, we introduce relative positive and negative likelihood ratios to factor in both power and size in the comparison of multiple tests. We derive sample size formulas for a comparative simulation study. As an example, we compared the performance of six statistical tests for small‐study effects in meta‐analyses of randomized controlled trials: Begg's rank correlation, Egger's regression, Schwarzer's method for sparse data, the trim‐and‐fill method, the arcsine‐Thompson test, and Lin and Chu's combined test. We illustrate that comparing power alone, or power adjusted or penalized for size, can be misleading, and how the proposed likelihood ratio approach enables accurate comparison of the trade‐off between power and size between competing tests. [ABSTRACT FROM AUTHOR]

The assumption of homoscedasticity is one of the most crucial assumptions for many parametric tests used in the biological sciences. The aim of this paper is to compare the empirical probability of type I error and the power of ten parametric and two non-parametric tests for homoscedasticity with simulations under different types of distributions, number of groups, number of samples per group, variance ratio and significance levels, as well as through empirical data from an agricultural experiment. According to the findings of the simulation study, when there is no violation of the assumption of normality and the groups have equal variances and equal number of samples, the Bhandary-Dai, Cochran’s C, Hartley’s Fmax, Levene (trimmed mean) and Bartlett tests are considered robust. The Levene (absolute and square deviations) tests show a high probability of type I error in a small number of samples, which increases as the number of groups rises. When data groups display a nonnormal distribution, researchers should utilize the Levene (trimmed mean), O’Brien and Brown-Forsythe tests. On the other hand, if the assumption of normality is not violated but diagnostic plots indicate unequal variances between groups, researchers are advised to use the Bartlett, Z-variance, Bhandary-Dai and Levene (trimmed mean) tests. Assessing the tests being considered, the test that stands out as the most well-rounded choice is the Levene’s test (trimmed mean), which provides satisfactory type I error control and relatively high power. According to the findings of the study and for the scenarios considered, the two non-parametric tests are not recommended. In conclusion, it is suggested to initially check for normality and consider the number of samples per group before choosing the most appropriate test for homoscedasticity. [ABSTRACT FROM AUTHOR]

Sample size estimation remains as a cornerstone in the meticulous planning and execution of clinical trials, pivotal for ensuring studies possess the requisite statistical power to discern meaningful treatment effects. Insufficient sample sizes compromise the robustness of findings, whereas excessively large samples inflate costs and compromise data integrity. This article meticulously explains the multifaceted factors that outline sample size determination, encompassing various factors such as research design, types of hypotheses, error thresholds, effect size considerations, validity and precision. It investigates into the scope of methodologies available for sample size computation, spanning from intricate statistical formulas to pragmatic tabular approaches. Moreover, it underscores the significance of post-hoc power analysis in retrospectively evaluating completed studies, shedding light on their statistical robustness. This literature review furnishes a nuanced understanding of the sample size estimation landscape in clinical trials, delineating their strengths, limitations, and real-world applications. Anticipating participant attrition assumes paramount importance for proactively adjusting sample sizes, ensuring studies remain methodologically sound. Equipped with a profound grasp of these principles, researchers are empowered to conduct scientifically rigorous and impactful clinical trials, furnishing compelling evidence to inform judicious decision-making in healthcare interventions.

Ordinary least squares (OLS) regression is widely employed for statistical prediction and theoretical explanation in psychology studies. However, OLS regression has a critical drawback: it becomes less accurate in the presence of outliers and non-random error distribution. Several robust regression methods have been proposed as alternatives. However, each robust regression has its own strengths and limitations. Consequently, researchers are often at a loss as to which robust regression method to use for their studies. This study uses a Monte Carlo experiment to compare different types of robust regression methods with OLS regression based on relative efficiency (RE), bias, root mean squared error (RMSE), Type 1 error, power, coverage probability of the 95% confidence intervals (CIs), and the width of the CIs. The results show that, with sufficient samples per predictor (n = 100), the robust regression methods are as efficient as OLS regression. When errors follow non-normal distributions, i.e., mixed-normal, symmetric and heavy-tailed (SH), asymmetric and relatively light-tailed (AL), asymmetric and heavy-tailed (AH), and heteroscedastic, the robust method (GM-estimation) seems to consistently outperform OLS regression.

Testing the association between objects is central in ecology, evolution, and quantitative sciences in general. Two types of variables can describe the relationships between objects: point variables (measured on individual objects), and distance variables (measured between pairs of objects). The Mantel test and derived methods have been extensively used for distance variables. Yet, these methods have been criticized due to low statistical power and inflated type I error when spatial autocorrelation is present. Here, we assessed the statistical power between different types of tested variables and the type I error rate over a wider range of autocorrelation intensities than previously assessed, both on univariate and multivariate data. We also illustrated the performance of distance matrix statistics through computational simulations of genetic diversity. We show that the Mantel test and derived methods are not affected by inflated type I error when spatial autocorrelation affects only one variable when investigating correlations, or when either the response or the explanatory variable(s) is affected by spatial autocorrelation while investigating causal relationships. As previously noted, with autocorrelation affecting more variables, inflated type I error could be reduced by modifying the significance threshold. Additionally, the Mantel test has no problem of statistical power when the hypothesis is formulated in terms of distance variables. We highlight that transformation of variable types should be avoided because of the potential information loss and modification of the tested hypothesis. We propose a set of guidelines to help choose the appropriate method according to the type of variables and defined hypothesis. [ABSTRACT FROM AUTHOR]

Ordinary least squares (OLS) regression is widely employed for statistical prediction and theoretical explanation in psychology studies. However, OLS regression has a critical drawback: it becomes less accurate in the presence of outliers and non-random error distribution. Several robust regression methods have been proposed as alternatives. However, each robust regression has its own strengths and limitations. Consequently, researchers are often at a loss as to which robust regression method to use for their studies. This study uses a Monte Carlo experiment to compare different types of robust regression methods with OLS regression based on relative efficiency (RE), bias, root mean squared error (RMSE), Type 1 error, power, coverage probability of the 95% confidence intervals (CIs), and the width of the CIs. The results show that, with sufficient samples per predictor (n = 100), the robust regression methods are as efficient as OLS regression. When errors follow non-normal distributions, i.e., mixed-normal, symmetric and heavy-tailed (SH), asymmetric and relatively light-tailed (AL), asymmetric and heavy-tailed (AH), and heteroscedastic, the robust method (GM-estimation) seems to consistently outperform OLS regression. [ABSTRACT FROM AUTHOR]

The matching paradigm can take a number of forms and has been used in many areas of psychology. When participants are asked to match or order sets of objects, researchers must correctly account for the number of matches expected purely by chance. Not accounting for the expected chance matches can lead to incorrectly drawing conclusions based on one's data. This study demonstrated that the z test can be an appropriate and easy test to use in the analysis of matching data from studies that require pairs of objects to be matched with each other. This article proves that in a matching paradigm the expected number of chance matches is 1.0 and the associated variance is also 1.0. The z test is shown to maintain the Type I error close to the nominal significance level when the null hypothesis is true and the sample size is at least 80 or 110. To attain power of .80, a sample size larger than 80 may be needed depending upon the effect size associated with the area of interest and the hypothesized alternative probability distribution. The matching paradigm can take a number of forms and has been used in many areas of psychology. When participants are asked to match, or order, sets of objects researchers must correctly account for the number of matches expected purely by chance. Not accounting for the expected chance matches can lead to incorrectly drawing conclusions based on one's data. This study demonstrated that the simple one-sample z test can be an appropriate and easy test to use in the analysis of matching data from studies that require pairs of objects to be matched with each other. When the null hypothesis is true (i.e., matches are occurring by chance) and the sample size is at least 80 or 110, the z test is shown to maintain the Type I error close to the alpha (significance) level researchers selected. To attain power of .80 a sample size larger than 80 may be needed depending upon the effect size associated with the area of interest. [ABSTRACT FROM AUTHOR]

Copyright of Ardahan University Journal of the Faculty of Economics & Administrative Sciences / Ardahan Üniversitesi Iktisadi ve Idari Bilimler Fakültesi Dergisi is the property of Ardahan University Journal of the Faculty of Economics & Administrative Sciences and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The main purpose of this study is to examine the Type I error and statistical power ratios of Differential Item Functioning (DIF) techniques based on different theories under different conditions. For this purpose, a simulation study was conducted by using Mantel-Haenszel (MH), Logistic Regression (LR), Lord's ?2, and Raju's Areas Measures techniques. In the simulation-based research model, the two-parameter item response model, group's ability distribution, and DIF type were the fixed conditions while sample size (1800, 3000), rates of sample size (0.50, 1), test length (20, 80) and DIF-containing item rate (0, 0.05, 0.10) were manipulated conditions. The total number of conditions is 24 (2x2x2x3), and statistical analysis was performed in the R software. The current study found that the Type I error rates in all conditions were higher than the nominal error level. It was also demonstrated that MH had the highest error rate while Raju's Areas Measures had the lowest error rate. Also, MH produced the highest statistical power rates. The analysis of the findings of Type 1 error and statistical power rates illustrated that techniques based on both of the theories performed better in the 1800 sample size. Furthermore, the increase in the sample size affected techniques based on CTT rather than IRT. Also, the findings demonstrated that the techniques' Type 1 error rates were lower while their statistical power rates were higher under conditions where the test length was 80, and the sample sizes were not equal. [ABSTRACT FROM AUTHOR]

This work examined the sensitivity of randomization test and F‐test in completely randomized design, randomized complete block design and Latin square design in order to ascertain which is more robust in terms of power (sensitivity) of the test and type I error rate in cases with normal error, skewed error (Chi‐square and Exponential distributions), by varying the sample sizes. Using Monte Carlo simulation, response observations were generated for normal error, skewed error at various sample sizes for each of the experimental designs. The results showed that for the normal error, the power and type I error rate of the F‐test were slightly smaller and greater respectively in each of the designs than that of the randomization test. Furthermore, for the skewed error, the randomization test was more sensitive than the F‐test, and also, the type I error was smaller in each of the designs. [ABSTRACT FROM AUTHOR]

In this study, the methods based on Classical Test Theory and Item Response Theory were used comparatively to determine Type I error and power rates in Differential Item Functioning. Logistic regression, MantelHaenszel, Lord's X², Breslow-Day and Raju's area index methods were used for the analyses, which were conducted using the R programming language. Determination of Type I error and power rates of these methods under certain conditions was carried out by simulation study. For data generation, analyzes were made under eight conditions in total by examining different sample sizes and DIF rates created with the WinGen 3 program. The results of the study indicate that, in general when the ratio of items containing DIF increased, Type I error increased and the power ratio decreased. Among the methods based on Item Response Theory, Lord's X² and Raju's area index methods gave better results than other methods with low error and high power. [ABSTRACT FROM AUTHOR]

A practical solution based on the method of variance estimates recovery is proposed for the Behrens–Fisher problem. Unlike the classical solution, the proposed test is not based on the central limit theorem and, therefore, it can control Type I error well for small or moderate sample sizes. In this sense, it performs better than the most commonly used Welch's approximate t-test, which may be liberal for small or moderate sample sizes. On the other hand, the powers of the proposed test appears to be close to that of the Welch's approximate t-test and better than those of the Fisher's solution and the generalized p-value solution; the latter two have no fixed p-value and need to simulate. The proposed test has a meaningful interpretation within and not just across experiments. It can be a competitive alternative solution. [ABSTRACT FROM AUTHOR]

Label noise in data has long been an important problem in supervised learning applications as it affects the effectiveness of many widely used classification methods. Recently, important real-world applications, such as medical diagnosis and cybersecurity, have generated renewed interest in the Neyman–Pearson (NP) classification paradigm, which constrains the more severe type of error (e.g., the Type I error) under a preferred level while minimizing the other (e.g., the Type II error). However, there has been little research on the NP paradigm under label noise. It is somewhat surprising that even when common NP classifiers ignore the label noise in the training stage, they are still able to control the Type I error with high probability. However, the price they pay is excessive conservativeness of the Type I error and hence a significant drop in power (i.e., 1 - Type II error). Assuming that domain experts provide lower bounds on the corruption severity, we propose the first theory-backed algorithm that adapts most state-of-the-art classification methods to the training label noise under the NP paradigm. The resulting classifiers not only control the Type I error with high probability under the desired level but also improve power. [ABSTRACT FROM AUTHOR]

A commonly used test for the independence of two sets of variables from a normal multivariate population is the Wilks Lambda test or the multivariate likelihood ratio test (LRT). However, this test's performance is highly influenced by outliers and non-normality of the data and thus, robust test statistics should be used, allied to Monte-Carlo methods. In this paper, we proposed and evaluated three new likelihood ratio test for the independence of two sets of multivariate variables (LRTR, T and TR), replacing the traditional covariance matrix estimator with the robust comedian estimators. The results showed that the proposed test T and TR control type I error rates also for non-normal distributions outperforming the ordinary test. [ABSTRACT FROM AUTHOR]

Manipulations checks are postexperimental measures widely used to verify that subjects understood the treatment. Some researchers drop subjects who failed manipulation checks in order to limit the analyses to attentive subjects. This short report offers a novel illustration on how this practice may bias experimental results: in the present case, through confirming a hypothesis that is likely false. In a survey experiment, subjects were primed with a fictional news story depicting an economic decline versus prosperity. Subjects were then asked whether the news story depicted an economic decline or prosperity. Results indicate that responses to this manipulation check captured subjects' preexisting beliefs about the economic situation. As a consequence, dropping subjects who failed the manipulation check mixes the effects of preexisting and induced beliefs, increasing the risk of false positive findings. Researchers should avoid dropping subjects based on posttreatment measures and rely on pretreatment measures of attentiveness. [ABSTRACT FROM AUTHOR]

Abstract Background Due to the high cost and high failure rate of Phase III trials where a classical group sequential design (GSD) is usually used, seamless Phase II/III designs are more and more popular to improve trial efficiency. A potential attraction of Phase II/III design is to allow a randomized proof-of-concept stage prior to committing to the full cost of a Phase III trial. Population selection during the trial allows a trial to adapt and focus investment where it is most likely to provide patient benefit. Previous methods have been developed for this problem when there is a single primary endpoint and two possible populations. Methods To find the population that potentially benefits with one or two primary endpoints (e.g., progression free survival (PFS), overall survival (OS)), we propose a gated group sequential design for a seamless Phase II/III trial design with adaptive population selection. Results The investigated design controls the familywise error rate and allows multiple interim analyses to enable early stopping for efficacy or futility. Simulations and an illustrative example suggest that the proposed gated group sequential design has more power and requires less time and resources compared to the group sequential design and adaptive design. Conclusions Combining the group sequential design and adaptive design, the gated group sequential design has more power and higher efficiency while controlling for the familywise error rate. It has the potential to save drug development cost and more quickly fulfill unmet medical needs.