Search

One-sided inference should be used in some applications, but Stata has limited support for one-sided tests and confidence intervals for a single compari-son and almost no support for one-sided tests and confidence bands for multiple comparisons. In this article, we provide an introduction to one-sided tests and confidence intervals for a single hypothesis and to one-sided tests and confidence intervals for multiple comparisons. We also discuss extensions of the sotable com-mand introduced in Drukker (2023, Stata Journal 23: 518-544) to cover one-sided tests and confidence bands for a single comparison and for multiple comparisons. We also provide examples of how to use sotable to perform multiple tests against values other than zero and how to perform multiple tests after commands like margins and nlcom that support the post option. [ABSTRACT FROM AUTHOR]

Stata estimation commands that implement frequentist methods produce an output table that contains multiple tests and multiple confidence intervals. Presumably, the multiple tests and multiple confidence are designed to help determine which parameters are responsible for a possible rejection of the overall null hypothesis of no effect. When taken by itself, each test and each confidence interval provides valid inference about the null hypothesis of no effect for each parameter at the specified error rate. However, simultaneously using two or more of these tests or confidence intervals provides inference at an error rate that exceeds the one specified. In this article, I discuss the sotable command, which provides p -values that are adjusted for the multiple tests and a confidence band that can be used to simultaneously test multiple parameters for no effect after almost all frequentist estimation commands. I also provide an introduction to the literature on simultaneous inference. [ABSTRACT FROM AUTHOR]

Rank tests are known to be robust to outliers and violation of distributional assumptions. Two major issues besetting microarray data are violation of the normality assumption and contamination by outliers. In this article, we formulate the normal theory simultaneous tests and their aligned rank transformation (ART) analog for detecting differentially expressed genes. These tests are based on the least-squares estimates of the effects when data follow a linear model. Application of the two methods are then demonstrated on a real data set. To evaluate the performance of the aligned rank transform method with the corresponding normal theory method, data were simulated according to the characteristics of a real gene expression data. These simulated data are then used to compare the two methods with respect to their sensitivity to the distributional assumption and to outliers for controlling the family-wise Type I error rate, power, and false discovery rate. It is demonstrated that the ART generally possesses the robustness of validity property even for microarray data with small number of replications. Although these methods can be applied to more general designs, in this article the simulation study is carried out for a dye-swap design since this design is broadly used in cDNA microarray experiments. [ABSTRACT FROM AUTHOR]

A class of simultaneous tests based on the aligned rank transform (ART) statistics is proposed for linear functions of parameters in linear models. The asymptotic distributions are derived. The stability of the finite sample behaviour of the sampling distribution of the ART technique is studied by comparing the simulated upper quantiles of its sampling distribution with those of the multivariatet-distribution. Simulation also shows that the tests based on ART have excellent small sample properties and because of their robustness perform better than the methods based on the least-squares estimates. [ABSTRACT FROM AUTHOR]

Testing for unequal variances is usually performed in order to check the validity of the assumptions that underlie standard tests for differences between means (the t-test and anova). However, existing methods for testing for unequal variances (Levene's test and Bartlett's test) are notoriously non-robust to normality assumptions, especially for small sample sizes. Moreover, although these methods were designed to deal with one hypothesis at a time, modern applications (such as to microarrays and fMRI experiments) often involve parallel testing over a large number of levels (genes or voxels). Moreover, in these settings a shift in variance may be biologically relevant, perhaps even more so than a change in the mean. This paper proposes a parsimonious model for parallel testing of the equal variance hypothesis. It is designed to work well when the number of tests is large; typically much larger than the sample sizes. The tests are implemented using an empirical Bayes estimation procedure which `borrows information' across levels. The method is shown to be quite robust to deviations from normality, and to substantially increase the power to detect differences in variance over the more traditional approaches even when the normality assumption is valid. [ABSTRACT FROM AUTHOR]

Technologies of measuring millions of quantities at once have been rapidly developed and used in biological and biomedical research and other fields in the past decade. Yet a key issue remains unsettled, namely, how to control spurious findings for the ensuing massive number of hypothesis tests. An emerging consensus is to control false discovery rate (FDR), with FDR defined as the expected proportion of true nulls among discoveries, and a discovery the rejection of a null hypothesis. However, the very concept of counting true nulls, implicitly or explicitly, is problematic, for nulls are rarely true in reality. We propose an approach that is philosophically different from the FDR and other approaches. Taking advantage of the massive measurements, we can and should directly evaluate the reproducibility of a discovery by calculating the posterior probability of discovery (PPD) given observed data. A discovery with a great PPD is deemed to be highly reproducible, that is, not spurious. For a subset of hypotheses tested, mean PPD yields the expected rate of discovery (ERD), a measure useful for various applications such as subset enrichment analysis. We present here the rationale, theoretical basis, and an algorithm for computing PPDs and ERDs from data. Their validity, utility, and optimal performance are demonstrated using both simulated and real data. Supplementary material is available online. [ABSTRACT FROM AUTHOR]

In a randomized block design, robust versions based on scale-invariant M-statistics are proposed for single-step multiple comparisons procedures discussed by Tukey (1953) and Dunnett (1955). Although the symmetry of the underlying distribution is needed in the asymptotic theory of Huber's M-estimators, the proposed procedures do not demand the symmetry. It is found that the M-procedures are superior to the classical normal theory procedures except the case that an underlying distribution is normal. [ABSTRACT FROM AUTHOR]

The classic frequentist theory of hypothesis testing developed by Neyman, Pearson and Fisher has a claim to being the twentieth century's most influential piece of applied mathematics. Something new is happening in the twenty-first century: high-throughput devices, such as microarrays, routinely require simultaneous hypothesis tests for thousands of individual cases, not at all what the classical theory had in mind. In these situations empirical Bayes information begins to force itself upon frequentists and Bayesians alike. The two-groups model is a simple Bayesian construction that facilitates empirical Bayes analysis. This article concerns the interplay of Bayesian and frequentist ideas in the two-groups setting, with particular attention focused on Benjamini and Hochberg's False Discovery Rate method. Topics include the choice and meaning of the null hypothesis in large-scale testing situations, power considerations, the limitations of permutation methods, significance testing for groups of cases (such as pathways in microarray studies), correlation effects, multiple confidence intervals and Bayesian competitors to the two-groups model. [ABSTRACT FROM AUTHOR]

A new methodology is proposed for estimating the proportion of true null hypotheses in a large collection of tests. Each test concerns a single parameter δ whose value is specified by the null hypothesis. We combine a parametric model for the conditional cumulative distribution function (CDF) of the p-value given δ with a nonparametric spline model for the density g(δ) of δ under the alternative hypothesis. The proportion of true null hypotheses and the coefficients in the spline model are estimated by penalized least squares subject to constraints that guarantee that the spline is a density. The estimator is computed efficiently using quadratic programming. Our methodology produces an estimate of the density of δ when the null is false and can address such questions as “when the null is false, is the parameter usually close to the null or far away?” This leads us to define a falsely interesting discovery rate (FIDR), a generalization of the false discovery rate. We contrast the FIDR approach to Efron's (2004, Journal of the American Statistical Association 99, 96–104) empirical null hypothesis technique. We discuss the use of in sample size calculations based on the expected discovery rate (EDR). Our recommended estimator of the proportion of true nulls has less bias compared to estimators based upon the marginal density of the p-values at 1. In a simulation study, we compare our estimators to the convex, decreasing estimator of Langaas, Lindqvist, and Ferkingstad (2005, Journal of the Royal Statistical Society, Series B 67, 555–572). The most biased of our estimators is very similar in performance to the convex, decreasing estimator. As an illustration, we analyze differences in gene expression between resistant and susceptible strains of barley. [ABSTRACT FROM AUTHOR]

This paper argues that it is wrong to require that regressing the outputs of a trace-driven simulation on the observed real outcomes should give a 45 degree (unit slope) line through the origin (zero intercept). This note proposes instead an alternative requirement: the responses of the simulated and the real systems should have the same means and the same variances. To test statistically whether this requirement is satisfied, a novel procedure is derived: regress the differences between simulated and real responses on their associated sums, and test whether the resulting intercept and slope are both zero. This novel but simple test assumes identically, independently, and normally distributed outputs of the real system and the simulated system. The old and the new procedures are investigated in extensive Monte Carlo experiments that simulate M/M/1 queueing systems. The conclusions are: (i) the naive intuitive test rejects a valid simulation model substantially more often than the novel test does; (it) the naive test shows "perverse" behavior within a certain domain: the worse the simulation model, the higher its estimated probability of acceptance; and (iii) the novel test does not reject a valid simulation model too often (its type I error probability is correct), provided the queueing response is transformed appropriately to obtain (nearly) normally distributed responses. [ABSTRACT FROM AUTHOR]