Search

In this chapter, we develop theory and methodology to support mode adjustment and hindcasting/forecasting in the presence of different possible mode effect types, including additive effects and odds-multiplicative effects. Mode adjustment is particularly important if the ultimate goal is to report one aggregate estimate of response parameters, and also to allow for comparison to historical surveys performed with different modes. Effect type has important consequences for inferential validity when the baseline response changes over time (i.e. when there is a time trend or time effect). We present a methodology to provide inference for additive and odds-multiplicative effect types, and demonstrate its performance in a simulation study. We also show that if the wrong effect type is assumed, the resulting inference can be invalid as confidence interval coverage is greatly reduced and estimates can be biased.

Motivated by the goal of evaluating a biomarker for acute kidney injury, we consider the problem of assessing operating characteristics for a new biomarker when a true gold standard for disease status is unavailable. In this case, the biomarker is typically compared to another imperfect reference test, and this comparison is used to estimate the performance of the new biomarker. However, errors made by the reference test can bias assessment of the new test. Analysis methods like latent class analysis have been proposed to address this issue, generally employing some strong and unverifiable assumptions regarding the relationship between the new biomarker and the reference test. We investigate the conditional independence assumption that is present in many such approaches and show that for a given set of observed data, conditional independence is only possible for a restricted range of disease prevalence values. We explore the information content of the comparison between the new biomarker and the reference test, and give bounds for the true sensitivity and specificity of the new test when operating characteristics for the reference test are known. We demonstrate that in some cases these bounds may be tight enough to provide useful information, but in other cases these bounds may be quite wide.

Summary Occupancy models are employed in species distribution modelling to account for imperfect detection during field surveys. While this approach is popular in the literature, problems can occur when estimating the model parameters. In particular, the maximum likelihood estimates can exhibit bias and large variance for data sets with small sample sizes, which can result in estimated occupancy probabilities near 0 and 1 (‘boundary estimates’). In this paper, we explore strategies for estimating parameters based on maximizing a penalized likelihood. Penalized likelihood methods augment the usual likelihood with a penalty function that encodes information about what parameter values are undesirable. We introduce penalties for occupancy models that have analogues in ridge regression and Bayesian approaches, and we compare them to a penalty developed for occupancy models in prior work. We examine the bias, variance and mean squared error of parameter estimates obtained from each method on synthetic data. Across all of the synthetic data sets, the penalized estimation methods had lower mean squared error than the maximum likelihood estimates. We also provide an example of the application of these methods to point counts of avian species. Penalized likelihood methods show similar improvements when tested using empirical bird point count data. We discuss considerations for choosing among these methods when modelling occupancy. We conclude that penalized methods may be of practical utility for fitting occupancy models with small sample sizes, and we are releasing R code that implements these methods.

The function and extracellular location of cell envelope proteins make them attractive candidates for developing vaccines against bacterial diseases, including challenging drug-resistant pathogens, such as Neisseria gonorrhoeae . A proteomics-driven reverse vaccinology approach has delivered multiple gonorrhea vaccine candidates; however, the biological functions of many of them remain to be elucidated. Herein, the functions of six gonorrhea vaccine candidates—NGO2121, NGO1985, NGO2054, NGO2111, NGO1205, and NGO1344—in cell envelope homeostasis were probed using phenotype microarrays under 1,056 conditions and a Δ bamE mutant (Δ ngo1780 ) as a reference of perturbed outer membrane integrity. Optimal growth conditions for an N. gonorrhoeae phenotype microarray assay in defined liquid medium were developed, which can be useful in other applications, including rapid and thorough antimicrobial susceptibility assessment. Our studies revealed 91 conditions having uniquely positive or negative effects on one of the examined mutants. A cluster analysis of 37 and 57 commonly beneficial and detrimental compounds, respectively, revealed three separate phenotype groups: NGO2121 and NGO1985; NGO1344 and BamE; and the trio of NGO1205, NGO2111, and NGO2054, with the last protein forming an independent branch of this cluster. Similar phenotypes were associated with loss of these vaccine candidates in the highly antibiotic-resistant WHO X strain. Based on their extensive sensitivity phenomes, NGO1985 and NGO2121 appear to be the most promising vaccine candidates. This study establishes the principle that phenotype microarrays can be successfully applied to a fastidious bacterial organism, such as N. gonorrhoeae . IMPORTANCE Innovative approaches are required to develop vaccines against prevalent and neglected sexually transmitted infections, such as gonorrhea. Herein, we have utilized phenotype microarrays in the first such investigation into Neisseria gonorrhoeae to probe the function of proteome-derived vaccine candidates in cell envelope homeostasis. Information gained from this screening can feed the vaccine candidate decision tree by providing insights into the roles these proteins play in membrane permeability, integrity, and overall N. gonorrhoeae physiology. The optimized screening protocol can be applied in investigations into the function of other hypothetical proteins of N. gonorrhoeae discovered in the expanding number of whole-genome sequences, in addition to revealing phenotypic differences between clinical and laboratory strains.

Summary Many papers have introduced adaptive clinical trial methods that allow modifications to the sample size based on interim estimates of treatment effect. There has been extensive commentary on type I error control and efficiency considerations, but little research on estimation after an adaptive hypothesis test. We evaluate the reliability and precision of different inferential procedures in the presence of an adaptive design with pre-specified rules for modifying the sampling plan. We extend group sequential orderings of the outcome space based on the stage at stopping, likelihood ratio statistic, and sample mean to the adaptive setting in order to compute median-unbiased point estimates, exact confidence intervals, and P-values uniformly distributed under the null hypothesis. The likelihood ratio ordering is found to average shorter confidence intervals and produce higher probabilities of P-values below important thresholds than alternative approaches. The bias adjusted mean demonstrates the lowest mean squared error among candidate point estimates. A conditional error-based approach in the literature has the benefit of being the only method that accommodates unplanned adaptations. We compare the performance of this and other methods in order to quantify the cost of failing to plan ahead in settings where adaptations could realistically be pre-specified at the design stage. We find the cost to be meaningful for all designs and treatment effects considered, and to be substantial for designs frequently proposed in the literature.

Background Serum creatinine has been used as the diagnostic test for acute kidney injury (AKI) for decades despite having imperfect sensitivity and specificity. Novel tubular injury biomarkers may revolutionize the diagnosis of AKI; however, even if a novel tubular injury biomarker is 100% sensitive and 100% specific, it may appear inaccurate when using serum creatinine as the gold standard. Conclusions In general, the apparent diagnostic performance of a biomarker depends not only on its ability to detect injury but also on disease prevalence and the sensitivity and specificity of the imperfect gold standard. Apparent errors in diagnosis using a new biomarker may be a reflection of errors in the imperfect gold standard itself rather than poor performance of the biomarker.

We examined RNA-Seq data on 211 biological samples from 24 different Arabidopsis experiments carried out by different labs. We grouped the samples according to tissue types, and in each of the groups, we identified genes that are stably expressed across biological samples, treatment conditions, and experiments. We fit a Poisson log-linear mixed-effect model to the read counts for each gene and decomposed the total variance into between-sample, between-treatment and between-experiment variance components. Identifying stably expressed genes is useful for count normalization and differential expression analysis. The variance component analysis that we explore here is a first step towards understanding the sources and nature of the RNA-Seq count variation. When using a numerical measure to identify stably expressed genes, the outcome depends on multiple factors: the background sample set and the reference gene set used for count normalization, the technology used for measuring gene expression, and the specific numerical stability measure used. Since differential expression (DE) is measured by relative frequencies, we argue that DE is a relative concept. We advocate using an explicit reference gene set for count normalization to improve interpretability of DE results, and recommend using a common reference gene set when analyzing multiple RNA-Seq experiments to avoid potential inconsistent conclusions.

We introduce two simple methods for the statistical comparison of the temporal pattern of life-cycle events between two populations. The methods are based on a translation of stage-frequency data into individual 'times in stage'. For example, if the stage-k individuals in a set of samples consist of three individuals counted at time t(1) and two counted at time t(2), the observed times in stage k would be (t(1), t(1), t(1), t(2), t(2)). Times in stage then can be compared between two populations by performing stage-specific t-tests or by testing for equality of regression lines of time versus stage between the two populations. Simulations show that our methods perform at close to the nominal level, have good power against a range of alternatives, and have much better operating characteristics than a widely-used phenology model from the literature.

In their paper [1], Drs. Mehta and Pocock illustrate the use of a particular approach to revising the maximal sample size of a randomized clinical trial (RCT) by using an interim estimate of the treatment effect. Slightly extending the results of Gao, Ware, and Mehta [2], the authors define conditions on an adaptive rule such that one can know that the naive statistical hypothesis test that ignores the adaptation is conservative. They then use this knowledge to define an adaptive rule for a clinical trial. In our review of this paper, however, we do not find that such an adaptive rule confers any advantage by the usual criteria for clinical trial design. Rather, we find that the designs proposed in this paper are markedly inferior to alternative designs that the authors do not (but should) consider. By way of full disclosure, the first author of this commentary provided to the authors a signed referee’s report on an earlier version of this manuscript, and that report contained the substance (and most of the detail) of this review. In the comments to the editor accompanying that review, the first author described the dilemma that arose during that review. In essence, the methods described in the manuscript do not seem to us worthy of emulation. But on the other hand, the purpose of case studies in the statistical literature is to present an academic exposition of lessons that can be learned. From years of recreational spelunking, we have noted parallels between research and cave exploration. In both processes, explorers spend their time in the dark exploring the maze of potential leads, most often without a clear idea of where they will end up. Because the overwhelming majority of such leads are dead ends, the most useful companions to have along with you are the ones who will willingly explore the dead ends. However, they rapidly become the least useful companions if they have a tendency to explore the dead ends and then come back and tell you the leads went somewhere. Furthermore, the most important skill that any explorers can have is the ability to recognize when they are back at the beginning, lest they believe that the promising lead took them someplace new and become hopelessly lost. According to these criteria, then, the fact that we would not adopt some approach does not necessarily detract from the importance of a paper to the statistical literature. Instead, a paper’s value relates to the extent to which it contributes to our understanding of the methods, which can often be greatly enhanced by identifying dead ends and/or leads that take us back to the beginning. We note that there are several levels to what could be called the “recommended approach” in this paper. At the topmost level, it can be viewed merely as advocating the use of adaptive designs to assess the likelihood of future futility and efficacy of a clinical trial. But in illustrating that use, the authors seem also to advocate for adaptive methods resulting in sampling distributions that are less “heavy tailed” than analogous fixed sample designs (so that they can safely use naive analytic approaches), and they seem to fall prey to some of the difficulties in interpreting conditional power. We note that

Sequential analysis is frequently employed to address ethical and financial issues in clinical trials. Sequential analysis may be performed using standard group sequential designs, or, more recently, with adaptive designs that use estimates of treatment effect to modify the maximal statistical information to be collected. In the general setting in which statistical information and clinical trial costs are functions of the number of subjects used, it has yet to be established whether there is any major efficiency advantage to adaptive designs over traditional group sequential designs. In survival analysis, however, statistical information (and hence efficiency) is most closely related to the observed number of events, while trial costs still depend on the number of patients accrued. As the number of subjects may dominate the cost of a trial, an adaptive design that specifies a reduced maximal possible sample size when an extreme treatment effect has been observed may allow early termination of accrual and therefore a more cost-efficient trial. We investigate and compare the tradeoffs between efficiency (as measured by average number of observed events required), power, and cost (a function of the number of subjects accrued and length of observation) for standard group sequential methods and an adaptive design that allows for early termination of accrual. We find that when certain trial design parameters are constrained, an adaptive approach to terminating subject accrual may improve upon the cost efficiency of a group sequential clinical trial investigating time-to-event endpoints. However, when the spectrum of group sequential designs considered is broadened, the advantage of the adaptive designs is less clear.

Vibrio cholerae , an etiological agent of cholera, circulates between aquatic reservoirs and the human gastrointestinal tract. The type II secretion (T2S) system plays a pivotal role in both stages of the lifestyle by exporting multiple proteins, including cholera toxin. Here, we studied the kinetics of expression of genes encoding the T2S system and its cargo proteins. We have found that under laboratory growth conditions, the T2S complex was continuously expressed throughout V. cholerae growth, whereas there was growth phase-dependent transcriptional activity of genes encoding different cargo proteins. Moreover, exposure of V. cholerae to different environmental cues encountered by the bacterium in its life cycle induced transcriptional expression of T2S. Subsequent screening of a V. cholerae genomic library suggested that σ E stress response, phosphate metabolism, and the second messenger 3′,5′-cyclic diguanylic acid (c-di-GMP) are involved in regulating transcriptional expression of T2S. Focusing on σ E , we discovered that the upstream region of the T2S operon possesses both the consensus σ E and σ 70 signatures, and deletion of the σ E binding sequence prevented transcriptional activation of T2S by RpoE. Ectopic overexpression of σ E stimulated transcription of T2S in wild-type and isogenic Δ rpoE strains of V. cholerae , providing additional support for the idea that the T2S complex belongs to the σ E regulon. Together, our results suggest that the T2S pathway is characterized by the growth phase-dependent expression of genes encoding cargo proteins and requires a multifactorial regulatory network to ensure appropriate kinetics of the secretory traffic and the fitness of V. cholerae in different ecological niches.

We derive the analytical mean and variance of the score test statistic in gene-dropping simulations and approximate the null distribution of the test statistic by a normal distribution. We provide insights into the gene-dropping test by decomposing the test statistic into two components: the first component provides information about linkage, and the second component provides information about fine mapping under the linkage peak. We demonstrate our theoretical findings by applying the gene-dropping test to the simulated data set from Genetic Analysis Workshop 18 and comparing its performance with existing population and family-based association tests.

In many disease settings, it is likely that only a subset of the disease population will exhibit certain genetic or phenotypic differences from the healthy population. Therefore, when seeking to identify genes or other explanatory factors that might be related to the disease state, we might expect a mixture distribution of the variable of interest in the disease group. A number of methods have been proposed for performing tests to identify situations for which only a subgroup of samples or patients exhibit differential expression levels. Our discussion here focuses on how inattention to standard statistical theory can lead to approaches that exhibit some serious drawbacks. We present and discuss several approaches motivated by theoretical derivations and compare to an ad hoc approach based upon identification of outliers. We find that the outlier-sum statistic proposed by Tibshirani and Hastie offers little benefit over a t-test even in the most idealized scenarios and suffers from a number of limitations including difficulty of calibration, lack of robustness to underlying distributions, high false positive rates owing to its asymmetric treatment of groups, and poor power or discriminatory ability under many alternatives.

Many papers have introduced adaptive clinical trial methods that allow modifications to the sample size based on interim estimates of treatment effect. There has been extensive commentary on type I error control and efficiency considerations, but little research on estimation after an adaptive hypothesis test. We evaluate the reliability and precision of different inferential procedures in the presence of an adaptive design with pre-specified rules for modifying the sampling plan. We extend group sequential orderings of the outcome space based on the stage at stopping, likelihood ratio statistic, and sample mean to the adaptive setting in order to compute median-unbiased point estimates, exact confidence intervals, and P-values uniformly distributed under the null hypothesis. The likelihood ratio ordering is found to average shorter confidence intervals and produce higher probabilities of P-values below important thresholds than alternative approaches. The bias adjusted mean demonstrates the lowest mean squared error among candidate point estimates. A conditional error-based approach in the literature has the benefit of being the only method that accommodates unplanned adaptations. We compare the performance of this and other methods in order to quantify the cost of failing to plan ahead in settings where adaptations could realistically be pre-specified at the design stage. We find the cost to be meaningful for all designs and treatment effects considered, and to be substantial for designs frequently proposed in the literature.

When assessing differential gene expression from RNA sequencing data, commonly used statistical tests tend to have greater power to detect differential expression of genes encoding longer transcripts. This phenomenon, called “length bias”, will influence subsequent analyses such as Gene Ontology enrichment analysis. In the presence of length bias, Gene Ontology categories that include longer genes are more likely to be identified as enriched. These categories, however, are not necessarily biologically more relevant. We show that one can effectively adjust for length bias in Gene Ontology analysis by including transcript length as a covariate in a logistic regression model. The logistic regression model makes the statistical issue underlying length bias more transparent: transcript length becomes a confounding factor when it correlates with both the Gene Ontology membership and the significance of the differential expression test. The inclusion of the transcript length as a covariate allows one to investigate the direct correlation between the Gene Ontology membership and the significance of testing differential expression, conditional on the transcript length. We present both real and simulated data examples to show that the logistic regression approach is simple, effective, and flexible.

Clinicians have used serum creatinine in diagnostic testing for acute kidney injury for decades, despite its imperfect sensitivity and specificity. Novel tubular injury biomarkers may revolutionize the diagnosis of acute kidney injury; however, even if a novel tubular injury biomarker is 100% sensitive and 100% specific, it may appear inaccurate when using serum creatinine as the gold standard. Acute kidney injury, as defined by serum creatinine, may not reflect tubular injury, and the absence of changes in serum creatinine does not assure the absence of tubular injury. In general, the apparent diagnostic performance of a biomarker depends not only on its ability to detect injury, but also on disease prevalence and the sensitivity and specificity of the imperfect gold standard. Assuming that, at a certain cutoff value, serum creatinine is 80% sensitive and 90% specific and disease prevalence is 10%, a new perfect biomarker with a true 100% sensitivity may seem to have only 47% sensitivity compared with serum creatinine as the gold standard. Minimizing misclassification by using more strict criteria to diagnose acute kidney injury will reduce the error when evaluating the performance of a biomarker under investigation. Apparent diagnostic errors using a new biomarker may be a reflection of errors in the imperfect gold standard itself, rather than poor performance of the biomarker. The results of this study suggest that small changes in serum creatinine alone should not be used to define acute kidney injury in biomarker or interventional studies.

Adaptive clinical trial design has been proposed as a promising new approach that may improve the drug discovery process. Proponents of adaptive sample size re-estimation promote its ability to avoid 'up-front' commitment of resources, better address the complicated decisions faced by data monitoring committees, and minimize accrual to studies having delayed ascertainment of outcomes. We investigate aspects of adaptation rules, such as timing of the adaptation analysis and magnitude of sample size adjustment, that lead to greater or lesser statistical efficiency. Owing in part to the recent Food and Drug Administration guidance that promotes the use of pre-specified sampling plans, we evaluate alternative approaches in the context of well-defined, pre-specified adaptation. We quantify the relative costs and benefits of fixed sample, group sequential, and pre-specified adaptive designs with respect to standard operating characteristics such as type I error, maximal sample size, power, and expected sample size under a range of alternatives. Our results build on others' prior research by demonstrating in realistic settings that simple and easily implemented pre-specified adaptive designs provide only very small efficiency gains over group sequential designs with the same number of analyses. In addition, we describe optimal rules for modifying the sample size, providing efficient adaptation boundaries on a variety of scales for the interim test statistic for adaptation analyses occurring at several different stages of the trial. We thus provide insight into what are good and bad choices of adaptive sampling plans when the added flexibility of adaptive designs is desired.

The empirical likelihood method is a versatile approach for testing hypotheses and constructing confidence regions in a non-parametric setting. For testing the value of a vector mean, the empirical likelihood method offers the benefit of making no distributional assumptions beyond some mild moment conditions. However, in small samples or high dimensions the method is very poorly calibrated, producing tests that generally have a much higher type I error than the nominal level, and it suffers from a limiting convex hull constraint. Methods to address the performance of the empirical likelihood in the vector mean setting have been proposed in a number of papers, including a contribution that suggests supplementing the observed dataset with an artificial data point. We examine the consequences of this approach and describe a limitation of their method that we have discovered in settings when the sample size is relatively small compared with the dimension. We propose a new modification to the extra data approach that involves adding two points and changing the location of the extra points. We explore the benefits that this modification offers, and show that it results in better calibration, particularly in difficult cases. This new approach also results in a small-sample connection between the modified empirical likelihood method and Hotelling’s T-square test. We show that varying the location of the added data points creates a continuum of tests that range from the unmodified empirical likelihood statistic to Hotelling’s T-square statistic.

As the results of HIV prevention trials transition to public health practice, Susan Buchbinder and colleagues1 have provided a timely exploration of how to prioritise the targeting of pre-exposure prophylaxis (PrEP) to men who have sex with men (MSM). They have drawn attention to the need to examine the efficiency of PrEP in seroconversion prevention and the effect on the HIV epidemic that could be achieved through targeting potential subgroups of the MSM population. Buchbinder and colleagues used the efficiency metric number needed to treat and the effect metric of population-attributable fraction (PAF). However, in the context of PrEP delivery, PAF might not be the ideal metric for evaluation and interpretation of the effect of the intervention. Kenneth Rothman and colleagues2 provide a common definition of the PAF as “the reduction in incidence that would be achieved if the population had been entirely unexposed, compared to its current (actual) exposure pattern”. The PAF can be useful when assessing the effect of an intervention that modifies an exposure and potentially reduces the risk in the exposed subgroup to the level in the remaining unexposed individuals. As noted by Buchbinder and colleagues, the PAFs of modifiable traits and behaviours were previously assessed for MSM subgroups in Australia.3 The interpretation of PAF is less meaningful in a PrEP setting because the sub groups are defined according to risk factors that we do not plan to modify, but merely to use in targeting PrEP; and treatment with PrEP might be more or less efficacious than reducing the risk to the level in the non-targeted population. We suggest that the fraction of population incident cases that arise in each subgroup is more relevant and interpretable than is the PAF for judging the contribution of each subgroup to overall population incidence. An assessment of the potential effect of PrEP might then be obtained by multiplying each subgroup’s fraction of population incident cases by PrEP efficacy within that subgroup. Our suggested metric is thus given by: Fraction of incident cases averted by PrEP = (incident cases in targeted subgroup/incident cases in total population) × PrEP efficacy By contrast with the PAF, which assumes that only the excess infections in a particular subgroup can be averted, our proposed metric would assess the fraction of total infections potentially averted by targeting each subgroup. Using this metric in conjunction with the number needed to treat would allow a more direct assessment of the cost-benefit tradeoffs for targeting different subgroups of the population.