Search

AbstractData heterogeneity is a challenging issue for modern statistical data analysis. There are different types of data heterogeneity in practice. In this article, we consider potential structural changes and complicated tail distributions. There are various existing methods proposed to handle either structural changes or heteroscedasticity. However, it is difficult to handle them simultaneously. To overcome this limitation, we consider statistically and computationally efficient change point detection and localization in high-dimensional quantile regression models. Our proposed framework is general and flexible since the change points and the underlying regression coefficients are allowed to vary across different quantile levels. The model parameters, including the data dimension, the number of change points, and the signal jump size, can be scaled with the sample size. Under this framework, we construct a novel two-step estimation of the number and locations of the change points as well as the underlying regression coefficients. Without any moment constraints on the error term, we present theoretical results, including consistency of the change point number, oracle estimation of change point locations, and estimation for the underlying regression coefficients with the optimal convergence rate. Finally, we present simulation results and an application to the S&P 100 dataset to demonstrate the advantage of the proposed method. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work. [ABSTRACT FROM AUTHOR]

Statistical network analysis primarily focuses on inferring the parameters of an observed network. In many applications, especially in the social sciences, the observed data is the groups formed by individual subjects. In these applications, the network is itself a parameter of a statistical model. Zhao and Weko propose a model-based approach, called the hub model, to infer implicit networks from grouping behavior. The hub model assumes that each member of the group is brought together by a member of the group called the hub. The set of members which can serve as a hub is called the hub set. The hub model belongs to the family of Bernoulli mixture models. Identifiability of Bernoulli mixture model parameters is a notoriously difficult problem. This article proves identifiability of the hub model parameters and estimation consistency under mild conditions. Furthermore, this article generalizes the hub model by introducing a model component that allows hubless groups in which individual nodes spontaneously appear independent of any other individual. We refer to this additional component as the null component. The new model bridges the gap between the hub model and the degenerate case of the mixture model—the Bernoulli product. Identifiability and consistency are also proved for the new model. In addition, a penalized likelihood approach is proposed to estimate the hub set when it is unknown. for this article are available online. [ABSTRACT FROM AUTHOR]

Modern statistics provides an ever-expanding toolkit for estimating unknown parameters. Consequently, applied statisticians frequently face a difficult decision: retain a parameter estimate from a familiar method or replace it with an estimate from a newer or more complex one. While it is traditional to compare estimates using risk, such comparisons are rarely conclusive in realistic settings. In response, we propose the "c-value" as a measure of confidence that a new estimate achieves smaller loss than an old estimate on a given dataset. We show that it is unlikely that a large c-value coincides with a larger loss for the new estimate. Therefore, just as a small p-value supports rejecting a null hypothesis, a large c-value supports using a new estimate in place of the old. For a wide class of problems and estimates, we show how to compute a c-value by first constructing a data-dependent high-probability lower bound on the difference in loss. The c-value is frequentist in nature, but we show that it can provide validation of shrinkage estimates derived from Bayesian models in real data applications involving hierarchical models and Gaussian processes. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Testing large covariance matrices is of fundamental importance in statistical analysis with high-dimensional data. In the past decade, three types of test statistics have been studied in the literature: quadratic form statistics, maximum form statistics, and their weighted combination. It is known that quadratic form statistics would suffer from low power against sparse alternatives and maximum form statistics would suffer from low power against dense alternatives. The weighted combination methods were introduced to enhance the power of quadratic form statistics or maximum form statistics when the weights are appropriately chosen. In this article, we provide a new perspective to exploit the full potential of quadratic form statistics and maximum form statistics for testing high-dimensional covariance matrices. We propose a scale-invariant power-enhanced test based on Fisher's method to combine the p-values of quadratic form statistics and maximum form statistics. After carefully studying the asymptotic joint distribution of quadratic form statistics and maximum form statistics, we first prove that the proposed combination method retains the correct asymptotic size under the Gaussian assumption, and we also derive a new Lyapunov-type bound for the joint distribution and prove the correct asymptotic size of the proposed method without requiring the Gaussian assumption. Moreover, we show that the proposed method boosts the asymptotic power against more general alternatives. Finally, we demonstrate the finite-sample performance in simulation studies and a real application. for this article are available online. [ABSTRACT FROM AUTHOR]

A novel statistical method is proposed and investigated for estimating a heavy tailed density under mild smoothness assumptions. Statistical analyses of heavy-tailed distributions are susceptible to the problem of sparse information in the tail of the distribution getting washed away by unrelated features of a hefty bulk. The proposed Bayesian method avoids this problem by incorporating smoothness and tail regularization through a carefully specified semiparametric prior distribution, and is able to consistently estimate both the density function and its tail index at near minimax optimal rates of contraction. A joint, likelihood driven estimation of the bulk and the tail is shown to help improve uncertainty assessment in estimating the tail index parameter and offer more accurate and reliable estimates of the high tail quantiles compared to thresholding methods. for this article are available online. [ABSTRACT FROM AUTHOR]

The document is a response to a discussion about an article on the representativeness and generalizability of inference for samples of networks. The authors address feedback and agree with some suggestions, such as using simulation-based residuals and user-friendly score testing. They also suggest collecting information on family roles as predictors of contacts in future surveys. The document includes graphs and equations that analyze relationships and patterns between different age groups. It provides statistical data on age-related interactions and relationships, which can be useful for researchers. The text also discusses the challenges of using Exponential Random Graph Models (ERGMs) and provides guidance for researchers on choosing statistical models and software packages. [Extracted from the article]

The last two decades have seen considerable progress in foundational aspects of statistical network analysis, but the path from theory to application is not straightforward. Two large, heterogeneous samples of small networks of within-household contacts in Belgium were collected using two different but complementary sampling designs: one smaller but with all contacts in each household observed, the other larger and more representative but recording contacts of only one person per household. We wish to combine their strengths to learn the social forces that shape household contact formation and facilitate simulation for prediction of disease spread, while generalising to the population of households in the region. To accomplish this, we describe a flexible framework for specifying multi-network models in the exponential family class and identify the requirements for inference and prediction under this framework to be consistent, identifiable, and generalisable, even when data are incomplete; explore how these requirements may be violated in practice; and develop a suite of quantitative and graphical diagnostics for detecting violations and suggesting improvements to candidate models. We report on the effects of network size, geography, and household roles on household contact patterns (activity, heterogeneity in activity, and triadic closure). for this article are available online. [ABSTRACT FROM AUTHOR]

Massive sized survival datasets become increasingly prevalent with the development of the healthcare industry, and pose computational challenges unprecedented in traditional survival analysis use cases. In this work we analyze the UK-biobank colorectal cancer data with genetic and environmental risk factors, including a time-dependent coefficient, which transforms the dataset into "pseudo-observation" form, thus, critically inflating its size. A popular way for coping with massive datasets is downsampling them, such that the computational resources can be afforded by the researcher. Cox regression has remained one of the most popular statistical models for the analysis of survival data to-date. This work addresses the settings of right censored and possibly left-truncated data with rare events, such that the observed failure times constitute only a small portion of the overall sample. We propose Cox regression subsampling-based estimators that approximate their full-data partial-likelihood-based counterparts, by assigning optimal sampling probabilities to censored observations, and including all observed failures in the analysis. The suggested methodology is applied on the UK-biobank for building a colorectal cancer risk-prediction model, while reducing the computation time and memory requirements. Asymptotic properties of the proposed estimators are established under suitable regularity conditions, and simulation studies are carried out to evaluate their finite sample performance. for this article are available online. [ABSTRACT FROM AUTHOR]

In statistical analysis, researchers often perform coordinatewise Gaussianization such that each variable is marginally normal. The normal score transformation is a method for coordinatewise Gaussianization and is widely used in statistics, econometrics, genetics and other areas. However, few studies exist on the theoretical properties of the normal score transformation, especially in high-dimensional problems where the dimension p diverges with the sample size n. In this article, we show that the normal score transformation uniformly converges to its population counterpart even when log p = o (n / log n) . Our result can justify the normal score transformation prior to any downstream statistical method to which the theoretical normal transformation is beneficial. The same results are established for the Winsorized normal transformation, another popular choice for coordinatewise Gaussianization. We demonstrate the benefits of coordinatewise Gaussianization by studying its applications to the Gaussian copula model, the nearest shrunken centroids classifier and distance correlation. The benefits are clearly shown in theory and supported by numerical studies. Moreover, we also point out scenarios where coordinatewise Gaussinization does not help and even causes damages. We offer a general recommendation on how to use coordinatewise Gaussianization in applications. for this article are available online. [ABSTRACT FROM AUTHOR]

This article develops an incremental learning algorithm based on quadratic inference function (QIF) to analyze streaming datasets with correlated outcomes such as longitudinal data and clustered data. We propose a renewable QIF (RenewQIF) method within a paradigm of renewable estimation and incremental inference, in which parameter estimates are recursively renewed with current data and summary statistics of historical data, but with no use of any historical subject-level raw data. We compare our renewable estimation method with both offline QIF and offline generalized estimating equations (GEE) approach that process the entire cumulative subject-level data all together, and show theoretically and numerically that our renewable procedure enjoys statistical and computational efficiency. We also propose an approach to diagnose the homogeneity assumption of regression coefficients via a sequential goodness-of-fit test as a screening procedure on occurrences of abnormal data batches. We implement the proposed methodology by expanding existing Spark's Lambda architecture for the operation of statistical inference and data quality diagnosis. We illustrate the proposed methodology by extensive simulation studies and an analysis of streaming car crash datasets from the National Automotive Sampling System-Crashworthiness Data System (NASS CDS). for this article are available online. [ABSTRACT FROM AUTHOR]

Consider a finite sample from an unknown distribution over a countable alphabet. The missing mass refers to the probability of symbols that do not appear in the sample. Estimating the missing mass is a basic problem in statistics and related fields, which dates back to the early work of Laplace, and the more recent seminal contribution of Good and Turing. In this article, we introduce a generalized Good-Turing (GT) framework for missing mass estimation. We derive an upper-bound for the risk (in terms of mean squared error) and minimize it over the parameters of our framework. Our analysis distinguishes between two setups, depending on the (unknown) alphabet size. When the alphabet size is bounded from above, our risk-bound demonstrates a significant improvement compared to currently known results (which are typically oblivious to the alphabet size). Based on this bound, we introduce a numerically obtained estimator that improves upon GT. When the alphabet size holds no restrictions, we apply our suggested risk-bound and introduce a closed-form estimator that again improves GT performance guarantees. Our suggested framework is easy to apply and does not require additional modeling assumptions. This makes it a favorable choice for practical applications. for this article are available online. [ABSTRACT FROM AUTHOR]

Tests for structural breaks in time series should ideally be sensitive to breaks in the parameter of interest, while being robust to nuisance changes. Statistical analysis thus needs to allow for some form of nonstationarity under the null hypothesis of no change. In this article, estimators for integrated parameters of locally stationary time series are constructed and a corresponding functional central limit theorem is established, enabling change-point inference for a broad class of parameters under mild assumptions. The proposed framework covers all parameters which may be expressed as nonlinear functions of moments, for example kurtosis, autocorrelation, and coefficients in a linear regression model. To perform feasible inference based on the derived limit distribution, a bootstrap variant is proposed and its consistency is established. The methodology is illustrated by means of a simulation study and by an application to high-frequency asset prices. [ABSTRACT FROM AUTHOR]

In the present article, Dai et al. ([5]) studies FDR control of the GM and DS methods for generalized linear models (GLMs) under a unified framework. Specifically, Xing, Zhao, and Liu ([16]) develops the Gaussian Mirror (GM) method for controlling FDR in high-dimensional linear regressions. Wang and Ramdas ([15]) uses e-value and develops the e-BH procedure to control FDR. With the maturity of data collection techniques, the analysis of data with high-dimensional input is now routinely conducted. [Extracted from the article]

Modern scientific studies often collect datasets in the form of tensors. These datasets call for innovative statistical analysis methods. In particular, there is a pressing need for tensor clustering methods to understand the heterogeneity in the data. We propose a tensor normal mixture model approach to enable probabilistic interpretation and computational tractability. Our statistical model leverages the tensor covariance structure to reduce the number of parameters for parsimonious modeling, and at the same time explicitly exploits the correlations for better variable selection and clustering. We propose a doubly enhanced expectation–maximization (DEEM) algorithm to perform clustering under this model. Both the expectation-step and the maximization-step are carefully tailored for tensor data in order to maximize statistical accuracy and minimize computational costs in high dimensions. Theoretical studies confirm that DEEM achieves consistent clustering even when the dimension of each mode of the tensors grows at an exponential rate of the sample size. Numerical studies demonstrate favorable performance of DEEM in comparison to existing methods. [ABSTRACT FROM AUTHOR]

Predicting risks of chronic diseases has become increasingly important in clinical practice. When a prediction model is developed in a cohort, there is a great interest to apply the model to other cohorts. Due to potential discrepancy in baseline disease incidences between different cohorts and shifts in patient composition, the risk predicted by the model built in the source cohort often under- or over-estimates the risk in a new cohort. In this article, we assume the relative risks of predictors are the same between the two cohorts, and propose a novel weighted estimating equation approach to recalibrating the projected risk for the targeted population through updating the baseline risk. The recalibration leverages the knowledge about survival probabilities for the disease of interest and competing events, and summary information of risk factors from the target population. We establish the consistency and asymptotic normality of the proposed estimators. Extensive simulation demonstrate that the proposed estimators are robust, even if the risk factor distributions differ between the source and target populations, and gain efficiency if they are the same, as long as the information from the target is precise. The method is illustrated with a recalibration of colorectal cancer prediction model. [ABSTRACT FROM AUTHOR]

The Wald test remains ubiquitous in statistical practice despite shortcomings such as its inaccuracy in small samples and lack of invariance under reparameterization. This article develops on another but lesser-known shortcoming called the Hauck–Donner effect (HDE) whereby a Wald test statistic is no longer monotone increasing as a function of increasing distance between the parameter estimate and the null value. Resulting in an upward biased p-value and loss of power, the aberration can lead to very damaging consequences such as in variable selection. The HDE afflicts many types of regression models and corresponds to estimates near the boundary of the parameter space. This article presents several new results, and its main contributions are to (i) propose a very general test for detecting the HDE in the class of vector generalized linear models (VGLMs), regardless of the underlying cause; (ii) fundamentally characterize the HDE by pairwise ratios of Wald and Rao score and likelihood ratio test statistics for 1-parameter distributions with large samples; (iii) show that the parameter space may be partitioned into an interior encased by at least 5 HDE severity measures (faint, weak, moderate, strong, extreme); (iv) prove that a necessary condition for the HDE in a 2 by 2 table is a log odds ratio of at least 2; (v) give some practical guidelines about HDE-free hypothesis testing. Overall, practical post-fit tests can now be conducted potentially to any model estimated by iteratively reweighted least squares, especially the GLM and VGLM classes, the latter which encompasses many popular regression models. [ABSTRACT FROM AUTHOR]

One fundamental statistical question for research areas such as precision medicine and health disparity is about discovering effect modification of treatment or exposure by observed covariates. We propose a semiparametric framework for identifying such effect modification. Instead of using the traditional outcome models, we directly posit semiparametric models on contrasts, or expected differences of the outcome under different treatment choices or exposures. Through semiparametric estimation theory, all valid estimating equations, including the efficient scores, are derived. Besides doubly robust loss functions, our approach also enables dimension reduction in presence of many covariates. The asymptotic and non-asymptotic properties of the proposed methods are explored via a unified statistical and algorithmic analysis. Comparison with existing methods in both simulation and real data analysis demonstrates the superiority of our estimators especially for an efficiency improved version. for this article are available online. [ABSTRACT FROM AUTHOR]

We review the most important statistical ideas of the past half century, which we categorize as: counterfactual causal inference, bootstrapping and simulation-based inference, overparameterized models and regularization, Bayesian multilevel models, generic computation algorithms, adaptive decision analysis, robust inference, and exploratory data analysis. We discuss key contributions in these subfields, how they relate to modern computing and big data, and how they might be developed and extended in future decades. The goal of this article is to provoke thought and discussion regarding the larger themes of research in statistics and data science. [ABSTRACT FROM AUTHOR]

In 1975, John W. Tukey defined statistical data depth as a function that determines the centrality of an arbitrary point with respect to a data cloud or to a probability measure. During the last decades, this seminal idea of data depth evolved into a powerful tool proving to be useful in various fields of science. Recently, extending the notion of data depth to the functional setting attracted a lot of attention among theoretical and applied statisticians. We go further and suggest a notion of data depth suitable for data represented as curves, or trajectories, which is independent of the parameterization. We show that our curve depth satisfies theoretical requirements of general depth functions that are meaningful for trajectories. We apply our methodology to diffusion tensor brain images and also to pattern recognition of handwritten digits and letters. for this article are available online. [ABSTRACT FROM AUTHOR]

One example is of course Graph HT ht . I want to first reemphasize that there is nothing special about the original population (Graph HT ht ) with regard to curvature: it may better, it may be worse. There appears to be no method that adapts to each setting except for the oracle Graph HT ht method, which is of course not implementable. The flat-value-surface example that LLL pose at the end of their Section 2.2 can happen for I w i = I w i SB 0 sb just as much as it can happen for Graph HT ht . [Extracted from the article]

Introducing some notation, let us define an empirical distribution with values Graph HT ht as Graph HT ht , where we let I SB a sb i denote point mass at a value I a i . Suppose one wishes to obtain predictions now based on Graph HT ht . Then we begin by splitting treated individuals (of size Graph HT ht ) into training and test samples of size I n SB a sb i and I n SB b sb i , respectively, where Graph HT ht . [Extracted from the article]

Large-scale genome-wide association studies (GWAS) provide opportunities for developing genetic risk prediction models that have the potential to improve disease prevention, intervention or treatment. The key step is to develop polygenic risk score (PRS) models with high predictive performance for a given disease, which typically requires a large training dataset for selecting truly associated single nucleotide polymorphisms (SNPs) and estimating effect sizes accurately. Here, we develop a comprehensive penalized regression for fitting l1 regularized regression models to GWAS summary statistics. We propose incorporating pleiotropy and annotation information into PRS (PANPRS) development through suitable formulation of penalty functions and associated tuning parameters. Extensive simulations show that PANPRS performs equally well or better than existing PRS methods when no functional annotation or pleiotropy is incorporated. When functional annotation data and pleiotropy are informative, PANPRS substantially outperforms existing PRS methods in simulations. Finally, we applied our methods to build PRS for type 2 diabetes and melanoma and found that incorporating relevant functional annotations and GWAS of genetically related traits improved prediction of these two complex diseases. for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement. [ABSTRACT FROM AUTHOR]

Abstract–In functional data analysis, data are commonly assumed to be smooth functions on a fixed interval of the real line. In this work, we introduce a comprehensive framework for the analysis of functional data, whose domain is a two-dimensional manifold and the domain itself is subject to variability from sample to sample. We formulate a statistical model for such data, here called functions on surfaces, which enables a joint representation of the geometric and functional aspects, and propose an associated estimation framework. We assess the validity of the framework by performing a simulation study and we finally apply it to the analysis of neuroimaging data of cortical thickness, acquired from the brains of different subjects, and thus lying on domains with different geometries. for this article are available online. [ABSTRACT FROM AUTHOR]

The era of big data has witnessed an increasing availability of multiple data sources for statistical analyses. We consider estimation of causal effects combining big main data with unmeasured confounders and smaller validation data with on these confounders. Under the unconfoundedness assumption with completely observed confounders, the smaller validation data allow for constructing consistent estimators for causal effects, but the big main data can only give error-prone estimators in general. However, by leveraging the information in the big main data in a principled way, we can improve the estimation efficiencies yet preserve the consistencies of the initial estimators based solely on the validation data. Our framework applies to asymptotically normal estimators, including the commonly used regression imputation, weighting, and matching estimators, and does not require a correct specification of the model relating the unmeasured confounders to the observed variables. We also propose appropriate bootstrap procedures, which makes our method straightforward to implement using software routines for existing estimators. for this article are available online. [ABSTRACT FROM AUTHOR]

For a given Graph HT ht , we would like to predict what Graph HT ht would be. Later we relax this requirement and only assume that Graph HT ht relates to Graph HT ht the same way as Graph HT ht relates to Graph HT ht , but Graph HT ht follows a marginal distribution that is different from that of Graph HT ht . [Extracted from the article]

What does statistics have to offer science and society, in this age of massive data, machine learning algorithms, and multiple online sources of tools for data analysis? I recall a few situations where statistics made a real difference and reinforced the impact of our discipline on society. Sometimes the difference lay in the insightful analysis and inference enabled by ground-breaking methods in our field like hypothesis testing, likelihood ratios, Bayesian models, jackknife, and bootstrap. But perhaps more often, the impacts came from thoughtful analyses before data were collected, and the questions that arose after the statistical analysis. The impact of understanding the problem, designing the experiment and data collections, conducting the pilot surveys, and raising important questions, is substantial. Through sensible explorations following formal statistical procedures, statisticians have made contributions in many domains. In this presentation, I recall some examples which made a long-lasting impact. Some of them, like randomization in clinical trials, known and familiar to all, are so ingrained in our practice that the role of statistics has been forgotten. Others may be less familiar but nonetheless benefited greatly from the critical input of statisticians. All remind us that our field remains today not only relevant but critical to science and society. [ABSTRACT FROM AUTHOR]

The populations of interest in modern studies are very often heterogeneous. The population heterogeneity, the qualitative nature of the outcome variable and the high dimensionality of the predictors pose significant challenge in statistical analysis. In this article, we introduce a category-adaptive screening procedure with high-dimensional heterogeneous data, which is to detect category-specific important covariates. The proposal is a model-free approach without any specification of a regression model and an adaptive procedure in the sense that the set of active variables is allowed to vary across different categories, thus making it more flexible to accommodate heterogeneity. For response-selective sampling data, another main discovery of this article is that the proposed method works directly without any modification. Under mild regularity conditions, the newly procedure is shown to possess the sure screening and ranking consistency properties. Simulation studies contain supportive evidence that the proposed method performs well under various settings and it is effective to extract category-specific information. Applications are illustrated with two real datasets. for this article are available online. [ABSTRACT FROM AUTHOR]

To avoid measure-theoretic considerations when writing conditioning statements, I will consider the treatments Graph HT ht to be discrete. I will write observed outcomes as Graph HT ht , where, under the stable unit treatment value assumption (SUTVA), Graph HT ht . Throughout, I will consider models of the joint distribution Graph HT ht , which I will refer to as latent variable models. For example, the setting of Theorem 7 can be framed as a problem where the unobserved confounder I Z i is measured with proxies Graph HT ht . [Extracted from the article]

In particular, the mapping Graph HT ht is the inverse mapping of the data-generating Equation (1). The elegantly constructed I i in Proposition 3.2 has components proportional to the inverse of the exponentiated directed graph path "value", minimized over ratios of components Graph HT ht over all Graph HT ht . The exposition of the development of the ideas is a series of geometric arguments that directly relate ratios of components of I i to ratios of the values of components of the Graph HT ht via Graph HT ht . [Extracted from the article]

It should, however, be noted that such approximations have ultimately allowed the authors to conduct a comprehensive analysis of thousands of images from multiple studies, which further substantiates their findings. We congratulate the authors on an interesting paper which combines the ideas of shape analysis with longitudinal data analysis. This, of course, loses some of the elegance of the SRNF framework, but at a considerable ease of preserving the topology of the data and avoiding Euclidean approximations. [Extracted from the article]

Dialect variation is of considerable interest in linguistics and other social sciences. However, traditionally it has been studied using proxies (transcriptions) rather than acoustic recordings directly. We introduce novel statistical techniques to analyze geolocalized speech recordings and to explore the spatial variation of pronunciations continuously over the region of interest, as opposed to traditional isoglosses, which provide a discrete partition of the region. Data of this type require an explicit modeling of the variation in the mean and the covariance. Usual Euclidean metrics are not appropriate, and we therefore introduce the concept of d-covariance, which allows consistent estimation both in space and at individual locations. We then propose spatial smoothing for these objects which accounts for the possibly nonconvex geometry of the domain of interest. We apply the proposed method to data from the spoken part of the British National Corpus, deposited at the British Library, London, and we produce maps of the dialect variation over Great Britain. In addition, the methods allow for acoustic reconstruction across the domain of interest, allowing researchers to listen to the statistical analysis. for this article are available online. [ABSTRACT FROM AUTHOR]

Despite the risk of misspecification they are tied to, parametric models continue to be used in statistical practice because they are simple and convenient to use. In particular, efficient estimation procedures in parametric models are easy to describe and implement. Unfortunately, the same cannot be said of semiparametric and nonparametric models. While the latter often reflect the level of available scientific knowledge more appropriately, performing efficient inference in these models is generally challenging. The efficient influence function is a key analytic object from which the construction of asymptotically efficient estimators can potentially be streamlined. However, the theoretical derivation of the efficient influence function requires specialized knowledge and is often a difficult task, even for experts. In this article, we present a novel representation of the efficient influence function and describe a numerical procedure for approximating its evaluation. The approach generalizes the nonparametric procedures of Frangakis et al. and Luedtke, Carone, and van der Laan to arbitrary models. We present theoretical results to support our proposal and illustrate the method in the context of several semiparametric problems. The proposed approach is an important step toward automating efficient estimation in general statistical models, thereby rendering more accessible the use of realistic models in statistical analyses. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

This work is concerned with the analysis of marketing data on the activation of applications (apps) on mobile devices. Each application has a hashed identification number that is specific to the device on which it has been installed. This number can be registered by a platform at each activation of the application. Activations on the same device are linked together using the identification number. By focusing on activations that took place at a business location, one can create a capture-recapture dataset about devices, that is, users, that "visited" the business: the units are owners of mobile devices and the capture occasions are time intervals such as days. A unit is captured when she activates an application, provided that this activation is recorded by the platform providing the data. Statistical capture-recapture techniques can be applied to the app data to estimate the total number of users that visited the business over a time period, thereby providing an indirect estimate of foot traffic. This article argues that the robust design, a method for dealing with a nested mark-recapture experiment, can be used in this context. A new algorithm for estimating the parameters of a robust design with a fairly large number of capture occasions and a simple parametric bootstrap variance estimator are proposed. Moreover, new estimation methods and new theoretical results are introduced for a wider application of the robust design. This is used to analyze a dataset about the mobile devices that visited the auto-dealerships of a major auto brand in a U.S. metropolitan area over a period of 1 year and a half. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement. [ABSTRACT FROM AUTHOR]

Extraordinary amounts of data are being produced in many branches of science. Proven statistical methods are no longer applicable with extraordinary large datasets due to computational limitations. A critical step in big data analysis is data reduction. Existing investigations in the context of linear regression focus on subsampling-based methods. However, not only is this approach prone to sampling errors, it also leads to a covariance matrix of the estimators that is typically bounded from below by a term that is of the order of the inverse of the subdata size. We propose a novel approach, termed information-based optimal subdata selection (IBOSS). Compared to leading existing subdata methods, the IBOSS approach has the following advantages: (i) it is significantly faster; (ii) it is suitable for distributed parallel computing; (iii) the variances of the slope parameter estimators converge to 0 as the full data size increases even if the subdata size is fixed, that is, the convergence rate depends on the full data size; (iv) data analysis for IBOSS subdata is straightforward and the sampling distribution of an IBOSS estimator is easy to assess. Theoretical results and extensive simulations demonstrate that the IBOSS approach is superior to subsampling-based methods, sometimes by orders of magnitude. The advantages of the new approach are also illustrated through analysis of real data. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

This article investigates the problem of matrix completion from the corrupted data, when the additional covariates are available. Despite being seldomly considered in the matrix completion literature, these covariates often provide valuable information for completing the unobserved entries of the high-dimensional target matrix A0. Given a covariate matrix X with its rows representing the row covariates of A0, we consider a column-space-decomposition model A0 = Xβ0 + B0, where β0 is a coefficient matrix and B0 is a low-rank matrix orthogonal to X in terms of column space. This model facilitates a clear separation between the interpretable covariate effects (Xβ0) and the flexible hidden factor effects (B0). Besides, our work allows the probabilities of observation to depend on the covariate matrix, and hence a missing-at-random mechanism is permitted. We propose a novel penalized estimator for A0 by utilizing both Frobenius-norm and nuclear-norm regularizations with an efficient and scalable algorithm. Asymptotic convergence rates of the proposed estimators are studied. The empirical performance of the proposed methodology is illustrated via both numerical experiments and a real data application. [ABSTRACT FROM AUTHOR]

One potential disadvantage of this sketched method is that it may be difficult to correctly specify Graph HT ht so that Graph HT ht while the maximization problem remains easy to solve. All the above results can be easily adapted to the case where the estimand of interest is instead an AEE Graph HT ht for a reference rule I e SB r sb i . Let Graph HT ht be a true optimal ITR in Graph HT ht . [Extracted from the article]

Suppose we want to infer the conditional independence relationships among components of a random vector Graph HT ht . Then Graph HT ht if either Graph HT ht or Graph HT ht is nonzero. Alternatively, one may choose to include an edge only when both Graph HT ht and Graph HT ht are nonzero. [Extracted from the article]

Importantly, I y i are observations on I w i not on Graph HT ht : Graph HT ht (3) which is the "data model" in an HM. There are instances (e.g., when considering the output from numerical climate models) when it is the surface Graph HT ht , not I w i , that one wants to predict. Then for each decision I i , Graph HT ht is an ensemble of utilities, where we can weight each one by its posterior probability (as we do in (6)). Integrating (8) yields, Graph HT ht (9) which when maximized yields Graph HT ht that is a function of I y i . [Extracted from the article]

We present false discovery rate (FDR) smoothing, an empirical-Bayes method for exploiting spatial structure in large multiple-testing problems. FDR smoothing automatically finds spatially localized regions of significant test statistics. It then relaxes the threshold of statistical significance within these regions, and tightens it elsewhere, in a manner that controls the overall false discovery rate at a given level. This results in increased power and cleaner spatial separation of signals from noise. The approach requires solving a nonstandard high-dimensional optimization problem, for which an efficient augmented-Lagrangian algorithm is presented. In simulation studies, FDR smoothing exhibits state-of-the-art performance at modest computational cost. In particular, it is shown to be far more robust than existing methods for spatially dependent multiple testing. We also apply the method to a dataset from an fMRI experiment on spatial working memory, where it detects patterns that are much more biologically plausible than those detected by standard FDR-controlling methods. All code for FDR smoothing is publicly available in Python and R (https://github.com/tansey/smoothfdr). Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Transmission blocking vaccines for malaria are not designed to directly protect vaccinated people from malaria disease, but to reduce the probability of infecting other people by interfering with the growth of the malaria parasite in mosquitoes. Standard membrane-feeding assays compare the growth of parasites in mosquitoes from a test sample (using antibodies from a vaccinated person) compared to a control sample. There is debate about whether to estimate the transmission reducing activity (TRA) which compares the mean number of parasites between test and control samples, or transmission blocking activity (TBA) which compares the proportion of infected mosquitoes. TBA appears biologically more important since each mosquito with any parasites is potentially infective; however, TBA is less reproducible and may be an overly strict criterion for screening vaccine candidates. Through a statistical model, we show that the TBA estimand depends on μc, the mean number of parasites in the control mosquitoes, a parameter not easily experimentally controlled. We develop a standardized TBA estimator based on the model and a given target value for μc which has better mean squared error than alternative methods. We discuss types of statistical inference needed for using these assays for vaccine development. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

This article considers causal inference for treatment contrasts from a randomized experiment using potential outcomes in a finite population setting. Adopting a Neymanian repeated sampling approach that integrates such causal inference with finite population survey sampling, an inferential framework is developed for general mechanisms of assigning experimental units to multiple treatments. This framework extends classical methods by allowing the possibility of randomization restrictions and unequal replications. Novel conditions that are “milder” than strict additivity of treatment effects, yet permit unbiased estimation of the finite population sampling variance of any treatment contrast estimator, are derived. The consequences of departures from such conditions are also studied under the criterion of minimax bias, and a new justification for using the Neymanian conservative sampling variance estimator in experiments is provided. The proposed approach can readily be extended to the case of treatments with a general factorial structure. [ABSTRACT FROM AUTHOR]

This article investigates the problem whether the difference between two parametric models m1, m2 describing the relation between a response variable and several covariates in two different groups is practically irrelevant, such that inference can be performed on the basis of the pooled sample. Statistical methodology is developed to test the hypotheses H0: d(m1, m2) ⩾ ϵ versus H1: d(m1, m2) < ϵ to demonstrate equivalence between the two regression curves m1, m2 for a prespecified threshold ϵ, where d denotes a distance measuring the distance between m1 and m2. Our approach is based on the asymptotic properties of a suitable estimator of this distance. To improve the approximation of the nominal level for small sample sizes, a bootstrap test is developed, which addresses the specific form of the interval hypotheses. In particular, data have to be generated under the null hypothesis, which implicitly defines a manifold for the parameter vector. The results are illustrated by means of a simulation study and a data example. It is demonstrated that the new methods substantially improve currently available approaches with respect to power and approximation of the nominal level. [ABSTRACT FROM AUTHOR]

The author offers a commentary on an article by Professors J. Duchi, M. I. Jordan and M. J. Wainwright which offered an extensive mapping of the statistical power of differential privacy in the local model and an introduction to differential privacy in the language of modern statistics. He discusses the distinction between the centralized model and the local model, circumventing lower bounds in the local model, and implication of the article for the broader statistics community.

The authors offer a response to the commentaries made on their article on statistical estimation and privacy. They discuss the issues raised by the discussants including the advantages and disadvantages of local notions of privacy as compared to classical differential and approximate differential privacy, the issue of interactivity, and the importance of local differential privacy. The goal of statistical inference is also mentioned.

Numerical climate model simulations run at high spatial and temporal resolutions generate massive quantities of data. As our computing capabilities continue to increase, storing all of the data is not sustainable, and thus it is important to develop methods for representing the full datasets by smaller compressed versions. We propose a statistical compression and decompression algorithm based on storing a set of summary statistics as well as a statistical model describing the conditional distribution of the full dataset given the summary statistics. We decompress the data by computing conditional expectations and conditional simulations from the model given the summary statistics. Conditional expectations represent our best estimate of the original data but are subject to oversmoothing in space and time. Conditional simulations introduce realistic small-scale noise so that the decompressed fields are neither too smooth nor too rough compared with the original data. Considerable attention is paid to accurately modeling the original dataset—1 year of daily mean temperature data—particularly with regard to the inherent spatial nonstationarity in global fields, and to determining the statistics to be stored, so that the variation in the original data can be closely captured, while allowing for fast decompression and conditional emulation on modest computers. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

In these cases getting accurate confidence intervals is of first order importance, and the methods Ignatiadis and Wager develop are likely to be useful. I want to congratulate Nikolaos Ignatiadis and Stefan Wager on a very stimulating paper on a timely and important topic. [Extracted from the article]

We agree that converting data to similar resolutions would lose some statistical information from our data. We thank gratefully Professors Wei Tu, Bei Jiang, Linglong Kong, and Professor Sudipto Banerjee for their insightful comments and discussion of our paper. The inference of the occupancy status of houses sharing one location might require more information at the individual level, which is the primary data barrier in social science. [Extracted from the article]

The article presents comment on the study regarding generalized additive model (GAM) technique. It outlines the general settings and implementation of the model along with its computational tricks and inferential products. It also commends the researchers for developing a more sophisticated level of the methods to higher realms of generality and functionality.

The article presents response to several comments regarding the study on statistical methodology in personalized medicine research. It recognizes important insights and questions raised in the comments presented. It also provides responses to the most of the main points raised and concluding thoughts on the study.

The article presents comment on the study regarding the parameter and model selection for general smooth models. It explores the development of a framework which is flexible and more likely to shape the routine use of smooth models. It also commends the researchers for their significant contribution to the development of smooth models and estimation.