Search

Panel data model with fixed effects is widely used in economic and administrative applications. However, the presence of factors: measurement errors, data variability and outliers may potentially decrease the accuracy of the model prediction. In this paper, we use panel interval-valued data to represent measurement errors and data volatility of observations. Further, we propose a corresponding panel interval-valued data model with fixed effects, in which both the response and explanatory variables are interval-valued data. To reduce the impact of outliers on our model, we propose a robust estimation method based on the iterative weighted least squares technique. Later, Monte Carlo simulation and empirical application demonstrate that our model is a suitable tool for analyzing the behaviour of panel interval-valued data. [ABSTRACT FROM AUTHOR]

Bayesian phylogenetic inference is powerful but computationally intensive. Researchers may find themselves with two phylogenetic posteriors on overlapping data sets and may wish to approximate a combined result without having to re-run potentially expensive Markov chains on the combined data set. This raises the question: given overlapping subsets of a set of taxa (e.g. species or virus samples), and given posterior distributions on phylogenetic tree topologies for each of these taxon sets, how can we optimize a probability distribution on phylogenetic tree topologies for the entire taxon set? In this paper we develop a variational approach to this problem and demonstrate its effectiveness. Specifically, we develop an algorithm to find a suitable support of the variational tree topology distribution on the entire taxon set, as well as a gradient-descent algorithm to minimize the divergence from the restrictions of the variational distribution to each of the given per-subset probability distributions, in an effort to approximate the posterior distribution on the entire taxon set. [ABSTRACT FROM AUTHOR]

It is of great interest to compute optimal exact confidence intervals for the success probability (p) in a binomial distribution, the number of subjects with a certain attribute (M) or the total number of subjects (N) in a hypergeometric distribution, and the mean λ of a Poisson distribution. In this paper, efficient algorithms are proposed to compute an admissible exact interval for each of the four parameters when the sample size (n) or the random observation X is large. The algorithms are utilized in four practical examples: evaluating the relationship between two diseases, certifying companies, estimating the proportion of drug users, and analyzing earthquake frequency. The intervals computed by the algorithms are shorter, and the calculations are faster, demonstrating the accuracy of the results and the time efficiency of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Recently, two-way or longitudinal functional data analysis has attracted much attention in many fields. However, little is known on how to appropriately characterize the association between two-way functional predictor and scalar response. Motivated by a mortality study, in this paper, we propose a novel two-way functional linear model, where the response is a scalar and functional predictor is two-way trajectory. The model is intuitive, interpretable and naturally captures relationship between each way of two-way functional predictor and scalar-type response. Further, we develop a new estimation method to estimate the regression functions in the framework of weak separability. The main technical tools for the construction of the regression functions are product functional principal component analysis and iterative least square procedure. The solid performance of our method is demonstrated in extensive simulation studies. We also analyze the mortality dataset to illustrate the usefulness of the proposed procedure. [ABSTRACT FROM AUTHOR]

This paper provides a new perspective for robust estimation and inference methods based on Huber regression. An efficient distributed learning with sparsity is proposed, in which the observations are randomly divided among different machines. We modify the proximal alternating direction method of multiplies (ADMM) algorithm to calculate the sparse penalized Huber regression. The convergence property of the algorithm is established. Computationally, our proposed approach requires only a central machine to solve a SCAD or adaptive lasso penalized M-estimation problem, and other node machines to calculate gradients based on local datasets. In terms of communication, the estimation error of our method decreases to that of the centralized method in several rounds of communication. Simulation studies are conducted to evaluate the finite-sample performance of our proposed method. The empirical study shows that our method has good practicability in the practice of a Communities and Crime analysis. [ABSTRACT FROM AUTHOR]

Classification trees are one of the most common models in interpretable machine learning. Although such models are usually built with greedy strategies, in recent years, thanks to remarkable advances in mixed-integer programming (MIP) solvers, several exact formulations of the learning problem have been developed. In this paper, we argue that some of the most relevant ones among these training models can be encapsulated within a general framework, whose instances are shaped by the specification of loss functions and regularizers. Next, we introduce a novel realization of this framework: specifically, we consider the logistic loss, handled in the MIP setting by a piece-wise linear approximation, and couple it with ℓ 1 -regularization terms. The resulting optimal logistic classification tree model numerically proves to be able to induce trees with enhanced interpretability properties and competitive generalization capabilities, compared to the state-of-the-art MIP-based approaches. [ABSTRACT FROM AUTHOR]

Biclustering is widely used in different kinds of fields including gene information analysis, text mining, and recommendation system by effectively discovering the local correlation between samples and features. However, many biclustering algorithms will collapse when facing heavy-tailed data. In this paper, we propose a robust version of convex biclustering algorithm with Huber loss. Yet, the newly introduced robustification parameter brings an extra burden to selecting the optimal parameters. Therefore, we propose a tuning-free method for automatically selecting the optimal robustification parameter with high efficiency. The simulation study demonstrates the more fabulous performance of our proposed method than traditional biclustering methods when encountering heavy-tailed noise. A real-life biomedical application is also presented. The R package RcvxBiclustr is available at https://github.com/YifanChen3/RcvxBiclustr. [ABSTRACT FROM AUTHOR]

With the advent of massive data sets, much of the computational science and engineering community has moved toward data-intensive approaches in regression and classification. However, these present significant challenges due to increasing size, complexity, and dimensionality of the problems. In particular, covariance matrices in many cases are numerically unstable, and linear algebra shows that often such matrices cannot be inverted accurately on a finite precision computer. A common ad hoc approach to stabilizing a matrix is application of a so-called nugget. However, this can change the model and introduce error to the original solution. It is well known from numerical analysis that ill-conditioned matrices cannot be accurately inverted. In this paper, we develop a multilevel computational method that scales well with the number of observations and dimensions. A multilevel basis is constructed adapted to a kd-tree partitioning of the observations. Numerically unstable covariance matrices with large condition numbers can be transformed into well-conditioned multilevel ones without compromising accuracy. Moreover, it is shown that the multilevel prediction exactly solves the best linear unbiased predictor (BLUP) and generalized least squares (GLS) model, but is numerically stable. The multilevel method is tested on numerically unstable problems of up to 25 dimensions. Numerical results show speedups of up to 42,050 times for solving the BLUP problem, but with the same accuracy as the traditional iterative approach. For very ill-conditioned cases, the speedup is infinite. In addition, decay estimates of the multilevel covariance matrices are derived based on high dimensional interpolation techniques from the field of numerical analysis. This work lies at the intersection of statistics, uncertainty quantification, high performance computing, and computational applied mathematics. [ABSTRACT FROM AUTHOR]

The mixed distribution model is often used to extract information from heterogeneous data and perform modeling analysis. When the density function of mixed distribution is complicated or the variable dimension is high, it usually brings challenges to the parameter estimation of the mixed distribution model. The application of MM algorithm can avoid complex expectation calculations, and can also solve the problem of high-dimensional optimization by decomposing the objective function. In this paper, MM algorithm is applied to the parameter estimation problem of mixed distribution model. The method of assembly and decomposition is used to construct the substitute function with separable parameters, which avoids the problems of complex expectation calculations and the inversion of high-dimensional matrices. [ABSTRACT FROM AUTHOR]

We propose a new time series model called Rational Gaussian Noise (rGn) to describe a pattern of long-range dependence. The rGn model is shown to be an extension of the traditional fractional Gaussian noise (fGn). Theoretical formulas such as autocorrelation function, and some properties of rGn are derived and compared to that of fGn. Transformed S&P500 daily excessive return data is used as a case study where parameters for both the rGn and fGn models are estimated. [ABSTRACT FROM AUTHOR]

In recent years, supervised machine learning methods have increased attention to extracting clinically relevant information from complex health data. Ensemble learning methods enable the establishment of more successful models by training multiple learners jointly to solve the same problem. Herein, we aimed to compare the performance of classification algorithms after data preprocessing to problems such as missing data, class noise, and class imbalance that may be encountered in the datasets used to make an accurate disease diagnosis. To this end, we used random forest and weighted subspace random forest as bagging algorithms while additive logistic regression and gradient boosted machines algorithms were used as boosting algorithms. The performance and running time of the algorithms were also calculated. Our findings indicated that the performance of algorithms increased after data preprocessing and the performance of boosting algorithms yielded higher results than the bagging algorithms. We also observed that the boosting algorithms were the longest-running ones. In conclusion, complementing existing studies, our work highlights the importance and effect of using multiple data preprocessing methods together. [ABSTRACT FROM AUTHOR]

Several statistical models have been proposed in recent years, among them is the semiparametric regression. In medicine, there are several situations in which it is impracticable to consider a linear regression for statistical modeling, especially when the data contain explanatory variables that present a nonlinear relationship with the response variable. Another common situation is when the response variable does not have a unimodal shape, and it is not possible to adopt distributions belonging to the symmetric or asymmetric classes. In this context, a semiparametric heteroskedastic regression is proposed based on an extension of the normal distribution. Then, we show the usefulness of this model to analyze the cost of prostate cancer surgery. The predictor variables refer to two groups of patients such that one group receives a multimodal local anesthetic solution (Preemptive Target Anesthetic Solution) and the second group is treated with neuraxial blockade (spinal anesthesia/traditional standard). The other relevant predictor variables are also evaluated, thus allowing for the in-depth interpretation of the predictor variables with a nonlinear effect on the dependent variable cost. The penalized maximum likelihood method is adopted to estimate the model parameters. The new regression is a useful statistical tool for analyzing medical data. [ABSTRACT FROM AUTHOR]

Despite the vast advantages of making antenatal care visits, the service utilization among pregnant women in Nigeria is suboptimal. A five-year monitoring estimate indicated that about 24% of the women who had live births made no visit. The non-utilization induced excessive zeroes in the outcome of interest. Thus, this study adopted a zero-inflated negative binomial model within a Bayesian framework to identify the spatial pattern and the key factors hindering antenatal care utilization in Nigeria. We overcome the intractability associated with posterior inference by adopting a Pólya-Gamma data-augmentation technique to facilitate inference. The Gibbs sampling algorithm was used to draw samples from the joint posterior distribution. Results revealed that type of place of residence, maternal level of education, access to mass media, household work index, and woman's working status have significant effects on the use of antenatal care services. Findings identified substantial state-level spatial disparity in antenatal care utilization across the country. Cost-effective techniques to achieve an acceptable frequency of utilization include the creation of a community-specific awareness to emphasize the importance and benefits of the appropriate utilization. Special consideration should be given to older pregnant women, women in poor antenatal utilization states, and women residing in poor road network regions. [ABSTRACT FROM AUTHOR]

An efficient random variable generator for a truncated gamma distribution with shape parameter greater than 1 is designed using an acceptance-rejection algorithm. Based on an approximation to a transformed gamma density function by the standard normal density, numerical information for the standard normal density is prepared in advance, and the calculation is performed with reference to that information. An improvement via a squeezing method is proposed to reduce the computational burden and time. The algorithm's acceptance rate for generating truncated gamma variables is very high and almost 1 when the truncated distribution is unimodal. Numerical experiments for one- and two-sided truncated domain cases are conducted to measure the execution time, including the parameter setup time. Compared with existing truncated gamma variate generators, the proposed method performs better when the distribution is unimodal and the shape parameter is equal to or greater than 3.3. [ABSTRACT FROM AUTHOR]

Current methods for clustering adult obesity prevalence by state focus on creating a single map of obesity prevalence for a given year in the United States. Comparing these maps for different years may limit our understanding of the progression of state and regional obesity prevalence over time for the purpose of developing targeted regional health policies. In this application note, we adopt the non-parametric Dynamic Time Warping method for clustering longitudinal time series of obesity prevalence by state. This method captures the lead and lag relationship between the time series as part of the temporal alignment, allowing us to produce a single map that captures the regional and temporal clusters of obesity prevalence from 1990 to 2019 in the United States. We identify six regions of obesity prevalence in the United States and forecast future estimates of obesity prevalence based on ARIMA models. [ABSTRACT FROM AUTHOR]

Many imputation methods have been developed over the years and tested mostly under ideal settings. Surprisingly, there is no detailed research on how imputation methods perform when the idealized assumptions about the distribution of data and/or model assumptions are partly not fulfilled. This research looks into the susceptibility of imputation techniques, particularly in relation to outliers, misclassifications, and incorrect model specifications. This is crucial knowledge about how well the methods convince in everyday life because, in reality, conditions are usually not ideal, and model assumptions may not hold. The data may not fit the defined models well. Outliers distort the estimates, and misclassifications reduce the quality of most imputation methods. Several different evaluation measures are discussed, from comparing imputed values with true values or comparing certain statistics, from the performance of classifiers to the variance of estimated parameters. Some well-known imputation methods are compared based on real data and simulations. It turns out that robust conditional imputation methods outperform other methods for real data and simulation settings. [ABSTRACT FROM AUTHOR]

In this paper, we consider aggregated functional data composed by a linear combination of component curves and the problem of estimating these component curves. We propose the application of a bayesian wavelet shrinkage rule based on a mixture of a point mass function at zero and the logistic distribution as prior to wavelet coefficients to estimate mean curves of components. This procedure has the advantage of estimating component functions with important local characteristics such as discontinuities, spikes and oscillations for example, due the features of wavelet basis expansion of functions. Simulation studies were done to evaluate the performance of the proposed method, and its results are compared with a spline-based method. An application on the so-called Tecator dataset is also provided. [ABSTRACT FROM AUTHOR]

We develop a new approach to creating covariance functions for Gaussian random fields via point processes on the complex plane. We present two approaches to construct valid covariance functions by exploiting Bochner's theorem and then modeling the characteristic function of a covariance function. In particular, we use a complex point process (CPP) to model the Fourier coefficients and illustrate how to estimate the covariance function of a Gaussian random field model from data. We further illustrate our construction approaches and compare several algorithms via simulations. The methods are exemplified via applications to real-life research data in wheat yields and earthquake studies. [ABSTRACT FROM AUTHOR]

Nowadays, to improve animal well being in livestock farming application, a wireless video sensor network (WVSN) can be deployed to early detect injury and monitor animals. They are composed of small embedded video and camera motes that capture video frames periodically and send them to a specific node called a sink. Sending all the captured images to the sink consumes a lot of energy on every sensor and may cause a bottleneck at the sink level. Energy consumption and bandwidth limitation are two important challenges in WVSNs because of the limited energy resources of the nodes and the medium scarcity. In this work, we introduce two mechanisms to reduce the overall number of frames sensed and sent to the sink. The first approach is applied on each sensor node, where the FRABID algorithm, a joint data reduction, and frame rate adaptation on sensing and transmission phases mechanism is introduced. This approach reduces the number of sensed frames based on a similarity method. The aim is to adapt the number of sensed frames based on the degree of difference between two consecutive sensed frames in each period. This adaptation technique maintains the accuracy of the video while capturing frames holding new information. This approach is validated through simulations using real data-sets from video sensors (Wang et al. in: 2014 IEEE conference on computer vision and pattern recognition workshops, pp 393–400, 2014). The results show that the amount of sensed data is reduced by more than 70% compared to a recent algorithm in Christian et al. (Multimed Tools Appl 79(3):1801–1819, 2020) while guaranteeing the detection of all the critical events at the sensor node level. The second approach exploits the Spatio-temporal correlation between neighboring nodes to reduce the number of captured frames. For that purpose, Synchronization with Frame Rate Adaptation SFRA algorithm is introduced where overlapping nodes capture frames in a synchronized fashion every N - 1 periods, where N is the number of overlapping sensor nodes. The results show more than 90% data reduction, surpassing other techniques in the literature at the level of the number of sensed frames by 20% at least. [ABSTRACT FROM AUTHOR]

The choice of an adequate mathematical model is a key step in solving problems in many different fields. When more than one model is available to represent a given phenomenon, a poor choice might result in loss of precision and efficiency. Well-known strategies for comparing mathematical models can be found in many previous works, but seldom regarding several models with uncertain parameters at once. In this work, we present a novel approach for measuring the similarity among any given number of mathematical models, so as to support decision making regarding model selection. The strategy consists in defining a new general model composed of all candidate models and a uniformly distributed random variable, whose sampling selects the candidate model employed to evaluate the response. Global Sensitivity Analysis (GSA) is then performed to measure the sensitivity of the response with respect to this random variable. The result indicates the level of discrepancy among the mathematical models in the stochastic context. We also demonstrate that the proposed approach is related to the Root Mean Square (RMS) error when only two models are compared. The main advantages of the proposed approach are: (i) the problem is cast in the sound framework of GSA, (ii) the approach also quantifies if the discrepancy among the mathematical models is significant in comparison to uncertainties/randomness of the parameters, an analysis that is not possible with RMS error alone. Numerical examples of different disciplines and degrees of complexity are presented, showing the kind of insight we can get from the proposed approach. [ABSTRACT FROM AUTHOR]

We introduce a newly developed R package AZIAD for analysing zero-inflated or zero-altered data. Compared with existing R packages, AZIAD covers a much larger class of zero-inflated and hurdle models, including both discrete and continuous cases. It provides more accurate parameter estimates, along with the corresponding Fisher information matrix and confidence intervals. It achieves significantly larger power for model identification and selection. To facilitate the potential users, in this paper we provide detailed formulae and theoretical justifications for AZIAD, as well as new theoretical results on zero-inflated and zero-altered models. We use simulation studies to show the advantages of AZIAD functions over existing R packages and provide real data examples and executable R code to illustrate how to use our package for sparse data analysis and model selection. [ABSTRACT FROM AUTHOR]

In the context of both cognitive and affective tests, items are usually designed to involve more than two responses, for which polytomous models are applicable. The purpose of this paper is to propose a highly effective Pólya-Gamma Gibbs sampling algorithm based on auxiliary variables to estimate the multidimensional graded response model that has been widely used in psychological, educational, and health-related assessment. The strategy is based on the Pólya Gamma family of distributions which provides a closed-form posterior distribution for logistic-based models. With the introduction of the two latent variables, the full conditional distributions are tractable, and consequently the Gibbs sampling is easy to implement. Nice features including empirical performance of the proposed methodology are demonstrated by simulation studies. Finally, two empirical data sets were analysed to demonstrate the efficiency and utility of the proposed method. [ABSTRACT FROM AUTHOR]

A modelling framework suitable for detecting shape shifts in functional profiles combining the notion of the Fréchet mean and the concept of deformation models is developed and proposed. The generalized mean sense offered by the Fréchet mean notion is employed to capture the typical pattern of the profiles under study, while the concept of deformation models, and in particular of the shape-invariant model, allows for interpretable parameterizations of the profile's deviations from the typical shape. The EWMA-type control charts compatible with the functional nature of data and the employed deformation model are built and proposed, exploiting certain shape characteristics of the profiles under study with respect to the generalized mean sense, allowing for the identification of potential shifts concerning the shape and/or the deformation process. Potential shifts in the shape deformation process are further distinguished into significant shifts with respect to amplitude and/or the phase of the profile under study. The proposed modeling and shift detection framework is implemented to a real-world case study, where daily concentration profiles concerning air pollutants from an area in the city of Athens are modeled, while profiles indicating hazardous concentration levels are successfully identified in most cases. [ABSTRACT FROM AUTHOR]

Randomized controlled trials (RCTs) admit unconfounded design-based inference – randomization largely justifies the assumptions underlying statistical effect estimates – but often have limited sample sizes. However, researchers may have access to big observational data on covariates and outcomes from RCT nonparticipants. For example, data from A/B tests conducted within an educational technology platform exist alongside historical observational data drawn from student logs. We outline a design-based approach to using such observational data for variance reduction in RCTs. First, we use the observational data to train a machine learning algorithm predicting potential outcomes using covariates and then use that algorithm to generate predictions for RCT participants. Then, we use those predictions, perhaps alongside other covariates, to adjust causal effect estimates with a flexible, design-based covariate-adjustment routine. In this way, there is no danger of biases from the observational data leaking into the experimental estimates, which are guaranteed to be exactly unbiased regardless of whether the machine learning models are “correct” in any sense or whether the observational samples closely resemble RCT samples. We demonstrate the method in analyzing 33 randomized A/B tests and show that it decreases standard errors relative to other estimators, sometimes substantially.

Within the theory of non-negative integer valued multivariate infinitely divisible distributions, the multivariate Poisson distribution plays a key role. As in the univariate case, any non-negative integer valued infinitely divisible multivariate distribution can be approximated by a multivariate distribution belonging to the compound Poisson family. The multivariate Poisson distribution is an important member of this family. In recent years, the multivariate Poisson distributions also has gained practical importance, since they serve as models to describe counting data having a positive covariance structure. However, due to the computational complexity of computing the multivariate Poisson probability mass function (pmf) and its corresponding cumulative distribution function (cdf), their use within these counting models is limited. Since most of the theoretical properties of the multivariate Poisson probability distribution seem already to be known, the main focus of this paper is on proposing more efficient algorithms to compute this pmf. Using a well known property of a Poisson multivariate distributed random vector, we propose in this paper a direct approach to calculate this pmf based on finding all solutions of a system of linear Diophantine equations. This new approach complements an already existing procedure depending on the use of recurrence relations existing for the pmf. We compare our new approach with this already existing approach applied to a slightly different set of recurrence relations which are easier to evaluate. A proof of this new set of recurrence relations is also given. As a result, several algorithms are proposed where some of them are based on the new approach and some use the recurrence relations. To test these algorithms, we provide an extensive analysis in the computational section. Based on the experiments in this section, we conclude that the approach finding all solutions of a set of linear Diophantine equations is computationally more efficient than the approach using the recurrence relations to evaluate the pmf of a multivariate Poisson distributed random vector. [ABSTRACT FROM AUTHOR]

A myriad of physical, biological and other phenomena are better modeled with semi-infinite distribution families, in which case not knowing the population minimum becomes a hassle when performing parametric inference. Ad hoc methods to deal with this problem exist, but are suboptimal and sometimes unfeasible. Besides, having the statistician handcraft solutions in a case-by-case basis is counterproductive. In this paper, we propose a framework under which the issue can be analyzed, and perform an extensive search in the literature for methods that could be used to solve the aforementioned problem; we also propose a method of our own. Simulation experiments were then performed to compare some methods from the literature and our proposal. We found that the straightforward method, which is to infer the population minimum by maximum likelihood, has severe difficulty in giving a good estimate for the population minimum, but manages to achieve very good inferred models. The other methods, including our proposal, involve estimating the population minimum, and we found that our method is superior to the other methods of this kind, considering the distributions simulated, followed very closely by the endpoint estimator by Alves et al. (Stat Sin 24(4):1811–1835, 2014). Although these two give much more accurate estimates for the population minimum, the straightforward method also displays some advantages, so choosing between these three methods will depend on the problem domain. [ABSTRACT FROM AUTHOR]

Leveraging external controls – relevant individual patient data under control from external trials or real-world data – has the potential to reduce the cost of randomized controlled trials (RCTs) while increasing the proportion of trial patients given access to novel treatments. However, due to lack of randomization, RCT patients and external controls may differ with respect to covariates that may or may not have been measured. Hence, after controlling for measured covariates, for instance by matching, testing for treatment effect using external controls may still be subject to unmeasured biases. In this article, we propose a sensitivity analysis approach to quantify the magnitude of unmeasured bias that would be needed to alter the study conclusion that presumed no unmeasured biases are introduced by employing external controls. Whether leveraging external controls increases power or not depends on the interplay between sample sizes and the magnitude of treatment effect and unmeasured biases, which may be difficult to anticipate. This motivates a combined testing procedure that performs two highly correlated analyses, one with and one without external controls, with a small correction for multiple testing using the joint distribution of the two test statistics. The combined test provides a new method of sensitivity analysis designed for data fusion problems, which anchors at the unbiased analysis based on RCT only and spends a small proportion of the type I error to also test using the external controls. In this way, if leveraging external controls increases power, the power gain compared to the analysis based on RCT only can be substantial; if not, the power loss is small. The proposed method is evaluated in theory and power calculations, and applied to a real trial.

In a pairwise exchangeable dependence setting for test statistics, the cutoffs of Sarkar et al. (2016) may be viewed as a first iteration improvement of Holm (1979)’s classical cutoffs under a convexity condition on the copula. The cutoffs of Seneta and Chen (1997) which improve Holm’s in the present exchangeability setting, are shown, after an analogous first iteration step, to lead to a refinement of Sarkar et al. (2016). Further, we show that the convexity condition can be circumvented in practice, computationally. Improvement by iteration limit of cutoffs is considered for both procedures. Comparisons between the effects of the several cutoff sets are made by way of plots of the familywise error rate against correlation ρ in the classic setting of the multivariate Normal; and the distributional setting of the multivariate Generalized Hyperbolic for the important Variance Gamma type subfamily, for which a convexity condition cannot be analytically verified. [ABSTRACT FROM AUTHOR]

We present exact solutions to the birthday and generalized occupancy problems using multiple approaches. We start with two alternative ways to solve the classical birthday problem for uniform distributions by iterating over people and iterating over days, respectively. The iterate-over-day method is used to find the probabilities of not only the birthday problem but also several variants, including the exact number of people sharing a birthday and the strong birthday problem, for non-uniform distributions. This method decomposes multinomial based problems as a recursive sum of binomial distributions and leads readily to a unifying solution for generalized occupancy and coincidence problems for any probability distribution and arbitrary sets of occupancies. The birthday, strong birthday and exact number of coincidences with arbitrary probability distribution and varying number of coincidences/occupancies can all be found from the generalized solution. The generalized solution also allows us to study the details of specific coincidences. We define a z-value, which is similar to the commonly used p-value but is calculated from the generalized occupancy model, to test the conformity of two distributions and show that z-value is more effective than p-value in hypothesis testing when one category (out of five) is an outlier. Several examples are used to illustrate the practical applications of the generalized occupancy model in real-world situations. [ABSTRACT FROM AUTHOR]

Background: The spectrum of mutations in a collection of cancer genomes can be described by a mixture of a few mutational signatures. The mutational signatures can be found using non-negative matrix factorization (NMF). To extract the mutational signatures we have to assume a distribution for the observed mutational counts and a number of mutational signatures. In most applications, the mutational counts are assumed to be Poisson distributed, and the rank is chosen by comparing the fit of several models with the same underlying distribution and different values for the rank using classical model selection procedures. However, the counts are often overdispersed, and thus the Negative Binomial distribution is more appropriate. Results: We propose a Negative Binomial NMF with a patient specific dispersion parameter to capture the variation across patients and derive the corresponding update rules for parameter estimation. We also introduce a novel model selection procedure inspired by cross-validation to determine the number of signatures. Using simulations, we study the influence of the distributional assumption on our method together with other classical model selection procedures. We also present a simulation study with a method comparison where we show that state-of-the-art methods are highly overestimating the number of signatures when overdispersion is present. We apply our proposed analysis on a wide range of simulated data and on two real data sets from breast and prostate cancer patients. On the real data we describe a residual analysis to investigate and validate the model choice. Conclusions: With our results on simulated and real data we show that our model selection procedure is more robust at determining the correct number of signatures under model misspecification. We also show that our model selection procedure is more accurate than the available methods in the literature for finding the true number of signatures. Lastly, the residual analysis clearly emphasizes the overdispersion in the mutational count data. The code for our model selection procedure and Negative Binomial NMF is available in the R package SigMoS and can be found at https://github.com/MartaPelizzola/SigMoS. [ABSTRACT FROM AUTHOR]

We derive a novel stochastic representation of exchangeable Marshall–Olkin distributions based on their death-counting processes. We show that these processes are Markov. Furthermore, we provide a numerically stable approximation of their infinitesimal generator matrices in the extendible case. This approach uses integral representations of Bernstein functions to calculate the generator’s first row, and then uses a recursion to calculate the remaining rows. Combining the Markov representation with the numerically stable approximation of corresponding generators allows us to sample extendible Marshall–Olkin distributions with a flexible simulation algorithm derived from known Markov sampling strategies. Finally, we benchmark an implementation of this Markov-based simulation algorithm against alternative simulation algorithms based on the Lévy frailty model, the Arnold model, and the exogenous shock model.

Researchers across many fields routinely analyze trial data using Null Hypothesis Significance Tests with zero null and p < 0.05. To promote thoughtful statistical testing, we propose a visualization tool that highlights practically meaningful effects when calculating sample sizes. The tool re-purposes and adapts funnel plots, originally developed for meta-analyses, after generalizing them to cater for meaningful effects. As with traditional sample size calculators, researchers must nominate anticipated effect sizes and variability alongside the desired power. The advantage of our tool is that it simultaneously presents sample sizes needed to adequately power tests for equivalence, for non-inferiority and for superiority, each considered at up to three alpha levels and in positive and negative directions. The tool thus encourages researchers at the design stage to think about the type and level of test in terms of their research goals, costs of errors, meaningful effect sizes and feasible sample sizes. An R-implementation of the tool is available on-line. [ABSTRACT FROM AUTHOR]

A common practice in statistics is to take the log transformation of highly skewed data and construct confidence intervals for the population average on the basis of transformed data. However, when computed based on log-transformed data, the confidence interval is for the geometric instead of the arithmetic average and neglecting this can lead to misleading conclusions. In this paper, we consider an approach based on a regression of the two sample averages to convert the confidence interval for the geometric average in a confidence interval for the arithmetic average of the original untransformed data. The proposed approach is substantially simpler to implement when compared to the existing methods and the extensive Monte Carlo and bootstrapping simulation study suggests outperforming in terms of coverage probabilities even at very small sample sizes. Some real data examples have been analyzed, which support the simulation findings of the paper. [ABSTRACT FROM AUTHOR]

Abstract This paper reports on our study of a regularization path algorithm for a regression spline estimator based on B-splines and total variation penalty. The total variation-penalized regression spline estimation method enjoys a spatial adaptation property, since the penalty function enables data-adaptive knot selection. Although the theoretical properties of the related methods have been well established in the literature, the existing implementation algorithms are usually based on techniques with a high computational cost. In particular, the selection of the optimal complexity parameter, required to strike a balance in bias-variance tradeoff, is based on the computationally intensive grid search method. In this study, we propose an efficient path algorithm by formulating an equivalent ℓ 1 penalized optimization problem. The least angle regression algorithm is applied to the reformulated problem to obtain an entire path of the jump size of B-spline derivatives that represents the knot removal condition. Numerical studies based on real and simulated data are provided to illustrate the advantages of the proposed algorithm. The simulation result shows that the proposed algorithm is significantly faster than an existing algorithm. The proposed algorithm has an additional benefit of obtaining an entire regularization path, which can provide additional statistical insights into the given problem. [ABSTRACT FROM AUTHOR]

Multifidelity methods are widely used for estimating quantities of interest (QoI) in computational science by employing numerical simulations of differing costs and accuracies. Many methods approximate numerical-valued statistics that represent only limited information, e.g., scalar statistics, about the QoI. Further quantification of uncertainty, e.g., for risk assessment, failure probabilities, or confidence intervals, requires estimation of the full distributions. In this paper, we generalize the ideas in (Xu et al. in SIAM J Sci Comput 44(1):A150–A175, 2022) to develop a multifidelity method that approximates the full distribution of scalar-valued QoI. The main advantage of our approach compared to alternative methods is that we require no particular relationships among the high and lower-fidelity models (e.g. model hierarchy), and we do not assume any knowledge of model statistics including correlations and other cross-model statistics before the procedure starts. Under suitable assumptions in the framework above, we achieve provable 1-Wasserstein metric convergence of an algorithmically constructed distributional emulator via an exploration–exploitation strategy. We also prove that crucial policy actions taken by our algorithm are budget-asymptotically optimal. Numerical experiments are provided to support our theoretical analysis. [ABSTRACT FROM AUTHOR]

This research develops a scalable computing method based on quasi-Newton algorithm for several estimation problems in dynamic generalised linear models (DGLMs). The new method is developed by applying the principle of maximising a pointwise penalised quasi-likelihood (PPQ) for a DGLM for observed data often of massive size. Statistical and computational challenges involved in this development have been effectively tackled by exploiting the specific block structure and sparsity involved in the underlying projection matrix. The obtained maximum PPQ estimator of the state vector in the DGLM has been shown to be consistent and asymptotically normal under regularity conditions. Numerical studies and real data applications are conducted to assess the performance of the developed method. [ABSTRACT FROM AUTHOR]

In the context of estimating stochastically ordered distribution functions, the pool-adjacent-violators algorithm (PAVA) can be modified such that the computation times are reduced substantially. This is achieved by studying the dependence of antitonic weighted least squares fits on the response vector to be approximated. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of variance estimation of predictions obtained from a subsampling-based ensemble estimator, such as subbagging and sub-random forest. We first recognize that a subsampling-based ensemble can be written as an infinite-order U-statistic of degree k n , where k n is the subsample size that may depend on the learning sample size n. As a result, one can study the uncertainty of predictions obtained from a subsampling-based ensemble under a U-statistic framework, such as approximating its asymptotic variance. However, existing methods used to estimate the asymptotic variance relies on some regularity conditions. In addition, they tend to yield variance estimations with large bias in finite sample scenarios. Motivated by the work of Wang and Lindsay (2014), we propose to construct an unbiased variance estimator for a subsampling-based ensemble. It is efficient to realize with the help of a partition-resampling scheme. We show by simulation studies that the proposed variance estimator yields better performance in terms of mean, standard deviation, and mean squared error compared to both the infinitesimal jackknife and internal variance estimation methods under either a simple linear regression model or a multivariate adaptive regression splines model. Furthermore, we present how to construct an asymptotic confidence interval for the expected prediction at a given test instance using the proposed variance estimator, and compare its coverage probability to that of competing methods. In the end, we demonstrate the practical applications of the methodology using two real data examples. [ABSTRACT FROM AUTHOR]

This study focused on comparing selected commonly used marginal models with marginal-conditional models for analyzing correlated longitudinal binary data. A simulation study shows that for explaining the relationship among the covariates and the repeated outcomes, each of the proposed models show competitive results in terms of bias and coverage probability as compared to the marginal models. If the repeated outcomes are associated or if the distribution of outcome variables are not identical at different follow-ups, the marginal-conditional models give better results in terms of bias and coverage probability of the estimates. For keeping the number of parameters to be estimated as small as possible, the regressive model is suggested for data with more than three follow-ups. The methods are illustrated with an example using Health and Retirement Study data. [ABSTRACT FROM AUTHOR]

In the Euclidean setting the proximal gradient method and its accelerated variants are a class of efficient algorithms for optimization problems with decomposable objective. In this paper, we develop a Riemannian proximal gradient method (RPG) and its accelerated variant (ARPG) for similar problems but constrained on a manifold. The global convergence of RPG is established under mild assumptions, and the O(1/k) is also derived for RPG based on the notion of retraction convexity. If assuming the objective function obeys the Rimannian Kurdyka–Łojasiewicz (KL) property, it is further shown that the sequence generated by RPG converges to a single stationary point. As in the Euclidean setting, local convergence rate can be established if the objective function satisfies the Riemannian KL property with an exponent. Moreover, we show that the restriction of a semialgebraic function onto the Stiefel manifold satisfies the Riemannian KL property, which covers for example the well-known sparse PCA problem. Numerical experiments on random and synthetic data are conducted to test the performance of the proposed RPG and ARPG. [ABSTRACT FROM AUTHOR]