Search

Customization of cardiac action potential models has become increasingly important with the recognition of patient-specific models and virtual patient cohorts as valuable predictive tools. Nevertheless, developing customized models by fitting parameters to data poses technical and methodological challenges: despite noise and variability associated with real-world datasets, traditional optimization methods produce a single "best-fit" set of parameter values. Bayesian estimation methods seek distributions of parameter values given the data by obtaining samples from the target distribution, but in practice widely known Bayesian algorithms like Markov chain Monte Carlo tend to be computationally inefficient and scale poorly with the dimensionality of parameter space. In this paper, we consider two computationally efficient Bayesian approaches: the Hamiltonian Monte Carlo (HMC) algorithm and the approximate Bayesian computation sequential Monte Carlo (ABC-SMC) algorithm. We find that both methods successfully identify distributions of model parameters for two cardiac action potential models using model-derived synthetic data and an experimental dataset from a zebrafish heart. Although both methods appear to converge to the same distribution family and are computationally efficient, HMC generally finds narrower marginal distributions, while ABC-SMC is less sensitive to the algorithmic settings including the prior distribution., (© 2022. International Federation for Medical and Biological Engineering.)

Background: Boosting algorithms to simultaneously estimate and select predictor effects in statistical models have gained substantial interest during the last decade., Objectives: This review highlights recent methodological developments regarding boosting algorithms for statistical modelling especially focusing on topics relevant for biomedical research., Methods: We suggest a unified framework for gradient boosting and likelihood-based boosting (statistical boosting) which have been addressed separately in the literature up to now., Results: The methodological developments on statistical boosting during the last ten years can be grouped into three different lines of research: i) efforts to ensure variable selection leading to sparser models, ii) developments regarding different types of predictor effects and how to choose them, iii) approaches to extend the statistical boosting framework to new regression settings., Conclusions: Statistical boosting algorithms have been adapted to carry out unbiased variable selection and automated model choice during the fitting process and can nowadays be applied in almost any regression setting in combination with a large amount of different types of predictor effects.

This article is part of a For-Discussion-Section of Methods of Information in Medicine about the papers "The Evolution of Boosting Algorithms - From Machine Learning to Statistical Modelling" and "Extending Statistical Boosting - An Overview of Recent Methodological Developments", written by Andreas Mayr and co-authors. It is introduced by an editorial. This article contains the combined commentaries invited to independently comment on the Mayr et al. papers. In subsequent issues the discussion can continue through letters to the editor.

Background: The concept of boosting emerged from the field of machine learning. The basic idea is to boost the accuracy of a weak classifying tool by combining various instances into a more accurate prediction. This general concept was later adapted to the field of statistical modelling. Nowadays, boosting algorithms are often applied to estimate and select predictor effects in statistical regression models., Objectives: This review article attempts to highlight the evolution of boosting algorithms from machine learning to statistical modelling., Methods: We describe the AdaBoost algorithm for classification as well as the two most prominent statistical boosting approaches, gradient boosting and likelihood-based boosting for statistical modelling. We highlight the methodological background and present the most common software implementations., Results: Although gradient boosting and likelihood-based boosting are typically treated separately in the literature, they share the same methodological roots and follow the same fundamental concepts. Compared to the initial machine learning algorithms, which must be seen as black-box prediction schemes, they result in statistical models with a straight-forward interpretation., Conclusions: Statistical boosting algorithms have gained substantial interest during the last decade and offer a variety of options to address important research questions in modern biomedicine.

We propose a novel hybrid quantum computing strategy for parallel MCMC algorithms that generate multiple proposals at each step. This strategy makes the rate-limiting step within parallel MCMC amenable to quantum parallelization by using the Gumbel-max trick to turn the generalized accept-reject step into a discrete optimization problem. When combined with new insights from the parallel MCMC literature, such an approach allows us to embed target density evaluations within a well-known extension of Grover's quantum search algorithm. Letting P denote the number of proposals in a single MCMC iteration, the combined strategy reduces the number of target evaluations required from O (P) to O (P) . In the following, we review the rudiments of quantum computing, quantum search and the Gumbel-max trick in order to elucidate their combination for as wide a readership as possible. [ABSTRACT FROM AUTHOR]

Quantile regression is a popular method with a wide range of scientific applications, but the computation for quantile regression is challenging. Hunter and Lange proposed an MM algorithm for solving optimization problems in parametric quantile regression models. For nonparametric and semiparametric quantile regression, their algorithm can be applied to estimate unknown quantile functions in a pointwise manner. However, the resulting estimates may suffer from drawbacks like nonsmooth with discontinuous points and unstable at extreme quantile levels. To remedy the above issues, we propose a new MM algorithm and show that it yields continuous, smoother, and faster estimated quantile functions. We systematically study the new MM algorithm using the local linear quantile regression model. We prove that the proposed algorithm preserves the monotone descent property in an asymptotic sense. We then extend it to some popular nonparametric and semiparametric quantile regression models. For semiparametric models, we propose new efficient backfitting algorithms based on the new MM algorithm. Compared to traditional backfitting algorithms, the new procedures can significantly reduce computational costs for fully iterative backfitting. The performance of the proposed algorithms is demonstrated via extensive simulation studies and a real data example. for this article are available online. [ABSTRACT FROM AUTHOR]

This paper introduces the Group Linear Algorithm with Sparse Principal decomposition, an algorithm for supervised variable selection and clustering. Our approach extends the Sparse Group Lasso regularization to calculate clusters as part of the model fit. Therefore, unlike Sparse Group Lasso, our idea does not require prior specification of clusters between variables. To determine the clusters, we solve a particular case of sparse Singular Value Decomposition, with a regularization term that follows naturally from the Group Lasso penalty. Moreover, this paper proposes a unified implementation to deal with, but not limited to, linear regression, logistic regression, and proportional hazards models with right-censoring. Our methodology is evaluated using both biological and simulated data, and details of the implementation in R and hyperparameter search are discussed. [ABSTRACT FROM AUTHOR]

Smoothing splines provide a powerful and flexible means for nonparametric estimation and inference. With a cubic time complexity, fitting smoothing spline models to large data is computationally prohibitive. In this paper, we use the theoretical optimal eigenspace to derive a low‐rank approximation of the smoothing spline estimates. We develop a method to approximate the eigensystem when it is unknown and derive error bounds for the approximate estimates. The proposed methods are easy to implement with existing software. Extensive simulations show that the new methods are accurate, fast and compare favourably against existing methods. [ABSTRACT FROM AUTHOR]

We present a kernel-independent method that applies hierarchical matrices to the problem of maximum likelihood estimation for Gaussian processes. The proposed approximation provides natural and scalable stochastic estimators for its gradient and Hessian, as well as the expected Fisher information matrix, that are computable in quasilinear O (n log 2 n) complexity for a large range of models. To accomplish this, we (i) choose a specific hierarchical approximation for covariance matrices that enables the computation of their exact derivatives and (ii) use a stabilized form of the Hutchinson stochastic trace estimator. Since both the observed and expected information matrices can be computed in quasilinear complexity, covariance matrices for maximum likelihood estimators (MLEs) can also be estimated efficiently. In this study, we demonstrate the scalability of the method, show how details of its implementation effect numerical accuracy and computational effort, and validate that the resulting MLEs and confidence intervals based on the inverse Fisher information matrix faithfully approach those obtained by the exact likelihood. for this article are available online. [ABSTRACT FROM AUTHOR]

A common divide-and-conquer approach for Bayesian computation with big data is to partition the data, perform local inference for each piece separately, and combine the results to obtain a global posterior approximation. While being conceptually and computationally appealing, this method involves the problematic need to also split the prior for the local inferences; these weakened priors may not provide enough regularization for each separate computation, thus eliminating one of the key advantages of Bayesian methods. To resolve this dilemma while still retaining the generalizability of the underlying local inference method, we apply the idea of expectation propagation (EP) as a framework for distributed Bayesian inference. The central idea is to iteratively update approximations to the local likelihoods given the state of the other approximations and the prior. The present paper has two roles: we review the steps that are needed to keep EP algorithms numerically stable, and we suggest a general approach, inspired by EP, for approaching data partitioning problems in a way that achieves the computational benefits of parallelism while allowing each local update to make use of relevant information from the other sites. In addition, we demonstrate how the method can be applied in a hierarchical context to make use of partitioning of both data and parameters. The paper describes a general algorithmic framework, rather than a specific algorithm, and presents an example implementation for it. [ABSTRACT FROM AUTHOR]

We propose two measures of performance for a confidence interval for a binomial proportion p: the root mean squared error and the mean absolute deviation. We also devise a confidence interval for p based on the actual coverage function that combines several existing approximate confidence intervals. This "Ensemble" confidence interval has improved statistical properties over the constituent confidence intervals. Software in an R package, which can be used in devising and assessing these confidence intervals, is available on CRAN. [ABSTRACT FROM AUTHOR]

We consider alternate formulations of recently proposed hierarchical nearest neighbor Gaussian process (NNGP) models for improved convergence, faster computing time, and more robust and reproducible Bayesian inference. Algorithms are defined that improve CPU memory management and exploit existing high-performance numerical linear algebra libraries. Computational and inferential benefits are assessed for alternate NNGP specifications using simulated datasets and remotely sensed light detection and ranging data collected over the U.S. Forest Service Tanana Inventory Unit (TIU) in a remote portion of Interior Alaska. The resulting data product is the first statistically robust map of forest canopy for the TIU. Supplemental materials for this article are available online. [ABSTRACT FROM AUTHOR]

Monte Carlo simulations have become a common practice to evaluate a proposed statistical procedure, particularly when it is analytically intractable. Validity of any simulation study relies heavily on the goodness of random variate generators for some specified distributions, which in turn is based on the successful generation of independent variates from the uniform distribution. However, a typical computer-generated pseudo-random number generator ( PRNG) is a deterministic algorithm and we know that no PRNG is capable of generating a truly random uniform sequence. Since the foundation of a simulation study is built on the PRNG used, it is extremely important to design a good PRNG. We review some recent developments on PRNGs with nice properties such as high-dimensional equi-distribution, efficiency, long period length, portability, and efficient parallel implementations. WIREs Comput Stat 2017, 9:e1404. doi: 10.1002/wics.1404 For further resources related to this article, please visit the . [ABSTRACT FROM AUTHOR]

One of the core problems of modern statistics is to approximate difficult-to-compute probability densities. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation involving the posterior density. In this article, we review variational inference (VI), a method from machine learning that approximates probability densities through optimization. VI has been used in many applications and tends to be faster than classical methods, such as Markov chain Monte Carlo sampling. The idea behind VI is to first posit a family of densities and then to find a member of that family which is close to the target density. Closeness is measured by Kullback–Leibler divergence. We review the ideas behind mean-field variational inference, discuss the special case of VI applied to exponential family models, present a full example with a Bayesian mixture of Gaussians, and derive a variant that uses stochastic optimization to scale up to massive data. We discuss modern research in VI and highlight important open problems. VI is powerful, but it is not yet well understood. Our hope in writing this article is to catalyze statistical research on this class of algorithms. Supplementary materials for this article are available online. [ABSTRACT FROM PUBLISHER]

Background: For the statistical analysis of dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data, compartment models are a commonly used tool. By these models, the observed uptake of contrast agent in some tissue over time is linked to physiologic properties like capillary permeability and blood flow. Up to now, models of different complexity have been used, and it is still unclear which model should be used in which situation. In previous studies, it has been found that for DCE-MRI data, the number of compartments differs for different types of tissue, and that in cancerous tissue, it might actually differ over a region of voxels of one DCE-MR image. Objectives: To find the appropriate number of compartments and estimate the parameters of a regression model for each voxel in an DCE-MR image. With that, tumors in an DCE-MR image can be located, and for example therapy success can be assessed. Methods: The observed uptake of contrast agent in a voxel of an image of some tissue is described by a concentration time curve. This curve can be modeled using a nonlinear regression model. We present a boosting approach with nonlinear regression as base procedure, which allows us to estimate the number of compartments and the related parameters for each voxel of an DCE-MR image. In addition, a spatially regularized version of this approach is proposed. Results: With the proposed approach, the number of compartments - and with that the complexity of the model - per voxel is not fixed but data-driven, which allows us to fit models of adequate complexity to the concentration time curves of all voxels. The parameters of the model remain nevertheless interpretable because of the underlying compartment model. Conclusions: The proposed boosting approaches outperform all competing methods considered in this paper regarding the correct localization of tumors in DCE-MR images as well as the spatial homogeneity of the estimated number of compartments across the image, and the definition of the tumor edge. [ABSTRACT FROM AUTHOR]

\Background: Boosting algorithms to simultaneously estimate and select predictor effects in statistical models have gained substantial interest during the last decade. Objectives: This review highlights recent methodological developments regarding boosting algorithms for statistical modelling especially focusing on topics relevant for biomedical research. Methods: We suggest a unified framework for gradient boosting and likelihood-based boosting (statistical boosting) which have been addressed separately in the literature up to now. Results: The methodological developments on statistical boosting during the last ten years can be grouped into three different lines of research: i) efforts to ensure variable selection leading to sparser models, ii) developments regarding different types of predictor effects and how to choose them, iii) approaches to extend the statistical boosting framework to new regression settings. Conclusions: Statistical boosting algorithms have been adapted to carry out unbiased variable selection and automated model choice during the fitting process and can nowadays be applied in almost any regression setting in combination with a large amount of different types of predictor effects. [ABSTRACT FROM AUTHOR]

The generalized median (GM) estimator is a family of robust estimators that balances the competing demands of statistical efficiency and robustness. By choosing a kernel that is efficient for the parameter, the GM estimator gains robustness by computing the median of the kernel evaluated at all possible subsets from the sample. The GM estimator is often computationally infeasible because the number of subsets can be large for even modest sample sizes. Writing the estimator in terms of the quantile function facilitates an approximation using a sample of all possible subsets. While both sampling with and without replacement are feasible, sampling without replacement is preferred because of the reduction in variance from the sampling fraction. The proposed algorithm uses sequential sampling to compute an approximation within a user-chosen margin of error. [ABSTRACT FROM AUTHOR]

In this article, we focus on computational aspects of spectral clustering algorithms that have recently shown promising results in machine learning, statistics, and computer vision. These algorithms cluster observations (of sizen) into groups by investigating eigenvectors of an affinity matrix or its Laplacian matrix, both of which are sizen×n. However, when the sample size is large, the computation involved in the matrix eigen-decomposition is expensive or even infeasible. To overcome the computation hurdle, subsampling techniques, such as the Nyström extension, have been used to approximate eigenvectors of large matrices. We study statistical properties of this approximation and their influence on the accuracy of various spectral clustering algorithms. We show that the perturbation of the spectrum due to subsampling could lead to a large discrepancy among clustering results. In order to provide accurate and stable results for large datasets, we propose a method to combine multiple subsamples using data spectroscopic clustering and the Nyström extension. In addition, we propose a sparse approximation of the eigenvectors to further speed up computation. Simulation and experiments on real datasets show that our approaches work quickly and provide reasonable results that are more stable across samples than the single sample approach. This article has supplementary material online. [ABSTRACT FROM AUTHOR]

In the parametric regression model, the covariate missing problem under missing at random is considered. It is often desirable to use flexible parametric or semiparametric models for the covariate distribution, which can reduce a potential misspecification problem. Recently, a completely nonparametric approach was developed by [H.Y. Chen, Nonparametric and semiparametric models for missing covariates in parameter regression, J. Amer. Statist. Assoc. 99 (2004), pp. 1176-1189; Z. Zhang and H.E. Rockette, On maximum likelihood estimation in parametric regression with missing covariates, J. Statist. Plann. Inference 47 (2005), pp. 206-223]. Although it does not require a model for the covariate distribution or the missing data mechanism, the proposed method assumes that the covariate distribution is supported only by observed values. Consequently, their estimator is a restricted maximum likelihood estimator (MLE) rather than the global MLE. In this article, we show the restricted semiparametric MLE could be very misleading in some cases. We discuss why this problem occurs and suggest an algorithm to obtain the global MLE. Then, we assess the performance of the proposed method via some simulation experiments. [ABSTRACT FROM AUTHOR]

An innovative algorithm is developed for obtaining spreadsheet regression measures used in computing out-of-sample statistics. This algorithm alleviates the leave-one-out computational simulation complexity and memory size problems perceived in computing these statistics. Hence, the purpose of this article is to describe a computationally enhanced algorithm that gives spreadsheet users advanced regression capabilities thereby adding a new dimension to spreadsheet regression operations. These statistics include diagonals of the hat matrix, legitimate forecasting intervals, and PRESS residuals. These computational innovations promote learning while eliminating spreadsheet inadequacies thereby making spreadsheet regression attractive to academicians in teaching and practitioners in acquiring further application competence. [ABSTRACT FROM AUTHOR]

Capturing the human hand motion from video involves the estimation of the rigid global hand pose as well as the nonrigid finger articulation. The complexity induced by the high degrees of freedom of the articulated hand challenges many visual tracking techniques. For example, the particle filtering technique is plagued by the demanding requirement of a huge number of particles and the phenomenon of particle degeneracy. This paper presents a novel approach to tracking the articulated hand in video by learning and integrating natural hand motion priors. To cope with the finger articulation, this paper proposes a powerful sequential Monte Carlo tracking algorithm based on importance sampling techniques, where the importance function is based on an initial manifold model of the articulation configuration space learned from motion-captured data. In addition, this paper presents a divide-and-conquer strategy that decouples the hand poses and finger articulations and integrates them in an iterative framework to reduce the complexity of the problem. Our experiments show that this approach is effective and efficient for tracking the articulated hand. This approach can be extended to track other articulated targets. [ABSTRACT FROM AUTHOR]

The non-central gamma distribution can be regarded as a general form of non-central χ² distributions whose computations were thoroughly investigated (Ruben, H., 1974, Non-central chi-square and gamma revisited. Communications in Statistics, 3(7), 607-633: Knüsel, L., 1986, Computation of the chi-square and Poisson distribution. SIAM Journal on Scientific and Statistical Computing, 7, 1022-1036; Voit, E.O. and Rust, P.F., 1987, Noncentral chi-square distributions computed by S-system differential equations. Proceedings of the Statistical Computing Section, ASA, pp. 118-121; Rust. P.F. and Voit, E.O., 1990, Statistical densities, cumulatives, quantiles, and power obtained by S-systems differential equations. Journal of the American Statistical Association, 85, 572-578; Chattamvelli, R., 1994. Another derivation of two algorithms for the noncentral χ² and F distributions. Journal of Statistical Computation and Simulation, 49, 207-214; Johnson. N.J., Kotz, S. and Balakrishnan, N., 1995, Continuous Univariate Distributions, Vol. 2 (2nd edn) (New York: Wiley). Both distributional function forms are usually in terms of weighted infinite series of the central one. The ad hoc approximations to cumulative probabilities of non-central gamma were extended or discussed by Chattamvelli, Knüsel and Bablok (Knüsel, L. and Bablok, B., 1996, Computation of the noncentral gamma distribution, SIAM Journal on Scientific Computing, 17, 1224-1231), and Ruben (Ruben, H., 1974, Non-central chi-square and gamma revisited. Communications in Statistics, 3(7), 607-633). However, they did not implement and demonstrate proposed numerical procedures. Approximations to noncentral densities and quantiles are not available. In addition, its S-system formulation has not been derived. Here, approximations to cumulative probabilities, density, and quantiles based on the method of Knüsel and Bablok are derived and implemented in R codes. Furthermore, two alternate S-system forms are recast on the basis of techniques of Savageau and Voit (Savageau, M.A. and Voit, E.O., 1987. Recasting nonlinear differential equations as S-systems: A canonical nonlinear form. Mathematical Biosciences, 87, 83-115) as well as Chen (Chen, Z.-Y., 2003, Computing the distribution of the squared sample multiple correlation coefficient with S-Systems. Communications in Statistics-Simulation and Computation, 32(3), 873-898.) and Chen and Chou (Chen, Z.-Y. and Chou, Y.-C., 2000, Computing the noncentral beta distribution with S-system. Computational Statistics and Data Analysis, 33, 343-360.). Statistical densities, cumulative probabilities, quantiles can be evaluated by only one numerical solver power low analysis and simulation (PLAS). With the newly derived S-systems of non-central gamma, the specialized non-central χ² distributions are demonstrated under five cases in the same three situations studied by Rust and Voit. Both numerical values in pairs are almost equal. Based on these, nine cases in three similar situations arc designed for demonstration and evaluation. In addition, exact values in finite significant digits are provided for comparison. Demonstrations are conducted by R package and PLAS solver in the same PC system. By doing these, very accurate and consistent numerical results are obtained by three methods in two groups. On the other hand, these three methods are performed competitively with respect to speed of computation. Numerical advantages of S-systems over the ad hoc approximation and related properties are also discussed. [ABSTRACT FROM AUTHOR]

Experiments to assess the performance characteristics, both of statistical methods and of computer programs that implement them, are best performed using test data with controlled statistical and numerical properties. We describe an algorithm for constructing test data for any multivariate linear model that provides complete control (subject only to the requirement of mutual consistency) over the following factors: regression coefficients; regression and residual sums of squares and products matrices; means, standard deviations, and correlations of the independent and dependent variables, residuals, and predicted values; and canonical correlations or multiple correlation. The algorithm permits aspects of the underlying data structure including high-leverage points, outliers, and the residual distribution, to be controlled by specifying the components of the singular-value decompositions of the independent variables, the dependent variables, or the error space. Some features of the generated test data may be matched to real data, while others are experimentally controlled or randomly generated. [ABSTRACT FROM AUTHOR]

In geostatistics, inference on spatial covariance parameters of the Gaussian process is often critical to scientists for understanding structural dependence in data. Finite‐sample inference customarily proceeds either using posterior distributions from fully a Bayesian approach or via resampling/subsampling techniques in a frequentist setting. Resampling methods, in particular, the bootstrap, have become more attractive in the modern age of big data as, unlike Bayesian models that require sequential sampling from Markov chain Monte Carlo, they naturally lend themselves to parallel computing resources. However, a spatial bootstrap involves an expensive Cholesky decomposition to decorrelate the data. In this manuscript, we develop a highly scalable parametric spatial bootstrap that uses sparse Cholesky factors for parameter estimation and decorrelation. The proposed bootstrap for rapid inference on spatial covariances (BRISC) algorithm requires linear memory and computations and is embarrassingly parallel, thereby delivering substantial scalability. Simulation studies highlight the accuracy and computational efficiency of our approach. Analysing large satellite temperature data, BRISC produces inference that closely matches that delivered from a state‐of‐the‐art Bayesian approach, while being several times faster. The R package BRISC is now available for download from GitHub (https://github.com/ArkajyotiSaha/BRISC) and will be available on CRAN soon. Copyright © 2018 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

This tutorial presents probability theory techniques for boosting linear algorithms. The approach is based on statistics and uses educated guesses instead of comprehensive calculations. Because estimates can be calculated in sublinear time, many algorithms can benefit from statistical estimation. Several examples show how to significantly boost linear algorithms without negative effects on their results. These examples involve a Ransac algorithm, an image-processing algorithm, and a geometrical reconstruction. The approach exploits that, in many cases, the amount of information in a dataset increases asymptotically sublinearly if its size or sampling density increases. Conversely, an algorithm with expected sublinear running time can extract the most information. [ABSTRACT FROM PUBLISHER]