Search

Renewable energy optimisation poses computationally-intensive challenges. Yet, often the continuous nature of the decision space precludes the use of many emerging, non-von-Neumann computing platforms such as quantum annealing, which are limited to discrete problems. We propose Quantum Annealing Continuous Optimisation (QuAnCO), a Trust Region (TR)-based algorithm, where the TR Newton sub-problem is transformed into Quadratic Unconstrained Binary Optimisation (QUBO), thereby allowing the use of Ising solvers such as D-Wave's quantum annealer. This transformation to QUBO is done by 1) using a hyper-rectangular shape for the TR, 2) discrete representation of each continuous dimension using an interval-bounded integer, and 3) binary encoding of the resulting bounded integers. We tackle a real-world challenge of optimising the biomass mix selection for Nature Energy, the largest biogas producer in Europe, thus providing evidence of feasibility and performance advantage in using QuAnCO in green energy production, and beyond., Comment: 35 pages, 13 figures, 2 tables

Combining matching and regression for causal inference provides double-robustness in removing treatment effect estimation bias due to confounding variables. In most real-world applications, however, treatment and control populations are not large enough for matching to achieve perfect or near-perfect balance on all confounding variables and their nonlinear/interaction functions, leading to trade-offs. [this fact is independent of regression, so a bit disjointed from first sentence.] Furthermore, variance is as important of a contributor as bias towards total error in small samples, and must therefore be factored into the methodological decisions. In this paper, we develop a mathematical framework for quantifying the combined impact of matching and linear regression on bias and variance of treatment effect estimation. The framework includes expressions for bias and variance in a misspecified linear regression, theorems regarding impact of matching on bias and variance, and a constrained bias estimation approach for quantifying misspecification bias and combining it with variance to arrive at total error. Methodological decisions can thus be based on minimization of this total error, given the practitioner's assumption/belief about an intuitive parameter, which we call `omitted R-squared'. The proposed methodology excludes the outcome variable from analysis, thereby avoiding overfit creep and making it suitable for observational study designs. All core functions for bias and variance calculation, as well as diagnostic tools for bias-variance trade-off analysis, matching calibration, and power analysis are made available to researchers and practitioners through an open-source R library, MatchLinReg.

The R package sns implements Stochastic Newton Sampler (SNS), a Metropolis-Hastings Monte Carlo Markov Chain algorithm where the proposal density function is a multivariate Gaussian based on a local, second-order Taylor series expansion of log-density. The mean of the proposal function is the full Newton step in Newton-Raphson optimization algorithm. Taking advantage of the local, multivariate geometry captured in log-density Hessian allows SNS to be more efficient than univariate samplers, approaching independent sampling as the density function increasingly resembles a multivariate Gaussian. SNS requires the log-density Hessian to be negative-definite everywhere in order to construct a valid proposal function. This property holds, or can be easily checked, for many GLM-like models. When initial point is far from density peak, running SNS in non-stochastic mode by taking the Newton step, augmented with with line search, allows the MCMC chain to converge to high-density areas faster. For high-dimensional problems, partitioning of state space into lower-dimensional subsets, and applying SNS to the subsets within a Gibbs sampling framework can significantly improve the mixing of SNS chains. In addition to the above strategies for improving convergence and mixing, sns offers diagnostics and visualization capabilities, as well as a function for sample-based calculation of Bayesian predictive posterior distributions.

The R package RegressionFactory provides expander functions for constructing the high-dimensional gradient vector and Hessian matrix of the log-likelihood function for generalized linear models (GLMs), from the lower-dimensional base-distribution derivatives. The software follows a modular implementation using the chain rule of derivatives. Such modularity offers a clear separation of case-specific components (base distribution functional form and link functions) from common steps (e.g., matrix algebra operations needed for expansion) in calculating log-likelihood derivatives. In doing so, RegressionFactory offers several advantages: 1) It provides a fast and convenient method for constructing log-likelihood and its derivatives by requiring only the low-dimensional, base-distribution derivatives, 2) The accompanying definiteness-invariance theorem allows researchers to reason about the negative-definiteness of the log-likelihood Hessian in the much lower-dimensional space of the base distributions, 3) The factorized, abstract view of regression suggests opportunities to generate novel regression models, and 4) Computational techniques for performance optimization can be developed generically in the abstract framework and be readily applicable across all the specific regression instances. We expect RegressionFactory to facilitate research and development on optimization and sampling techniques for GLM log-likelihoods as well as construction of composite models from GLM lego blocks, such as Hierarchical Bayesian models.

The R package MfUSampler provides Monte Carlo Markov Chain machinery for generating samples from multivariate probability distributions using univariate sampling algorithms such as Slice Sampler and Adaptive Rejection Sampler. The sampler function performs a full cycle of univariate sampling steps, one coordinate at a time. In each step, the latest sample values obtained for other coordinates are used to form the conditional distributions. The concept is an extension of Gibbs sampling where each step involves, not an independent sample from the conditional distribution, but a Markov transition for which the conditional distribution is invariant. The software relies on proportionality of conditional distributions to the joint distribution to implement a thin wrapper for producing conditionals. Examples illustrate basic usage as well as methods for improving performance. By encapsulating the multivariate-from-univariate logic, MfUSampler provides a reliable library for rapid prototyping of custom Bayesian models while allowing for incremental performance optimizations such as utilization of conjugacy, conditional independence, and porting function evaluations to compiled languages.

Single-Instruction, Multiple-Data (SIMD) random number generators (RNGs) take advantage of vector units to offer significant performance gain over non-vectorized libraries, but they often rely on batch production of deviates from distributions with fixed parameters. In many statistical applications such as Gibbs sampling, parameters of sampled distributions change from one iteration to the next, requiring that random deviates be generated one-at-a-time. This situation can render vectorized RNGs inefficient, and even inferior to their scalar counterparts. The C++ class BatchRNG uses buffers of base distributions such uniform, Gaussian and exponential to take advantage of vector units while allowing for sequences of deviates to be generated with varying parameters. These small buffers are consumed and replenished as needed during a program execution. Performance tests using Intel Vector Statistical Library (VSL) on various probability distributions illustrates the effectiveness of the proposed batching strategy.

Computational intensity and sequential nature of estimation techniques for Bayesian methods in statistics and machine learning, combined with their increasing applications for big data analytics, necessitate both the identification of potential opportunities to parallelize techniques such as MCMC sampling, and the development of general strategies for mapping such parallel algorithms to modern CPUs in order to elicit the performance up the compute-based and/or memory-based hardware limits. Two opportunities for Single-Instruction Multiple-Data (SIMD) parallelization of MCMC sampling for probabilistic graphical models are presented. In exchangeable models with many observations such as Bayesian Generalized Linear Models, child-node contributions to the conditional posterior of each node can be calculated concurrently. In undirected graphs with discrete nodes, concurrent sampling of conditionally-independent nodes can be transformed into a SIMD form. High-performance libraries with multi-threading and vectorization capabilities can be readily applied to such SIMD opportunities to gain decent speedup, while a series of high-level source-code and runtime modifications provide further performance boost by reducing parallelization overhead and increasing data locality for NUMA architectures. For big-data Bayesian GLM graphs, the end-result is a routine for evaluating the conditional posterior and its gradient vector that is 5 times faster than a naive implementation using (built-in) multi-threaded Intel MKL BLAS, and reaches within the striking distance of the memory-bandwidth-induced hardware limit. The proposed optimization strategies improve the scaling of performance with number of cores and width of vector units (applicable to many-core SIMD processors such as Intel Xeon Phi and GPUs), resulting in cost-effectiveness, energy efficiency, and higher speed on multi-core x86 processors.

We present MH-MGT, a multivariate technique for sampling from twice-differentiable, log-concave probability density functions. MH-MGT is Metropolis-Hastings sampling using asymmetric, multivariate Gaussian proposal functions constructed from Taylor-series expansion of the log-density function. The mean of the Gaussian proposal function represents the full Newton step, and thus MH-MGT is the stochastic counterpart to Newton optimization. Convergence analysis shows that MH-MGT is well suited for sampling from computationally-expensive log-densities with contributions from many independent observations. We apply the technique to Gibbs sampling analysis of a Hierarchical Bayesian marketing effectiveness model built for a large US foodservice distributor. Compared to univariate slice sampling, MH-MGT shows 6x improvement in sampling efficiency, measured in terms of `function evaluation equivalents per independent sample'. To facilitate wide applicability of MH-MGT to statistical models, we prove that log-concavity of a twice-differentiable distribution is invariant with respect to 'linear-projection' transformations including, but not restricted to, generalized linear models.