Search

This paper presents the R package MitISEM (mixture of t by importance sampling weighted expectation maximization) which provides an automatic and flexible two-stage method to approximate a non-elliptical target density kernel - typically a posterior density kernel - using an adaptive mixture of Student-t densities as approximating density. In the first stage a mixture of Student-t densities is fitted to the target using an expectation maximization (EM) algorithm where each step of the optimization procedure is weighted using importance sampling. In the second stage this mixture density is a candidate density for efficient and robust application of importance sampling or the Metropolis-Hastings (MH) method to estimate properties of the target distribution. The package enables Bayesian inference and prediction on model parameters and probabilities, in particular, for models where densities have multi-modal or other non-elliptical shapes like curved ridges. These shapes occur in research topics in several scientific fields. For instance, analysis of DNA data in bioinformatics, obtaining loans in the banking sector by heterogeneous groups in financial economics and analysis of education's effect on earned income in labor economics. The package MitISEM provides also an extended algorithm, 'sequential MitISEM', which substantially decreases computation time when the target density has to be approximated for increasing data samples. This occurs when the posterior or predictive density is updated with new observations and/or when one computes model probabilities using predictive likelihoods. We illustrate the MitISEM algorithm using three canonical statistical and econometric models that are characterized by several types of non-elliptical posterior shapes and that describe well-known data patterns in econometrics and finance. We show that MH using the candidate density obtained by MitISEM outperforms, in terms of numerical efficiency, MH using a simpler candidate, as well as the Gibbs sampler. The MitISEM approach is also used for Bayesian model comparison using predictive likelihoods.

This paper presents the parallel computing implementation of the MitISEM algorithm, labeled Parallel MitISEM. The basic MitISEM algorithm, introduced by Hoogerheide, Opschoor and Van Dijk (2012), provides an automatic and flexible method to approximate a non-elliptical target density using adaptive mixtures of Student-t densities, where only a kernel of the target density is required. The approximation can be used as a candidate density in Importance Sampling or Metropolis Hastings methods for Bayesian inference on model parameters and probabilities. We present and discuss four canonical econometric models using a Graphics Processing Unit and a multi-core Central Processing Unit version of the MitISEM algorithm. The results show that the parallelization of the MitISEM algorithm on Graphics Processing Units and multi-core Central Processing Units is straightforward and fast to program using MATLAB. Moreover the speed performance of the Graphics Processing Unit version is much higher than the Central Processing Unit one.

A novel dynamic asset-allocation approach is proposed where portfolios as well as portfolio strategies are updated at every decision period based on their past performance. For modeling, a general class of models is specified that combines a dynamic factor and a vector autoregressive model and includes stochastic volatility, denoted by FAVAR-SV. Next, a Bayesian strategy combination is introduced in order to deal with a set of strategies. Our approach extends the mixture of the experts analysis by allowing the strategic weights to be dependent between strategies as well as over time and to further allow for strategy incompleteness. Our approach results in a combination of different portfolio strategies: a model-based and a residual momentum strategy. The estimation of this modeling and strategy approach can be done using an extended and modified version of the forecast combination methodology of Casarin, Grassi, Ravazzolo and Van Dijk(2016). Given the complexity of the non-linear and non-Gaussian model used a new and efficient filter is introduced based on the MitISEM approach by Hoogerheide, Opschoor and Van Dijk (2013). Using US industry portfolios between 1926M7 and 2015M6 as data, our empirical results indicate that time-varying combinations of flexible models in the FAVAR-SV class and two momentum strategies lead to better return and risk features than very simple and very complex models. Combinations of two strategies help, in particular, to reduce risk features like volatility and largest loss, which indicates that complete densities provide useful information for risk.

In this paper we consider two cases of pairs trading strategies: a conditional statistical arbitrage method and an implicit statistical arbitrage method. We use a simulation-based Bayesian procedure for predicting stable ratios, defined in a cointegration model, of pairs of stock prices. We show the effect that using an encompassing prior under an orthogonal normalization has for the selection of pairs of cointegrated stock prices and for the estimation and prediction of the spread between cointegrated stock prices and its uncertainty. An empirical application is done using stocks that are ingredients of the Dow Jones Composite Average index. The results show that the normalization has little effect on the selection of pairs of cointegrated stocks on the basis of Bayes factors. However, the results stress the importance of the orthogonal normalization for the estimation and prediction of the spread – the deviation from the equilibrium relationship – which leads to better results in terms of profit per capital engagement and risk than using a standard linear normalization

Accurate prediction of risk measures such as Value at Risk (VaR) and Expected Shortfall (ES) requires precise estimation of the tail of the predictive distribution. Two novel concepts are introduced that offer a specific focus on this part of the predictive density: the censored posterior, a posterior in which the likelihood is replaced by the censored likelihood; and the censored predictive likelihood, which is used for Bayesian Model Averaging. We perform extensive experiments involving simulated and empirical data. Our results show the ability of these new approaches to outperform the standard posterior and traditional Bayesian Model Averaging techniques in applications of Value-at-Risk prediction in GARCH models.

A class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student-t densities that approximates accurately the target distribution - typically a posterior distribution, of which we only require a kernel - in the sense that the Kullback-Leibler divergence between target and mixture is minimized. We label this approach Mixture of t by Importance Sampling and Expectation Maximization (MitISEM). The constructed mixture is used as a candidate density for quick and reliable application of either Importance Sampling (IS) or the Metropolis-Hastings (MH) method. We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner. Second, we introduce a permutation-augmented MitISEM approach. Third, we propose a partial MitISEM approach, which aims at approximating the joint distribution by estimating a product of marginal and conditional distributions. This division can substantially reduce the dimension of the approximation problem, which facilitates the application of adaptive importance sampling for posterior simulation in more complex models with larger numbers of parameters. Our results indicate that the proposed methods can substantially reduce the computational burden in econometric models like DCC or mixture GARCH models and a mixture instrumental variables model.

We discuss Bayesian inferential procedures within the family of instrumental variables regression models and focus on two issues: existence conditions for posterior moments of the parameters of interest under a flat prior and the potential of Direct Monte Carlo (DMC) approaches for efficient evaluation of such possibly highly onelliptical posteriors. We show that, for the general case of m endogenous variables under a flat prior, posterior moments of order r exist for the coefficients reflecting the endogenous regressors’ effect on the dependent variable, if the number of instruments is greater than m+r, even though there is an issue of local non-identification that causes non-elliptical shapes of the posterior. This stresses the need for efficient Monte Carlo integration methods. We introduce an extension of DMC that incorporates an acceptance-rejection sampling step within DMC. This Acceptance-Rejection within Direct Monte Carlo (ARDMC) method has the attractive property that the generated random drawings are independent, which greatly helps the fast convergence of simulation results, and which facilitates the evaluation of the numerical accuracy. The speed of ARDMC can be easily further improved by making use of parallelized computation using multiple core machines or computer clusters. We note that ARDMC is an analogue to the well-known 'Metropolis-Hastings within Gibbs' sampling in the sense that one 'more difficult' step is used within an 'easier' simulation method. We compare the ARDMC approach with the Gibbs sampler using simulated data and two empirical data sets, involving the settler mortality instrument of Acemoglu et al. (2001) and father's education's instrument used by Hoogerheide et al. (2012a). Even without making use of parallelized computation, an efficiency gain is observed both under strong and weak instruments, where the gain can be enormous in the latter case.

A Direct Monte Carlo (DMC) approach is introduced for posterior simulation in theInstrumental Variables (IV) model with one possibly endogenous regressor, multipleinstruments and Gaussian errors under a flat prior. This DMC method can also beapplied in an IV model (with one or multiple instruments) under an informativeprior for the endogenous regressor's effect. This DMC approach can not be appliedto more complex IV models or Simultaneous Equations Models with multiple endogenous regressors. An Approximate DMC (ADMC) approach is introduced thatmakes use of the proposed Hybrid Mixture Sampling (HMS) method, which facilitates Metropolis-Hastings (MH) or Importance Sampling from a proper marginalposterior density with highly non-elliptical shapes that tend to infinity for a pointof singularity. After one has simulated from the irregularly shaped marginal distri-bution using the HMS method, one easily samples the other parameters from theirconditional Student-t and Inverse-Wishart posteriors. An example illustrates theclose approximation and high MH acceptance rate. While using a simple candidatedistribution such as the Student-t may lead to an infinite variance of ImportanceSampling weights. The choice between the IV model and a simple linear model un-der the restriction of exogeneity may be based on predictive likelihoods, for whichthe efficient simulation of all model parameters may be quite useful. In future workthe ADMC approach may be extended to more extensive IV models such as IV withnon-Gaussian errors, panel IV, or probit/logit IV.

In this short paper we summarize the computational steps of Adaptive Radial-Based Direction Sampling (ARDS), which can be used for Bayesian analysis of ill behaved target densities. We consider one simulation experiment in order to illustrate the good performance of ARDS relative to the independence chain MH algorithm and importance sampling.