Search

In this thesis, we incorporate information theory into statistical signal processing and machine learning in order to achieve robust learning in presence of outliers (or generally in environments with non-Gaussian structure). In other words, by incorporating information theory especially when structure of environment is non-Gaussian, the efficiency of information extraction from data and consequently precision of the learning are increased. Note that, although the well-known central limit theorem creates the expectations that we should see the Gaussian distribution everywhere in the real world, this is not necessarily true. Indeed central limit theorem is concluded under some assumptions that do not hold in many real scenarios. For instance, in many real scenarios heavy-tailed distributions arise which result in outliers. The problem of learning a system affected by these types of distributions is of great importance inasmuch as conventional learning approaches that are mostly based on Gaussian assumption are not effective anymore. In this thesis, we address this problem by means of information theory. Our work finds broad applications from communication channel estimation, adaptive equalization, adaptive echo noise cancellation, system identification, underwater and satellite communications, blind source separation to physics, biology, computer science, the social sciences, and beyond where real-world environments may change in time and are conducive to non-Gaussian probability density functions. In such environments higher order statistics of data is needed, therefore conventional methodologies are not efficient anymore. Specifically, the problem of online linear regression (or interchangeably linear adaptive filtering) in environments corrupted by non-Gaussian noise is addressed in this thesis. In such environments, the error between system outputs and labels (or desired responses) does not follow a Gaussian distribution and there might exist abnormally large error samples (or outliers). The main challenge we face here is how to keep our supervised learning problem least affected by these unwanted and misleading outliers. Information theory helps us with correntropy and entropy as two robust information-theoretic criteria. Error correntropy is a similarity measure between system output and label and consequently should be maximized while error entropy denotes the uncertainty about a system and should be minimized. Although each of these two criteria has an advantage over the other one, both of them show superior performance compared to conventional mean square error in aforementioned non-Gaussian environments. In this thesis, we discuss the shortcomings of these algorithms and compare them. Moreover, we improve the algorithms based on error correntropy and error entropy by excluding major outliers from the learning process after detecting them in each iteration. Note that major outliers may occur frequently due to the nature of non-Gaussian environments and the system parameters should not be updated based on them. We use a filter to detect major outliers whose width is updated in each learning iteration based on running quartiles of the error samples. More precisely, quartiles of a data set are robust quantities of data against outliers and we use them to define boundaries by which we determine if an error sample is an outlier or not. These quartiles are functions of time and should be updated once a new error sample is available. This means that we need to deal with a running quartile problem. We propose an efficient technique for running quantile estimation based on the non-uniform quantization of error samples. This method does not need to store and sort all error samples in each iteration. Using this method alongside algorithms based on correntropy and entropy, we reject major outliers and achieve improved convergence speed and/or steady-state misalignment in the learning curves of both algorithms.

The critical behavior of systems belonging to the universality class of the three-dimensional Ising model has been studied theoretically. A three-dimensional Ising-like system with exponentially decreasing interaction potential and in the presence of a homogeneous external field was considered in the framework of the collective variables method. A specific feature in the calculation of the partition function and the free energy of a uniaxial magnet consists in singling out a reference system. The role of the latter is played by the molecular-field Hamiltonian. A method to describe the critical behavior with the use of a singled out reference system is developed on the basis of a non-Gaussian (quartic) distribution of order-parameter fluctuations (the p4 model)., Роботу присвячено теоретичному вивченню критичної поведiнки систем класу унiверсальностi тривимiрної моделi Iзинга. Тривимiрна iзингоподiбна система з експоненцiйно спадним потенцiалом взаємодiї дослiджується в методi колективних змiнних за наявностi однорiдного зовнiшнього поля. Характерною особливiстю розрахунку статистичної суми та вiльної енергiї одновiсного магнетика є видiлення системи вiдлiку. Роль останньої вiдiграє гамiльтонiан молекулярного поля. Метод опису критичної поведiнки з видiленою системою вiдлiку розвинуто на основi негаусового (четвiрного) розподiлу флуктуацiй параметра порядку (моделi p4).

Gouriéroux and Zakoïan (2013) propose to use noncausal models to parsimoniously capture nonlinear features often observed in financial time series and in particular bubble phenomena. In order to distinguish causal autoregressive processes from purely noncausal or mixed causal-noncausal ones, one has to depart from the Gaussianity assumption on the error distribution. Financial (and to a large extent macroeconomic) data are characterized by large and sudden changes that cannot be capturedby the Normal distribution, which explains why leptokurtic error distributions are often considered in empirical finance. By means of Monte Carlo simulations, this paper investigates the identication of mixed causal-noncausal models in finite samples for different values of the excess kurtosis of the error process. We compare the performance of the MLE, assuming a t-distribution, with that of theLAD estimator that we propose in this paper. Similar to Davis, Knight and Liu (1992) we find that for infinite variance autoregressive processes both the MLE and LAD estimator converge faster. We further specify the general asymptotic normality results obtained in Andrews, Breidt and Davis (2006)for the case of t-distributed and Laplacian distributed error terms. We first illustrate our analysis by estimating mixed causal-noncausal autoregressions to model the demand for solar panels in Belgium over the last decade. Then we look at the presence of potential noncausal components in daily realizedvolatility measures for 21 equity indexes. The presence of a noncausal component is confirmed in both empirical illustrations.

Geostatistical analysis has been introduced over half a century ago to allow quantifying seemingly random spatial variations in earth quantities such as rock mineral content or permeability. The traditional approach has been to view such quantities as multivariate Gaussian random functions characterized by one or a few well-defined spatial correlation scales. There is, however, mounting evidence that many spatially varying quantities exhibit non-Gaussian behavior over a multiplicity of scales. The purpose of this minireview is not to paint a broad picture of the subject and its treatment in the literature. Instead, we focus on very recent advances in the recognition and analysis of this ubiquitous phenomenon, which transcends hydrology and the Earth sciences, brought about largely by our own work. In particular, we use porosity data from a deep borehole to illustrate typical aspects of such scalable non-Gaussian behavior, describe a very recent theoretical model that (for the first time) captures all these behavioral aspects in a comprehensive manner, show how this allows generating random realizations of the quantity conditional on sampled values, point toward ways of incorporating scalable non-Gaussian behavior in hydrologic analysis, highlight the significance of doing so, and list open questions requiring further research.

We apply the Atkinson (1970) inequality index to time series of asset returns to offer a novel measure of financial risk consistent with expected-utility theory. This measure is converted to a certainty-equivalent return serving as a performance measure. We extend the Atkinson index to HARA utility and derive closed-form solutions to our measures for a number of preference-return combinations. Further, we establish relationships between risk aversion and the weights assigned to the cumulants of the return distribution for our performance measure. Using data from hedge funds and asset-pricing anomalies, we find that our performance measure contains additional, economically meaningful information.

We propose a novel semi-nonparametric distribution that is feasibly parameterized to represent the non-Gaussianities of the asset return distributions. Our Moments Expansion (ME) density presents gains in simplicity attributable to its innovative polynomials, which are defined by the difference between the nth power of the random variable and the nth moment of the density used as the basis. We show that the Gram-Charlier distribution is a particular case of the ME-type of densities. The latter being more tractable and easier to implement when quadratic transformations are used to ensure positiveness. In an empirical application to asset returns, the ME model outperforms both standard and non-Gaussian GARCH models along several risk forecasting dimensions.

We apply the Atkinson (1970) inequality index to time series of asset returns to offer a novel measure of financial risk consistent with expected-utility theory. This measure is converted to a certainty-equivalent return serving as a performance measure. We extend the Atkinson index to HARA utility and derive closed-form solutions to our measures for a number of preference-return combinations. Further, we establish relationships between risk aversion and the weights assigned to the cumulants of the return distribution for our performance measure. Using data from hedge funds and asset-pricing anomalies, we find that our performance measure contains additional, economically meaningful information.

Hyperspectral imaging (HSI) extends from the fact that for any given material, the amount of emitted radiation varies with wavelength. HSI sensors measure the radiance of the materials within each pixel area at a very large number of contiguous spectral bands and provide image data containing both spatial and spectral information. Classical adaptive detection schemes assume that the background is zero-mean Gaussian or with known mean vector that can be exploited. However, when the mean vector is unknown, as it is the case for hyperspectral imaging, it has to be included in the detection process. We propose in this work an extension of classical detection methods when both covariance matrix and mean vector are unknown.However, the actual multivariate distribution of the background pixels may differ from the generally used Gaussian hypothesis. The class of elliptical distributions has already been popularized for background characterization in HSI. Although these non-Gaussian models have been exploited for background modeling and detection schemes, the parameters estimation (covariance matrix, mean vector) is usually performed using classical Gaussian-based estimators. We analyze here some robust estimation procedures (M-estimators of location and scale) more suitable when non-Gaussian distributions are assumed. Jointly used with M-estimators, these new detectors allow to enhance the target detection performance in non-Gaussian environment while keeping the same performance than the classical detectors in Gaussian environment. Therefore, they provide a unified framework for target detection and anomaly detection in HSI., L'imagerie hyperspectrale (HSI) repose sur le fait que, pour un matériau donné, la quantité de rayonnement émis varie avec la longueur d'onde. Les capteurs HSI mesurent donc le rayonnement des matériaux au sein de chaque pixel pour un très grand nombre de bandes spectrales contiguës et fournissent des images contenant des informations à la fois spatiale et spectrale. Les méthodes classiques de détection adaptative supposent généralement que le fond est gaussien à vecteur moyenne nul ou connu. Cependant, quand le vecteur moyen est inconnu, comme c'est le cas pour l'image hyperspectrale, il doit être inclus dans le processus de détection. Nous proposons dans ce travail d'étendre les méthodes classiques de détection pour lesquelles la matrice de covariance et le vecteur de moyenne sont tous deux inconnus.Cependant, la distribution statistique multivariée des pixels de l'environnement peut s'éloigner de l'hypothèse gaussienne classiquement utilisée. La classe des distributions elliptiques a été déjà popularisée pour la caractérisation de fond pour l’HSI. Bien que ces modèles non gaussiens aient déjà été exploités dans la modélisation du fond et dans la conception de détecteurs, l'estimation des paramètres (matrice de covariance, vecteur moyenne) est encore généralement effectuée en utilisant des estimateurs conventionnels gaussiens. Dans ce contexte, nous analysons de méthodes d’estimation robuste plus appropriées à ces distributions non-gaussiennes : les M-estimateurs. Ces méthodes de détection couplées à ces nouveaux estimateurs permettent d'une part, d'améliorer les performances de détection dans un environment non-gaussien mais d'autre part de garder les mêmes performances que celles des détecteurs conventionnels dans un environnement gaussien. Elles fournissent ainsi un cadre unifié pour la détection de cibles et la détection d'anomalies pour la HSI.

It is well known that time series of returns are characterized by volatility clustering and excess kurtosis. Therefore, when modelling the dynamic behavior of returns, inference and prediction methods, based on independent and/or Gaussian observations may be inadequate. As bootstrap methods are not, in general, based on any particular assumption on the distribution of the data, they are well suited for the analysis of returns. This paper reviews the application of bootstrap procedures for inference and prediction of financial time series. In relation to inference, bootstrap techniques have been applied to obtain the sample distribution of statistics for testing, for example, autoregressive dynamics in the conditional mean and variance, unit roots in the mean, fractional integration in volatility and the predictive ability of technical trading rules. On the other hand, bootstrap procedures have been used to estimate the distribution of returns which is of interest, for example, for Value at Risk (VaR) models or for prediction purposes. Although the application of bootstrap techniques to the empirical analysis of financial time series is very broad, there are few analytical results on the statistical properties of these techniques when applied to heteroscedastic time series. Furthermore, there are quite a few papers where the bootstrap procedures used are not adequate. Publicado

Non-Gaussian limiting distributions of the rescaling solutions of the heat equation for non-Gaussian initial data with long-range dependence are discribed in terms of their multiple stochastic integral representations.

In the first part of this thesis, I address the classical problem of asset price dynamics based on a new theoretical framework developed for nonequilibrium physical systems. This problem is mainly relevant for two reasons. First, because understanding the true distribution of returns is important for asset allocation, risk management, and option pricing. Second, because in spite of all the effort in determining the origin of non-Gaussian returns no conclusive result has been achieved yet. The most important result of this part is the demostration that the non-Gaussian shape and stable scaling of the returns distribution are due to slow, but significant, fluctuations in volatility. Futhermore, this result suggests that stock price fluctuations are universal, and that return distributions can be described by one functional form. In the second part, I present an empirical study about the execution of large orders in two stock exchanges: the London Stock Exchange, and the Spanish Stock Exchange. This type of orders can cause a tremendous impact because they are larger than the available liquidity in the order book at a time. For this reason, they are split to minimize transaction costs. Market price impact is the basic factor of these costs, so an accurate description of its functional form is necessary to any optimal execution. The most important result in this part is the empirical determination of this functional form in two markets and the finding of a common behavior in both markets that can be summarized into a concave temporary impact, roughly described by a square root function of the hidden order size, and a price reversion after the completion of the hidden order making permanent impact equal to roughly half of the temporary impact.

In the first part of this thesis, I address the classical problem of asset price dynamics based on a new theoretical framework developed for nonequilibrium physical systems. This problem is mainly relevant for two reasons. First, because understanding the true distribution of returns is important for asset allocation, risk management, and option pricing. Second, because in spite of all the effort in determining the origin of non-Gaussian returns no conclusive result has been achieved yet. The most important result of this part is the demostration that the non-Gaussian shape and stable scaling of the returns distribution are due to slow, but significant, fluctuations in volatility. Futhermore, this result suggests that stock price fluctuations are universal, and that return distributions can be described by one functional form. In the second part, I present an empirical study about the execution of large orders in two stock exchanges: the London Stock Exchange, and the Spanish Stock Exchange. This type of orders can cause a tremendous impact because they are larger than the available liquidity in the order book at a time. For this reason, they are split to minimize transaction costs. Market price impact is the basic factor of these costs, so an accurate description of its functional form is necessary to any optimal execution. The most important result in this part is the empirical determination of this functional form in two markets and the finding of a common behavior in both markets that can be summarized into a concave temporary impact, roughly described by a square root function of the hidden order size, and a price reversion after the completion of the hidden order making permanent impact equal to roughly half of the temporary impact.

The reduced form of the local level model with conditionally heteroscedastic GARCH(1,1) noises is analyzed. We show that the IMA-GARCH model is a good alternative but its conditional heteroscedasticity is weaker than this of the unobserved disturbances. Financial support from Project SEJ2006-03919 by the Spanish Government is gratefully acknowledged Publicado

As GARCH models and stable Paretian distributions have been revisited in the recent past with the papers of Hansen and Lunde (J Appl Econom 20: 873–889, 2005) and Bidarkota and McCulloch (Quant Finance 4: 256–265, 2004), respectively, in this paper we discuss alternative conditional distributional models for the daily returns of the US, German and Portuguese main stock market indexes, considering ARMA-GARCH models driven by Normal, Student’s t and stable Paretian distributed innovations. We find that a GARCH model with stable Paretian innovations fits returns clearly better than the more popular Normal distribution and slightly better than the Student’s t distribution. However, the Student’s t outperforms the Normal and stable Paretian distributions when the out-of-sample density forecasts are considered. info:eu-repo/semantics/acceptedVersion

This paper introduces a new family of multivariate distributions based on Gram–Charlier and Edgeworth expansions. This family encompasses many of the univariate semi-non-parametric densities proposed in financial econometrics as marginal of its different formulations. Within this family, we focus on the analysis of the specifications that guarantee positivity to obtain well-defined multivariate semi-non-parametric densities. We compare two different multivariate distributions of the family with the multivariate Edgeworth–Sargan, Normal, Student's t and skewed Student's t in an in- and out-of-sample framework for financial returns data. Our results show that the proposed specifications provide a reasonably good performance, and would therefore be of interest for applications involving the modelling and forecasting of heavy-tailed distributions.

A new bootstrap procedure to obtain prediction densities of returns and volatilities of GARCH processes is proposed. Financial market participants have shown an increasing interest in prediction intervals as measures of uncertainty. Furthermore, accurate predictions of volatilities are critical for many financial models. The advantages of the proposed method are that it allows incorporation of parameter uncertainty and does not rely on distributional assumptions. The finite sample properties are analyzed by an extensive Monte Carlo simulation. Finally, the technique is applied to the Madrid Stock Market index, IBEX-35. Acknowledgements: We are very grateful for their helpful comments by three anonymous referees, the editor Stephen Pollock and seminar participants at the Universities of Valladolid, New South Wales and Canterbury and the June 2001 Time Series Workshop of Arrabida, the September 2001 International Conference on Modelling Volatility (Perth) and the June 2002 International Symposium on Forecasting (Dublin). We are also grateful to Gregorio Serna for providing the data set analyzed in this paper and to Dolores Redondas for helping us with the figures. Financial support was provided by projects DGES PB96-0111 and BEC2002-03720 from the Spanish Government and Cátedra de Calidad from BBVA Publicado

We describe financial systems as condensates, similar to Bose-Einstein condensates, and calculate equilibrium statistical distributions following from the model. The calculated distribution of investments into speculated financial asset is exponentially truncated Pareto distribution, and the calculated distribution of the price moves is exponentially truncated Levy distribution. The calculated from the model distributions correspond well to the empirically observed distributions.

In this paper, we propose a bootstrap procedure to construct prediction intervals for future values of a variable after a linear ARIMA model has been fitted to its power transformation. The procedure is easy to implement and provides a useful tool in empirical applications given that it is often the case that, for example, the log-transformation is modeled when the variable of interest for prediction is the original one. The advantages over existing methods for computing prediction intervals of power-transformed time series are that the proposed bootstrap intervals incorporate the variability due to parameter estimation and do not rely on distributional assumptions neither on the original variable nor on the transformed one. We derive the asymptotic distribution and show the good behavior of the bootstrap approach versus alternative procedures by means of Monte Carlo experiments. Finally, the procedure is illustrated by analyzing three real-time series data sets. The authors are very grateful to Lutz Kilian for providing the RXR and NBRX series and to the associate editor Antonio García Ferrer and two anonymous referees for valuable comments that have been very helpful to improve the paper. Financial support was provided by projects DGES PB96-0111 and BEC 2002-03720 from the Spanish Government and Catedra de Calidad from BBV. Publicado

The distribution of resistance fluctuations of conducting thin films with granular structure near electrical breakdown is studied by numerical simulations. The film is modeled as a resistor network in a steady state determined by the competition between two biased processes, breaking and recovery. Systems of different sizes and with different levels of internal disorder are considered. Sharp deviations from a Gaussian distribution are found near breakdown and the effect increases with the degree of internal disorder. However, we show that in general this non-Gaussianity is related to the finite size of the system and vanishes in the large size limit. Nevertheless, near the critical point of the conductor-insulator transition, deviations from Gaussianity persist when the size is increased and the distribution of resistance fluctuations is well fitted by the universal Bramwell-Holdsworth-Pinton distribution., 8 pages, 6 figures; accepted for publication on Physica A

In this study, we propose a new bootstrap strategy to obtain prediction intervals for autoregressive integrated moving-average processes. Its main advantage over other bootstrap methods previously proposed for autoregressive integrated processes is that variability due to parameter estimation can be incorporated into prediction intervals without requiring the backward representation of the process. Consequently, the procedure is very flexible and can be extended to processes even if their backward representation is not available. Furthermore, its implementation is very simple. The asymptotic properties of the bootstrap prediction densities are obtained. Extensive finite-sample Monte Carlo experiments are carried out to compare the performance of the proposed strategy vs. alternative procedures. The behaviour of our proposal equals or outperforms the alternatives in most of the cases. Furthermore, our bootstrap strategy is also applied for the first time to obtain the prediction density of processes with moving-average components. Publicado