Search

By making use of Kullback-Leibler information, we develop a new approach for the quickest detection problem for general statistical models with dependent observations and unknown postchange distributions; the postchange distribution depends on either unknown informative parameters or unknown nonparametric infinite-dimensional nuisance functions. For such models, we introduce a robust risk as the supremum of the mean detection delay over the class of postchange distributions. On the basis of the window-limited cumulative sum rules developed by Lai in 1988, we propose new detection procedures, making use of the noise density that minimizes the Kullback-Leibler divergence. Then for the constructed detection procedures, we provide sufficient conditions on the considered statistical models that ensure minimax optimality properties with respect to the robust risk. We apply the developed methods to the quick detection problems for both scalar and multivariate autoregressive processes with unknown postchange parameters and unknown noise distributions.

International audience; Discretization of continuous time autoregressive (AR) processes driven by a Brownian motion and embedding of discrete time AR sequences driven by a Gaussian white noise are classical issues. The article aims at establishing and using such discretization and embedding formulae between extended AR continuous time processes and discrete time sequences. The continuous-time processes are driven by either Brownian or jump processes, and may have random coefficients depending on time; Levy-driven processes are also considered. The innovation of the discrete time processes may be of many types - including Gaussian. In one way, observing the continuous time AR process at discrete times leads the AR dynamics of the discretized process to be characterized. The other way round, AR sequences can be embedded, in the almost sure sense, into continuous time AR processes with the same dynamics. Illustration is provided through many examples and simulation.

We introduce a methodology to study the possible matter flows of an ecosystem defined by observational biomass data and realistic biological constraints. The flows belong to a polyhedron in a multi dimensional space that may make statistical exploration difficult in practice; instead, we propose to solve a convex optimization problem. Seven criteria based on ecological network indices have been selected to be used as convex goal functions. Numerical results show that the method is fast and can be used for large systems. Minimum flow solutions are analyzed using flow decomposition in paths and circuits. Their consistency is also tested by introducing a system of differential equations for the biomasses and examining the stability of the biomass fixed point. The method is illustrated and explained throughout the text on an ecosystem toy model. It is also applied to realistic food models.

This paper mainly aims at unifying as a unique goodness-of-fit procedure the tests based on Shannon entropy–called S-tests–introduced by Vasicek in 1976, and the tests based on relative entropy–or Kullback-Leibler divergence, called KL-tests–introduced by Song in 2002. While Vasicek’s procedure is widely used in the literature, Song’s has remained more confidential. Both tests are known to have good power properties and to lead to straightforward computations. However, some asymptotic properties of the S-tests have never been checked and the link between the two procedures has never been highlighted. Mathematical justification of both tests is detailed here, leading to show their equivalence for testing any parametric composite null hypothesis of maximum entropy distributions. For testing any other distribution, the KL-tests are still reliable goodness-of-fit tests, whereas the S-tests become tests of entropy level. Moreover, for simple null hypothesis, only the KL-tests can be considered. The met...

International audience; Zighera (App Stoch Mod Data Anal 1:93–108 1985) introduced a new parameterization of log-linear models for analyzing categorical data, directly linked to a thorough analysis of discrimination information through Kullback-Leibler divergence. The method mainly aims at quantifying in terms of information the variations of a binary variable of interest, by comparing two contingency tables – or sub-tables – through effects of explanatory categorical variables. The present paper settles the mathematical background necessary to rigorously apply Zighera’s parameterization to any categorical data. In particular, identifiability and good properties of asymptotically χ 2-distributed test statistics are proven to hold. Determination of parameters and all tests of effects due to explanatory variables are simultaneous. Application to classical data sets illustrates contribution with respect to existing methods.

[Departement_IRSTEA]Eaux [TR1_IRSTEA]QUASARE [ADD1_IRSTEA]Dynamique et fonctionnement des écosystèmes(IF 4.49 [2018]; Q1); International audience; A combination of modelling tools was applied to simulate the impacts of the future Courseulles-sur-mer offshore wind farm (OWF) construction (Bay of Seine, English Channel) on the ecosystem structure and functioning. To do so, food-web models of the ecosystem under three scenarios were constructed to investigate the effect caused by the OWF of added substrate (reef effect), fishing restriction (reserve effect), and their combined effect. Further, Ecological Network Analysis indices and Mean Trophic Level were derived to investigate their suitability for detecting changes in the ecosystem state. Our analysis suggests changes in the ecosystem structure and functioning after the OWF construction, the ecosystem maturity was predicted to increase, but no alterations in its overall resilience capacity.

International audience; Based on rescaling by some suitable sequence instead of the number of time units, the usual notion of divergence rate is here extended to define and determine meaningful generalized divergence rates. Rescaling entropy rates appears as a special case. Suitable rescaling is naturally induced by the asymptotic behavior of the marginal divergences. Closed form formulas are obtained as soon as the marginal divergences behave like powers of some analytical functions. A wide class of countable Markov chains is proven to satisfy this property. Most divergence and entropy functionals defined in the literature are concerned, e.g., the classical Shannon, Kullback-Leibler, R´enyi, Tsallis. For illustration purposes, Ferreri or Basu-Harris-Hjort-Jones – among others – are also considered.

International audience; Optimality properties of decision procedures are studied for the quickest detection of a change-point of parameters in autoregressive and other Markov type sequences. The limit of the normalized conditional log-likelihood ratios is shown to exist for Markov chains satisfying the ergodic theorem of information theory. Then closed-form expressions for this limit are derived by making use of the time average rate of Kullback-Leibler divergence. The good properties of the detection procedures based on a sequential analysis approach are proven to hold thanks to geometric ergodicity properties of the observation processes. In particular, the window-limited CUSUM rule is shown to be optimal for detecting the disruption point in autore-gressive models. Sparre Andersen models are specifically studied.

This chapter is devoted to jump Markov processes and finite semi-Markov processes. In both cases, the index is considered as the calender time, continuously counted over the positive real line. Markov processes are continuous-time processes that share the Markov property with the discrete-time Markov chains. Their future evolution conditional to the past depends only on the last occupied state. Their extension to the so-called semi-Markov processes naturally arises in many types of applications. The future evolution of a semi-Markov process given the past depends on the occupied state too, but also on the time elapsed since the last transition.

Random sequences are understood here as real random processes indexed by discrete time. Advanced notions on sequences of real random variables are presented. Apart for their intrinsic value, they are thought as a tool box for the other chapters. They will be completed in Chap. 4 by definitions and properties common to all real random processes, either indexed by continuous or by discrete time.

In this chapter, conditional distributions and expectations are presented, following steps of increasing difficulty. Conditional distributions are first supposed to exist, which is true for all random variables taking values in \(\mathbb {R}^d\). Still, conditional expectation is defined in the most general case. A section is dedicated to determining practically conditional distributions and expectations, and another to the linear model in which a random phenomenon is assumed to be linearly related to other simultaneously observed phenomena.

Closed-form expressions for generalized entropy rates of Markov chains are obtained through pertinent averaging. First, the rates are expressed in terms of Perron-Frobenius eigenvalues of perturbations of the transition matrices. This leads to a classification of generalized entropy functionals into five exclusive types. Then, a weighted expression is obtained in which the associated Perron-Frobenius eigenvectors play the same role as the stationary distribution in the well-known weighted expression of Shannon entropy rate. Finally, all terms are shown to bear a meaning in terms of dynamics of an auxiliary absorbing Markov chain through the notion of quasi-limit distribution. Illustration of important properties of the involved spectral elements is provided through application to binary Markov chains.

A stochastic process represents a system, usually evolving along time, which incorporates an element of randomness, as opposed to a deterministic process.

For Markov chains with transition probabilities \(p_{ij}\), the Shannon entropy rate is well-known to be equal to the sum of the \(-\sum _j p_{ij}\log p_{ij}\) weighted by the stationary distribution. This expression derives from the chain rule specific to Shannon entropy. For an ergodic Markov chain, the stationary distribution is the limit of the marginal distributions of the chain.

00000; International audience; During 2002–2005, a new container terminal in the commercial harbour of Le Havre, named “Port2000”, was built on the northern flank of the Seine estuary, northern France. This extension is already known to have modified the estuary current and sediment dynamics, as well as reducing biomass of the suprabenthos assemblage, for the whole downstream part of the system. However, studies on other biotic communities were largely inconclusive, and an ecosystem-wide analysis was still lacking. Here, we performed a before/after study of ecosystem dynamics of the different habitats of the Seine estuary, using a Linear Inverse Modelling technique (LIM-MCMC) to estimate all flows occurring in the food web. Ecological Network Analysis indices were calculated, summarising ecosystem functioning traits and giving indications about the habitat health status. Results showed that the southern flank (FS, Fosse Sud) exhibits all characteristics to be considered as the least stressed habitat of the estuary: system activity and functional specialisation of flows were stable between periods, ecosystem recycling processes and detrital dynamics were also stable; an increase in trophic specialisation (decrease in system omnivory) was the only change confirming a general ecological succession. The northern flank (FN, Fosse Nord), where the actual terminal was built, showed a food web with increased importance of lower trophic levels (increased detritivory and carbon recycling), increased stability and flow efficiency, but possibly regressed to a previous step in ecological succession. In the central navigation channel (CH), patterns of network indices were overall inconclusive and the general image is one of a constantly shifting food web, a condition possibly caused by the year-round dredging activities. The functioning of the Seine estuary – especially of FN and FS – seems to have been modified by the combination of harbour construction and the related mitigation measures. Network indices partially captured this combination of changes and, although not fully operational yet, they are promising tools to comply with the European Union mandate of defining ecosystem health status.

The aim of this paper is to give an overview of the structure of the class of discrete time wide Markov processes, either periodically correlated or multivariate stationary. We show many properties of their covariance, correlation and reflection coefficients matrices. We characterize these processes chiefly in terms of autoregressive models of order one. Illustrative numerical examples are given.

We determine the form of spectral densities of multidimensional scalar processes which minimize a relative entropy under a finite number of general moment-type constraints. The obtained theoretical results are applied to spectral densities of weakly stationary processes under covariances, inverse covariances and cepstral or impulse response constraints. Invariance properties of the class of autoregressive moving-average (ARMA) processes are shown to hold under the relative entropy minimization principle for many choices of entropy.

We consider here ergodic homogeneous Markov chains with countable state spaces. The entropy rate of the chain is an explicit function of its transition and stationary distributions. We construct estimators for this entropy rate and for the entropy of the stationary distribution of the chain, in the parametric and nonparametric cases. We study estimation from one sample with long length and from many independent samples with given length. In the parametric case, the estimators are deduced by plug-in from the maximum likelihood estimator of the parameter. In the nonparametric case, the estimators are deduced by plug-in from the empirical estimators of the transition and stationary distributions. They are proven to have good asymptotic properties.

The literature about maximum of entropy for Markov processes deals mainly with discrete-time Markov chains. Very few papers dealing with continuous-time jump Markov processes exist and none dealing with semi-Markov processes. It is the aim of this paper to contribute to fill this lack. We recall the basics concerning entropy for Markov and semi-Markov processes and we study several problems to give an overview of the possible directions of use of maximum entropy in connection with these processes. Numeric illustrations are presented, in particular in application to reliability.

We are concerned with introducing entropy in the field of countable discrete-time semi-Markov process theory. We define the entropy of the finite distributions of the semi-Markov chain and obtain explicitly its entropy rate by extending the Shannon–McMillan–Breiman theorem to this class of non-stationary discrete-time processes. We also define the relative entropy rate between two semi-Markov chains. We then develop some maximum entropy methods for these processes.

The aim of this paper is to define the entropy of a finite semi-Markov process. We define the entropy of the finite distributions of the process, and obtain explicitly its entropy rate by extending the Shannon–McMillan–Breiman theorem to this class of nonstationary continuous-time processes. The particular cases of pure jump Markov processes and renewal processes are considered. The relative entropy rate between two semi-Markov processes is also defined.

A one-to-one relationship exists between scalar periodically correlated nonstationary processes and multivariate stationary processes. This fact allows us to transfer results proven for ones to the others. We are interested in a probabilistic approach of results sometimes already known in a different (analytical or numerical) context, in order to simplify, generalize and unify them. We use a probabilistic approach of generalized reflection coefficients to give a constructive condition of extension of partial covariance sequences, achieved by an autoregressive model. We develop a Trench–Zohar recursion for the nonstationary case which leads to an economical algorithm to solve the associated Yule–Walker equations. Shannon and Burg entropies are linked through a Szego type theorem. A numerical example is given.

A parallel is made between the role played by covariances in the determination of auto-regressive models and the role played by impulse responses in the determination of ARMA models. Auto-regressive models are known to maximize the Burg-entropy under covariance constraints. Auto-regressive-moving-average models give the maximum of the Burg-entropy among processes sharing the same covariances and impulse responses up to a certain lag. Such models are constructed by iterative or algebraic methods under the different constraints. A new recursive method of identification of the order of an ARMA model is also developed, based on the generalized reflection coefficients.

International audience; —We study entropy rates of random sequences for general entropy functionals including the classical Shannon and Rényi entropies and the more recent Tsallis and Sharma-Mittal ones. In the first part, we obtain an explicit formula for the entropy rate for a large class of entropy functionals, as soon as the process satisfies a regularity property known in dynamical systems theory as the quasi-power property. Independent and identically distributed sequence of random variables naturally satisfy this property. Markov chains are proven to satisfy it too, under simple explicit conditions on their transition probabilities. All the entropy rates under study are thus shown to be either infinite or zero except at a threshold where they are equal to Shannon or Rényi entropy rates up to a multiplicative constant. In the second part, we focus on the estimation of the marginal generalized entropy and entropy rate for parametric Markov chains. Estimators with good asymptotic properties are built through a plug-in procedure using a maximum likelihood es-timation of the parameter.

The aim of this paper is to present methods of maximization of entropy rate in the context of birth‐death Markov processes used in queueing theory. Several types of birth‐death queueing systems with exponential arrivals and service distributions are investigated.

The aim of this paper is to define the entropy rate of a semi-Markov process with a Borel state space by extending the strong asymptotic equirepartition property (also called the ergodic theorem of information theory or Shannon-McMillan-Breiman theorem) to this class of non-stationary processes. The mean asymptotic equirepartition property (also called the Shannon-McMillan theorem) is also proven to hold. The relative entropy rate between two semi-Markov processes is defined. All earlier results concerning entropy for semi-Markov processes, jump Markov processes and Markov chains thus appear as special cases. Two invariance principles are established for entropy, one for the central limit theorem and the other for the law of the iterated logarithm.

The purpose of this paper is to review the different generalizations and extensions of the ergodic theorem of information theory in terms of reference measure, state space, index set and required properties (ergodicity, stationarity, etc.) of the process, from the original Shannon-McMillan-Breiman version to its latest developments.

The entropy rate of an ergodic homogeneous Markov chain taking only two values is an explicit function of its transition probabilities. We study a plug‐in estimator of this entropy rate obtained from the observation of one trajectory with long length. Its exact asymptotic distribution is given. The case of uniform transition probabilities is especially considered. A detailed numerical study using simulation results is provided.

We present an extended definition of the second-order stationarity concept. This is based on the theory of harmonic analysis for semigroups with involution. It provides a spectral representation for a wide class of processes which are non-stationary in the usual weak sense, and allows miscellaneous spectral representation results to be unified. Many applications are given to illustrate the concept. Most of these are already known, %as symmetric, locally %stationary, stationary reducible by space transformation, %multiplicative-stationary processes or processes with independent %increments. but some are new, such as the multiplicative-symmetric processes. We are less concerned with proving fundamental results than with opening up a new field of investigation for spectral representation of non-stationary processes.

We consider zero mean weakly stationary multidimensional scalar time series. We determine the form of the spectral density which minimizes a relative entropy under trigonometric moment constraints, as covariance, impulse responses or cepstral coefficients. It often yields autoregressive moving average models giving one more justification to their widespread use.

Let K be a finite subset of ℤr including 0 = (0,…, 0). Let Φ = (φ k ) k∈K be a family of measurable and linearly independent complex or real functions denned over a compact \(I \subset \mathbb{R}^{r}\), with \(\phi _{0}\equiv 1\). Let p be a reference measure over I.

International audience; We begin with two possible extensions of Stam's inequality and of de Bruijn's identity. In both cases, a generalized q-Gaussian plays the same role as the standard Gaussian in the classical case. These generalized q-Gaussians are important in several areas of physics and mathematics. A generalized Fisher information also pops up, playing the same role as the classical Fisher information, but for the extended identity and inequality. In the estimation theory context, we give several extensions of the Cramér-Rao inequality in the multivariate case, with matrix versions as well as versions for general norms. We define new forms of Fisher information, that reduce to the classical one in special cases. In the case of a translation parameter, the general Cramér-Rao inequalities lead to an inequality for distributions, which involves the same generalized Fisher information as in the generalized de Bruijn's identity and Stam's inequality. This Cramér-Rao inequality is saturated by generalized q-Gaussian distributions. This shows that the generalized q-Gaussians also minimize the generalized Fisher information among dis-tributions with a fixed moment. Similarly, the generalized q-Gaussians also minimize the generalized Fisher information among distributions with a given q-entropy.

This PhD report deals with the estimation of both Shannon entropy of distributions from independent and Markovian data and entropy rate of pure jump Markov processes with finite state space. In the latter case, different schemes of continuous and discrete observation of the processes are considered. Several related problems are studied. Kullback-Leibler information geometry linked with escort transformations come ahead of estimation. Others appear as applications of the estimation results. Tests on the entropy level of a distribution are derived from a large deviation principle satisfied by the sequence of empirical estimators of the entropy of a distribution. Properties related to entropy of birth and death Markovian queueing systems are also considered., On étudie à la fois l'estimation de l'entropie de Shannon d'une probabilité à partir d'observations indépendantes ou markoviennes, et l'estimation du taux d'entropie d'un processus markovien de sauts d'espace d'état fini, à partir d'observations continues ou discrètes. Plusieurs problèmes connexes sont traités. Certains apparaissent en amont de l'estimation, comme l'étude de la géométrie de la divergence de Kullback-Leibler en lien avec la transformation escorte. D'autres apparaissent comme des applications des résultats d'estimation obtenus. On construit ainsi des tests sur le niveau d'entropie d'une probabilité, à partir d'un principe de grandes déviations pour la suite des estimateurs empiriques de l'entropie d'une suite de variables indépendantes. On étudie également diverses propriétés en lien avec l'estimation de l'entropie et du taux d'entropie de files d'attente modélisées par des processus markoviens de naissance et de mort.