Search

This study aims to estimate the parameters of a stochastic exposed-infected epidemiological model for the transmission dynamics of notifiable infectious diseases, based on observations related to isolated cases counts only. We use the setting of hidden multi-chain Markov models and adapt the Baum-Welch algorithm to the special structure of the multi-chain. From the estimated transition matrix, we retrieve the parameters of interest (contamination rates, incubation rate, and isolation rate) from analytical expressions of the moments and Monte Carlo simulations. The performance of this approach is investigated on synthetic data, together with an analysis of the impact of using a model with one less compartment to fit the data in order to help for model selection.

Designing patient-specific follow-up strategy is a crucial step towards personalized medicine in cancer. Tools to help doctors deciding on treatment allocation together with next visit date, based on patient preferences and medical observations, would be particularly beneficial. Such tools should be based on realistic models of disease progress under the impact of medical treatments, involve the design of (multi-)objective functions that a treatment strategy should optimize along the patient's medical journey, and include efficient resolution algorithms to optimize personalized follow-up by taking the patient's history and preferences into account. We propose to model cancer evolution with a Piecewise Deterministic Markov Process where patients alternate between remission and relapse phases with disease-specific tumor evolution. This model is controlled via the online optimization of a long-term cost function accounting for treatment side-effects, hospital visits burden and disease impact on the quality of life. Optimization is based on noisy measurements of blood markers at visit dates. We leverage the Partially-Observed Monte-Carlo Planning algorithm to solve this continuous-time, continuous-state problem, taking advantage of the nearly-deterministic nature of cancer evolution. We show that this approximate solution approach of the exact model performs better than the counterpart exact resolution of the discrete model, while allowing for more versatility in the cost function model.

In this paper, we use a linear birth and death process with immigration to model infectious disease propagation when contamination stems from both person-to-person contact and contact with the environment. Our aim is to estimate the parameters of the process. The main originality and difficulty comes from the observation scheme. Counts of infected population are hidden. The only data available are periodic cumulated new retired counts. Although very common in epidemiology, this observation scheme is mathematically challenging even for such a standard stochastic process. We first derive an analytic expression of the unknown parameters as functions of well-chosen discrete time transition probabilities. Second, we extend and adapt the standard Baum-Welch algorithm in order to estimate the said discrete time transition probabilities in our hidden data framework. The performance of our estimators is illustrated both on synthetic data and real data of typhoid fever in Mayotte.

Designing efficient and rigorous numerical methods for sequential decision-making under uncertainty is a difficult problem that arises in many applications frameworks. In this paper we focus on the numerical solution of a subclass of impulse control problem for piecewise deterministic Markov process (PDMP) when the jump times are hidden. We first state the problem as a partially observed Markov decision process (POMDP) on a continuous state space and with controlled transition kernels corresponding to some specific skeleton chains of the PDMP. Then we proceed to build a numerically tractable approximation of the POMDP by tailor-made discretizations of the state spaces. The main difficulty in evaluating the discretization error come from the possible random or boundary jumps of the PDMP between consecutive epochs of the POMDP and requires special care. Finally we extensively discuss the practical construction of discretization grids and illustrate our method on simulations.

We study the long time behavior of an asymmetric size-structured measure-valued growth-fragmentation branching process that models the dynamics of a population of cells taking into account physiological and morphological asymmetry at division. We show that the process exhibits a Malthusian behavior; that is that the global population size grows exponentially fast and that the trait distribution of individuals converges to some stable distribution. The proof is based on a generalization of Lyapunov function techniques for non-conservative semi-groups. We then investigate the fluctuations of the growth rate with respect to the parameters guiding asymmetry. In particular, we exhibit that, under some special assumptions, asymmetric division is optimal in a Darwinian sense.

This paper investigates the random horizon optimal stopping problem for measure-valued piecewise deterministic Markov processes (PDMPs). This is motivated by population dynamics applications, when one wants to monitor some characteristics of the individuals in a small population. The population and its individual characteristics can be represented by a point measure. We first define a PDMP on a space of locally finite measures. Then we define a sequence of random horizon optimal stopping problems for such processes. We prove that the value function of the problems can be obtained by iterating some dynamic programming operator. Finally we prove on a simple counter-example that controlling the whole population is not equivalent to controlling a random lineage.

We consider a change-point detection problem for a simple class of Piecewise Deterministic Markov Processes (PDMPs). A continuous-time PDMP is observed in discrete time and through noise, and the aim is to propose a numerical method to accurately detect both the date of the change of dynamics and the new regime after the change. To do so, we state the problem as an optimal stopping problem for a partially observed discrete-time Markov decision process taking values in a continuous state space and provide a discretization of the state space based on quantization to approximate the value function and build a tractable stopping policy. We provide error bounds for the approximation of the value function and numerical simulations to assess the performance of our candidate policy.

We present in this paper a numerical method which computes the optimal trajectory of a underwater vehicle subject to some mission objectives. The method is applied to a submarine whose goal is to best detect one or several targets, or/and to minimize its own detection range perceived by the other targets. The signal considered is acoustic propagation attenuation. Our approach is based on dynamic programming of a finite horizon Markov decision process. A quantization method is applied to fully discretize the problem and allows a numerically tractable solution. Different scenarios are considered. We suppose at first that the position and the velocity of the targets are known and in the second we suppose that they are unknown and estimated by a Kalman type filter in a context of bearings-only tracking.

This paper deals with the general discounted impulse control problem of a piecewise deterministic Markov process. We investigate a new family of epsilon-optimal strategies. The construction of such strategies is explicit and only necessitates the previous knowledge of the cost of the no-impulse strategy. In particular, it does not require the resolution of auxiliary optimal stopping problem or the computation of the value function at each point of the state space. This approach is based on the iteration of a single-jump-or-intervention operator associated to the piecewise deterministic Markov process.

This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization procedure based on optimal quantization. First,we discretize the state space of the unobserved variable by quantizing an underlying reference measure. Then we jointly discretize the resulting approximate filter and the observation process. We obtain a fully computable approximation of the value function with explicit error bounds for its convergence towards the true value fonction.

This paper is dedicated to the numerical study of the optimization of an industrial launcher integration process. It is an original case of inventory-production system where a calendar plays a crucial role. The process is modeled using the Markov Decision Processes (MDPs) framework. Classical optimization procedures for MDPs cannot be used because of specificities of the transition law and cost function. Two simulation-based algorithms are tuned to fit this special case. We obtain a non trivial optimal policy that can be applied in practice and significantly outperforms reference policies.

New linear minimum mean square estimators are introduced in this paper by considering a cluster information structure in the filter design. The set of filters constructed in this way can be ordered in a lattice according to the refines of clusters of the Markov chain, including the linear Markovian estimator at one end (with only one cluster) and the Kalman filter at the other hand (with as many clusters as Markov states). The higher is the number of clusters, the heavier are pre-compuations and smaller is the estimation error, so that the cluster cardinality allows for a trade-off between performance and computational burden. In this paper we propose the estimator, give the formulas for pre-computation of gains, present some properties, and give an illustrative numerical example.

The data we analyze derives from the observation of numerous cells of the bacterium Escherichia coli (E. coli) growing and dividing. Single cells grow and divide to give birth to two daughter cells, that in turn grow and divide. Thus, a colony of cells from a single ancestor is structured as a binary genealogical tree. At each node the measured data is the growth rate of the bacterium. In this paper, we study two different data sets. One set corresponds to small complete trees, whereas the other one corresponds to long specific sub-trees. Our aim is to compare both sets. This paper is accessible to post graduate students and readers with advanced knowledge in statistics.

The aim of this paper is to propose a new numerical approximation of the Kalman-Bucy filter for semi-Markov jump linear systems. This approximation is based on the selection of typical trajectories of the driving semi-Markov chain of the process by using an optimal quantization technique. The main advantage of this approach is that it makes pre-computations possible. We derive a Lipschitz property for the solution of the Riccati equation and a general result on the convergence of perturbed solutions of semi-Markov switching Riccati equations when the perturbation comes from the driving semi-Markov chain. Based on these results, we prove the convergence of our approximation scheme in a general infinite countable state space framework and derive an error bound in terms of the quantization error and time discretization step. We employ the proposed filter in a magnetic levitation example with markovian failures and compare its performance with both the Kalman-Bucy filter and the Markovian linear minimum mean squares estimator.

We present a numerical method to compute an optimal maintenance date for the test case of the heated hold-up tank. The system consists of a tank containing a fluid whose level is controlled by three components: two inlet pumps and one outlet valve. A thermal power source heats up the fluid. The failure rates of the components depends on the temperature, the position of the three components monitors the liquid level in the tank and the liquid level determines the temperature. Therefore, this system can be modeled by a hybrid process where the discrete (components) and continuous (level, temperature) parts interact in a closed loop. We model the system by a piecewise deterministic Markov process, propose and implement a numerical method to compute the optimal maintenance date to repair the components before the total failure of the system., Comment: arXiv admin note: text overlap with arXiv:1101.1740

This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of the partially observed optimal stopping problem. Then, we propose a numerical method, based on the quantization of the discrete-time filter process and the inter-jump times, to approximate the value function and to compute an actual $\epsilon$-optimal stopping time. We prove the convergence of the algorithms and bound the rates of convergence.

A rigorous methodology is proposed to study cell division data consisting in several observed genealogical trees of possibly different shapes. The procedure takes into account missing observations, data from different trees, as well as the dependence structure within genealogical trees. Its main new feature is the joint use of all available information from several data sets instead of single data set estimation, to avoid the drawbacks of low accuracy for estimators or low power for tests on small single-trees. The data is modeled by an asymmetric bifurcating autoregressive process and possibly missing observations are taken into account by modeling the genealogies with a two-type Galton-Watson process. Least-squares estimators of the unknown parameters of the processes are given and symmetry tests are derived. Results are applied on real data of Escherichia coli division and an empirical study of the convergence rates of the estimators and power of the tests is conducted on simulated data.

This paper presents a model of asymmetric bifurcating autoregressive process with random coefficients. We couple this model with a Galton Watson tree to take into account possibly missing observations. We propose least-squares estimators for the various parameters of the model and prove their consistency with a convergence rate, and their asymptotic normality. We use both the bifurcating Markov chain and martingale approaches and derive new important general results in both these frameworks.

We present symmetry tests for bifurcating autoregressive processes (BAR) when some data are missing. BAR processes typically model cell division data. Each cell can be of one of two types \emph{odd} or \emph{even}. The goal of this paper is to study the possible asymmetry between odd and even cells in a single observed lineage. We first derive asymmetry tests for the lineage itself, modeled by a two-type Galton-Watson process, and then derive tests for the observed BAR process. We present applications on both simulated and real data.

We present a numerical method to compute expectations of functionals of a piecewise-deterministic Markov process. We discuss time dependent functionals as well as deterministic time horizon problems. Our approach is based on the quantization of an underlying discrete-time Markov chain. We obtain bounds for the rate of convergence of the algorithm. The approximation we propose is easily computable and is flexible with respect to some of the parameters defining the problem. Two examples illustrate the paper., Comment: 41 pages

We present a numerical method to compute the optimal maintenance time for a complex dynamic system applied to an example of maintenance of a metallic structure subject to corrosion. An arbitrarily early intervention may be uselessly costly, but a late one may lead to a partial/complete failure of the system, which has to be avoided. One must therefore find a balance between these too simple maintenance policies. To achieve this aim, we model the system by a stochastic hybrid process. The maintenance problem thus corresponds to an optimal stopping problem. We propose a numerical method to solve the optimal stopping problem and optimize the maintenance time for this kind of processes.

We present a numerical method to compute the survival function and the moments of the exit time for a piecewise-deterministic Markov process (PDMP). Our approach is based on the quantization of an underlying discrete-time Markov chain related to the PDMP. The approximation we propose is easily computable and is even flexible with respect to the exit time we consider. We prove the convergence of the algorithm and obtain bounds for the rate of convergence in the case of the moments. An academic example and a model from the reliability field illustrate the paper.

We estimate the unknown parameters of an asymmetric bifurcating autoregressive process (BAR) when some of the data are missing. In this aim, we model the observed data by a two-type Galton-Watson process consistent with the binary tree structure of the data. Under independence between the process leading to the missing data and the BAR process and suitable assumptions on the driven noise, we establish the strong consistency of our estimators on the set of non-extinction of the Galton-Watson, via a martingale approach. We also prove a quadratic strong law and the asymptotic normality.

This paper presents a numerical method to calculate the value function for a general discounted impulse control problem for piecewise deterministic Markov processes. Our approach is based on a quantization technique for the underlying Markov chain defined by the post jump location and inter-arrival time. Convergence results are obtained and more importantly we are able to give a convergence rate of the algorithm. The paper is illustrated by a numerical example., Comment: This work was supported by ARPEGE program of the French National Agency of Research (ANR), project "FAUTOCOES", number ANR-09-SEGI-004

We propose a numerical method to approximate the value function for the optimal stopping problem of a piecewise deterministic Markov process (PDMP). Our approach is based on quantization of the post jump location---inter-arrival time Markov chain naturally embedded in the PDMP, and path-adapted time discretization grids. It allows us to derive bounds for the convergence rate of the algorithm and to provide a computable $\epsilon$-optimal stopping time. The paper is illustrated by a numerical example., Comment: Published in at http://dx.doi.org/10.1214/09-AAP667 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

We study the asymptotic behavior of the least squares estimators of the unknown parameters of bifurcating autoregressive processes. Under very weak assumptions on the driven noise of the process, namely conditional pair-wise independence and suitable moment conditions, we establish the almost sure convergence of our estimators together with the quadratic strong law and the central limit theorem. All our analysis relies on non-standard asymptotic results for martingales.

Let Y be an Ornstein-Uhlenbeck diffusion governed by a stationary and ergodic Markov jump process X: dY_t=a(X_t)Y_t dt+\sigma(X_t) dW_t, Y_0=y_0. Ergodicity conditions for Y have been obtained. Here we investigate the tail propriety of the stationary distribution of this model. A characterization of either heavy or light tail case is established. The method is based on a renewal theorem for systems of equations with distributions on R., Comment: Published at http://dx.doi.org/10.1214/105051604000000828 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)