Search

Στη συγκεκριμένη διατριβή εξετάζουμε τις συνθήκες κάτω από τις οποίες ένα μοντέλο με πολλά στοιχεία μπορεί να προσεγγιστεί από ένα άλλο, πιο απλό μοντέλο αποτελούμενο από λιγότερα στοιχεία, χωρίς το τελευταίο να χάνει τις ιδιότητες του αρχικού μοντέλου. Οι εν λόγω συνθήκες διευκολύνουν την ανάλυση μεγάλων και πολύπλοκων συστημάτων. Εφαρμόζουμε την έρευνά μας σε δύο πολύ βασικά εργαλεία διαχείρισης πιστωτικού κινδύνου: τις πιστωτικές διαβαθμίσεις (credit ratings) και το την πιστωτική βαθμολόγηση (credit scoring). Αρχικά, παρουσιάζουμε τον ορισμό και κάποιες βασικές ιδιότητες των διαδικασιών Markov. Στη συνέχεια εκθέτουμε τις συνθήκες κάτω από τις οποίες μία αλυσίδα Markov, που περιγράφει την στοχαστική ανέλιξη των πιστωτικών διαβαθμίσεων, μπορεί να συσσωρευτεί σε μία μικρότερη αλυσίδα η οποία διατηρεί τη Μαρκοβιανή ιδιότητα, να είναι δηλ. «lumpable». Επιπλέον, παραθέτουμε τον ορισμό και τις ιδιότητες μιας απόλυτα συσσωρευόμενης («Exactly Lumpable») αλυσίδας Markov και, μέσω ενός παραδείγματος, αποδεικνύουμε ότι γενικά οι συγκεκριμένες ιδιότητες δεν ικανοποιούνται πάντα. Στο πλαίσιο αυτό, εισάγουμε την έννοια προσεγγιστικής συσσώρευσης («Approximate Lumpability»), και προτείνουμε μία διαδικασία ανεύρεσης αλυσίδας Markov, η οποία να είναι αφενός συσσωρεύσιμη και αφετέρου όσο το δυνατόν πιο κοντά στην αρχική αλυσίδα που δεν έχει τη συγκεκριμένη ιδιότητα. Ακόμα, εξετάζουμε το ερώτημα της ύπαρξης ή μη ενός αξιόπιστου κανόνα πιστωτικής βαθμολόγησης ενός συνόλου πελατών που αιτούνται δάνειο. Η έρευνά μας, που βασίζεται σε ένα θεώρημα διαχωρισμού υπερεπιπέδων του Kirchberger, απαντάει στο εν λόγω ερώτημα, καθώς τέτοιος κανόνας είναι εφικτό να δημιουργηθεί μέσω της εξέτασης ενός μικρού μόνο δείγματος δανειστών-πελατών, και όχι ολόκληρου του συνόλου τους, καταλήγοντας έτσι σε ένα γρήγορο και αξιόπιστο μοντέλο πιστωτικής βαθμολόγησης. Η μέθοδός μας αυτή παρουσιάζεται σε σχετική εκτενή εφαρμογή.

Papers presented at a workshop held January 1990 (location unspecified) cover just about all aspects of solving Markov models numerically. There are papers on matrix generation techniques and generalized stochastic Petri nets; the computation of stationary distributions, including aggregation/disagg

This article is motivated by the quantitative study of the exponential growth of Markov-driven bifurcating processes [see Hervé et al., ESAIM: PS 23 (2019) 584–606]. In this respect, a key property is the multiplicative ergodicity, which deals with the asymptotic behaviour of some Laplace-type transform of nonnegative additive functional of a Markov chain. We establish a spectral version of this multiplicative ergodicity property in a general framework. Our approach is based on the use of the operator perturbation method. We apply our general results to two examples of Markov chains, including linear autoregressive models. In these two examples the operator-type assumptions reduce to some expected finite moment conditions on the functional (no exponential moment conditions are assumed in this work).

In this chapter, the concept of probability is introduced. Both independent and disjoint events are covered, as are basic properties of probability. Bernoulli events and counting (combinatorics) are also discussed. Markov chains used in modeling are also presented. Applications are given as well.

A random point process is, roughly speaking, a countable random set of points of the real line. In most applications to engineering and operations research, a point of a point process is the time of occurrence of some event, and this is why points are also called events. For instance, the arrival times of customers at the desk of a post office or of jobs at the central processing unit of a computer are point process events.

We consider a Markov chain X with invariant distribution π and investigate conditions under which the distribution of X n converges to π as n→∞. Essentially it is necessary and sufficient that the state space of the chain cannot be decomposed into subspaces that the chain does not leave, or that are visited by the chain periodically; e.g., only for odd n or only for even n.

In this paper, we study a fast algorithm for finding stationary joint probability distributions of sparse Markov chains or multilinear PageRank vectors which arise from data mining applications. In these applications, the main computational problem is to calculate and store solutions of many unknowns in joint probability distributions of sparse Markov chains. Our idea is to approximate large-scale solutions of such sparse Markov chains by two components: the sparsity component and the rank-one component. Here the non-zero locations in the sparsity component refer to important associations in the joint probability distribution and the rank-one component refers to a background value of the solution. We propose to determine solutions by formulating and solving sparse and rank-one optimization problems via closed form solutions. The convergence of the truncated power method is established. Numerical examples of multilinear PageRank vector calculation and second-order web-linkage analysis are presented to show the efficiency of the proposed method. It is shown that both computation and storage are significantly reduced by comparing with the traditional power method.

This article is devoted to the strong law of large numbers and the entropy ergodic theorem for non homogeneous M-bifurcating Markov chains indexed by a M-branch Cayley tree, which generalizes the relevant results of tree-indexed nonhomogeneous bifurcating Markov chains. Meanwhile, our proof is quite different from the traditional method.

In this paper, we present examples of a class of Markov chains that occur frequently, but whose associated matrices are a challenge to construct efficiently. These are Markov chains that arise as a result of several identical Markov chains running in parallel. Specifically for the cases considered, both the infinitesimal generator matrix for the continuous case, and more so the transition probability matrix for the discrete equivalent, are complex to construct effectively and efficiently. We summarize the algorithms for constructing the associated matrices and present examples of applications, ranging from special queueing problems to reliability issues and order statistics. MATLAB subroutines are provided in an online supplement for the implementation of the algorithms. The online supplement is available at https://doi.org/10.1287/ijoc.2017.0759 .

This article presents the definition of Markov chain indexed by homogeneous trees in Markovian environment. Then we mainly study the strong limit theorems for a Markov chain indexed by homogeneous ...

In this paper, we are devoted to singular perturbation analysis for discrete-time or continuous-time Markov chains. We modify and extend the drift condition method, well known for regular perturbation, to develop a new framework for singular perturbation analysis. Our results extend and improve the corresponding ones in [2] for singularly perturbed Markov chains by allowing a general perturbation form, less restrictive conditions, and more computable bounds. Our analysis covers the regular perturbation analysis, and hence unifies singular and regular perturbation analysis. Furthermore, our results are illustrated by two two-dimensional Markov chains, including a discrete-time queue and a continuous-time level dependent quasi-birth-death process.

Motivated by various applications in queueing theory, this article is devoted to the stochastic monotonicity and comparability of Markov chains with block-monotone transition matrices. First, we introduce the notion of block-increasing convex order for probability vectors, and characterize the block-monotone matrices in the sense of the block-increasing order and block-increasing convex order. Second, we characterize the Markov chain with general transition matrix by martingale and provide a stochastic comparison of two block-monotone Markov chains under the two block-monotone orders. Third, the stochastic comparison results for the Markov chains corresponding to the discrete-time GI/G/1 queue with different service distributions under the two block-monotone orders are given, and the lower bound and upper bound of the Markov chain corresponding to the discrete-time GI/G/1 queue in the sense of the block-increasing convex order are found.

We study two well-known linear-time metrics on Markov chains (MCs), namely, the strong and strutter trace distances. Our interest in these metrics is motivated by their relation to the probabilistic linear temporal logic (LTL)-model checking problem: we prove that they correspond to the maximal differences in the probability of satisfying the same LTL and LTL−X(LTL without next operator) formulas, respectively.The threshold problem for these distances (whether their value exceeds a given threshold) is NP-hard and not known to be decidable. Nevertheless, we provide an approximation schema where each lower and upper approximant is computable in polynomial time in the size of the MC.The upper approximants are bisimilarity-like pseudometrics (hence, branching-time distances) that converge point-wise to the linear-time metrics. This convergence is interesting in itself, because it reveals a non-trivial relation between branching and linear-time metric-based semantics that does not hold in equivalence-based semantics.

For a Markov chain described by an irreducible stochastic matrix A of order n, the mean first passage time m i , j measures the expected time for the Markov chain to reach state j for the first time given that the system begins in state i, thus quantifying the short-term behaviour of the chain. In this article, a lower bound for the maximum mean first passage time is found in terms of the stationary distribution vector of A, and some matrices for which equality is attained are produced. The main objective of this article is to characterise the directed graphs for which any stochastic matrix A respecting this directed graph attains equality in this lower bound, producing a class of Markov chains with optimal short-term behaviour.

In this paper we contribute to the theory of conditional Markov chains (CMCs) that take finitely many values and that admit intensity. We provide a method for constructing a CMC with given intensity and with given conditional initial law, and which is also a doubly stochastic Markov chain. We provide a martingale characterization for such process, and we discuss other useful properties. We define and give sufficient and necessary conditions for strong Markovian consistency and weak Markovian consistency of a multivariate CMC. We use these results to model structured dependence between univariate CMCs, that is, to model a multivariate CMC whose components are univariate CMCs with given laws. An example of potential application of our theory is presented.

In order to fully characterize the state-transition behaviour of finite Markov chains one needs to provide the corresponding transition matrix P. In many applications such as molecular simulation and drug design, the entries of the transition matrix P are estimated by generating realizations of the Markov chain and determining the one-step conditional probability Pij for a transition from one state i to state j. This sampling can be computational very demanding. Therefore, it is a good idea to reduce the sampling effort. The main purpose of this paper is to design a sampling strategy, which provides a partial sampling of only a subset of the rows of such a matrix P. Our proposed approach fits very well to stochastic processes stemming from simulation of molecular systems or random walks on graphs and it is different from the matrix completion approaches which try to approximate the transition matrix by using a low-rank-assumption. It will be shown how Markov chains can be analyzed on the basis of a partial sampling. More precisely. First, we will estimate the stationary distribution from a partially given matrix P. Second, we will estimate the infinitesimal generator Q of P on the basis of this stationary distribution. Third, from the generator we will compute the leading invariant subspace, which should be identical to the leading invariant subspace of P. Forth, we will apply Robust Perron Cluster Analysis (PCCA+) in order to identify metastabilities using this subspace.

It is well known that, starting from a transient state in a finite absorbing Markov chain, the mean and variance in the number of visits to any transient state can be expressed in terms of entries ...

We firstly define a Markov chain indexed by a homogeneous tree in a finite i.i.d random environment. Then, we prove the strong law of large numbers and Shannon-McMillan theorem for finite Markov chains indexed by a homogeneous tree in the finite i.i.d random environment.

We study the dynamics of countable state topological Markov chains with holes, where the hole is a countable union of 1-cylinders. For a large class of positive recurrent potentials and under natural assumptions on the surviving dynamics, we prove the existence of a limiting conditionally invariant distribution, which is the unique limit of regular densities under the renormalized dynamics conditioned on non-escape. We also prove the existence of a Gibbs measure on the survivor set, the set of points that never enter the hole, which is an equilibrium measure for the punctured potential of the open system. We prove that the Gurevic pressure on the survivor set equals the exponential escape rate from the open system. These results extend to the non-compact setting results previously available for finite state topological Markov chains.

Model checking of Markov chains using logics like CSL or asCSL proves whether a logical formula holds for a state of the Markov chain. It has been developed in the last decade to a widely used approach to express performance and dependability quantities for models from a wide range of application areas. In this paper, model checking is extended to prove formulas for distributions rather than single states. This is a very natural way to express certain performance or dependability measures that depend on the state of the system rather than on a specific state in the state space of the Markov chain. It is shown that the mentioned logics can be easily extended from states to distributions and model checking algorithms can also be easily adopted. Furthermore, new equivalences will be introduced that are weaker than bisimulation but still characterize the extended logics.

Analogous to the traditional Markov chains which have been studied extensively and have many successful applications, fuzzy Markov chains have been proposed for decision-making in an environment of uncertainty and imprecision for decades. It is known that results of fuzzy Markov chains depend on the transition matrix as well as the algebraic composition involved. In their study of max–min fuzzy Markov chains, Avrachenkov and Sanchez raised an open question for finding conditions to ensure the ergodicity of max–min fuzzy Markov chains. In this paper, we provide sufficient conditions for the ergodicity of both max–min and max-product fuzzy Markov chains. It is not surprising that such sufficient conditions are very different because of the max–min and max-product compositions.

Consider a Markov chain with finite alphabets. In this paper, we study the asymptotic properties of moving average, harmonic mean, and strong deviation theorems (limit theorems expressed by inequalities) of moving geometric average of random transition probabilities and the generalized entropy ergodic theorem for Markov chains in single infinite Markovian environments. It is shown that, under some mild conditions, the sequence of the generalized relative entropy density converges almost surely and in . The trick of the proofs is the construction of random variables with a parameter and the application of Borel–Cantelli lemma.

In previous work, the embedding problem is examined within the entire set of discrete-time Markov chains. However, for several phenomena, the states of a Markov model are ordered categories and the transition matrix is state-wise monotone. The present paper investigates the embedding problem for the specific subset of state-wise monotone Markov chains. We prove necessary conditions on the transition matrix of a discrete-time Markov chain with ordered states to be embeddable in a state-wise monotone Markov chain regarding time-intervals with length 0.5: A transition matrix with a square root within the set of state-wise monotone matrices has a trace at least equal to 1.

It was conjectured in the paper “Stationary probability vectors of higher-order Markov chains” (Li and Zhang, 2015 [7] ) that if the set of stationary vectors of the second-order Markov chain contains k-interior points of the ( k − 1 ) -dimensional face of the simplex Ω n then every vector in the ( k − 1 ) -dimensional face is a stationary vector. In this paper, we provide counterexamples to this conjecture.

Publisher Summary This chapter focuses on the continuous-time analog probability models of the Markov chains that have a wide variety of applications in the real world. The Poisson process is a continuous-time Markov chain having states 0, 1, 2, … and always proceeds from state n to state n + 1, where n ≥ 0. A continuous-time Markov chain is a stochastic process that moves from state to state in accordance with a discrete-time Markov chain, but is such that the amount of time it spends in each state, before proceeding to the next state, is exponentially distributed. In addition, the amount of time the process spends in state i and the next state visited must be independent random variables.

Publisher Summary This chapter focuses on Markov chains of the probability theory. It presents a stochastic process {Xn, n = 0, 1, 2…} that takes on a finite or countable number of possible values. This set of possible values of the process is denoted by the set of nonnegative integers {0, 1, 2…}. If Xn = i, the process is said to be in state i at time n. Such a stochastic process is known as a Markov chain. The value Pij represents the probability that the process will, when in state i, next make a transition into state j. The Chapman–Kolmogorov equations provide a method for computing these n-step transition probabilities. In a finite-state Markov chain, all recurrent states are positive recurrent.

We construct Cheeger-type bounds for the second eigenvalue of a substochastic transition probability matrix in terms of the Markov chain's conductance and metastability (and vice versa) with respect to its quasistationary distribution, extending classical results for stochastic transition matrices.

We suggest a novel approach to model continuous time processes of the interactions of independent elements. The model assumes a finite number of independent Markov chains, each representing an element. Chains move among a common space of states. Sometimes chains intersect, being in the same state at the same time. These intersections relate the chains with each other and imply many interesting processes. In this paper, we examine our new approach in the context of network evolution. Our analytic study achieves a closed solution for the expected time until a node has any specific degree. Our numerical study demonstrates properties which are in agreement with real world networks. Thus we show the potential of our approach.

This paper gives the definition of tree-indexed Markov chains in random environment with discrete state space, and then studies some equivalent theorems of tree-indexed Markov chains in random environment. Finally, we give the equivalence on tree-indexed Markov chains in Markov environment and double Markov chains indexed by a tree.

We present a general functional central limit theorem started at a point also known under the name of quenched. As a consequence, we point out several new classes of stationary processes, defined via projection conditions, which satisfy this type of asymptotic result. One of the theorems shows that if a Markov chain is stationary ergodic and reversible, this result holds for bounded additive functionals of the chain which have a martingale coboundary in L_1 representation. Our results are also well adapted for strongly mixing sequences providing for this case an alternative, shorter approach to some recent results in the literature., Dedicated to the memory of Mikhail Gordin, 20 pages The paper will appear in Stochastic Processes and Their Applications

We consider time-homogeneous Markov chains with state space Eki¾?{0,1,',k} and initial distribution concentrated on the state 0. For pairs of such Markov chains, we study the Stochastic Tail Order and the stochastic order in the usual sense between the respective first passage times in the state k. On this purpose, we will develop a method based on a specific relation between two stochastic matrices on the state space Ek. Our method provides comparisons that are simpler and more refined than those obtained by the analysis based on the spectral gaps. Copyright © 2016 John Wiley & Sons, Ltd.

We first consider a stochastic system described by an absorbing semi-Markov chain (SMC) with finite state space, and we introduce the absorption probability to a class of recurrent states. Afterwar...

International audience; Markov chains are very efficient models and have been extensively applied in a wide range of fields covering queuing theory, signal processing, performance evaluation, time series, and finance. For discrete finite first-order Markov chains, which are among the most used models of this family, the transition matrix can be seen as the model parameter, since it encompasses the set of probabilities governing the system state. Estimating such a matrix is, however, not an easy task due to possible opposing expert reports or variability of conditions under which the estimation process is carried out. In this paper, we propose an original approach to infer a consensus transition matrix, defined in accordance with the theory ofevidence, from a family of data samples or transition matrices. To validate our method, experiments are conducted on nonstationary label images and daily rainfall data. The obtained results confirm the interest of the proposed evidential modeling with respect to the standard Bayesian one

We develop some sufficient conditions for the usual stochastic ordering between hitting times, of a fixed state, for two finite Markov chains with the same state-space. Our attention will be focused on the so called skip-free case and, for the proof of our results, we develop a special type of coupling. We also analyze some special cases of applications in the frame of reliability degradation and of times to occurrence of words under random sampling of letters from a finite alphabet. As will be briefly discussed such fields give rise, in a natural way, to skip-free Markov chains.

The problem of designing various radio devices, such as filters, delay lines, random antennas with a given radiation pattern and so on, requires the development of methods for constructing random sequences of the parameters of these systems having specified correlation properties. An adequate mathematical approach for solving such problems is the Markov chains of higher orders. Statistical characteristics of these objects are completely determined by their conditional probability function that, in general, can be very complicated. The purpose of this paper is to present the decomposition procedure for the conditional probability function of random sequences with long-range correlationtions in a form convenient for their numerical generation. Here we restrict ourselves to the case of the state space of the system of such kind, when random values of its elements belong to the finite abstract set. The developed theory opens the way to build a more consistent and nuanced approach for the description of systems with long-range correlations. In the limiting case of weak (by value, but not the distance) correlations memory function is uniquely expressed in terms of higher-order correlation functions, allowing us to generate a random sequence with a given long-range correlations. As an example of the analytical results obtained, which can be used in practical applications, we present an example of the numerical realization of the method of construction of random sequence with specified correlators of the second and third orders.

This paper is concerned with some long-run average cost problems for controlled Markov chains with a denumerable state space. The criterion to be optimized contains both reward and penalty functions. As a trade-off between reward and penalty, we observe certain phase transition phenomena. Our results also provide a stochastic optimal control interpretation for phase transitions of discrete homopolymers with finite attracting potentials.

We present sufficient conditions for two Markov chains to be similar, thus confirming a conjecture of Pollett (2001). Our proof technique also shows one of the sufficient conditions given in Theorem 2 of Lenin et al. (2000) for two birth–death processes to be similar is unnecessary.

In this paper, the resilient model approximation problem for a class of discrete-time Markov jump time-delay systems with input sector-bounded nonlinearities is investigated. A linearised reduced-order model is determined with mode changes subject to domination by a hierarchical Markov chain containing two different nonhomogeneous Markov chains. Hence, the reduced-order model obtained not only reflects the dependence of the original systems but also model external influence that is related to the mode changes of the original system. Sufficient conditions formulated in terms of bilinear matrix inequalities for the existence of such models are established, such that the resulting error system is stochastically stable and has a guaranteed l2-l∞ error performance. A linear matrix inequalities optimisation coupled with line search is exploited to solve for the corresponding reduced-order systems. The potential and effectiveness of the developed theoretical results are demonstrated via a numerical example.

In this chapter we present various examples of Markov chains. We will often use these examples in the sequel to illustrate the results we will develop. Most of our examples are derived from time series models or Monte Carlo simulation methods. Many time series models belong to the class of random iterative functions that are introduced in Section 2.1. We will establish in this section some properties of these models and in particular will provide conditions under which these models have an invariant probability. In Section 2.2, we introduce the so-called observation-driven models, which have many applications in econometrics in particular.

We analyze the covariance operator from a Central Limit Theorem for functionals on countable ergodic Markov chains. We provide an algorithm for characterizing the covariance operator and determining its kernel. We present examples, including countable and finite Markov chains.