Search

We define a class of not necessarily linear C 0 -semigroups (P t) t ≥ 0 on C b (E) (more generally, on C κ (E) : = 1 κ C b (E) , for some bounded function κ , which is the pointwise limit of a decreasing sequence of continuous functions) equipped with the mixed topology τ 1 M for a large class of topological (not necessarily Polish) state spaces E. If these semigroups are linear, classical theory of operator semigroups on locally convex spaces as well as the theory of bicontinuous semigroups apply to them. In particular, they are infinitesimally generated by their generator (L , D (L)) and thus reconstructable through an Euler formula from their strong derivative at zero in (C b (E) , τ 1 M). In the linear case, we prove that such (P t) t ≥ 0 can be characterized as integral operators given by measure kernels satisfying certain tightness properties. As a consequence, transition semigroups of Markov processes are C 0 -semigroups on (C b (E) , τ 1 M) , if they leave C b (E) invariant and they are jointly weakly continuous in space and time. Hence, they can be reconstructed from their strong derivative at zero and thus have a fully infinitesimal description. This solves an open problem for Markov processes. We show that our results apply to a large number of Markov processes, e.g., those given as the laws of solutions to SDEs and SPDEs, including the stochastic 2D Navier-Stokes equations and the stochastic fast and slow diffusion porous media equations. Furthermore, we introduce the notion of a Markov core operator (L 0 , D (L 0)) for the above generators (L , D (L)) and prove that uniqueness of the Fokker-Planck-Kolmogorov equations corresponding to (L 0 , D (L 0)) for all Dirac initial conditions implies that (L 0 , D (L 0)) is a Markov core operator for (L , D (L)). As a consequence, we identify the Kolmogorov operators of a large number of SDEs on finite and infinite dimensional state spaces as Markov core operators for the infinitesimal generators of the C 0 -semigroups on (C κ (E) , τ κ M) given by their transition semigroups. Furthermore, if each P t is merely convex, we prove that (P t) t ≥ 0 gives rise to viscosity solutions to the Cauchy problem given by its associated (nonlinear) infinitesimal generator. We also show that value functions of optimal control problems, both, in finite and infinite dimensions are particular instances of convex C 0 -semigroups on (C κ (E) , τ κ M). [ABSTRACT FROM AUTHOR]

We give Hoeffding- and Bernstein-type concentration inequalities for the largest eigenvalue of sums of random matrices arising from a Markov chain. We consider time-dependent matrix-valued functions on a general state space, generalizing previous results that had only considered Hoeffding-type inequalities, and only for time-independent functions on a finite state space. In particular, we study a kind of non-commutative moment generating function, provide tight bounds on this object and use a method of Garg et al. (A matrix expander Chernoff bound. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2018, page 1102–1114, New York, NY, USA, 2018. Association for Computing Machinery) to turn this into tail bounds. Our proof proceeds spectrally, bounding the norm of a certain perturbed operator. In the process we make an interesting connection to dynamical systems and Banach space theory to prove a crucial result on the limiting behaviour of our moment generating function that may be of independent interest. [ABSTRACT FROM AUTHOR]

In this article, we introduce a new type of pseudo-differential equations naturally connected with non-archimedean pseudo-differential operators and whose symbols are new classes of negative definite functions in the p-adic context and in arbitrary dimension. These equations are proposed as a mathematical models to study the spreading of infectious diseases (say COVID-19) through a random walk on a complex energy landscape and taking into account social clusters in a situation of extreme social isolation. [ABSTRACT FROM AUTHOR]

We study Markov operators T, A, and T * of general Markov chains on an arbitrary measurable space. The operator, T, is defined on the Banach space of all bounded measurable functions. The operator A is defined on the Banach space of all bounded countably additive measures. We construct an operator T * , topologically conjugate to the operator T, acting in the space of all bounded finitely additive measures. We prove the main result of the paper that, in general, a Markov operator T * is quasi-compact if and only if T is quasi-compact. It is proved that the conjugate operator T * is quasi-compact if and only if the Doeblin condition (D) is satisfied. It is shown that the quasi-compactness conditions for all three Markov operators T, A, and T * are equivalent to each other. In addition, we obtain that, for an operator T * to be quasi-compact, it is necessary and sufficient that it does not have invariant purely finitely additive measures. A strong uniform reversible ergodic theorem is proved for the quasi-compact Markov operator T * in the space of finitely additive measures. We give all the proofs for the most general case. A detailed analysis of Lin's counterexample is provided. [ABSTRACT FROM AUTHOR]

In this work, we characterize the fixed points of the doubly stochastic operators defined on discrete spaces $ \ell ^p(I) , $ ℓ p (I) , where I is a nonempty set and $ p\in [1,+\infty). $ p ∈ [ 1 , + ∞). We use our results to characterize all Markov chains with infinite state space and doubly stochastic transition probability matrix that have at least one stationary distribution vector. [ABSTRACT FROM AUTHOR]

In this work, we consider a class of centred kernel quadratic stochastic operators. We prove that in this class a centred kernel quadratic stochastic operator convergences almost surely and in L² with an exponential L²-rate to its limit distribution. We propose an approximation scheme for this class of quadratic stochastic operators and describe three algorithms for simulating them. We consider in detail an example where the kernel is a Guassian one. [ABSTRACT FROM AUTHOR]

In this paper, we derive stability margins for optimal and inverse optimal stochastic feedback regulators. Specifically, gain, sector, and disk margin guarantees are obtained for discrete-time nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal controllers that minimise a nonlinear-nonquadratic performance criterion. Furthermore, using the newly developed notion of stochastic dissipativity we derive a return difference inequality to provide connections between stochastic dissipativity and optimality of nonlinear controllers for discrete-time stochastic dynamical systems. In particular, using extended Kalman-Yakubovich-Popov conditions characterising stochastic dissipativity we show that our optimal feedback control law satisfies a return difference inequality predicated on a difference operator of a controlled Markov dispersion process and is stochastically dissipative with respect to a specific quadratic supply rate. [ABSTRACT FROM AUTHOR]

In this paper, we consider general Markov chains with discrete time in an arbitrary measurable (phase) space. Markov chains are given by a classical transition function that generates a pair of conjugate linear Markov operators in a Banach space of measurable bounded functions and in a Banach space of bounded finitely additive measures. We study sequences of Cesaro means of powers of Markov operators on the set of finitely additive probability measures. It is proved that the set of all limit measures (points) of such sequences in the weak topology generated by the preconjugate space is non-empty, weakly compact, and all of them are invariant for this operator. We also show that the well-known Doeblin condition (D) for the ergodicity of a Markov chain is equivalent to condition (∗) : all invariant finitely additive measures of the Markov chain are countably additive, i.e., there are no invariant purely finitely additive measures. We give all the proofs for the most general case. [ABSTRACT FROM AUTHOR]

We present and develop different approaches to study the asymptotic behavior of the distribution functions in the odd continued fractions case. Firstly, by considering the transition operator of the Markov chain associated with these expansions on a certain Banach space of complex-valued functions of bounded variation, we make a brief survey of the solution in the Gauss-Kuzmin-type problem. Secondly, we use the method of Szüsz to obtain a similar asymptotic result and to give a good estimate of the convergence rate involved. [ABSTRACT FROM AUTHOR]

In this study, we consider general Markov chains (MC) defined by a transition probability (kernel) that is finitely additive. These Markov chains were constructed by S. Ramakrishnan within the concepts and symbolism of game theory. Here, we study these MCs by using the operator approach. In our work, the state space (phase space) of the MC has any cardinality and the sigma-algebra is discrete. The construction of a phase space allows us to decompose the Markov kernel (and the Markov operators that it generates) into the sum of two components: countably additive and purely finitely additive kernels. We show that the countably additive kernel is atomic. Some properties of Markov operators with a purely finitely additive kernel and their invariant measures are also studied. A class of combined finitely additive MC and two of its subclasses are introduced, and the properties of their invariant measures are proven. Some asymptotic regularities of such MCs were revealed. [ABSTRACT FROM AUTHOR]

In this paper, we establish the global optimality and convergence rate of an off-policy actor critic algorithm in the tabular setting without using density ratio to correct the discrepancy between the state distribution of the behavior policy and that of the target policy. Our work goes beyond existing works on the optimality of policy gradient methods in that existing works use the exact policy gradient for updating the policy parameters while we use an approximate and stochastic update step. Our update step is not a gradient update because we do not use a density ratio to correct the state distribution, which aligns well with what practitioners do. Our update is approximate because we use a learned critic instead of the true value function. Our update is stochastic because at each step the update is done for only the current state action pair. Moreover, we remove several restrictive assumptions from existing works in our analysis. Central to our work is the finite sample analysis of a generic stochastic approximation algorithm with time-inhomogeneous update operators on time-inhomogeneous Markov chains, based on its uniform contraction properties. [ABSTRACT FROM AUTHOR]

The aim of this paper is to provide approximation results for space-time non-local equations with general non-local (and fractional) operators in space and time. We consider a general Markov process time changed with general subordinators or inverses to general subordinators. Our analysis is based on Bernstein symbols and Dirichlet forms, where the symbols characterize the time changes, and the Dirichlet forms characterize the Markov processes. [ABSTRACT FROM AUTHOR]

We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators and distributions behind the iterates of stochastic algorithms, and in particular the regularity of Markov operators and rates of convergence of the distributions of the corresponding Markov chains. This provides a detailed characterization of the moments of the sequences beyond just the expected behavior. This also serves as a case study of how randomization restores favorable properties to algorithms that iterations of only partial information destroys. We demonstrate this on stochastic blockwise implementations of the forward–backward and Douglas–Rachford algorithms for nonconvex (and, as a special case, convex), nonsmooth optimization. [ABSTRACT FROM AUTHOR]

We develop an analytical framework and numerical approach to obtain the coefficient of self-diffusivity for the transport of a rarefied gas in channels in the limit of large Knudsen number. This framework provides a method for determining the influence of channel surface microstructure on the value of diffusivity that is particularly effective when the microstructure exhibits relatively low roughness. This method is based on the observation that the Markov transition (scattering) operator determined by the microstructure, under the condition of weak surface scattering, has a universal form given, up to a multiplicative constant, by the classical Legendre differential operator. We also show how characteristic numbers of the system--namely, geometric parameters of the microstructure, the spectral gap of a Markov operator, and the tangential momentum accommodation coefficient of a commonly used model of surface scattering--are all related. Examples of microstructures are investigated to illustrate the relation of these quantities numerically and analytically. [ABSTRACT FROM AUTHOR]

In the past decade, many Bayesian shrinkage models have been developed for linear regression problems where the number of covariates, p, is large. Computation of the intractable posterior is often done with three-block Gibbs samplers (3BG), based on representing the shrinkage priors as scale mixtures of Normal distributions. An alternative computing tool is a state of the art Hamiltonian Monte Carlo (HMC) method, which can be easily implemented in the Stan software. However, we found both existing methods to be inefficient and often impractical for large p problems. Following the general idea of Rajaratnam et al., we propose two-block Gibbs samplers (2BG) for three commonly used shrinkage models, namely, the Bayesian group lasso, the Bayesian sparse group lasso, and the Bayesian fused lasso models. We demonstrate with simulated and real data examples that the Markov chains underlying 2BG's converge much faster than that of 3BG's, and no worse than that of HMC. At the same time, the computing costs of 2BG's per iteration are as low as that of 3BG's, and can be several orders of magnitude lower than that of HMC. As a result, the newly proposed 2BG is the only practical computing solution to do Bayesian shrinkage analysis for datasets with large p. Further, we provide theoretical justifications for the superior performance of 2BG's. We establish geometric ergodicity of Markov chains associated with the 2BG for each of the three Bayesian shrinkage models. We also prove, for most cases of the Bayesian group lasso and the Bayesian sparse group lasso model, the Markov operators for the 2BG chains are trace-class. Whereas for all cases of all three Bayesian shrinkage models, the Markov operator for the 3BG chains is not even Hilbert–Schmidt. for this article are available online. [ABSTRACT FROM AUTHOR]

This paper focuses on the state estimation problem for a type of coupled neural networks with multiple time delays and markovian jumping communication topologies. To avoid unnecessary resources consuming, a novel state estimator is designed based on event-triggered mechanism, in which the control input of each node is only updated when the measurement output error exceeds a predefined threshold. The event-triggering time sequence is a subset of the switching time sequence, which can naturally excludes the Zeno-behavior. By utilizing an appropriate Lyapunov-Krasovskii functional, as well as the weak infinitesimal operator of Markov process and some algebraic inequalities, an easy-to-check sufficient criterion is derived to ensure the exponential ultimate boundedness of the estimation error. Finally, a simulation example is presented to illustrate the applications and effectiveness of the theoretical results. [ABSTRACT FROM AUTHOR]

Normally, one thinks of probabilistic transition systems as taking an initial probability distribution over the state space into a new probability distribution representing the system after a transition. We, however, take a dual view of Markov processes as transformers of bounded measurable functions. This is very much in the same spirit as a "predicate-transformer" view, which is dual to the state-transformer view of transition systems. We redevelop the theory of labelled Markov processes from this viewpoint; in particular, we explore approximation theory. We obtain three main results. (i) It is possible to define bisimulation on general measure spaces and show that it is an equivalence relation. The logical characterization of bisimulation can be done straightforwardly and generally. (ii) A new and flexible approach to approximation based on averaging can be given. This vastly generalizes and streamlines the idea of using conditional expectations to compute approximations. (iii) We show that there is a minimal process bisimulation-equivalent to a given process, and this minimal process is obtained as the limit of the finite approximants. [ABSTRACT FROM AUTHOR]

The response of a low-frequency mode of climate variability, El Niño–Southern Oscillation, to stochastic forcing is studied in a high-dimensional model of intermediate complexity, the fully-coupled Cane–Zebiak model (Zebiak and Cane 1987), from the spectral analysis of Markov operators governing the decay of correlations and resonances in the power spectrum. Noise-induced oscillations excited before a supercritical Hopf bifurcation are examined by means of complex resonances, the reduced Ruelle–Pollicott (RP) resonances, via a numerical application of the reduction approach of the first part of this contribution (Chekroun et al. 2019) to model simulations. The oscillations manifest themselves as peaks in the power spectrum which are associated with RP resonances organized along parabolas, as the bifurcation is neared. These resonances and the associated eigenvectors are furthermore well described by the small-noise expansion formulas obtained by Gaspard (2002) and made explicit in the second part of this contribution (Tantet et al. 2019). Beyond the bifurcation, the spectral gap between the imaginary axis and the real part of the leading resonances quantifies the diffusion of phase of the noise-induced oscillations and can be computed from the linearization of the model and from the diffusion matrix of the noise. In this model, the phase diffusion coefficient thus gives a measure of the predictability of oscillatory events representing ENSO. ENSO events being known to be locked to the seasonal cycle, these results should be extended to the non-autonomous case. More generally, the reduction approach theorized in Chekroun et al. (2019), complemented by our understanding of the spectral properties of reference systems such as the stochastic Hopf bifurcation, provides a promising methodology for the analysis of low-frequency variability in high-dimensional stochastic systems. [ABSTRACT FROM AUTHOR]

We prove that certain asymptotic moments exist for some random distance expanding dynamical systems and Markov chains in random dynamical environment, and compute them in terms of the derivatives at 0 of an appropriate pressure function. It will follow that these moments satisfy the relations that the asymptotic moments γ k = lim n → ∞ n - [ k 2 ] E (∑ i = 1 n X i) k of sums of independent and identically distributed centered random variables satisfy. Under certain mixing conditions we will also estimate the convergence rate towards these limits. The arguments in the proof of these results yield that the partial sums generated by the random Ruelle–Perron–Frobenius triplets and all of their parametric derivatives (considered as functions on the base) corresponding to appropriate random transfer or Markov operators satisfy several probabilistic limit theorems such as the central limit theorem. We will also obtain certain (Edgeworth) asymptotic expansions related to the central limit theorem for such processes. Our proofs rely on a (parametric) random complex Ruelle-Perron-Frobenius theorem, which replaces some of the spectral techniques used in literature in order to obtain limit theorems for deterministic dynamical systems and Markov chains. [ABSTRACT FROM AUTHOR]

Staggered quantum walks on graphs are based on the concept of graph tessellation and generalize some well-known discrete-time quantum walk models. In this work, we address the class of 2-tessellable quantum walks with the goal of obtaining an eigenbasis of the evolution operator. By interpreting the evolution operator as a quantum Markov chain on an underlying multigraph, we define the concept of quantum detailed balance, which helps to obtain the eigenbasis. A subset of the eigenvectors is obtained from the eigenvectors of the double discriminant matrix of the quantum Markov chain. To obtain the remaining eigenvectors, we have to use the quantum detailed balance conditions. If the quantum Markov chain has a quantum detailed balance, there is an eigenvector for each fundamental cycle of the underlying multigraph. If the quantum Markov chain does not have a quantum detailed balance, we have to use two fundamental cycles linked by a path in order to find the remaining eigenvectors. We exemplify the process of obtaining the eigenbasis of the evolution operator using the kagome lattice (the line graph of the hexagonal lattice), which has symmetry properties that help in the calculation process. [ABSTRACT FROM AUTHOR]

The stationary probability vectors of a second order Markov chain on the (n-1)- dimensional standard simplex are considered. In 2015, Li and Zhang gave a characterization of the second order Markov chain such that every vector in the simplex is a stationary vector. A modification of the characterization is presented in the paper. Some sufficient conditions are derived for any facet of the simplex such that every vector of the facet is a stationary vector. [ABSTRACT FROM AUTHOR]

We develop a Thermodynamic Formalism for bounded continuous potentials defined on the sequence space X ≡ EN, where E is a general standard Borel space. In particular, we introduce meaningful concepts of entropy and pressure for shifts acting on X and obtain the existence of equilibrium states as finitely additive probability measures for any bounded continuous potential. Furthermore, we establish convexity and other structural properties of the set of equilibrium states, prove a version of the Perron-Frobenius-Gruelled theorem under additional assumptions on the regularity of the potential and show that the Yoshida-Hewitt decomposition of these equilibrium states does not have a purely finite additive part. We then apply our results to the construction of invariant measures of time-homogeneous Markov chains taking values on a general Borel standard space and obtain exponential asymptotic stability for a class of Markov operators. We also construct conformal measures for an infinite collection of interacting random paths which are associated to a potential depending on infinitely many coordinates. Under an additional differentiability hypothesis, we show how this process is related after a proper scaling limit to a certain infinite-dimensional diffusion. [ABSTRACT FROM AUTHOR]

A nonlinear Markov chain is a discrete time stochastic process whose transitions may depend on both the current state and the current distribution of the process. The nonlinear Markov chain over a finite state space can be identified by a continuous mapping (the so-called nonlinear Markov operator) defined on a set of all probability distributions (which is a simplex) of the finite state space and by a family of transition matrices depending on occupation probability distributions of states. In this paper, we introduce a notion of Dobrushin's ergodicity coefficients for stochastic hypermatrices and provide a criterion for the contraction nonlinear Markov operator by means of Dobrushin's ergodicity coefficients. We also introduce a notion of p − majorizing nonlinear Markov operators associated with stochastic hypermatrices and provide a criterion for strong ergodicity of such kind of operator. We show that the p − majorizing nonlinear Markov operators associated with scrambling , Sarymsakov , and Wolfowitz stochastic hypermatrices are strongly ergodic. These classes of p − majorizing nonlinear Markov operators assure an existence of a residual set of strongly ergodic nonlinear Markov operators which are not contractions. Some supporting examples are also provided. [ABSTRACT FROM AUTHOR]

The paper is concerned with special one-dimensional Markov processes, which are L'evy processes defined on a finite interval and reflected from the boundary points of the interval. It is shown that in this setting, in addition to the standard semigroup of operators generated by the Markov process, there also appears the family of "boundary" random operators that send functions defined on the boundary of the interval to elements of the space L2 on the entire interval. In the case when the original process is a Wiener process, we show that these operators can be expressed in terms of the local time of the process on the boundary of the interval. [ABSTRACT FROM AUTHOR]

In this article we analyze the M/G/l retrial queue with two-way communication and n types of outgoing calls from a stochastic comparison viewpoint. The main idea is that given a complex Markov chain that cannot be analyzed numerically, we propose to bound it by a new Markov chain, which is easier to solve by using a stochastic comparison approach. Particularly, we study the monotonicity of the transition operator of the embedded Markov chain relative to the stochastic and convex orderings. Bounds are also obtained for the stationary distribution of the embedded Markov chain at departure epochs. Additionally, the performance measures of the considered system can be estimated by those of an M/M/l retrial queue with two-way communication and n types of outgoing calls when the service time distribution is NBUE (respectively, NWUE). Finally, we test numerically the accuracy of the proposed bounds. [ABSTRACT FROM AUTHOR]

Copyright of Universitas Scientiarum is the property of Pontificia Universidad Javeriana and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

We introduce partial loop-erasing operators. We show that by applying a refinement sequence of partial loop-erasing operators to a finite Markov chain, we get a process equivalent to the chronological loop-erased Markov chain. As an application, we construct loop-erased random paths on bounded domains of resistance spaces as the weak limit of the loop erasure of the Markov chains on a sequence of finite sets approximating the space, and the limit is independent of the approximating sequences. The random paths we constructed are simple paths almost surely, and they can be viewed as the loop-erasure of the paths of the diffusion process. Finally, we show that the scaling limit of the loop-erased random walks on the Sierpiński carpet graphs exists, and is equivalent to the loop-erased random paths on the Sierpińksi carpet. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we prove a slight, but practically useful, generalisation of a criterion on asymptotic stability for Markov e-chains by T. Szarek, which is based on the so-called lower bound technique, developed by A. Lasota and J. York. Simultaneously, we present an alternative proof of this theorem using an asymptotic coupling method introduced by M. Hairer. Our main result is illustrated by an application to random iterated function systems, which are not contracting on average. [ABSTRACT FROM AUTHOR]

A birth process model proposed by Dixon and Robinson has been widely used in football spread betting market. However, multiple goals in a minute are permitted in the model, which does not conform to historical record. Moreover, it is difficult to calculate the outcome probability of the process accurately. The article presents a discrete-time and finite-state Markov chain model for real-time forecast of football matches and a recursive algorithm is derived to calculate the outcome probability accurately. The empirical study shows that the proposed model outperforms the models of Dixon and Robinson and Dixon and Coles. [ABSTRACT FROM AUTHOR]

Markov chains have been widely employed as a fundamental model in the studies of probabilistic and stochastic communicating and concurrent systems. It is well-understood that decomposition techniques play a key role in reachability analysis and model-checking of Markov chains. (Discrete-time) quantum Markov chains have been introduced as a model of quantum communicating systems [1] and also a semantic model of quantum programs [2] . The BSCC (Bottom Strongly Connected Component) and stationary coherence decompositions of quantum Markov chains were introduced in [3–5] . This paper presents a new decomposition technique, namely periodic decomposition, for quantum Markov chains. We further establish a limit theorem for them. As an application, an algorithm to find a maximum dimensional noiseless subsystem of a quantum communicating system is given using decomposition techniques of quantum Markov chains. [ABSTRACT FROM AUTHOR]

We study the long time behavior of the stochastic quantization equation. Extending recent results by Mourrat andWeber (Global well-posedness of the dynamic φ4 in the plane (2015) Preprint) we first establish a strong non-linear dissipative bound that gives control of moments of solutions at all positive times independent of the initial datum. We then establish that solutions give rise to a Markov process whose transition semigroup satisfies the strong Feller property. Following arguments by Chouk and Friz (Support theorem for a singular SPDE: the case of gPAM (2016) Preprint) we also prove a support theorem for the laws of the solutions. Finally all of these results are combined to show that the transition semigroup satisfies the Doeblin criterion which implies exponential convergence to equilibrium. Along the way we give a simple direct proof of the Markov property of solutions and an independent argument for the existence of an invariant measure using the Krylov--Bogoliubov existence theorem. Our method makes no use of the reversibility of the dynamics or the explicit knowledge of the invariant measure and it is therefore in principle applicable to situations where these are not available, e.g. the vector-valued case. [ABSTRACT FROM AUTHOR]

Let π denote the intractable posterior density that results when the likelihood from a multivariate linear regression model with errors from a scale mixture of normals is combined with the standard non-informative prior. There is a simple data augmentation algorithm (based on latent data from the mixing density) that can be used to explore π . Let h and d denote the mixing density and the dimension of the regression model, respectively. Hobert et al. (2018) have recently shown that, if h converges to 0 at the origin at an appropriate rate, and ∫ 0 ∞ u d ∕ 2 h ( u ) d u < ∞ , then the Markov chains underlying the data augmentation (DA) algorithm and an alternative Haar parameter expanded DA (PX-DA) algorithm are both geometrically ergodic. Their results are established using probabilistic techniques based on drift and minorization conditions. In this paper, spectral analytic techniques are used to establish that something much stronger than geometric ergodicity often holds. In particular, it is shown that, under simple conditions on h , the Markov operators defined by the DA and Haar PX-DA Markov chains are trace-class , i.e., compact with summable eigenvalues. Many standard mixing densities satisfy the conditions developed in this paper. Indeed, the new results imply that the DA and Haar PX-DA Markov operators are trace-class whenever the mixing density is generalized inverse Gaussian, log-normal, Fréchet (with shape parameter larger than d ∕ 2 ), or inverted Gamma (with shape parameter larger than d ∕ 2 ). [ABSTRACT FROM AUTHOR]

We study a single-unit deteriorating system under condition monitoring for which collected signals are only stochastically related to the actual level of degradation. Failure replacement is costlier than preventive replacement and there is a delay (lead time) between the initiation of the maintenance setup and the actual maintenance, which is closely related to the process of spare parts inventory and/or maintenance setup activities. We develop a dynamic control policy with a two-dimensional decision space, referred to as a warning-replacement policy, which jointly optimizes the replacement time and replacement setup initiation point (maintenance ordering time) using online condition monitoring data. The optimization criterion is the long-run expected average cost per unit of operation time. We develop the optimal structure of such a dynamic policy using a partially observable semi-Markov decision process and provide some important results with respect to optimality and monotone properties of the optimal policy. We also discuss how to find the optimal values of observation/inspection interval and lead time using historical condition monitoring data. Illustrative numerical examples are provided to show thatour joint policy outperforms conventional suboptimal policies commonly used in theliterature. [ABSTRACT FROM AUTHOR]

In this paper we develop a new approach for decentralized multi-robot belief space planning in high-dimensional state spaces while operating in unknown environments. State of the art approaches often address related problems within a sampling based motion planning paradigm, where robots generate candidate paths and are to choose the best paths according to a given objective function. As exhaustive evaluation of all candidate path combinations from different robots is computationally intractable, a commonly used (sub-optimal) framework is for each robot, at each time epoch, to evaluate its own candidate paths while only considering the best paths announced by other robots. Yet, even this approach can become computationally expensive, especially for high-dimensional state spaces and for numerous candidate paths that need to be evaluated. In particular, upon an update in the announced path from one of the robots, state of the art approaches re-evaluate belief evolution for all candidate paths and do so from scratch. In this work we develop a framework to identify and efficiently update only those paths that are actually impacted as a result of an update in the announced path. Our approach is based on appropriately propagating belief evolution along impacted paths while employing insights from factor graph and incremental smoothing for efficient inference that is required for evaluating the utility of each impacted path. We demonstrate our approach in a synthetic simulation. [ABSTRACT FROM AUTHOR]

This paper proposes a hierarchical dual filtering (HDF) algorithm to estimate the spatial region between a Cloud of Things (CoT) gateway and an Internet of Things (IoT) device. The accuracy of the spatial region estimation is important for autonomous CoT clustering. We conduct spatial region estimation using a hidden Markov model (HMM) with a raw Bluetooth received signal strength indicator (RSSI). However, the accuracy of the region estimation using the validation data is only 53.8%. To increase the accuracy of the spatial region estimation, the HDF algorithm removes the high‐frequency signals hierarchically, and alters the parameters according to whether the IoT device moves. The accuracy of spatial region estimation using a raw RSSI, Kalman filter, and HDF are compared to evaluate the effectiveness of the HDF algorithm. The success rate and root mean square error (RMSE) of all regions are 0.538, 0.622, and 0.75, and 0.997, 0.812, and 0.5 when raw RSSI, a Kalman filter, and HDF are used, respectively. The HDF algorithm attains the best results in terms of the success rate and RMSE of spatial region estimation using HMM. [ABSTRACT FROM AUTHOR]

Ordinal outcomes collected at multiple follow-up visits are common in clinical trials. Sometimes, one visit is chosen for the primary analysis and the scale is dichotomized amounting to loss of information. Multistate Markov models describe how a process moves between states over time. Here, simulation studies are performed to investigate the Type I error and power characteristics of multistate Markov models for panel data with limited non-adjacent state transitions. The results suggest that the multistate Markov models preserve the Type I error and adequate power is achieved with modest sample sizes for panel data with limited non-adjacent state transitions. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of the optimal timing to initiate a medical treatment. In the absence of treatment, we model the disease evolution as a semi-Markov process. The optimal time to initiate the treatment is a stopping time, which maximizes the total expected reward for the patient. We propose a stochastic dynamic programming formulation to find this stopping time. Under some plausible conditions, we show that the maximum total expected reward at the start of a health state will be smaller when the patient is in a more severe state. We then prove that the optimal policy for initializing the treatment is determined by a time threshold for each given health state. That is, in each health state, the treatment should be planned to start, when the patient's duration time in the health state reaches (or exceeds, in the case of a late observation of the patient's health status) a certain threshold level. We also present numerical examples to illustrate our model and to provide managerial insights. [ABSTRACT FROM AUTHOR]

It is important to consider the changing states in hedging. The Markov regime-switching dynamic correlation multivariate stochastic volatility (MRS-DC-MSV) model was proposed to solve this issue. DC-MSV model and MRS-DC-MSV model were used to calculate the time-varying hedging ratios and compare the hedging performance. The Markov chain Monte Carlo (MCMC) method was used to estimate the parameters. The results showed that, there were obviously two economic states in Chinese financial market. Two models all did well in hedging, but the performance of MRS-DC-MSV model was better. It could reduce risk by nearly 90 %. Thus, in the hedging period, changing states is a factor that cannot be neglected. [ABSTRACT FROM AUTHOR]

We study a mixing condition for entangled Markov chains, so-called ψ -mixing property. We prove that every entangled Markov chain, whose Markov operators satisfy the Markov–Dobrushin condition, is ψ -mixing. Moreover, we show the restriction of the underlying QMC to the diagonal algebra gives rise to a classical mixing Markov chains with the same exponential rate of convergence. [ABSTRACT FROM AUTHOR]

The article discusses approach of Markov chain Monte Carlo (MCMC) algorithms to evaluate posterior density in using data augmentation (DA) algorithm. Topics discussed include approach of sandwich algorithm better than data augmentation (DA) algorithm theoretically, extension of sandwich algorithm based on the Markov chain and establishing geometric drift condition to prove DA Markov chain to be geometrically ergodic.

The exponential rate of convergence and the Central Limit Theorem for some Markov operators are established. These operators were efficiently used in some biological models (see Hille, Horbacz & Szarek (2015) [8] ), which generalize the cell cycle model given by Lasota & Mackey (1999) [12] . [ABSTRACT FROM AUTHOR]

This work concerns with semi-Markov decision chains on a finite state space. Assuming that the controller has a constant and positive risk-sensitive coefficient, an optimality equation for the corresponding (long-run) risk-sensitive average cost index is formulated and, under suitable continuity-compactness conditions, it is shown that a solution of such an equation determines the optimal average cost, as well as an optimal stationary policy. Additionally, if the underlying Markov chain is communicating, then it is proved that the optimality equation has a solution. [ABSTRACT FROM AUTHOR]

In this article, we define and study the affine and cyclotomic Yokonuma-Hecke algebras. These algebras generalize at the same time the Ariki-Koike and affine Hecke algebras and the Yokonuma-Hecke algebras. We study the representation theory of these algebras and construct several bases for them. We then show how we can define Markov traces on them, which we in turn use to construct invariants for framed and classical knots in the solid torus. Finally, we study the Markov trace with zero parameters on the cyclotomic Yokonuma-Hecke algebras and determine the Schur elements with respect to that trace. [ABSTRACT FROM AUTHOR]

Spam pages boost themselves to approach the top of search result, which makes users not satisfied with search results. Hidden Markova Model (HMM) could predict user's preference according to user's comments or records, and filter these spam pages. However, HMM consumes lots of time in getting parameters for each user. In this paper, a K-Nearest Neighbor based HMM (KNN-HMM) is proposed to improve efficiency of HMM. For similar users could share parameters of HMM, KNN-HMM groups similar users into a cluster, and personalizes parameter of HMM for a cluster instead of each user's HMM parameter. Experimental results on AOL dataset demonstrate the accuracy of KNN-HMM method is better than that of page rank based methods and the efficiency is better than that of HMM based method. [ABSTRACT FROM AUTHOR]

Consider a constrictive Markov operator $$T:L^1(X, \Sigma , \mu ) \rightarrow L^1(X, \Sigma , \mu )$$ defined on a finite measure space $$(X, \Sigma , \mu )$$ . We give a necessary and sufficient condition for a constrictive Markov operator T which is an integral operator with stochastic kernel satisfying $$T\mathbf {1}_X=\mathbf {1}_X$$ . [ABSTRACT FROM AUTHOR]