Search

This article presents an approximation of discrete Markov decision processes with small noise on Borel spaces with an infinite horizon and an expected total discounted cost by the corresponding deterministic Markov process. In both cases, the dynamics evolve through a system consisting of two coupled difference equations. It is assumed that the difference equations of the system are perturbed by a small noise. Under our assumptions, a bound for the stability index is given, and the optimal cost convergence rate is estimated using a small perturbation parameter. Moreover, the convergence of the optimal policy on compact subsets is verified. Finally, two examples are presented to illustrate the developed theory.

In this article, new iterative algorithms for solving the discrete Riccati and Lyapunov equations are derived in the case where the transition matrix is diagonalizable with real eigenvalues. It is shown that the proposed iterative algorithms are faster than the classical ones, even for a small number of iterations.