Search

Abstract: Facing the pressing needs of many applications in network and control systems, this paper introduces a class of nonlinear systems with random time delays and derives conditions on moment exponential stability of the underlying systems. The system model is versatile and can accommodate a wide variety of situations. The stability analysis to date in the literature is mostly delay independent. To highlight the role of random delay for stability, this paper focuses on delay-dependent stability. Dependence of stability on random time delays introduces technical difficulties beyond the existing literature. We model the random time delays by a continuous-time Markov chain involving two-time scales defined by a small parameter . leading to a two-time scale framework. The random delays change their values with a fast varying mode and a slowly evolving effect. Under broad conditions, the stability of the system is studied using a limit system in the sense of weak convergence of probability measures. Using the limit system as a bridge, this paper establishes the Razumikhin-type criteria on the moment exponential stability. These criteria show that the mean of the random time delay with respect to the stationary distribution of the fast changing part of the Markov chain plays an important role in the moment exponential stability, which presents a novel feature of our work. In particular, we show that the overall system may be stabilized by the Markov switching even when some of the underlying subsystems are unstable, which shows that the Markov chain may serve as a stabilization factor. Explicit conditions for moment exponential stability are derived when the system is linear. Examples are given to illustrate our results. [Copyright &y& Elsevier]

Abstract: This paper develops numerical methods for finding optimal dividend pay-out and reinsurance policies. A generalized singular control formulation of surplus and discounted payoff function is introduced, where the surplus is modeled by a regime-switching process subject to both regular and singular controls. To approximate the value function and optimal controls, Markov chain approximation techniques are used to construct a discrete-time controlled Markov chain. The proofs of the convergence of the approximation sequence to the surplus process and the value function are given. Examples of proportional and excess-of-loss reinsurance are presented to illustrate the applicability of numerical methods. [Copyright &y& Elsevier]

Abstract: This work develops numerical approximation algorithms for solutions of stochastic differential equations with Markovian switching. The existing numerical algorithms all use a discrete-time Markov chain for the approximation of the continuous-time Markov chain. In contrast, we generate the continuous-time Markov chain directly, and then use its skeleton process in the approximation algorithm. Focusing on weak approximation, we take a re-embedding approach, and define the approximation and the solution to the switching stochastic differential equation on the same space. In our approximation, we use a sequence of independent and identically distributed (i.i.d.) random variables in lieu of the common practice of using Brownian increments. By virtue of the strong invariance principle, we ascertain rates of convergence in the pathwise sense for the weak approximation scheme. [Copyright &y& Elsevier]

Abstract: This paper is concerned with asymptotic properties of consensus-type algorithms for networked systems whose topologies switch randomly. The regime-switching process is modeled as a discrete-time Markov chain with a finite state space. The consensus control is achieved by using stochastic approximation methods. In the setup, the regime-switching process (the Markov chain) contains a rate parameter in the transition probability matrix that characterizes how frequently the topology switches. On the other hand, the consensus control algorithm uses a stepsize that defines how fast the network states are updated. Depending on their relative values, three distinct scenarios emerge. Under suitable conditions, we show that when , a continuous-time interpolation of the iterates converges weakly to a system of randomly switching ordinary differential equations modulated by a continuous-time Markov chain. In this case a scaled sequence of tracking errors converges to a system of switching diffusion. When , the network topology is almost non-switching during consensus control transient intervals, and hence the limit dynamic system is simply an autonomous differential equation. When , the Markov chain acts as a fast varying noise, and only its averaged network matrices are relevant, resulting in a limit differential equation that is an average with respect to the stationary measure of the Markov chain. Simulation results are presented to demonstrate these findings. [Copyright &y& Elsevier]

This work develops an approximation procedure to find optimal annuity-purchasing strategies for minimizing the probability of lifetime ruin. The wealth is modelled as a regime-switching diffusion modulated by a continuous-time Markov chain. Based on Markov chain approximation techniques, a discrete-time controlled Markov chain with two components is constructed. Under simple conditions, the convergence of the approximation sequence to the wealth process is obtained. The convergence of the approximation to the value function is also established. Several examples are provided to demonstrate the performance of the algorithms. [ABSTRACT FROM AUTHOR]

This work is concerned with weak convergence of non-Markov random processes modulated by a Markov chain. The motivation of our study stems from a wide variety of applications in actuarial science, communication networks, production planning, manufacturing and financial engineering. Owing to various modelling considerations, the modulating Markov chain often has a large state space. Aiming at reduction of computational complexity, a two-time-scale formulation is used. Under this setup, the Markov chain belongs to the class of nearly completely decomposable class, where the state space is split into several subspaces. Within each subspace, the transitions of the Markov chain varies rapidly, and among different subspaces, the Markov chain moves relatively infrequently. Aggregating all the states of the Markov chain in each subspace to a single super state leads to a new process. It is shown that under such aggregation schemes, a suitably scaled random sequence converges to a switching diffusion process. [ABSTRACT FROM AUTHOR]

Abstract: This work develops an approximation procedure for portfolio selection with bounded constraints. Based on the Markov chain approximation techniques, numerical procedures are constructed for the utility optimization task. Under simple conditions, the convergence of the approximation sequences to the wealth process and the optimal utility function is established. Numerical examples are provided to illustrate the performance of the algorithms. [Copyright &y& Elsevier]

Stochastic optimization/approximation algorithms are widely used to recursively estimate the optimum of a suitable function or its root under noisy observations when this optimum or root is a constant or evolves randomly according to slowly time-varying continuous sample paths. In comparison, this paper analyzes the asymptotic properties of stochastic optimization/approximation algorithms for recursively estimating the optimum or root when it evolves rapidly with nonsmooth (jump-changing) sample paths. The resulting problem falls into the category of regime-switching stochastic approximation algorithms with two-time scales. Motivated by emerging applications in wireless communications, and system identification, we analyze asymptotic behavior of such algorithms. Our analysis assumes that the noisy observations contain a (nonsmooth) jump process modeled by a discrete-time Markov chain whose transition frequency varies much faster than the adaptation rate of the stochastic optimization algorithm. Using stochastic averaging, we prove convergence of the algorithm. Rate of convergence of the algorithm is obtained via bounds on the estimation errors and diffusion approximations. Remarks on improving the convergence rates through iterate averaging, and limit mean dynamics represented by differential inclusions are also presented. [ABSTRACT FROM AUTHOR]

This work is concerned with rates of convergence of Markov chain approximation methods for controlled switching diffusions. The cost function is defined on an infinite horizon with stopping times and without discount. Displaying both continuous dynamics and discrete events, the discrete events are modeled by continuous-time Markov chains to delineate a random environment and other random factors that cannot be represented by diffusion processes. This paper presents a first attempt using a probabilistic approach to treat such rates of convergence problems. In addition, in contrast to the significant developments in the literature using partial differential equation (PDE) methods for the approximation of controlled diffusions, there do not yet appear to be any PDE results to date for rates of convergence of numerical solutions for controlled switching diffusions, to the best of our knowledge. Although some of the working conditions in this paper such as the one-dimensional continuous state variable, nondegenerate diffusions, and control only on the drift may be seemingly strong, they are adequate as the starting point for using this new approach to treat the rates of convergence problems. Moreover, in the literature, to prove the convergence using Markov chain approximation methods for control problems involving cost functions with stopping (even for uncontrolled diffusion without switching), an added assumption was used to avoid the socalled tangency problem. As a by-product of our approach, by modifying the value function, it is demonstrated that the anticipated tangency problem will not arise in the sense of convergence in probability and in the sense of L1. [ABSTRACT FROM AUTHOR]

This paper analyzes the tracking properties of the least mean squares (LMS) algorithm when the underlying parameter evolves according to a finite-state Markov chain with infrequent jumps. First, using perturbed Liapunov function methods, mean-square error estimates are obtained for the tracking error. Then using recent results on two-time-scale Markov chains, mean ordinary differential equation and diffusion approximation results are obtained. It is shown that a sequence of the centered tracking errors converges to an ordinary differential equation. Moreover, a suitably scaled sequence of the tracking errors converges weakly to a diffusion process. It is also shown that iterate averaging of the tracking algorithm results in optimal asymptotic convergence rate in an appropriate sense. Two application examples, analysis of the performance of an adaptive multiuser detection algorithm in a direct-sequence code-division multiple-access (DS/CDMA) system, and tracking analysis of the state of a hidden Markov model (HMM) with infrequent jumps, are presented. [ABSTRACT FROM AUTHOR]

Aiming at reduction of complexity, this work is concerned with two-time-scale Markov chains and applications to quasi-birth-death queues. Asymptotic expansions of probability vectors are constructed and justified. Lumping all states of the Markov chain in each subspace into a single state, an aggregated process is shown to converge to a continuous-time Markov chain whose generator is an average with respect to the stationary measures. Then a suitably scaled sequence is shown to converge to a switching diffusion process. Extensions of the results are presented together with examples of quasi-birth-death queues. [ABSTRACT FROM AUTHOR]

This work is devoted to near-optimal controls of large-scale discrete-time nonlinear dynamic systems driven by Markov chains; the underlying problem is to minimize an expected cost function. Our main goal is to reduce the complexity of the underlying systems. To achieve this goal, discrete-time control models under singularly-perturbed Markov chains are introduced. Using a relaxed control representation, our effort is devoted to finding near-optimal controls. Lumping the states in each irreducible class into a single state gives rise to a limit system. Applying near-optimal controls of the limit system to the original system, near-optimal controls of the original system are derived. [ABSTRACT FROM AUTHOR]

Abstract: This work develops numerical approximation methods for quantile hedging involving mortality components for contingent claims in incomplete markets, in which guaranteed minimum death benefits (GMDBs) could not be perfectly hedged. A regime-switching jump-diffusion model is used to delineate the dynamic system and the hedging function for GMDBs, where the switching is represented by a continuous-time Markov chain. Using Markov chain approximation techniques, a discrete-time controlled Markov chain with two component is constructed. Under simple conditions, the convergence of the approximation to the value function is established. Examples of quantile hedging model for guaranteed minimum death benefits under linear jumps and general jumps are also presented. [ABSTRACT FROM AUTHOR]