Search

Abstract: This paper is concerned with asymptotic properties of consensus-type algorithms for networked systems whose topologies switch randomly. The regime-switching process is modeled as a discrete-time Markov chain with a finite state space. The consensus control is achieved by using stochastic approximation methods. In the setup, the regime-switching process (the Markov chain) contains a rate parameter in the transition probability matrix that characterizes how frequently the topology switches. On the other hand, the consensus control algorithm uses a stepsize that defines how fast the network states are updated. Depending on their relative values, three distinct scenarios emerge. Under suitable conditions, we show that when , a continuous-time interpolation of the iterates converges weakly to a system of randomly switching ordinary differential equations modulated by a continuous-time Markov chain. In this case a scaled sequence of tracking errors converges to a system of switching diffusion. When , the network topology is almost non-switching during consensus control transient intervals, and hence the limit dynamic system is simply an autonomous differential equation. When , the Markov chain acts as a fast varying noise, and only its averaged network matrices are relevant, resulting in a limit differential equation that is an average with respect to the stationary measure of the Markov chain. Simulation results are presented to demonstrate these findings. [Copyright &y& Elsevier]

Abstract: This paper is concerned with persistent system identification for plants that are equipped with binary sensors whose unknown parameter is a random process represented by a Markov chain. We treat two classes of problems. In the first class, the parameter is a stochastic process modeled by an irreducible and aperiodic Markov chain with transition rates sufficiently faster than adaptation rates of identification algorithms. In this case, an averaged behavior of the parameter process can be derived from the stationary measure of the Markov chain and can be estimated with empirical measures. Upper and lower error bounds are established that explicitly show impact of unmodeled dynamics. In the second class of problems, the state switches values infrequently. A moving-window maximum a posterior (MAP) algorithm is introduced for tracking the time-varying parameters. Numerical results are presented to illustrate the tracking performance of the MAP algorithm and compare it with the widely used Viterbi algorithm. [Copyright &y& Elsevier]