ASYNCHRONOUS STOCHASTIC APPROXIMATION ALGORITHMS FOR NETWORKED SYSTEMS: REGIME-SWITCHING TOPOLOGIES AND MULTISCALE STRUCTURE.

This work develops asynchronous stochastic approximation (SA) algorithms for networked systems with multiagents and regime-switching topologies to achieve consensus control. There are several distinct features of the algorithms: (1) In contrast to most of the existing consensus algorithms, the participating agents compute and communicate in an asynchronous fashion without using a global clock. (2) The agents compute and communicate at random times. (3) The regimeswitching process is modeled as a discrete-time Markov chain with a finite state space. (4) The functions involved are allowed to vary with respect to time; hence, nonstationarity can be handled. (5) Multiscale formulation enriches the applicability of the algorithms. In the setup, the switching process contains a rate parameter ε > 0 in the transition probability matrix that characterizes how frequently the topology switches. The algorithm uses a step-size μ that defines how fast the network states are updated. Depending on their relative values, three distinct scenarios emerge. Under suitable conditions, it is shown that a continuous-time interpolation of the iterates converges weakly either to a system of randomly switching ordinary differential equations modulated by a continuous-time Markov chain or to a system of differential equations (an average with respect to certain measure). In addition, a scaled sequence of tracking errors converges to either a switching diffusion or a diffusion. Simulation results are presented to demonstrate these findings. [ABSTRACT FROM AUTHOR]