Search

Stochastic dynamic teams and games are rich models for decentralized systems and challenging testing grounds for multiagent learning. Previous work that guaranteed team optimality assumed stateless dynamics, or an explicit coordination mechanism, or joint-control sharing. In this article, we present an algorithm with guarantees of convergence to team optimal policies in teams and common interest games. The algorithm is a two-timescale method that uses a variant of Q-learning on the finer timescale to perform policy evaluation while exploring the policy space on the coarser timescale. Agents following this algorithm are “independent learners”: they use only local controls, local cost realizations, and global state information, without access to controls of other agents. The results presented here are the first, to the best of our knowledge, to give formal guarantees of convergence to team optimality using independent learners in stochastic dynamic teams and common interest games. [ABSTRACT FROM AUTHOR]

This article adopts a reinforcement learning (RL) method to solve infinite horizon continuous-time stochastic linear quadratic problems, where the drift and diffusion terms in the dynamics may depend on both the state and control. Based on the Bellman’s dynamic programming principle, we presented an online RL algorithm to attain optimal control with partial system information. This algorithm computes the optimal control, rather than estimates the system coefficients, and solves the related Riccati equation. It only requires local trajectory information, which significantly simplifies the calculation process. We shed light on our theoretical findings using two numerical examples. [ABSTRACT FROM AUTHOR]

This article presents ConVex optimization-based Stochastic steady-state Tracking Error Minimization (CV-STEM), a new state feedback control framework for a class of Itô stochastic nonlinear systems and Lagrangian systems. Its innovation lies in computing the control input by an optimal contraction metric, which greedily minimizes an upper bound of the steady-state mean squared tracking error of the system trajectories. Although the problem of minimizing the bound is nonconvex, its equivalent convex formulation is proposed utilizing SDC parameterizations of the nonlinear system equation. It is shown using stochastic incremental contraction analysis that the CV-STEM provides a sufficient guarantee for exponential boundedness of the error for all time with ${\bf \mathcal {L}_2}$ -robustness properties. For the sake of its sampling-based implementation, we present discrete-time stochastic contraction analysis with respect to a state- and time-dependent metric along with its explicit connection to continuous-time cases. We validate the superiority of the CV-STEM to PID, $\mathcal {H}_\infty$ , and baseline nonlinear controllers for spacecraft attitude control and synchronization problems. [ABSTRACT FROM AUTHOR]

• A stochastic optimal control is proposed to stabilize electrostatic MEMS resonators with noise disturbance. • The controller can be used to suppress the noise-activated pull-in instability. • The controller shows superior performance on rejecting the deterministic pull-in instability. • Time delay existing in the optimal control has a periodic impact on the control effectiveness. Noise-induced motions are a significant source of uncertainty in the response of electrostatic MEMS where electrical and mechanical sources contribute to noise and can result in sudden and dramatic loss of stability. In this manuscript, we present an analysis for the stabilization of an electrostatically actuated MEMS resonator in the presence of noise processes using a stochastic optimal control scheme. A stochastic lumped-mass model that accounts for the uncertainty in mass, mechanical restoring force, bias voltage, and AC voltage amplitude of the resonator is presented. The It o ^ equations, describing the averaged modulations of the resonator's total energy and phase difference, are obtained via the stochastic averaging of energy envelope. The stochastic optimal control law aiming at minimization the pull-in probability of the resonator is determined via the stochastic dynamic programming equations associated with the It o ^ equations. We show that the resulting stochastic optimal feedback control, with a careful selection of its control gain, can be used to suppress the noise-activated pull-in instability of the disturbed resonator. The impact of the time delay inherent in the feedback control input on the control effectiveness is investigated. In addition, the control scheme also shows superior performance in counteracting the deterministic pull-in instability of the resonator operating in the dynamic pull-in frequency band. Good agreement between the theoretical results and numerical simulation is demonstrated. [ABSTRACT FROM AUTHOR]