Search

This article adopts a reinforcement learning (RL) method to solve infinite horizon continuous-time stochastic linear quadratic problems, where the drift and diffusion terms in the dynamics may depend on both the state and control. Based on the Bellman’s dynamic programming principle, we presented an online RL algorithm to attain optimal control with partial system information. This algorithm computes the optimal control, rather than estimates the system coefficients, and solves the related Riccati equation. It only requires local trajectory information, which significantly simplifies the calculation process. We shed light on our theoretical findings using two numerical examples. [ABSTRACT FROM AUTHOR]

We analyze a tree search problem with an underlying Markov decision process, in which the goal is to identify the best action at the root that achieves the highest cumulative reward. We present a new tree policy that optimally allocates a limited computing budget to maximize a lower bound on the probability of correctly selecting the best action at each node. Compared to widely used upper confidence bound (UCB) tree policies, the new tree policy presents a more balanced approach to manage the exploration and exploitation tradeoff when the sampling budget is limited. Furthermore, UCB assumes that the support of reward distribution is known, whereas our algorithm relaxes this assumption. Numerical experiments demonstrate the efficiency of our algorithm in selecting the best action at the root. [ABSTRACT FROM AUTHOR]

In this article, we study the problem of information sharing among rational self-interested agents as a dynamic game of asymmetric information. We assume that the agents imperfectly observe a Markov chain, and they are called to decide whether they will share their noisy observations or not at each time instant. We utilize the notion of conditional mutual information to evaluate the information being shared among the agents. The challenges that arise due to the interdependence of agents’ information structure and decision making are exhibited. For the finite horizon game, we prove that agents do not have incentive to share information. In contrast, we show that cooperation can be sustained in the infinite-horizon case by devising appropriate punishment strategies, which are defined over the agents’ beliefs on the system state. We show that these strategies are closed under the best-response mapping and that cooperation can be the optimal choice in some subsets of the state belief simplex. We characterize these equilibrium regions, prove uniqueness of a maximal equilibrium region, and devise an algorithm for its approximate computation. [ABSTRACT FROM AUTHOR]

• A stochastic optimal control is proposed to stabilize electrostatic MEMS resonators with noise disturbance. • The controller can be used to suppress the noise-activated pull-in instability. • The controller shows superior performance on rejecting the deterministic pull-in instability. • Time delay existing in the optimal control has a periodic impact on the control effectiveness. Noise-induced motions are a significant source of uncertainty in the response of electrostatic MEMS where electrical and mechanical sources contribute to noise and can result in sudden and dramatic loss of stability. In this manuscript, we present an analysis for the stabilization of an electrostatically actuated MEMS resonator in the presence of noise processes using a stochastic optimal control scheme. A stochastic lumped-mass model that accounts for the uncertainty in mass, mechanical restoring force, bias voltage, and AC voltage amplitude of the resonator is presented. The It o ^ equations, describing the averaged modulations of the resonator's total energy and phase difference, are obtained via the stochastic averaging of energy envelope. The stochastic optimal control law aiming at minimization the pull-in probability of the resonator is determined via the stochastic dynamic programming equations associated with the It o ^ equations. We show that the resulting stochastic optimal feedback control, with a careful selection of its control gain, can be used to suppress the noise-activated pull-in instability of the disturbed resonator. The impact of the time delay inherent in the feedback control input on the control effectiveness is investigated. In addition, the control scheme also shows superior performance in counteracting the deterministic pull-in instability of the resonator operating in the dynamic pull-in frequency band. Good agreement between the theoretical results and numerical simulation is demonstrated. [ABSTRACT FROM AUTHOR]