Search

Recently, Thompson sampling has been shown to achieve good theoretical performance guarantees for stochastic control problems with parameter uncertainty when the state, control, and parameter spaces are all finite. Much less is known however about the performance of Thompson sampling when applied to continuous or more general spaces, which constitutes an important class of problems in practice. In this paper, we study Thompson sampling when applied to a broad class of average cost stochastic control problems where the state, control, and parameter spaces are all general measurable spaces. The main contributions of our paper are establishing theoretical performance guarantees for Thompson sampling as measured by: first, expected posterior sampling error; and second, average per period regret. [ABSTRACT FROM AUTHOR]

Devising efficient algorithms to solve continuously-varying strongly convex optimization programs is key in many applications, from control systems to signal processing and machine learning. In this context, solving means to find and track the optimizer trajectory of the continuously-varying convex optimization program. Recently, a novel prediction–correction methodology has been put forward to set up iterative algorithms that sample the continuously-varying optimization program at discrete time steps and perform a limited amount of computations to correct their approximate optimizer with the new sampled problem and predict how the optimizer will change at the next time step. Prediction–correction algorithms have been shown to outperform more classical strategies, i.e., correction-only methods. Typically, prediction–correction methods have asymptotical tracking errors of the order of $h^2$ , where $h$ is the sampling period, whereas classical strategies have order of $h$. Up to now, prediction–correction algorithms have been developed in the primal space, both for unconstrained and simply constrained convex programs. In this paper, we show how to tackle linearly constrained continuously-varying problem by prediction–correction in the dual space and we prove similar asymptotical error bounds as their primal versions. [ABSTRACT FROM AUTHOR]

This paper concentrates on the integration of relative localization and the formation control over leader–follower networks. First, we develop a consensus-like relative localization scheme for each agent to estimate the real-time relative positions of its neighbors by merely using velocity as well as distance-related measurements and local communications under the assumption that the local frames of all agents share a common orientation. It is shown that the relative position estimate exponentially converges to its true value under a persistent excitation condition on relative velocity between the two agents. Second, an integrated relative localization and a leader–follower formation control are developed by combining the proposed relative localization scheme and a complex Laplacian-based formation control scheme. It is proven that the integrated system globally asymptotically converges to a stationary similar formation in the case of error-free initial relative position estimates while globally uniformly bounds in a neighborhood of the prescribed formation in the case with initial relative position estimation errors. Furthermore, based on the two degrees of freedom of similar formation on translation, the integrated scheme is also generalized to achieve moving similar formation control. Finally, simulations are presented to illustrate the effectiveness of our theoretical results. [ABSTRACT FROM AUTHOR]

Inspired by the interesting findings regarding the hierarchy and coordination of pigeons in Nagy et al.  [Nature, 2010], we revisit the discrete-time leader–follower consensus problem with switching topologies. The purpose of this paper is to investigate the impact of hierarchical topologies and followers’ self-loops on the convergence performance of leader–follower consensus, including the convergence rate and robustness to switching topologies. We first study the fixed topology case, and show that the followers converge to the leader's state in finite time if and only if each follower has no self-loops and the topology is hierarchical. However, we show via counterexample, that leader–follower consensus may not be achieved when some followers have no self-loops and the topology is switching; even if each interaction graph has a spanning tree rooted at the leader. With the aid of binary relation theory, we further develop a new approach to present a novel sufficient condition for leader–follower consensus with switching topologies and with some followers having no self-loops. We prove that when no followers have self-loops and the same hierarchical organization is kept under the switching topologies, then the fastest rate of convergence in leader–follower consensus can be achieved; even in the presence of complex dynamic topologies. This is consistent with the natural phenomena found in pigeons by Nagy et al. [ABSTRACT FROM AUTHOR]

The present work considers the question of convergence of Wang–Landau, multi-canonical and stochastic approximation Monte Carlo algorithms, which was previously investigated on very different levels. In the case of the multi-canonical algorithms, a convergence behavior similar to either the original Wang–Landau or the stochastic approximation Monte Carlo algorithms is shown, depending on details of the algorithm. I suggest a method for error estimation based only on the results of simulations, not involving an information about the exact density of states, which is unknown for most practically interesting problems. [ABSTRACT FROM AUTHOR]

In addition to being observable, systems composed of interconnected subsystems must typically exhibit a particular subsystem structure, if their state is to be estimated successfully by a partition-based moving horizon estimator (PMHE). Suitable subsystem structures—also called system partitionings—can be unknown, may not be desirable for a particular application, or may not exist at all. In this contribution, we present the first PMHE without any restrictions on the system partitioning. We analyze the convergence of its iterations at each time step and the stability of its estimation error dynamics. The benefits of our solution are illustrated on a numerical example. [ABSTRACT FROM AUTHOR]

The paper proposes a technique to estimate the angular velocity of a rigid body from single vector measurements. Compared to the approaches presented in the literature, it does not use attitude information nor rate gyros as inputs. Instead, vector measurements are directly filtered through a nonlinear observer estimating the angular velocity. Convergence is established using a detailed analysis of a linear-time varying dynamics appearing in the estimation error equation. This equation stems from the classic Euler equations and measurement equations. As is proven, the case of free-rotation allows one to relax the persistence of excitation assumption. Simulation results are provided to illustrate the method. [ABSTRACT FROM PUBLISHER]

A novel type of iterative partition-based moving horizon estimators (PMHE) is proposed, the estimates of which approach those of a centralized moving horizon estimator as the number of iterations increases. It uses a deterministic setting and can handle known inputs as well as bounds on the estimated state. We derive conditions on the system, its partitions, and the scalar regularization parameter, which guarantee convergence towards the optimal centralized state estimate as well as stability of the estimation error dynamics, even with a finite number of iterations at each time step. Finally, numerical simulations demonstrate both the features and competitive estimation quality of the proposed method. [ABSTRACT FROM AUTHOR]

Tackling large approximate dynamic programming or reinforcement learning problems requires methods that can exploit regularities of the problem in hand. Most current methods are geared towards exploiting the regularities of either the value function or the policy. We introduce a general classification-based approximate policy iteration (CAPI) framework that can exploit regularities of both. We establish theoretical guarantees for the sample complexity of CAPI-style algorithms, which allow the policy evaluation step to be performed by a wide variety of algorithms, and can handle nonparametric representations of policies. Our bounds on the estimation error of the performance loss are tighter than existing results. [ABSTRACT FROM AUTHOR]

This technical note presents new results on probabilistic characterization of identification errors in their relationships to data sizes and accuracy requirements. Employing the moderate deviations principle, this technical note shows that if the identification accuracy progressively increases with a suitable rate, the probability of an estimate going outside the precision bounds decays exponentially with the data size. The precise rate of the decaying probability is obtained. System identification under regular, binary, and quantized observations are considered. Impact of unmodeled dynamics is also investigated. [ABSTRACT FROM AUTHOR]

A hybrid observer to estimate the frequencies of a signal composed of a finite number of sinusoids and measured through saturation and deadzone nonlinearities is proposed. The observer design exploits the Immersion and Invariance technique combined with a cl assic high-gain approach, yielding semi-global exponential convergence to zero of the estimation errors. Simulation results are provided to illustrate the effectiveness of the observer. [ABSTRACT FROM AUTHOR]

We propose a formation control strategy based on distributed position estimation for single-integrator modeled agents by using relative position measurements. Under the proposed strategy, the estimated and the actual formations globally exponentially converge to the actual and the desired formations, respectively, up to translation, if and only if the interaction graph for the agents has a spanning tree. A sufficient condition is provided for the case that the edge weights of the interaction graph are time-varying. Further, the proposed strategy is applied to formation control of unicycle-like agents. [ABSTRACT FROM AUTHOR]

This brief presents an analysis of the performance of kernel smoothing models used to estimate an unknown target function, addressing the case where the choice of the training set is part of the learning process. In particular, we consider a choice of the points at which the function is observed based on low-discrepancy sequences, which is a family of sampling methods commonly employed for efficient numerical integration. We prove that, under suitable regularity assumptions, consistency of the empirical risk minimization is guaranteed with a good rate of convergence of the estimation error, as well as the convergence of the approximation error. Simulation results confirm, in practice, the good theoretical properties given by the combination of kernel smoothing models with low-discrepancy sampling. [ABSTRACT FROM AUTHOR]

Integrative design of communication mechanism and coordinated control law is an interesting and important problem for multi-agent networks. In this paper, we consider distributed coordination of discrete-time second-order multi-agent systems with partially measurable state and a limited communication data rate. A quantized-observer based encoding-decoding scheme is designed, which integrates the state observation with encoding/decoding. A distributed coordinated control law is proposed for each agent which is given in terms of the states of its encoder and decoders. It is shown that for a connected network, 2-bit quantizers suffice for the exponential asymptotic synchronization of the states of the agents. The selection of controller parameters and the performance limit are discussed. It is shown that the algebraic connectivity and the spectral radius of the Laplacian matrix of the communication graph play key roles in the closed-loop performance. The spectral radius of the Laplacian matrix is related to the selection of control gains, while the algebraic connectivity is related to the spectral radius of the closed-loop state matrix. Furthermore, it is shown that as the number of agents increases, the asymptotic convergence rate can be approximated as a function of the number of agents, the number of quantization levels (communication data rate) and the ratio of the algebraic connectivity to the spectral radius of the Laplacian matrix of the communication graph. [ABSTRACT FROM AUTHOR]