Search

This note is concerned with the stability analysis problem of stochastic neutral-type time-delay systems with multiple delays. A less restrictive constraint is imposed to ensure that the difference operator (which appears on the left hand side of the stochastic neutral-type time-delay systems) possesses the property that "exponentially converging input implies exponentially converging state". Such a constraint is necessary and sufficient for the strong stability of the difference equation associated with the system in the deterministic setting, and thus cannot be further relaxed. [ABSTRACT FROM AUTHOR]

This paper presents weighted stochastic Riccati (WSR) equations for designing multiple types of controllers for linear stochastic systems. The system matrices are independent and identically distributed (i.i.d.) to represent noise in the systems. While the stochasticity can invoke unpredictable control results, it is essentially difficult to design controllers for systems with i.i.d. matrices because the controllers can be solutions to non-algebraic equations. Although an existing method has tackled this difficulty, the method has not realized the generality because it relies on the special form of cost functions for risk-sensitive linear (RSL) control. Furthermore, designing controllers over an infinite-horizon remains challenging because many iterations of solving nonlinear optimization is needed. To overcome these problems, the proposed WSR equations employ a weighted expectation of stochastic equations. Solutions to the WSR equations provide multiple types of controllers characterized by the weight, which contain stochastic optimal and RSL controllers. Two approaches calculating simple recursive formulas are proposed to solve the WSR equations without solving the nonlinear optimization. Moreover, designing the weight yields a novel controller termed the robust RSL controller that has both a risk-sensitive policy and robustness to randomness occurring in stochastic controller design. [ABSTRACT FROM AUTHOR]

In this paper, finite-time stabilization is investigated for a class of non-local Lipschitzian large-scale stochastic nonlinear systems with two types of uncertainties, including parametric uncertainties and uncertain interactions. First, we present a new stochastic finite-time stability theorem on stochastic adaptive finite-time control, by which we obtain an existing finite-time stability result for the interconnected stochastic nonlinear systems. Then, for a class of large-scale stochastic nonlinear systems with uncertainties, an adaptive finite-time controller is constructively designed. It is proved by the developed finite-time stability theorem that there exists a global weak solution to the closed-loop system, the trivial weak solution of the closed-loop system is globally stable in probability and the states of the system almost surely converge to the origin in finite time. At last, a simulation example is given to show the effectiveness of the proposed design procedure. [ABSTRACT FROM AUTHOR]

Recent years have witnessed a booming interest in data-driven control of dynamical systems. However, the implicit data-driven output predictors are vulnerable to uncertainty such as process disturbance and measurement noise, causing unreliable predictions and unexpected control actions. In this brief, we put forward a new data-driven approach to output prediction of stochastic linear time-invariant (LTI) systems. By utilizing the innovation form, the uncertainty in stochastic LTI systems is recast as innovations that can be readily estimated from input–output data without knowing system matrices. In this way, by applying the fundamental lemma to the innovation form, we propose a new innovation-based data-driven output predictor (OP) of stochastic LTI systems, which bypasses the need for identifying state–space matrices explicitly and building a state estimator. The boundedness of the second moment of prediction errors in closed-loop is established under mild conditions. The proposed data-driven OP can be integrated into optimal control design for better performance. Numerical simulations demonstrate the outperformance of the proposed innovation-based methods in output prediction and control design over existing formulations. [ABSTRACT FROM AUTHOR]

The problem of online change point detection is to detect abrupt changes in properties of time series, ideally as soon as possible after those changes occur. Existing work on online change point detection either assumes i.i.d. data, focuses on asymptotic analysis, does not present theoretical guarantees on the trade-off between detection accuracy and detection delay, or is only suitable for detecting single change points. In this work, we study the online change point detection problem for linear dynamical systems with unknown dynamics, where the data exhibits temporal correlations and the system could have multiple change points. We develop a data-dependent threshold that can be used in our test that allows one to achieve a pre-specified upper bound on the probability of making a false alarm. We further provide a finite-sample-based bound for the probability of detecting a change point. Our bound demonstrates how parameters used in our algorithm affect the detection probability and delay, and provides guidance on the minimum required time between changes to guarantee detection. [ABSTRACT FROM AUTHOR]

This paper is concerned with a class of linear–quadratic stochastic large-population problems with partial information, where the individual agent only has access to a noisy observation process related to the state. The dynamics of each agent follows a linear stochastic differential equation driven by the individual noise, and all agents are coupled together via the control average term. By studying the associated mean-field game and using the backward separation principle with a state decomposition technique, the decentralized optimal control can be obtained in the open-loop form through a forward–backward stochastic differential equation with the conditional expectation. The optimal filtering equation is also provided. Thanks to the decoupling method, the decentralized optimal control can also be further presented as the feedback of state filtering via the Riccati equation. The explicit solution of the control average limit is given, and the consistency condition system is discussed. Moreover, the related ɛ -Nash equilibrium property is verified. To illustrate the good performance of theoretical results, an example in finance is studied. [ABSTRACT FROM AUTHOR]

In all the references on stochastic fixed-time stability, the customary treatment of the stochastic noise in the worst-case sense is that it is treated as the unfavorable factor for system stability. Consequently, stochastic fixed-time Lyapunov-type conditions are rather restrictive. Realizing this limitation, we revisit stochastic fixed-time stability and present a generalized fixed-time stability theorem for stochastic differential equations. This theorem completely removes the assumption in these references that the differential operator of the Lyapunov function must be strictly negative, and reveals a positive role of the stochastic noise in stochastic fixed-time stability. As the application of this theorem and its corollary, we solve the problem of fixed-time stabilization for stochastic nonlinear systems with high-order and low-order nonlinearities. [ABSTRACT FROM AUTHOR]

This paper is concerned with approximately solving the optimal predictor-feedback control problem of multiplicative-noise systems with input delay in infinite horizon. The optimal predictor-feedback control, provided by the analytical method, is determined by Riccati–ZXL equations and is hard to obtain in the case of unknown system dynamics. We aim to propose a policy iteration (PI) algorithm for solving the optimal solution by approximate dynamic programming. For convergence analysis of the algorithm, we first develop a necessary and sufficient stabilizing condition, in the form of several new Lyapunov-type equations, which parameterizes all predictor-feedback controllers and can be seen as an important addition to Lyapunov stability theory. We then propose an iterative scheme for the Riccati–ZXL equations computations, along with convergence analysis, based on the condition. Inspired by this scheme, a data-driven online PI algorithm, convergence implied in that of the iterative scheme, is proposed for the optimal predictor-feedback control problem without full system dynamics. Finally, a numerical example is used to evaluate the proposed PI algorithm. [ABSTRACT FROM AUTHOR]

This article studies a fundamental problem of security of cyber–physical systems (CPSs): that of detecting, almost surely, the presence of malicious components in the CPS. We assume that some of the actuators may be malicious while all sensors are honest. We introduce a novel idea of separability of state trajectories generated by CPSs in two situations: those under the nominal no-attack situation and those under the influence of an attacker. We establish its connection to the security of CPSs, particularly in detecting the presence of malicious actuators (if any) in them. As primary contributions, we establish necessary and sufficient conditions for the aforementioned detection in CPSs modeled as Markov decision processes (MDPs). Moreover, we focus on the mechanism of perturbing the pre-determined control policies of the honest agents in CPSs modeled as stochastic linear systems, by injecting a certain class of random process called private excitation ; sufficient conditions for detectability and non-detectability of the presence of malicious actuators, assuming that the policies are randomized history-dependent and randomized Markovian, are established. Several technical aspects of our results are discussed extensively. [ABSTRACT FROM AUTHOR]

Available methods for identification of stochastic dynamical systems from input–output data generally impose restricting structural assumptions on either the noise structure in the data-generating system or the possible state probability distributions. In this paper, we introduce a novel identification method of such systems, which results in a dynamical model that is able to produce the time-varying output distribution accurately without taking restrictive assumptions on the data-generating process. The method is formulated by first deriving a novel and exact representation of a wide class of nonlinear stochastic systems in a so-called meta-state–space form, where the meta-state can be interpreted as a parameter vector of a state probability function space parameterization. As the resulting representation of the meta-state dynamics is deterministic, we can capture the stochastic system based on a deterministic model, which is highly attractive for identification. The meta-state–space representation often involves unknown and heavily nonlinear functions, hence, we propose an artificial neural network (ANN)-based identification method capable of efficiently learning nonlinear meta-state–space models. We demonstrate that the proposed identification method can obtain models with a log-likelihood close to the theoretical limit even for highly nonlinear, highly stochastic systems. [ABSTRACT FROM AUTHOR]

This paper is concerned with the multi-player stochastic differential pursuit-evasion game problem based on directed graphs. To solve the challenges posed by limited communication and ubiquitous noise, a novel Riccati equation is proposed based on the linear–quadratic exponential cost function under both complete and partial observations. The optimal strategies for pursuers and evaders are obtained based on the direct method of completing the square and Radon–Nikodym derivative, without the need to solve the complex Hamilton–Jacobi–Isaacs equation. The strategy presented in this paper is distributed and can be implemented without requiring any global information, under the constraints of a directed topology. Additionally, the proposed partial algorithm introduces an information filter to estimate partial observations. We also demonstrate that the proposed strategy constitutes a Nash equilibrium. Numerical simulations confirm the effectiveness of our strategy in both complete and partial observation scenarios. [ABSTRACT FROM AUTHOR]

This paper presents a novel filter design for the single-output stochastic non-linear systems subjected to non-Gaussian noises and the proposed assumptions. Based on a state transformation, the unmeasurable states of the systems can be estimated where non-linear terms in the systems have been eliminated. It has been shown that the estimation error is linearly dynamical regarding to the presented vector-valued filter gain which can be optimised by minimising the entropy-based performance criterion. In addition, the convergence of the presented algorithm is analysed in mean-square sense and a numerical example is given to verify the effectiveness of the presented filtering algorithm. Meanwhile, the extended Kalman filter, unscented particle filter and minimum entropy filter are given for the comparisons of the filtering performance. Following the presented framework, some extensions of the presented filtering algorithm are discussed to indicate the flexibility of the filter design. The contribution of this paper can be summarised as establishing a novel minimum entropy filtering framework which consists of model transformation, entropy optimisation and convergence analysis. [ABSTRACT FROM AUTHOR]

This paper is concerned with the issue of smooth control for semi-Markov jump linear systems. The performance metric on the control smoothness for general non-deterministic switched systems is renewed for the studied stochastic switched system by exploiting the statistical information including transition probabilities and sojourn-time probability mass functions. The practical asynchronous switching phenomenon in the system is considered. A piecewise control law with flexible transition-dependent duration for control smoothness is proposed as a first attempt, which outperforms the traditional fixed duration that is either less- or over-designed in terms of smoothness. A receding algorithm is developed to eliminate the nonlinearity curse caused by the terms of power of matrices in the derived existence criterion of controller. Illustrative examples including a class of hybrid terrestrial/aerial bicopters are presented to demonstrate the validity and applicability of the theoretical findings [ABSTRACT FROM AUTHOR]

This paper studies the stabilization problem of networked control systems (NCSs) with random packet dropouts caused by stochastic channels. To describe the effects of stochastic channels on the information transmission, the transmission times are assumed to be deterministic, whereas the packet transmission is assumed to be random. We first propose a stochastic scheduling protocol to model random packet dropouts, and address the properties of the proposed stochastic scheduling protocol. The proposed scheduling protocol provides a unified modelling framework for a general class of random packet dropouts due to different stochastic channels. Next, the proposed scheduling protocol is embedded into the closed-loop system, which leads to a stochastic hybrid model for NCSs with random packet dropouts. Based on this stochastic hybrid model, we follow the emulation approach to establish sufficient conditions to guarantee uniform global asymptotical stability in probability. In particular, an upper bound on the maximally allowable transmission interval is derived explicitly for all stochastic protocols satisfying Lyapunov conditions that guarantee uniform global asymptotic stability in probability. Finally, two numerical examples are presented to demonstrate the derived results. [ABSTRACT FROM AUTHOR]

In this paper, we address fixed time stability in probability of discrete-time stochastic dynamical systems. Unlike finite time stability in probability, wherein the finite time almost sure convergence behavior of the dynamical system depends on the system initial conditions, fixed time stability in probability involves finite time stability in probability for which the stochastic settling-time is guaranteed to be independent of the system initial conditions. More specifically, we develop Lyapunov theorems for fixed time stability in probability for Itô-type stationary nonlinear stochastic difference equations including a Lyapunov theorem that involves a Lyapunov difference satisfying an exponential inequality of the Lyapunov function that gives rise to a minimum bound on the average stochastic settling-time characterized by the primary and secondary branches of the Lambert W function. [ABSTRACT FROM AUTHOR]

This study is concerned with the computationally efficient robust model predictive control (MPC) for discrete-time stochastic switching systems. Aiming at achieving optimal control synthesis under the requirement of bumpless transfer control (BTC), the min–max MPC formulation is extended to the transition-dependent paradigm, such that the abrupt variation of mode-dependent gains can be mitigated via BTC performance optimization. To address the high computational complexity caused by the transition-dependent variables and constraints, the MPC algorithm is developed with an offline-to-online synthesis strategy, achieving comparable online computation cost to non-switching MPC. Meanwhile, a class of more general stochastic switching signals is considered, where the sojourn time may follow any form of distribution, and the recursive feasibility and mean-square stability are theoretically guaranteed. Compared with existing studies on switching MPC and the ones on BTC, this work avoids the assumption of the Markov property on mode switching and reduces conservatism by exploiting the statistical information of sojourn time. An illustrative example is provided to show the potential of the obtained results. [ABSTRACT FROM AUTHOR]

We study a partially observable nonlinear stochastic system with unknown parameters, where the given time scales of the states and measurements may be distinct. The proposed setting is inspired by disease management, particularly leukemia. [ABSTRACT FROM AUTHOR]

This paper gives a new perspective for the stability analysis of a class of two-layer stochastic switched systems (TLSSSs). In the considered TLSSS, the upper layer system can be regarded as a semi-Markov jump system, in which the arbitrary switching is embedded in the corresponding latent subsystems. A new stability analysis method based on the switched transfer function matrix and ergodic property of the semi-Markov process is proposed to tackle difficulties caused by the complex nature of TLSSSs. Under such a framework, auxiliary systems are constructed for upper layer systems through the corresponding eigenvalues, and the latent subsystems of each auxiliary system are associated with the switched transfer function matrix. Then, the relations between the Lyapunov functions and prescribed system parameters are constructed to deal with the instability caused by the mode jumping. By comprehensively utilizing the property of the semi-Markov process and the above-mentioned methods, sufficient conditions are further presented to ensure the exponential mean-square (EMS) stability and exponential almost sure (EAS) stability, respectively. A numerical example is given to illustrate the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

This paper studies the p th moment stability of discrete-time Markov jump systems by utilizing the extended system method, which is composed of the operator spectrum technique and the ℋ -representation technique. Firstly, some extended systems are constructed by the operator spectrum technique and the ℋ -representation technique to transform the discrete-time Markov jump systems into the extended systems. Furthermore, the relationship in p th moment stability between the extended systems and the stochastic systems is discussed. Next, with the help of the extended systems, the necessary and sufficient conditions for the even-order moment stability, and several sufficient conditions/necessary conditions for the odd-order moment stability are obtained, respectively. Additionally, as applications of the extended system method, observability and detectability in p th moment sense are discussed, and the necessary and sufficient conditions for observability and detectability in even-order moment are obtained, respectively. Finally, several examples are given to illustrate the validity of the results. [ABSTRACT FROM AUTHOR]

This work discusses the sampled-data sliding mode control problem for Markovian jump systems in the presence of Denial-of-Service (DoS) attacks, where the state signals are sampled stochastically and transmitted via vulnerable communication networks. The successful transmitted sequence with double randomness is established, and the probability distribution on both the attacked signals and stochastic delays are given. With the aid of the coordinate transformation, a linear sliding surface is constructed and the mode-dependent sliding mode controller is designed by means of only successfully transmitted state signals. Moreover, the resultant closed-loop systems composed of partial state and sliding mode variable are described as a continuous time-delay system with double randomness depending on the probability distribution characteristics of both the sampling periods and DoS attacks. Both the mean-square exponential ultimate boundedness of the closed-loop system and the reachability of the sliding surface are analyzed and the corresponding sufficient conditions are derived. Meanwhile, an optimization problem with genetic algorithm is formulated for achieving fast convergence of the state trajectory, small ultimate bound of the system state and small sliding bound. Finally, the simulation results via the numerical example are provided to illustrate the proposed sampled-data sliding mode control scheme. [ABSTRACT FROM AUTHOR]

We develop a method to approximate the moments of a discrete-time stochastic polynomial system. Our method is built upon Carleman linearization with truncation. Specifically, we take a stochastic polynomial system with finitely many states and transform it into an infinite-dimensional system with linear deterministic dynamics, which describe the exact evolution of the moments of the original polynomial system. We then truncate this deterministic system to obtain a finite-dimensional linear system, and use it for moment approximation by iteratively propagating the moments along the finite-dimensional linear dynamics across time. We provide efficient online computation methods for this propagation scheme with several error bounds for the approximation. Our results also show that precise values of certain moments at a given time step can be obtained when the truncated system is sufficiently large. Furthermore, we investigate techniques to reduce the offline computation load using reduced Kronecker power. Based on the obtained approximate moments and their errors, we also provide hyperellipsoidal regions that are safe for some given probability bound. Those bounds allow us to conduct probabilistic safety analysis online through convex optimization. We demonstrate our results on a logistic map with stochastic dynamics and a vehicle dynamics subject to stochastic disturbance. [ABSTRACT FROM AUTHOR]

This paper formulates and solves new problems of inverse optimal stabilization and inverse optimal gain assignment for stochastic nonlinear systems driven by Lévy processes. First, a theorem is developed to design inverse optimal stabilizers based on inverse pre-optimal stabilization controllers for stochastic systems with known noise characteristics, where it does not require to solve a Hamilton–Jacobi–Bellman equation. Second, another theorem is developed to design inverse optimal gain assignment controllers for stochastic systems with unknown noise characteristics, where there is no need to solve a Hamilton–Jacobi–Isaacs equation. Third, the results are applied to design inverse optimal stabilizers for Euler–Lagrange systems. [ABSTRACT FROM AUTHOR]

This paper studies the leader-following consensus problem for high-order stochastic nonlinear multi-agent systems. Each agent is in the strict feedback form with nonlinear functions in drift and diffusion terms and admitting time-varying incremental rates. A novel distributed observer-type consensus protocol is proposed based only on the relative output measurements of neighboring agents. The dynamic gains in controller are designed to compensate for the time-varying coefficients of nonlinear functions. By using appropriate state transformation, it is proved that the 1th moment exponential leader-following consensus is guaranteed by the proposed protocol. Different from some existing approaches, the proposed method only requires the output information of neighboring agents to be shared and does not need the observers' full state variables to be transmitted, thus the network bandwidth is saved. In addition, the controller is constructed without using the traditional backstepping method, which makes the design procedure simple and convenient for use. Finally, a simulation example is given to illustrate the effectiveness of the theoretical results. [ABSTRACT FROM AUTHOR]

We show that stochastic programming provides a framework to design hierarchical model predictive control (MPC) schemes for periodic systems. This is based on the observation that, if the state policy of an infinite-horizon problem is periodic, the problem can be cast as a stochastic program (SP). This reveals that it is possible to update periodic state targets by solving a retroactive optimization problem that progressively accumulates historical data. Moreover, we show that the retroactive problem is a statistical approximation of the SP and thus delivers optimal targets in the long run. Notably, the computation of the optimal targets can be achieved without data forecasts. The SP setting also reveals that the retroactive problem can be seen as a high-level hierarchical layer that provides targets to guide a low-level MPC controller that operates over a short period at high time resolution. We derive a retroactive scheme tailored to linear systems by using cutting plane techniques and suggest strategies to handle nonlinear systems and to analyze stability properties. [ABSTRACT FROM AUTHOR]

This paper is concerned with a compositional approach for constructing infinite abstractions of interconnected discrete-time stochastic control systems. The proposed approach uses the interconnection matrix and joint dissipativity-type properties of subsystems and their abstractions described by new notions of so-called stochastic storage functions. The interconnected abstraction framework is based on new notions of so-called stochastic simulation functions, constructed compositionally using stochastic storage functions of components. Using stochastic simulation functions, one can quantify the distance between original interconnected stochastic systems and interconnected abstractions in the probabilistic setting. Accordingly, one can leverage the proposed results to perform analysis and synthesis over abstract interconnected systems, and then carry the results back over concrete ones. In the first part of the paper, we derive dissipativity-type compositional reasoning for the quantification of the distance in probability between the interconnection of stochastic control subsystems and that of their abstractions. Moreover, we focus on a class of discrete-time nonlinear stochastic control systems with independent noises in the abstract and concrete subsystems, and propose a computational scheme to construct abstractions together with their corresponding stochastic storage functions. In the second part of the paper, we consider specifications expressed as syntactically co-safe linear temporal logic formulae and show how a synthesized policy for the abstract system can be refined to a policy for the original system while providing a guarantee on the probability of satisfaction. We demonstrate the effectiveness of the proposed results by constructing an abstraction (totally 3 dimensions) of the interconnection of three discrete-time nonlinear stochastic control subsystems (together 222 dimensions) in a compositional fashion such that the compositionality condition does not require any constraint on the number or gains of the subsystems. We also employ the constructed abstraction as a substitute to synthesize a controller enforcing a syntactically co-safe linear temporal logic specification. [ABSTRACT FROM AUTHOR]

In this paper, a new finite-horizon H ∞ control problem is considered for a class of time-varying systems with stochastic nonlinearities, measurements degradation and chance constraints. The purpose of the addressed problem is to design the time-varying controller such that the closed-loop system satisfies the prespecified H ∞ disturbance attenuation requirement and certain chance constraints on the controlled output vector z k (i.e., the probability of the controlled output z k belonging to a given set is larger than a prescribed value). A modified maximum-volume-inscribed-ellipsoid (MVIE) method is employed to convert the chance constraint into some tractable inequalities, which could be conveniently handled by the recursive linear matrix inequality (RLMI) approach. Then, by using stochastic control analysis method, sufficient conditions are derived for the existence of the desired multi-objective controller and, furthermore, the gains of the desired controllers are characterized by means of RLMIs. Finally, two illustrative examples and a practical system are proposed to highlight the effectiveness and applicability of the presented controller design technology. [ABSTRACT FROM AUTHOR]

This paper deals with the stability of linear stochastic systems in the presence of sporadic measurements and communication delays. Motivated by considerations about the efficient use of the available resources, two so-called dynamic discrete-time event-triggered control (DTETC) strategies tailored to the system are designed, including some existing periodic event-triggered control strategies as particular cases. An impulsive switched system approach and a switched time-delay system approach are developed for modeling and stability analysis of the resulting closed-loop systems, respectively. In addition, it is shown that under the same decay rate, the designed dynamic DTETC may further reduce the conservatism than their static counterparts. Interestingly, the proposed impulsive switched system approach can capture the stabilizing effect of the communication delays. A numerical example is provided to illustrate the theoretical results. [ABSTRACT FROM AUTHOR]

We consider the problem of state-feedback control of linear continuous-time stochastic systems with nonlinear diffusion terms affected by time-varying input delays or, equivalently, by time-varying state measurement delays. We propose a finite-dimensional control law based on closed-loop predictors and derive sufficient delay bounds for the exponential stability in mean and exponential mean square stability with prescribed rate. [ABSTRACT FROM AUTHOR]

This paper investigates the problem of ℋ − index for stochastic linear continuous-time systems involving Markov jump and multiplicative noise. A set of generalized differential Riccati equations (GDREs) are presented and it is shown that the solvability of GDREs is necessary and sufficient for the feasibility of an ℋ − index larger than γ > 0. Our results extend the corresponding deterministic cases to stochastic systems. Furthermore, the infinite horizon ℋ − index problem for s q u a r e systems is considered. Finally, the effectiveness of the obtained results is tested by two examples. [ABSTRACT FROM AUTHOR]

The estimation problem for stochastic parametric nonlinear dynamical models is recognized to be challenging. The main difficulty is the intractability of the likelihood function and the optimal one-step ahead predictor. In this paper, we present relatively simple prediction error methods based on non-stationary predictors that are linear in the outputs. They can be seen as extensions of the linear prediction error methods for the case where the hypothesized model is stochastic and nonlinear. The resulting estimators are defined by analytically tractable objective functions in several common cases. It is shown that, under certain identifiability and standard regularity conditions, the estimators are consistent and asymptotically normal. We discuss the relationship between the suggested estimators and those based on second-order equivalent models as well as the maximum likelihood method. The paper is concluded with a numerical simulation example as well as a real-data benchmark problem. [ABSTRACT FROM AUTHOR]

Different from the existing delay-dependent stability criteria which depend on the size of the delays in both the drift and diffusion parts of stochastic time-delay systems (STDSs), this paper shows that the delay-dependent stability criterion depends only on the drift part and works for arbitrary large delays of the diffusion part of STDSs. By developing the stochastic version descriptor system approach, simple stability criterion is further derived for aperiodic sampled-data stochastic systems. The theoretical results are illustrated by means of a numerical example. [ABSTRACT FROM AUTHOR]

Abstract Markov decision processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize costs) for a stochastic dynamical system. In this paper, we introduce a new type of sensor measurement to the MDP model that provides additional information about the stochastic process, and hence that information can be incorporated in the decision policy to increase the performance. The new model is tailored for environments with high uncertainty. With the additional measurements, more refined information on the possible state transition is provided in real-time before taking an action. This new MDP model with sequential measurements is referred to as sequentially-observed MDP (SO-MDP). We show that the SO-MDP shares some similar properties with a standard MDP; among randomized history dependent policies, deterministic Markovian policies are still optimal. Optimal SO-MDP policies have the advantage of producing better total rewards than standard MDP policies due to the additional measurements, however computing these policies is more complex. We present two algorithms for solving the finite-horizon SO-MDP problem: the first algorithm is based on linear-programming, and the second algorithm is based on dynamic programming. We show that the complexity of computing optimal policies of the SO-MDP model with perfect sensors is the same as standard MDP. Simulations demonstrate that the SO-MDP model outperforms the standard MDP model in the presence of high environmental uncertainty. [ABSTRACT FROM AUTHOR]

Abstract In many control contexts, such as vision-based control, data-processing methods are needed to distill information from measurement data (such as images). These data-processing methods introduce several undesirable effects such as delays, measurement inaccuracies and possible absence of information, which limit closed-loop performance. Typically, a single processing method with an appropriate compromise between these effects is chosen in practice. Instead of settling for a compromise using only one fixed processing method, we propose to break the design trade-off by switching on-line between several data-processing methods having different delay, accuracy, and data-loss characteristics. We provide a modeling framework for sensing and data-processing methods that is suitable for control applications and incorporates the characteristics of the undesirable effects mentioned above. Using the models provided by the framework, we provide explicit policies for switching on-line between sensing methods with different characteristics based on a modified rollout strategy. Our approach formally guarantees that an LQG-type infinite horizon performance is better than, or at least not worse than, non-switching approaches. The advantages of the proposed methodology are further highlighted via a numerical example. [ABSTRACT FROM AUTHOR]

Abstract This paper is concerned with the mean-square exponential stability and performance analysis, stabilization, dissipative control and dissipativity-based sliding mode control (SMC) problems for discrete-time switched stochastic hybrid systems. Such systems can be also regarded as switched hybrid systems with stochastic perturbation. Event detectors, which can save computational resource, are designed to determine whether the current data should be transmitted or not in the design process. Firstly, by proposing an average dwell time method combining with a switched (mode-dependent) Lyapunov function, a sufficient condition is established under the event-triggering scheme, which guarantees that the considered switched stochastic system is mean-square exponentially stable, and then an explicit parametrization of the desired stabilization controller is also given. Secondly, the event-triggering dissipativity analysis and the dissipative control problems are investigated. A dissipativity performance condition is proposed to guarantee the mean-square exponential stability and strict dissipativity for the discrete-time switched stochastic system, and then the dissipative controller is implemented by the state feedback. Thirdly, a switched observer is constructed to estimate unavailable state variables, based on which a differential-type sliding surface function is designed, with which the sliding mode dynamical equations can be easily obtained. The resulting closed-loop system with sliding mode dynamics is an autonomous switched stochastic system. Dissipativity analysis and synthesis are both investigated for the closed-loop system, and consequently sufficient conditions are derived, which pave the way for solving the event-triggering observer-based dissipative sliding mode control problem. Moreover, an event-triggering output-based SMC law is first synthesized to drive the system trajectories to enter a predefined sliding vicinity. Finally, illustrative examples are employed to validate the proposed controller design schemes. [ABSTRACT FROM AUTHOR]

Abstract In this paper, stability and stabilizability of discrete-time dual switching linear systems is investigated. The switched systems under consideration have two switching variables. One of them is stochastic, described by an underlying Markov chain; the other one can be regarded either as a deterministic disturbance or as a control input, leading to stability or stabilizability problems, respectively. For the considered class of systems, sufficient conditions for mean square stability (with or without control gain synthesis) and mean square stabilizability are provided in terms of matrix inequalities. When the stochastic switching is driven by an independent identically distributed sequence, we establish simpler conditions without additional conservatism. Then, it is shown how the proposed framework can be used to study aperiodic sampled-data systems with stochastic computation times. The results are illustrated on examples borrowed from the literature. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we study the problem of synchronization over a network, with nonlinear components dynamics modeled in Lure form, and linear stochastic interaction among the components. To study this problem we utilize the stochastic version of Positive Real Lemma (PRL), which is used to provide a sufficient condition for synchronization of stochastic network systems. This sufficiency condition is a function of nominal (mean coupling) Laplacian eigenvalues, and the statistics of link uncertainty in the form of coefficient of dispersion (CoD). Robust control-based small-gain interpretation is provided for the derived sufficiency condition which allows us to define the margin of synchronization. The margin of synchronization is used to understand the important tradeoff between the component dynamics, network topology, and uncertainty characteristics for network synchronization. Our results indicate that significant role played by both the largest and the second smallest eigenvalue of the nominal Laplacian in synchronization of stochastic networks. Furthermore, for a special class of network system connected over torus topology we provide an analytical expression for the tradeoff between the number of neighbors and the dimension of the torus. Similarly, by exploiting the identical nature of component dynamics computationally efficient sufficient condition, independent of network size, is provided for general class of network system. Simulation results for network of coupled oscillators with stochastic link uncertainty are presented to verify the developed theoretical framework. [ABSTRACT FROM AUTHOR]

Taking different system structures in different Markovian modes into consideration, this paper studies the structured stabilisation of a class of superlinear hybrid stochastic delay systems by feedback control based on discrete-time state observations. The controller is designed in a bounded state area, rather than every observable state, in order to reduce control cost. The time delay is more general in terms of the classical differentiability assumption being relaxed. Compared with the existing papers on discrete-state-feedback stabilisation problem, a new method to estimate the difference between current-time state and discrete-time state is presented, as a result of which the conditions imposed on the underlying system and the control function are less restrictive. Meanwhile, the Lyapunov functional used in this paper is modified to adapt to this change. Finally, an application to stochastic structured neural networks is given to demonstrate the practicability of the developed theory. [ABSTRACT FROM AUTHOR]

In this paper, we consider the distributed state estimation problem of continuous-time linear time-invariant (LTI) systems, where communication channels are attacked by stochastic impulsive sequences. In a networked system, the communication channel between an agent and its neighbor is vulnerable to being attacked. Continuous attacks have been widely studied so far, but with the development of attack techniques, intermittent attacks have been emerging, which are more difficult to be identified and detected, and have not been well portrayed and studied. Therefore, this paper investigates a random intermittent injection attack, which leads to abrupt and random changes of the state information transmitted by the neighbors. First, this attack is modeled as a stochastic impulsive sequential attack, where both its impulsive time and impulsive intensity are random. Then a distributed observer under stochastic impulsive sequential attack is proposed. Since the impulsive attack destroys the stability of the system, in order to attenuate the impact of the impulsive attack, a large feedback gain is necessary. By using the graph theory, matrix theory and stochastic analysis, sufficient conditions for the existence of the distributed observer in the sense of mean square and in the sense of almost sure are presented, respectively. Finally, the effectiveness of the proposed distributed observer is verified by numerical simulations. [ABSTRACT FROM AUTHOR]

The paper focuses on active fault diagnosis of stochastic large-scale systems decomposed into several coupled subsystems, where the subsystem fault-free and faulty behavior is described in the multiple-model framework. In the active approach, the detector generates optimal excitation input to improve the diagnosis. This paper proposes a solution to the problems with cost functions in a generally non-separable form. Unlike the separable form, the generally non-separable form facilitates penalizing missed detections, false alerts, and incorrect, false identifications involving several subsystems simultaneously. Three approaches are proposed to treat such cost functions in the offline stage of the active fault diagnosis algorithm. Their performance is illustrated using a simple example and an elaborate example involving a power network system. [ABSTRACT FROM AUTHOR]

In this work, we study verification and synthesis problems for safety specifications over unknown discrete-time stochastic systems. When a model of the system is available, barrier certificates have been successfully applied for ensuring the satisfaction of safety specifications. In this work, we formulate the computation of barrier certificates as a robust convex program (RCP). Solving the acquired RCP is hard in general because the model of the system that appears in one of the constraints of the RCP is unknown. We propose a data-driven approach that replaces the uncountable number of constraints in the RCP with a finite number of constraints by taking finitely many random samples from the trajectories of the system. We thus replace the original RCP with a scenario convex program (SCP) and show how to relate their optimizers. We guarantee that the solution of the SCP is a solution of the RCP with a priori guaranteed confidence when the number of samples is larger than a specific value. This provides a lower bound on the safety probability of the original unknown system together with a controller in the case of synthesis. We also discuss an extension of our verification approach to a case where the associated robust program is non-convex and show how a similar methodology can be applied. Finally, the applicability of our proposed approach is illustrated through three case studies. [ABSTRACT FROM AUTHOR]

In this paper, finite-time stabilization is investigated in depth for stochastic nonlinear systems with stochastic inverse dynamics. The contributions of this work are characterized by the following novel features: (i) Motivated by the concept of stochastic integral input-to-state stability (SiISS) and the existing stochastic finite-time stability theorem, a new concept of finite-time stochastic integral input-to-state stability (FT-SiISS) using Lyapunov function is first introduced, and an inclusion relationship among three types of stochastic stability, i.e., FT-SiISS, FT-SISS and SISS, is rigorously established. On the basis of FT-SiISS, we develop two new FT-SiISS small-gain conditions and discuss their relationship. Then, to analyze finite-time stability of stochastic nonlinear systems with FT-SiISS inverse dynamics, a stochastic finite-time stability theorem is improved. (ii) As the application of (i), we propose a unified finite-time control method, which can simultaneously handle stochastic nonlinear systems with FT-SiISS or FT-SISS inverse dynamics, and guarantee that the closed-loop system has an almost surely continuous solution, all the closed-loop signals are bounded almost surely, and its trivial solution is stochastically finite-time stable. [ABSTRACT FROM AUTHOR]

We consider a distributed learning framework, where there are a group of agents communicating with a centralized coordinator. The goal of the agents is to find the root of an operator composed of the local operators at the agents. Such a framework models many practical problems in different areas, including those in federated learning and reinforcement learning. For solving this problem, we study the popular distributed stochastic approximation. Over a series of time epoch, each agent runs a number of local stochastic approximation steps based on its own data, whose results are then aggregated at the centralized coordinator. Existing theoretical guarantees for the finite-time performance of local stochastic approximation are studied under the common assumption that the local data at each agent is sampled i.i.d. Such an assumption may not hold in many applications, where the data are temporally dependent, for example, they are sampled from some dynamical systems. In this paper, we study the setting where the data are generated from Markov random processes, which are often used to model the systems in stochastic control and reinforcement learning. Our main contribution is to characterize the finite-time performance of the local stochastic approximation under this setting. We provide explicit formulas for the rates of this method for both constant and time-varying step sizes when the local operators are strongly monotone. Our results show that these rates are within a logarithmic factor of the comparable bounds under independent data. We also provide a number of numerical simulations to illustrate our theoretical results by applying local SA in solving problems in robust identification and reinforcement learning over multi-agent systems. [ABSTRACT FROM AUTHOR]

LaSalle theorems, sometimes called as invariance-like theorems, have witnessed abundant applications as powerful tools of analysing system convergence. However, these theorems have twofold limitations: On the one hand, the uniformity of convergence with respect to initial state values cannot be offered, which indicates that the possibility of excessively slow convergence could not be ruled out for a given bounded set of initial state values. On the other hand, unbounded time-variations are not allowed in the systems, and meanwhile, the boundedness of all the states are required even if one merely concerns the convergence of partial states, precluding many scenarios with the unboundedness (e.g., induced by stochastic noise) for some states defined on the whole time horizon. Towards the limitations of LaSalle theorems, this paper seeks to further develop convergence theorems in the stochastic framework. First, a Lyapunov-like function based theorem on the convergence owning the uniformity with respect to initial state values is presented for Itô-type stochastic systems, with the infinitesimal of Lyapunov-like functions exploited more delicately than those in stochastic LaSalle theorems. Based on this, a framework of adaptive stabilization with the uniformity of convergence is established for uncertain stochastic nonlinear systems. In particular, the performance specifications involved are impossible to achieve by applying LaSalle theorems as in the related results. Second, an enlarged convergence theorem is established with Lyapunov-like conditions moderately relaxed, which allows the unboundedness for partial states defined on the whole time horizon and has potential applications in the presence of unbounded time-variations. What is notable, the enlarged convergence theorem is nontrivial to verify, which compels us to propose an extended Barbălat lemma with the conventional requirement of uniform continuity relaxed in the stochastic framework. [ABSTRACT FROM AUTHOR]

In this paper, we study the exact controllability of forward and backward stochastic difference system (FBSDS) with multiplicative noise. Different from the standard system governed by forward stochastic difference equation (FSDE), the inclusion of both initial and terminal conditions in FBSDS complicates the solvability which also leads to the challenge of exact controllability. To overcome the difficulty, we transform the FBSDS into an equivalent backward stochastic difference equation. Accordingly, we derive both the necessary and sufficient Gramian matrix criterion and the Rank criterion for the exact controllability of FBSDS, which is the main contribution of the paper. [ABSTRACT FROM AUTHOR]

Abstract Opacity is an important information-flow property that arises in security and privacy analysis of cyber–physical systems. Among many different notions of opacity, K -step opacity requires that the intruder can never determine unambiguously that the system was at a secret state for any specific instant within K steps prior to that particular instant. This notion becomes infinity-step opacity when K goes to infinity. Existing works on the analysis of infinite-step opacity and K -step opacity only provide a binary characterization, i.e., a system is either opaque or non-opaque. To analyze infinite-step and K -step opacity more quantitatively, in this paper, we investigate the verification of infinite-step and K -step opacity in the context of stochastic discrete-event systems. A new notion of opacity, called almost infinite-step opacity (respectively, almost K -step opacity), is proposed to capture whether or not the probability of violating infinite-step opacity (respectively, K -step opacity) is smaller than a given threshold. We also provide effective algorithms for the verification of almost infinite-step opacity and almost K -step opacity. [ABSTRACT FROM AUTHOR]

Abstract This paper is concerned with the issues of stability analysis and control synthesis for a class of linear stochastic switching systems in discrete-time domain. The switching dynamics are considered to be governed by a semi-Markov process and the sojourn time for each system mode is deemed to be finite. A mode-dependent control scheme is employed with an adaptation sense in the presence of time delays in the mode switching of controller, which is manifested as a constant lag between the system mode and the controller mode. A novel form of Lyapunov function is adopted, in which the Lyapunov matrix depends on the modes of both the system and the controller as well as the time since the occurrence of the last mode switching. On the basis of the new proposed σ -error mean-square stability that integrates the weights of all the system modes, numerically testable stability criteria are developed via the semi-Markov kernel approach. In virtue of certain techniques that can eliminate the terms containing powers or products of matrices, a desired mode-dependent stabilizing controller is designed such that the closed-loop system is σ -error mean-square stable by allowing a mode-unmatched controller to perform before the controller switches to a mode-matched one. Finally, the theoretical results are applied to a practical example of one joint of a space robot manipulator to demonstrate the effectiveness, applicability and superiority of the proposed control strategy as well as the necessity of considering the mode-switching delays in the designed controller. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to present a network realization theory for a class of mixed quantum–classical linear stochastic systems. Two forms, the standard form and the general form, of this class of linear mixed quantum–classical systems are proposed. Necessary and sufficient conditions for their physical realizability are derived. Based on these physical realizability conditions, a network synthesis theory for this class of linear mixed quantum–classical systems is developed, which clearly exhibits the quantum component, the classical component, and their interface. An example is used to illustrate the theory presented in this paper. [ABSTRACT FROM AUTHOR]

This paper presents a novel partial differential equation (PDE)-based framework for controlling an ensemble of robots, which have limited sensing and actuation capabilities and exhibit stochastic behaviors, to perform mapping and coverage tasks. We model the ensemble population dynamics as an advection–diffusion–reaction PDE model and formulate the mapping and coverage tasks as identification and control problems for this model. In the mapping task, robots are deployed over a closed domain to gather data, which is unlocalized and independent of robot identities, for reconstructing the unknown spatial distribution of a region of interest. We frame this task as a convex optimization problem whose solution represents the region as a spatially-dependent coefficient in the PDE model. We then consider a coverage problem in which the robots must perform a desired activity at a programmable probability rate to achieve a target spatial distribution of activity over the reconstructed region of interest. We formulate this task as an optimal control problem in which the PDE model is expressed as a bilinear control system, with the robots’ coverage activity rate and velocity field defined as the control inputs. We validate our approach with simulations of a combined mapping and coverage scenario in two environments with three target coverage distributions. [ABSTRACT FROM AUTHOR]

In this work, we investigate the problem of event-triggered stabilization for a class of stochastic nonlinear systems. An event-triggered control (ETC) approach is proposed by introducing an additional internal dynamic variable. The presented event-triggered mechanism (ETM) can guarantee the existence of a positive lower bound on inter-event times (or called inter-execution times). In addition, the presented technique can ensure the second moment asymptotic stability of the closed-loop stochastic nonlinear system. [ABSTRACT FROM AUTHOR]

Optimal control of bilinear systems has been a well-studied subject in the areas of mathematical and computational optimal control. However, effective methods for solving emerging optimal control problems involving an ensemble of deterministic or stochastic bilinear systems are underdeveloped. These burgeoning problems arise in diverse applications from quantum control and molecular imaging to neuroscience. In this work, we develop an iterative method to find optimal controls for an inhomogeneous bilinear ensemble system with free-endpoint conditions. The central idea is to represent the bilinear ensemble system at each iteration as a time-varying linear ensemble system, and then solve it in an iterative manner. We analyze convergence of the iterative procedure and discuss optimality of the convergent solutions. The method is directly applicable to solve the same class of optimal control problems involving a stochastic bilinear ensemble system driven by independent additive noise processes. We demonstrate the robustness and applicability of the developed iterative method through practical control designs in neuroscience and quantum control. [ABSTRACT FROM AUTHOR]