Search

Most cyber–physical systems (CPS) encounter a large volume of data which is added to the system gradually in real time and not altogether in advance. In this paper, we provide a theoretical framework that yields optimal control strategies for such CPS at the intersection of control theory and learning. In the proposed framework, we use the actual CPS, i.e., the "true" system that we seek to optimally control online, in parallel with a model of the CPS that is available. We then institute an information state for the system which does not depend on the control strategy. An important consequence of this independence is that for any given choice of a control strategy and a realization of the system's variables until time t , the information states at future times do not depend on the choice of the control strategy at time t but only on the realization of the decision at time t , and thus they are related to the concept of separation between estimation of the state and control. Namely, the future information states are separated from the choice of the current control strategy. Such control strategies are called separated control strategies. Hence, we can derive offline the optimal control strategy of the system with respect to the information state, which might not be precisely known due to model uncertainties or complexity of the system, and then use standard learning approaches to learn the information state online while data are added gradually to the system in real time. We show that after the information state becomes known, the separated control strategy of the CPS model derived offline is optimal for the actual system. We illustrate the proposed framework in a dynamic system consisting of two subsystems with a delayed sharing information structure. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we consider the stochastic optimal control problem of discrete-time linear systems subject to Markov jumps and multiplicative noises under two criteria. The first one is an unconstrained mean–variance trade-off performance criterion along the time, and the second one is a minimum variance criterion along the time with constraints on the expected output. We present explicit conditions for the existence of an optimal control strategy for the problems, generalizing previous results in the literature. We conclude the paper by presenting a numerical example of a multi-period portfolio selection problem with regime switching in which it is desired to minimize the sum of the variances of the portfolio along the time under the restriction of keeping the expected value of the portfolio greater than some minimum values specified by the investor. [Copyright &y& Elsevier]

Abstract: A new approach for finding an exact analytical solution to the modified Hamilton–Jacobi–Bellman equation is proposed. Together with the recently developed hybrid solution method, the proposed strategy allows to find a solution to a whole class of stochastic optimal control problems with bounded in magnitude control force. [Copyright &y& Elsevier]

Abstract: The standard approach in computer-controlled systems is to sample and control periodically. In certain applications, such as networked control systems or energy-constrained systems, it could be advantageous to instead use event-based control schemes. Aperiodic event-based control of first-order stochastic systems has been investigated in previous work. In any real implementation, however, it is necessary to have a well-defined minimum inter-event time. In this paper, we explore two such sporadic control schemes for first-order linear stochastic systems and compare the achievable performance to both periodic and aperiodic control. The results show that sporadic control can give better performance than periodic control in terms of both reduced process state variance and reduced control action frequency. [Copyright &y& Elsevier]

Abstract: A ‘proportional’ order-up-to policy reacting to ARMA demand is analyzed using stochastic optimal control theory. This policy is compared with a full-state-feedback order-up-to policy. Necessary conditions for an optimum of a weighted sum of the inventory and the ordering variances for both policies are formulated. Based on this a relatively simple expression for the ‘full-state’ policy is derived. The comparison between the two policies demonstrates that the ‘intuitively’ designed proportional policy does not fulfill the objective of controlling both the inventory and ordering variance for all parameter values of the demand model as well as the full-state-feedback policy. The full-state-feedback policy outperforms the proportional policy in several aspects. [Copyright &y& Elsevier]